TLDR; I deployed a django + celery app on heroku but I’m unable to see Celery logs on heroku logs.
django==5.0.1
celery==5.3.4
Here are the relevant files
Procfile
web: daphne ecom_proj.asgi:application --port $PORT --bind 0.0.0.0 -v2
worker: celery -A ecom_proj.celery_app worker -E -B --loglevel=INFO
settings.py
LOGGING = {
"version": 1,
"disable_existing_loggers": False,
"formatters": {
"base": {
"()": "django.utils.log.ServerFormatter",
"format": "[{server_time}] {name} {levelname} {module} {message}",
"style": "{",
}
},
"handlers": {
"console": {
"class": "logging.StreamHandler",
"formatter": "base",
"level": "INFO",
},
},
"loggers": {
"": {
"handlers": ["console"],
"level": "INFO",
},
"django": {
"handlers": ["console"],
"level": "INFO",
"propagate": False,
},
"celery": {
"handlers": ["console"],
"level": "INFO",
"propagate": False,
},
},
}
celery.py
import os
from celery import Celery
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "ecom_proj.settings")
celery_app = Celery("ecom_proj_celery")
celery_app.autodiscover_tasks()
myapp/tasks.py
import logging
from celery import shared_task
logger = logging.getLogger(__name__)
@shared_task
def my_task():
logger.info("task has started")
# Do something....
With all these files, I deploed the django project on heroku and able to see web process’s log messages however for worker process (Celery), I can’t see any logs that are in task functions. although I’m able to see all celery logs on local machine with same configuration.
I went through different SOF questions and most of them are outdated and based on django-heroku which is already deprecated.
I used celery signals as well like worker_process_init
, beat_init
and setup_logging
, below is the code. I tried to set CELERY_WORKER_HIJACK_ROOT_LOGGER = False
as well.
I tried different combinations of these approaches but none of those seems to be working. Not sure what exactly I’m missing. I’d really appreciate if someone can share the working configuration on these versions of django and celery or point me to the correct resource.
chola bhatura is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.