-
Notifications
You must be signed in to change notification settings - Fork 483
Closed
Description
Summary:
My app is growing and is now running more than 56K periodic tasks. I am using Django Celery Beat for this. I noticed that If I add a new periodic task, my celery beat is stuck.
- Celery Version: 5.1.2
- Celery-Beat Version: 2.2.1
Exact steps to reproduce the issue:
- Start celery beat with the +56K periodic tasks
- Add a new periodic task
- DatabaseScheduler is stuck for hours
Detailed information
When I add a periodic task, all the stored ones stop for a long time. Looking at the celery beat logs I realized that for several hours, my celery beat was stuck at this message :
[2021-11-04 11:46:31,548: INFO/MainProcess] DatabaseScheduler: Schedule changed.
[2021-11-04 11:46:31,702: INFO/MainProcess] DatabaseScheduler: Schedule changed.
[2021-11-04 11:46:31,849: INFO/MainProcess] DatabaseScheduler: Schedule changed.
[2021-11-04 11:46:32,383: INFO/MainProcess] DatabaseScheduler: Schedule changed.
[2021-11-04 11:46:32,529: INFO/MainProcess] DatabaseScheduler: Schedule changed.
[2021-11-04 11:46:32,690: INFO/MainProcess] DatabaseScheduler: Schedule changed.
[2021-11-04 11:46:32,814: INFO/MainProcess] DatabaseScheduler: Schedule changed.
[2021-11-04 11:46:32,864: INFO/MainProcess] DatabaseScheduler: Schedule changed.
[2021-11-04 11:46:33,004: INFO/MainProcess] DatabaseScheduler: Schedule changed.
[2021-11-04 11:46:33,145: INFO/MainProcess] DatabaseScheduler: Schedule changed.
[2021-11-04 11:46:33,282: INFO/MainProcess] DatabaseScheduler: Schedule changed.
.....
It's been 3 hours like this. Is there a maximum number of periodic tasks ? How can I handle this?
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
No labels