How safe is setting consumer_timeout to long values like 10 years #8382
-
|
Im guessing it's there for a reason, but I can't seem to find any answer to this question. In short I have a django app where users get set badges that months or even years down the line need to be auto removed for which I use celery. My question is will setting consumer_timeout to large values pose a risk for those actions to not be performed or performed early/late (mind you a delay of even a few hours shouldn't be a big deal for long duration tasks) |
Beta Was this translation helpful? Give feedback.
Replies: 1 comment
-
|
Messaging systems are not built for scheduling tasks for 10 years from now.
You cannot assume that a delivery to a consumer will remain unprocessed for 10 years. Nodes and clusters are usually restarted in a 10 year period. For delaying tasks by years you probably want a data store and a periodically triggered job, be it with Cron, one of the K8S-based schedulers, or anything else. Not a consumer from a queue. |
Beta Was this translation helpful? Give feedback.
Messaging systems are not built for scheduling tasks for 10 years from now.
consumer_timeoutis a guardrail that prevents buggy consumers from driving nodes out of disk space (when consumers consume from quorum queues and use manual acknowledgements).You cannot assume that a delivery to a consumer will remain unprocessed for 10 years. Nodes and clusters are usually restarted in a 10 year period.
For delaying tasks by years you probably want a data store and a periodically triggered job, be it with Cron, one of the K8S-based schedulers, or anything else. Not a consumer from a queue.