[Feature]: Clean up celery tasks if function map / run call fails #8408
Description
Is there an existing issue for this?
- I have searched the existing issues
User Story
When at the moment the function map/run call fails, there are possibly still celery workers running that are running the functions.
Ideally we have a mechanism to catch exceptions / context managers to let all the running celery workers clean up after themselves (including running comp services), and also make sure all the celery workers get a termination signal.
Definition of Done
When a function map/run fails, all the celery workers are killed and all the data they generated is cleaned.