🐛 Monitor Celery tasks cancellation

[giancarloromeo]

What do these changes do?

Add an abort monitor that checks for tasks abortion to cancel the corresponding task.

CeleryWorkers don't write progress in Celerys task metadata but on separate field of a Redis hash. E.g.:

celery-task-meta:     <- keys managed by Celery (execution)
    osparc:
        42:
          <task-uuid>
          
celery-task-info:     <- keys managed by us (Hash) (info)
    osparc:
        42:
          <task-uuid>
              metadata:
              progress:

Related issue/s

• fixes: Celery export_data tasks do not return when aborted #7505

How to test

Dev-ops checklist

• No ENV changes or I properly updated ENV (read the instruction)

[codecov]

Codecov Report

Attention: Patch coverage is 93.10345% with 8 lines in your changes missing coverage. Please review.

Project coverage is 87.27%. Comparing base (c5fe5bc) to head (1c15a29).
Report is 1 commits behind head on master.

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #7514      +/-   ##
==========================================
- Coverage   87.44%   87.27%   -0.18%     
==========================================
  Files        1740     1370     -370     
  Lines       67389    56951   -10438     
  Branches     1143      595     -548     
==========================================
- Hits        58931    49704    -9227     
+ Misses       8138     7061    -1077     
+ Partials      320      186     -134     

Flag	Coverage Δ
integrationtests	65.11% <ø> (-0.03%)	⬇️
unittests	86.30% <93.10%> (-0.34%)	⬇️

Components	Coverage Δ
api	∅ <ø> (∅)
pkg_aws_library	∅ <ø> (∅)
pkg_dask_task_models_library	∅ <ø> (∅)
pkg_models_library	∅ <ø> (∅)
pkg_notifications_library	∅ <ø> (∅)
pkg_postgres_database	∅ <ø> (∅)
pkg_service_integration	∅ <ø> (∅)
pkg_service_library	72.65% <50.00%> (-0.02%)	⬇️
pkg_settings_library	∅ <ø> (∅)
pkg_simcore_sdk	85.40% <ø> (ø)
agent	96.46% <ø> (ø)
api_server	90.02% <ø> (ø)
autoscaling	96.08% <ø> (ø)
catalog	92.52% <ø> (ø)
clusters_keeper	99.24% <ø> (ø)
dask_sidecar	91.29% <ø> (ø)
datcore_adapter	98.12% <ø> (ø)
director	76.78% <ø> (ø)
director_v2	91.30% <ø> (+0.02%)	⬆️
dynamic_scheduler	97.40% <ø> (ø)
dynamic_sidecar	90.11% <ø> (ø)
efs_guardian	89.79% <ø> (ø)
invitations	93.28% <ø> (ø)
payments	92.66% <ø> (ø)
resource_usage_tracker	89.12% <ø> (ø)
storage	87.84% <93.85%> (+<0.01%)	⬆️
webclient	∅ <ø> (∅)
webserver	85.94% <ø> (+0.06%)	⬆️

---

Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update c5fe5bc...1c15a29. Read the comment docs.

🚀 New features to boost your workflow:

• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[matusdrobuliak66]

Thanks! Can you please add some description to this PR explaining a bit your changes, thanks 👍

[odeimaiz]

Thanks!

[bisgaard-itis]

Very cool! Thanks a lot 🏊🏻‍♂️

[sonarqubecloud]

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

[Copilot]

Copilot reviewed 19 out of 19 changed files in this pull request and generated no comments.

Comments suppressed due to low confidence (2)

services/storage/tests/unit/test_rpc_handlers_simcore_s3.py:616

• Verify that the updated index (2) for accessing the progress value aligns with the new signature of CeleryTaskWorker.set_progress. If the call arguments change in the future, this index may need an update to avoid silent failures in progress reporting.

progress_updates = [x[0][2].actual_value for x in task_progress_spy.call_args_list]

services/storage/tests/unit/test_async_jobs.py:261

• [nitpick] Consider replacing the magic number (60 * 10) with a named constant or adding inline documentation to clarify its purpose, ensuring future maintainers understand this value is chosen to prevent hanging tests.

payload=60 * 10,  # test hangs if not cancelled properly

[pcrespov]

thx