⬆️ upgrading rabbitmq to 4.1.2 ⚠️🚨

[GitHK]

⚠️ devops

When this is released to a deployment:

• release login to RabbitMQ interface
• search for all exchanges containing deferred_manager and remove them (should be 2)
• search for all queues containing deferred_manager and dynamic-scheduler and remove them (should be 5 or more)
• after all queues have been removed, also restart dynamic-scheduler service.
• before releasing, enable all features

What do these changes do?

✅ Upgrading rabbitmq to 4.1.2 is free of risks with regard to the x-death header. The code always used it in the intended case.

✅ Removed the warning relative to transient_nonexcl_queues by making the queues durable, which has no negative impact:

• RPC backend
• DeferredTasks

✅ If declare_queue would create a transient_nonexcl_queues warning in RabbitMQ, it will now raise an error.

⚠️ Regarding Celery, it will still trigger transient_nonexcl_queues on the following:

$ rabbitmqctl list_queues name durable exclusive
celery@sto-worker-cpu-bound-neagu-wkst-1-fkcwzzmkh3bbvacwjvpg6kyod.celery.pidbox	false	false
celeryev.9c5251c6-d3f8-4bea-a2cd-8f6aba966a35	false	false
celeryev.bcb95510-6469-4ef8-b6af-0cf5bda4833c	false	false
celery@sto-worker-neagu-wkst-1-q9ijewz3pntvnpko041z56y5h.celery.pidbox	false	false

I found the following issue in the community: celery/kombu#2237

Related issue/s

• Upgrade to rabbit 4.1.X osparc-ops-environments#1077
• closes Rabbitmq dead letter: support both 3.13 and 4.1 rabbitmq versions #8077

How to test

Dev-ops

[codecov]

Codecov Report

❌ Patch coverage is 57.14286% with 3 lines in your changes missing coverage. Please review.
✅ Project coverage is 88.19%. Comparing base (26bc00b) to head (b089b78).
⚠️ Report is 40 commits behind head on master.

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #8109      +/-   ##
==========================================
- Coverage   88.21%   88.19%   -0.02%     
==========================================
  Files        1883     1883              
  Lines       72416    72419       +3     
  Branches     1272     1273       +1     
==========================================
- Hits        63883    63873      -10     
- Misses       8159     8170      +11     
- Partials      374      376       +2     

Flag	Coverage Δ
integrationtests	64.24% <ø> (-0.02%)	⬇️
unittests	86.81% <57.14%> (-0.01%)	⬇️

Components	Coverage Δ
pkg_aws_library	93.93% <ø> (ø)
pkg_celery_library	87.34% <ø> (ø)
pkg_dask_task_models_library	79.62% <ø> (ø)
pkg_models_library	93.26% <ø> (ø)
pkg_notifications_library	85.26% <ø> (ø)
pkg_postgres_database	88.22% <ø> (ø)
pkg_service_integration	70.17% <ø> (ø)
pkg_service_library	71.72% <57.14%> (-0.08%)	⬇️
pkg_settings_library	90.45% <ø> (ø)
pkg_simcore_sdk	85.16% <ø> (ø)
agent	93.81% <ø> (ø)
api_server	92.85% <ø> (ø)
autoscaling	95.88% <ø> (ø)
catalog	92.34% <ø> (ø)
clusters_keeper	99.13% <ø> (ø)
dask_sidecar	92.37% <ø> (ø)
datcore_adapter	97.94% <ø> (ø)
director	76.14% <ø> (ø)
director_v2	90.92% <ø> (ø)
dynamic_scheduler	96.27% <ø> (ø)
dynamic_sidecar	90.06% <ø> (ø)
efs_guardian	89.76% <ø> (ø)
invitations	91.44% <ø> (ø)
payments	92.60% <ø> (ø)
resource_usage_tracker	92.50% <ø> (ø)
storage	86.39% <ø> (-0.09%)	⬇️
webclient	∅ <ø> (∅)
webserver	88.52% <ø> (-0.03%)	⬇️

---

Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 26bc00b...b089b78. Read the comment docs.

🚀 New features to boost your workflow:

• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[YuryHrytsuk]

Thanks,

What are the effects of making these queues durable?

[pcrespov]

thx

[sonarqubecloud]

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

[sanderegg]

thanks

[GitHK]

Thanks,

What are the effects of making these queues durable?

It makes them persist to disk.

• the DeferredManager's queues now correctly durable, meaning: all it's data should survive a reboot of the rabbitmq service
• the RPC interface's has set auto_delete and durable to True, meaning: survives a server restart but will automatically disappear once it's no longer in use

[GitHK]

@Mergifyio queue

[mergify]

queue

✅ The pull request has been merged automatically

The pull request has been merged automatically at f976a56