✨Computational clusters: connect autoscaling to RabbitMQ ⚠️ #7485
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## master #7485 +/- ##
==========================================
+ Coverage 87.60% 87.63% +0.03%
==========================================
Files 1758 1751 -7
Lines 68116 67942 -174
Branches 1124 1124
==========================================
- Hits 59672 59540 -132
+ Misses 8135 8093 -42
Partials 309 309
Continue to review full report in Codecov by Sentry.
🚀 New features to boost your workflow:
|
sanderegg
force-pushed
the
clusters-keeper/connect-primary-to-rabbitmq
branch
2 times, most recently
from
6ab02df to
2bf6e94
Compare
sanderegg
requested review from
YuryHrytsuk,
giancarloromeo,
matusdrobuliak66,
mrnicegyu11 and
pcrespov
as code owners
services/clusters-keeper/src/simcore_service_clusters_keeper/utils/clusters.py
Show resolved
Hide resolved
pcrespov
approved these changes
Apr 9, 2025
services/clusters-keeper/src/simcore_service_clusters_keeper/core/settings.py
Show resolved
Hide resolved
sanderegg
force-pushed
the
clusters-keeper/connect-primary-to-rabbitmq
branch
3 times, most recently
from
a1c0e94 to
9c35879
Compare
YuryHrytsuk
approved these changes
Apr 14, 2025
6 tasks
sanderegg
force-pushed
the
clusters-keeper/connect-primary-to-rabbitmq
branch
from
9c35879 to
47f6be2
Compare
There was a problem hiding this comment.
Pull Request Overview
This PR integrates RabbitMQ settings for autoscaling in private clusters by introducing a new environment variable and updating related settings and tests.
- Added a new environment variable to inject RabbitMQ settings into Docker Compose.
- Updated the settings schema, tests, and related utilities to propagate and validate the RabbitMQ configuration.
Reviewed Changes
Copilot reviewed 10 out of 10 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
| services/docker-compose.yml | Added PRIMARY_EC2_INSTANCES_RABBIT_SETTINGS environment variable for RabbitMQ. |
| services/clusters-keeper/tests/unit/test_utils_clusters.py | Included new RabbitMQ fixtures and tests to verify RabbitMQ settings propagation. |
| services/clusters-keeper/tests/unit/test_modules_dask.py | Updated TLSAuthentication usage to the new model_json_schema() pattern. |
| services/clusters-keeper/tests/unit/conftest.py | Modified fixture to properly unset RABBIT_HOST when disabling RabbitMQ. |
| services/clusters-keeper/src/simcore_service_clusters_keeper/utils/clusters.py | Appended AUTOSCALING_RABBITMQ into environment variables using RabbitSettings. |
| services/clusters-keeper/src/simcore_service_clusters_keeper/data/docker-compose.yml | Injected AUTOSCALING_RABBITMQ in the deploy compose file. |
| services/clusters-keeper/src/simcore_service_clusters_keeper/core/settings.py | Added PRIMARY_EC2_INSTANCES_RABBIT field in the settings schema. |
| services/autoscaling/tests/unit/test_modules_dask.py | Reformatted async call for task results, along with TLSAuthentication update. |
| packages/settings-library/src/settings_library/rabbit.py | Revised allowed_schemes typing and JSON schema examples for Rabbit settings. |
| packages/models-library/src/models_library/clusters.py | Updated schema examples for TLSAuthentication and cluster configuration. |
Comments suppressed due to low confidence (1)
services/docker-compose.yml:223
- Environment variable 'PRIMARY_EC2_INSTANCES_RABBIT_SETTINGS' is inconsistent with the settings field 'PRIMARY_EC2_INSTANCES_RABBIT'. Consider renaming it to 'PRIMARY_EC2_INSTANCES_RABBIT' for consistency.
PRIMARY_EC2_INSTANCES_RABBIT_SETTINGS: ${PRIMARY_EC2_INSTANCES_RABBIT_SETTINGS}
giancarloromeo
approved these changes
Apr 23, 2025
|
1 task
This was referenced Apr 23, 2025
YuryHrytsuk
changed the title
YuryHrytsuk
changed the title
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What do these changes do?
PRIMARY_EC2_INSTANCES_RABBITfor setting RabbitMQ settings in user private clustersThis PR shall allow private cluster autoscaling to send logs to the main simcore rabbitMQ.
This is a pre-requisite to see logs about scaling up/down of clusters and also send information about the cluster current status.
Related issue/s
How to test
Dev-ops checklist