Skip to content

Conversation

@sanderegg
Copy link
Member

@sanderegg sanderegg commented Oct 24, 2025

What do these changes do?

Second trial at refactoring autoscaling like in #8423
Re-revert #8557

The problem was due to an infortunate refactoring in services/autoscaling/src/simcore_service_autoscaling/models.py/.
Added a note and an issue for refactoring that problem:
#8559

Related issue/s

How to test

Dev-ops

@sanderegg sanderegg added this to the Imparable milestone Oct 24, 2025
@sanderegg sanderegg self-assigned this Oct 24, 2025
@mergify
Copy link
Contributor

mergify bot commented Oct 24, 2025

🧪 CI Insights

Here's what we observed from your CI run for ffbbfb8.

✅ Passed Jobs With Interesting Signals

Pipeline Job Signal Health on master Retries 🔍 CI Insights 📄 Logs
CI integration-tests Base branch is broken, but the job passed. Looks like this might be a real fix 💪 Broken 0 View View

@sanderegg sanderegg force-pushed the autoscaling/dask-nthreads2 branch from 4c40854 to e3d2a06 Compare October 25, 2025 09:34
@codecov
Copy link

codecov bot commented Oct 25, 2025

Codecov Report

❌ Patch coverage is 96.31336% with 8 lines in your changes missing coverage. Please review.
✅ Project coverage is 87.44%. Comparing base (a2ecd48) to head (ffbbfb8).

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #8558      +/-   ##
==========================================
- Coverage   87.52%   87.44%   -0.08%     
==========================================
  Files        2011     1608     -403     
  Lines       78612    66977   -11635     
  Branches     1351      764     -587     
==========================================
- Hits        68802    58566   -10236     
+ Misses       9406     8158    -1248     
+ Partials      404      253     -151     
Flag Coverage Δ
integrationtests 63.97% <28.57%> (+0.02%) ⬆️
unittests 85.93% <96.31%> (-0.30%) ⬇️
Components Coverage Δ
pkg_aws_library 94.98% <100.00%> (+1.37%) ⬆️
pkg_celery_library ∅ <ø> (∅)
pkg_dask_task_models_library 79.00% <76.92%> (-0.34%) ⬇️
pkg_models_library ∅ <ø> (∅)
pkg_notifications_library ∅ <ø> (∅)
pkg_postgres_database ∅ <ø> (∅)
pkg_service_integration ∅ <ø> (∅)
pkg_service_library 70.98% <62.50%> (-0.01%) ⬇️
pkg_settings_library ∅ <ø> (∅)
pkg_simcore_sdk 84.95% <ø> (ø)
agent 93.10% <ø> (ø)
api_server 91.62% <ø> (ø)
autoscaling 95.83% <99.20%> (+0.83%) ⬆️
catalog 92.06% <ø> (ø)
clusters_keeper 99.14% <ø> (ø)
dask_sidecar 91.60% <ø> (-0.56%) ⬇️
datcore_adapter 97.95% <ø> (ø)
director 75.72% <ø> (ø)
director_v2 90.87% <100.00%> (-0.07%) ⬇️
dynamic_scheduler 96.66% <ø> (ø)
dynamic_sidecar 90.44% <ø> (ø)
efs_guardian 89.83% <ø> (ø)
invitations 90.90% <ø> (ø)
payments 92.80% <ø> (ø)
resource_usage_tracker 92.11% <ø> (ø)
storage 86.56% <ø> (-0.13%) ⬇️
webclient ∅ <ø> (∅)
webserver 87.11% <66.66%> (+0.03%) ⬆️

Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update a2ecd48...ffbbfb8. Read the comment docs.

🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@sanderegg sanderegg marked this pull request as ready for review October 25, 2025 09:57
@sonarqubecloud
Copy link

Copy link
Member

@pcrespov pcrespov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thx!

def as_flat_dict(self) -> dict[str, int | float | str]:
"""Like model_dump, but flattens generic_resources to top level keys"""
base = self.model_dump()
base.update(base.pop("generic_resources"))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

MIONR: just as a precaution in case one day you add another object-like field , here i would do it in a more general fashion. I know right now is perhaps an overkill, but it might be handy to add some helpers like this in common_library.dict_tools so that afterwards we could do something like

common_library.dict_tools.flatten(self.model_dump(mode='json'))

A draft courtesy of chatty (double check first!)

def _flatten(obj, prefix=""):
        items = {}
        if isinstance(obj, dict):
            for k, v in obj.items():
                items.update(_flatten(v, f"{prefix}{k}." if prefix else k))
        elif isinstance(obj, list):
            for i, v in enumerate(obj):
                items.update(_flatten(v, f"{prefix}{i}." if prefix else f"{i}"))
        else:
            items[prefix] = obj
        return items

def as_flat_dict(self) -> dict[str, int | float | str]:
    """Recursively flattens all nested dicts/lists into a single-level dict."""
    return _flatten(self.model_dump(mode='json'))

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Autoscaling: in non-billable systems the chosen machine type does not take in account the removed resources as the dask-sidecar does

2 participants