Skip to content

Conversation

@sanderegg
Copy link
Member

@sanderegg sanderegg commented Jan 23, 2025

What do these changes do?

After #6929 changes, a missing hot buffer would be replaced by a warm buffer if possible.
A bug in the implementation of this feature had for consequence that all the available warm buffers would be started to replace 1 missing hot buffer which leads to:

  • additional costs as too many machines are running
  • longer waiting times as all the warm buffers were used to replace 1 hot buffer and it takes time to re-create the missing warm buffers

driving test: test_warm_buffers_only_replace_hot_buffer_if_service_is_started_issue7071

@matusdrobuliak66 @YuryHrytsuk This should be hotfixed

Related issue/s

How to test

Dev-ops checklist

@sanderegg sanderegg added the a:autoscaling autoscaling service in simcore's stack label Jan 23, 2025
@sanderegg sanderegg added this to the Singularity milestone Jan 23, 2025
@sanderegg sanderegg self-assigned this Jan 23, 2025
@sanderegg sanderegg requested a review from pcrespov as a code owner January 23, 2025 17:58
@sanderegg sanderegg force-pushed the autoscaling/bugfix/scale-up-warm-buffer-too-much branch from f7447e3 to 711ff6d Compare January 23, 2025 17:59
@codecov
Copy link

codecov bot commented Jan 23, 2025

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 87.19%. Comparing base (e0a69be) to head (87299c0).
Report is 1 commits behind head on master.

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #7079      +/-   ##
==========================================
- Coverage   87.26%   87.19%   -0.08%     
==========================================
  Files        1634     1627       -7     
  Lines       64262    64074     -188     
  Branches     2051     2051              
==========================================
- Hits        56078    55868     -210     
- Misses       7848     7869      +21     
- Partials      336      337       +1     
Flag Coverage Δ
integrationtests 66.73% <ø> (-1.12%) ⬇️
unittests 85.83% <100.00%> (+0.27%) ⬆️
Components Coverage Δ
api ∅ <ø> (∅)
pkg_aws_library 93.49% <ø> (ø)
pkg_dask_task_models_library 97.09% <ø> (ø)
pkg_models_library 91.43% <ø> (ø)
pkg_notifications_library 84.57% <ø> (ø)
pkg_postgres_database 88.41% <ø> (ø)
pkg_service_integration 70.18% <ø> (ø)
pkg_service_library 74.13% <ø> (ø)
pkg_settings_library 90.49% <ø> (ø)
pkg_simcore_sdk 85.38% <ø> (-0.12%) ⬇️
agent 96.45% <ø> (ø)
api_server 90.55% <ø> (ø)
autoscaling 96.10% <100.00%> (+<0.01%) ⬆️
catalog 90.32% <ø> (ø)
clusters_keeper 99.24% <ø> (ø)
dask_sidecar 91.26% <ø> (ø)
datcore_adapter 93.18% <ø> (ø)
director 76.92% <ø> (ø)
director_v2 91.27% <ø> (-0.02%) ⬇️
dynamic_scheduler 97.21% <ø> (ø)
dynamic_sidecar 89.75% <ø> (-0.04%) ⬇️
efs_guardian 90.39% <ø> (ø)
invitations 93.42% <ø> (ø)
osparc_gateway_server ∅ <ø> (∅)
payments 92.66% <ø> (ø)
resource_usage_tracker 89.33% <ø> (+0.38%) ⬆️
storage 89.57% <ø> (ø)
webclient ∅ <ø> (∅)
webserver 84.56% <ø> (-0.36%) ⬇️

Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update e0a69be...87299c0. Read the comment docs.

Copy link
Member

@mguidon mguidon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

@sanderegg sanderegg changed the title 🐛Autoscaling: Fix issue where all warm buffers would be used to replace hot buffers 🐛Autoscaling: Fix issue where all warm buffers would be used to replace hot buffers (🚨) Jan 23, 2025
Copy link
Member

@pcrespov pcrespov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thx

@sanderegg sanderegg force-pushed the autoscaling/bugfix/scale-up-warm-buffer-too-much branch from 711ff6d to ff27450 Compare January 24, 2025 07:31
@sonarqubecloud
Copy link

@sanderegg sanderegg merged commit 7f218ea into ITISFoundation:master Jan 24, 2025
85 of 91 checks passed
@sanderegg sanderegg deleted the autoscaling/bugfix/scale-up-warm-buffer-too-much branch January 24, 2025 10:58
matusdrobuliak66 pushed a commit that referenced this pull request Jan 24, 2025
@matusdrobuliak66 matusdrobuliak66 mentioned this pull request Mar 6, 2025
63 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

a:autoscaling autoscaling service in simcore's stack

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Autoscaling: all warm buffers are started to replace hot buffer

5 participants