Skip to content

🐛Stop pruning /inputs folder after unzipping input port #8016

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 9 commits into from
Jul 1, 2025
Merged

🐛Stop pruning /inputs folder after unzipping input port #8016

merged 9 commits into from
Jul 1, 2025

Conversation

@sanderegg
Copy link
Member

@sanderegg sanderegg commented Jun 30, 2025 •
edited
Loading

What do these changes do?

https://github.com/ITISFoundation/osparc-simcore/pull/7697/files#diff-72320e9071f977f6ae6bb9a112730cdb7d5f1c8854cbfb08a2a80816196c5e83 introduced a bug in the dynamic-sidecar:

  • when an input file is a zip file and the port type is not application/zip, then it gets downloaded and extracted in the corresponding port_key/ subfolder.
  • Since the above mentioned PR, the folder was defined as a PrunableFolder which takes a snapshot of the current file tree under that folder. Once the prune function is called it deletes anything in that folder
  • in the case of the zip file, the folder assigned to the PrunableFolder object is 1 too high in the hierarchy, thus this deletes all the other input_x folders around

This PR fixes this.

Others:

  • add a more robust test around notifying service when comp_tasks table is modified

Related issue/s

How to test

Dev-ops

@sanderegg sanderegg added this to the Engage milestone Jun 30, 2025
@sanderegg sanderegg self-assigned this Jun 30, 2025
@sanderegg sanderegg added the a:webserver webserver's codebase. Assigning the area is particularly useful for bugs label Jun 30, 2025
@codecov
Copy link

codecov bot commented Jun 30, 2025 •
edited
Loading

Codecov Report

Attention: Patch coverage is 0% with 1 line in your changes missing coverage. Please review.

Project coverage is 60.87%. Comparing base (22374df) to head (05235b4).
Report is 1 commits behind head on master.

❗ There is a different number of reports uploaded between BASE (22374df) and HEAD (05235b4). Click for more details.

HEAD has 32 uploads less than BASE
Flag BASE (22374df) HEAD (05235b4)
unittests 33 1
Additional details and impacted files 
@@             Coverage Diff             @@
##           master    #8016       +/-   ##
===========================================
- Coverage   87.88%   60.87%   -27.01%     
===========================================
  Files        1853      696     -1157     
  Lines       71478    32808    -38670     
  Branches     1258      176     -1082     
===========================================
- Hits        62817    19972    -42845     
- Misses       8297    12778     +4481     
+ Partials      364       58      -306
Flag Coverage Δ *Carryforward flag
integrationtests 58.03% <0.00%> (-6.22%) ⬇️ Carriedforward from 7bdf96a
unittests 89.19% <0.00%> (+2.70%) ⬆️

*This pull request uses carry forward flags. Click here to find out more.

Components Coverage Δ
api ∅ <ø> (∅)
pkg_aws_library ∅ <ø> (∅)
pkg_celery_library ∅ <ø> (∅)
pkg_dask_task_models_library ∅ <ø> (∅)
pkg_models_library ∅ <ø> (∅)
pkg_notifications_library ∅ <ø> (∅)
pkg_postgres_database ∅ <ø> (∅)
pkg_service_integration ∅ <ø> (∅)
pkg_service_library ∅ <ø> (∅)
pkg_settings_library ∅ <ø> (∅)
pkg_simcore_sdk 76.98% <ø> (-8.13%) ⬇️
agent ∅ <ø> (∅)
api_server ∅ <ø> (∅)
autoscaling ∅ <ø> (∅)
catalog ∅ <ø> (∅)
clusters_keeper ∅ <ø> (∅)
dask_sidecar ∅ <ø> (∅)
datcore_adapter ∅ <ø> (∅)
director ∅ <ø> (∅)
director_v2 49.29% <ø> (-41.75%) ⬇️
dynamic_scheduler ∅ <ø> (∅)
dynamic_sidecar 90.09% <0.00%> (ø)
efs_guardian ∅ <ø> (∅)
invitations ∅ <ø> (∅)
payments ∅ <ø> (∅)
resource_usage_tracker ∅ <ø> (∅)
storage ∅ <ø> (∅)
webclient ∅ <ø> (∅)
webserver 58.91% <ø> (-28.76%) ⬇️

Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 22374df...05235b4. Read the comment docs.

🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@sanderegg sanderegg force-pushed the bugfix/i1942/ensure-outputs-are-transmitted branch from 7cdc4c5 to 0d2e582 Compare June 30, 2025 15:15
@sanderegg sanderegg changed the title 🐛added new test 🐛Do not delete everythin around in inputs folder Jun 30, 2025
@sanderegg sanderegg changed the title 🐛Do not delete everythin around in inputs folder 🐛Stop pruning /inputs folder after unzipping input port Jun 30, 2025
@sanderegg sanderegg marked this pull request as ready for review June 30, 2025 15:19
matusdrobuliak66
matusdrobuliak66 approved these changes Jun 30, 2025
View reviewed changes
Copy link
Collaborator

@matusdrobuliak66 matusdrobuliak66 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🥇 thanks

wvangeit
wvangeit approved these changes Jun 30, 2025
View reviewed changes
Copy link
Contributor

@wvangeit wvangeit left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice find. Thanks.

pcrespov
pcrespov approved these changes Jun 30, 2025
View reviewed changes
Copy link
Member

@pcrespov pcrespov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎉 thx so much!

@sanderegg sanderegg mentioned this pull request Jul 1, 2025
Merged
@sonarqubecloud
Copy link

sonarqubecloud bot commented Jul 1, 2025

Quality Gate Passed Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

@sanderegg sanderegg merged commit 123aa5a into ITISFoundation:master Jul 1, 2025
46 of 57 checks passed
@sanderegg sanderegg deleted the bugfix/i1942/ensure-outputs-are-transmitted branch July 1, 2025 06:51
sanderegg added a commit that referenced this pull request Jul 1, 2025
@sanderegg
🐛Stop pruning /inputs folder after unzipping input port (#8016)
3f92381
@pcrespov pcrespov mentioned this pull request Jul 2, 2025
Closed
@matusdrobuliak66 matusdrobuliak66 mentioned this pull request Aug 5, 2025
Closed
88 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@wvangeit wvangeit wvangeit approved these changes

@pcrespov pcrespov pcrespov approved these changes

@matusdrobuliak66 matusdrobuliak66 matusdrobuliak66 approved these changes

@GitHK GitHK Awaiting requested review from GitHK GitHK is a code owner

@Konohana0608 Konohana0608 Awaiting requested review from Konohana0608

@bisgaard-itis bisgaard-itis Awaiting requested review from bisgaard-itis

Assignees

@sanderegg sanderegg

Labels

a:webserver webserver's codebase. Assigning the area is particularly useful for bugs

Projects

None yet

Milestone

Engage

Development

Successfully merging this pull request may close these issues.

Bug: Inputs not downloaded to Service

4 participants

@sanderegg @wvangeit @pcrespov @matusdrobuliak66