Skip to content

Conversation

@bisgaard-itis
Copy link
Contributor

@bisgaard-itis bisgaard-itis commented Mar 7, 2025

What do these changes do?

This PR redesigns both the webservers api for the async jobs (currently: computing the size of a path and zipping). It also improves on the design of the rpc interface (currently in storage, but to be moved to servicelib in the future).

Webserver task api

  • Overwrite long running tasks api and move the "legacy" implementation to /v0/tasks-legacy which is not included in the openapi specs. The new interface does not break backwards compatibility. The new GET /v0/tasks endpoint retrives both the legacy long running tasks as well as the new async jobs. The HATEOS links will point to the right endpoints for getting the status/results or abort endpoints.
  • Add links (get_status, abort, get_result) to response bodies for async job endpoints in the webserver.

Storage async jobs rpc interface

  • Raise exceptions in get_result method (for the async jobs in the storage worker) when an error occurred while running the job, it was aborted or is not yet done.

  • Raise exceptions if status/result/abort is requested but the job doesn't exist.

  • Improve unit testing both in webserver and storage related to async jobs.

Related issue/s

How to test

Dev-ops checklist

@bisgaard-itis bisgaard-itis self-assigned this Mar 7, 2025
@bisgaard-itis bisgaard-itis added the a:webserver webserver's codebase. Assigning the area is particularly useful for bugs label Mar 7, 2025
@bisgaard-itis bisgaard-itis added this to the The Awakening milestone Mar 7, 2025
@codecov
Copy link

codecov bot commented Mar 7, 2025

Codecov Report

Attention: Patch coverage is 85.12821% with 29 lines in your changes missing coverage. Please review.

Project coverage is 87.15%. Comparing base (fb2ccf3) to head (253f65f).
Report is 1 commits behind head on master.

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #7320      +/-   ##
==========================================
- Coverage   87.30%   87.15%   -0.16%     
==========================================
  Files        1709     1546     -163     
  Lines       66361    62651    -3710     
  Branches     1125      891     -234     
==========================================
- Hits        57938    54601    -3337     
+ Misses       8103     7791     -312     
+ Partials      320      259      -61     
Flag Coverage Δ
integrationtests 65.32% <63.82%> (+0.08%) ⬆️
unittests 86.26% <85.12%> (-0.21%) ⬇️
Components Coverage Δ
api 76.84% <ø> (ø)
pkg_aws_library ∅ <ø> (∅)
pkg_dask_task_models_library ∅ <ø> (∅)
pkg_models_library 92.04% <100.00%> (+0.03%) ⬆️
pkg_notifications_library ∅ <ø> (∅)
pkg_postgres_database ∅ <ø> (∅)
pkg_service_integration 70.03% <ø> (ø)
pkg_service_library 72.26% <0.00%> (-0.07%) ⬇️
pkg_settings_library ∅ <ø> (∅)
pkg_simcore_sdk 85.46% <ø> (ø)
agent 96.46% <ø> (ø)
api_server 90.68% <ø> (ø)
autoscaling 96.08% <ø> (ø)
catalog 92.14% <ø> (ø)
clusters_keeper 99.24% <ø> (ø)
dask_sidecar 91.25% <ø> (ø)
datcore_adapter 98.11% <ø> (ø)
director 76.68% <ø> (+0.09%) ⬆️
director_v2 91.30% <ø> (ø)
dynamic_scheduler 97.33% <ø> (ø)
dynamic_sidecar 90.11% <ø> (ø)
efs_guardian 89.79% <ø> (ø)
invitations 93.28% <ø> (ø)
payments 92.66% <ø> (ø)
resource_usage_tracker 89.02% <ø> (-0.17%) ⬇️
storage 84.56% <96.55%> (+0.41%) ⬆️
webclient ∅ <ø> (∅)
webserver 85.84% <100.00%> (+0.03%) ⬆️

Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update fb2ccf3...253f65f. Read the comment docs.

🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Copy link
Contributor

@GitHK GitHK left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks ok with me, some minor things

@bisgaard-itis bisgaard-itis requested a review from odeimaiz March 19, 2025 14:00
Copy link
Member

@sanderegg sanderegg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

that is very nice thanks! I added a few comments. we can discuss if needed.

@sonarqubecloud
Copy link

@bisgaard-itis bisgaard-itis enabled auto-merge (squash) March 20, 2025 11:58
@bisgaard-itis bisgaard-itis merged commit b5230f1 into ITISFoundation:master Mar 20, 2025
141 of 144 checks passed
@bisgaard-itis bisgaard-itis deleted the introduce-links-for-webserver-async-jobs branch March 20, 2025 12:28
mrnicegyu11 pushed a commit to mrnicegyu11/osparc-simcore that referenced this pull request Mar 26, 2025
@matusdrobuliak66 matusdrobuliak66 mentioned this pull request Apr 15, 2025
56 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

a:webserver webserver's codebase. Assigning the area is particularly useful for bugs

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants