Skip to content

Conversation

@matusdrobuliak66
Copy link
Collaborator

@matusdrobuliak66 matusdrobuliak66 commented Apr 16, 2025

What do these changes do?

ReDoc

This PR introduces new webserver endpoints:

  • /computations/-/iterations/latest
  • /computations/{{project_id}}/iterations/latest/tasks

to list the latest iterations of computations and tasks, and adjusts the RPC and RabbitMQ setups in Director-v2 to support these features.

  • add DB indexes 🗃️ to comp_* tables

Related issue/s

How to test

  • services/director-v2/tests/unit/with_dbs/comp_scheduler/test_api_rpc_computations.py
  • services/web/server/tests/unit/with_dbs/01/test_director_v2_handlers.py

Dev-ops

NOne

@matusdrobuliak66 matusdrobuliak66 self-assigned this Apr 16, 2025
@matusdrobuliak66 matusdrobuliak66 added a:webserver webserver's codebase. Assigning the area is particularly useful for bugs a:director-v2 issue related with the director-v2 service labels Apr 16, 2025
@matusdrobuliak66 matusdrobuliak66 added this to the Pauwel Kwak milestone Apr 16, 2025
@codecov
Copy link

codecov bot commented Apr 16, 2025

Codecov Report

Attention: Patch coverage is 88.71595% with 29 lines in your changes missing coverage. Please review.

Project coverage is 87.48%. Comparing base (e6f13b1) to head (05789b5).
Report is 1 commits behind head on master.

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #7544      +/-   ##
==========================================
- Coverage   87.62%   87.48%   -0.15%     
==========================================
  Files        1752     1696      -56     
  Lines       67864    66384    -1480     
  Branches     1121     1024      -97     
==========================================
- Hits        59467    58074    -1393     
+ Misses       8089     8023      -66     
+ Partials      308      287      -21     
Flag Coverage Δ
integrationtests 65.07% <69.13%> (-0.06%) ⬇️
unittests 86.65% <88.71%> (-0.17%) ⬇️
Components Coverage Δ
api 76.84% <ø> (ø)
pkg_aws_library ∅ <ø> (∅)
pkg_dask_task_models_library ∅ <ø> (∅)
pkg_models_library 92.77% <97.14%> (+0.05%) ⬆️
pkg_notifications_library 85.26% <ø> (ø)
pkg_postgres_database 88.18% <ø> (ø)
pkg_service_integration 69.98% <ø> (ø)
pkg_service_library 72.80% <0.00%> (-0.31%) ⬇️
pkg_settings_library ∅ <ø> (∅)
pkg_simcore_sdk 85.40% <ø> (ø)
agent 96.46% <ø> (ø)
api_server 91.23% <ø> (ø)
autoscaling 96.08% <ø> (ø)
catalog 92.46% <ø> (ø)
clusters_keeper 99.24% <ø> (ø)
dask_sidecar 91.29% <ø> (ø)
datcore_adapter 98.12% <ø> (ø)
director 76.78% <ø> (ø)
director_v2 91.34% <97.77%> (+0.04%) ⬆️
dynamic_scheduler 97.40% <ø> (ø)
dynamic_sidecar 90.11% <ø> (ø)
efs_guardian 89.79% <ø> (ø)
invitations 93.28% <ø> (ø)
payments 92.66% <ø> (ø)
resource_usage_tracker 89.18% <ø> (-0.06%) ⬇️
storage 87.58% <ø> (-0.11%) ⬇️
webclient ∅ <ø> (∅)
webserver 86.08% <100.00%> (+0.04%) ⬆️

Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update e6f13b1...05789b5. Read the comment docs.

🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR introduces new endpoints and repository methods to list the latest iterations of computations and tasks, and adjusts the RPC and RabbitMQ setups to support these features.

  • Added web routes for listing the latest computation iterations and tasks.
  • Renamed repository methods from list to list_ and introduced new query methods for filtering user computations.
  • Updated tests and RPC interfaces to reflect the new endpoints and repository changes.

Reviewed Changes

Copilot reviewed 27 out of 28 changed files in this pull request and generated no comments.

Show a summary per file
File Description
services/web/server/src/simcore_service_webserver/director_v2/_computations_rest.py New endpoints for listing latest computation iterations and tasks.
services/director-v2/tests/unit/with_dbs/comp_scheduler/test_db_repositories_comp_runs.py Updated tests to use the renamed repository method list_.
services/director-v2/tests/unit/with_dbs/comp_scheduler/test_api_rpc_computations.py New tests for RPC endpoints for computations.
services/director-v2/tests/conftest.py Updated test fixtures to include rpc_client and patch RPC routes.
services/director-v2/src/simcore_service_director_v2/modules/rabbitmq.py Adjusted RabbitMQ client names and added rpc_server initialization/teardown.
services/director-v2/src/simcore_service_director_v2/modules/db/repositories/comp_tasks/_core.py Added new method to list computational tasks for frontend clients with pagination and ordering.
services/director-v2/src/simcore_service_director_v2/modules/db/repositories/comp_runs.py Renamed list method to list_ and added a new method for listing only latest iterations.
services/director-v2/src/simcore_service_director_v2/modules/comp_scheduler/_manager.py Updated scheduling to use the new list_ repository method.
services/director-v2/src/simcore_service_director_v2/core/application.py Integrated RPC API routes setup into app initialization.
API and RPC routes files Added new routes and dependencies for computations RPC.
Models-library changes Updated schemas to support the new computations endpoints.
Files not reviewed (1)
  • services/web/server/src/simcore_service_webserver/api/v0/openapi.yaml: Language not supported
Comments suppressed due to low confidence (1)

services/director-v2/src/simcore_service_director_v2/modules/db/repositories/comp_runs.py:210

  • The function 'desc' is undefined in this scope; please replace it with 'sa.desc' to ensure the ordering works correctly with SQLAlchemy.
desc(getattr(comp_runs.c, order_by.field)), comp_runs.c.run_id

@matusdrobuliak66 matusdrobuliak66 changed the title WIP: task manager listing task manager listing Apr 17, 2025
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot reviewed 30 out of 31 changed files in this pull request and generated 2 comments.

Files not reviewed (1)
  • services/web/server/src/simcore_service_webserver/api/v0/openapi.yaml: Language not supported
Comments suppressed due to low confidence (1)

services/director-v2/src/simcore_service_director_v2/modules/db/repositories/comp_runs.py:191

  • The function 'literal_column' is used without a proper namespace. Please either import it explicitly from 'sqlalchemy' or use 'sa.literal_column' to ensure the reference is correct.
                    comp_runs.c.iteration == literal_column("latest_runs.latest_iteration"),

@matusdrobuliak66 matusdrobuliak66 changed the title task manager listing ✨ task manager web-api listing Apr 17, 2025
@matusdrobuliak66 matusdrobuliak66 marked this pull request as ready for review April 17, 2025 12:12
@matusdrobuliak66 matusdrobuliak66 changed the title ✨ task manager web-api listing ✨ task manager web-api listing (🗃️) Apr 17, 2025
Copy link
Member

@pcrespov pcrespov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thx

Copy link
Member

@odeimaiz odeimaiz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

tested 👍

Copy link
Member

@sanderegg sanderegg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

cool! a few things that we can discuss tomorrow!

Copy link
Contributor

@GitHK GitHK left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK with me

@sonarqubecloud
Copy link

Copy link
Member

@sanderegg sanderegg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks for the checks and changes!

@sanderegg sanderegg merged commit d0f485b into ITISFoundation:master Apr 22, 2025
93 of 94 checks passed
@matusdrobuliak66 matusdrobuliak66 mentioned this pull request May 8, 2025
34 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

a:director-v2 issue related with the director-v2 service a:webserver webserver's codebase. Assigning the area is particularly useful for bugs

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add listing endpoint to webserver for Task Manager Define web.api openapi specs for the tasks table in the front-end

6 participants