-
Notifications
You must be signed in to change notification settings - Fork 32
🐛 Fixes webserver-api error on get_project_services when project services are missing in the catalog by allowing partial failure
#8486
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
🐛 Fixes webserver-api error on get_project_services when project services are missing in the catalog by allowing partial failure
#8486
Conversation
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## master #8486 +/- ##
==========================================
- Coverage 87.62% 87.35% -0.27%
==========================================
Files 1999 1654 -345
Lines 77828 67797 -10031
Branches 1338 1004 -334
==========================================
- Hits 68193 59227 -8966
+ Misses 9235 8260 -975
+ Partials 400 310 -90
*This pull request uses carry forward flags. Click here to find out more.
Continue to review full report in Codecov by Sentry.
🚀 New features to boost your workflow:
|
webserver-api 503 on projects/{project_id}/nodes/-/services when project services are missing in the catalog
webserver-api 503 on projects/{project_id}/nodes/-/services when project services are missing in the catalogwebserver-api error on get_project_services when project services are missing in the catalog by allowing partial failure
389ab4a to
a32fc97
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull Request Overview
This PR fixes a bug where the GET /v0/projects/{project_id}/nodes/-/services endpoint returned a 503 error when project services were missing from the catalog. The fix introduces support for partial failure handling by returning both found and missing services in the response.
- Introduces generic batch operations models for handling partial success scenarios
- Updates the API to return missing services in addition to found ones
- Refactors batch-get operations throughout the catalog service to use the new envelope pattern
Reviewed Changes
Copilot reviewed 27 out of 27 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| packages/models-library/src/models_library/batch_operations.py | Adds generic BatchGetEnvelope model and validation utilities for batch operations |
| packages/models-library/src/models_library/api_schemas_catalog/services.py | Introduces MyServicesRpcBatchGet schema using the new batch envelope pattern |
| packages/models-library/src/models_library/api_schemas_webserver/projects_nodes.py | Adds missing field to ProjectNodeServicesGet for tracking services not found in catalog |
| services/catalog/src/simcore_service_catalog/service/catalog_services.py | Refactors batch_get_user_services to return envelope with found and missing items |
| services/catalog/src/simcore_service_catalog/repository/services.py | Updates batch_get_services_access_rights to return None when no services found |
| services/web/server/src/simcore_service_webserver/projects/_controller/nodes_rest.py | Updates get_project_services to handle the new batch envelope response |
| Multiple test files | Updates tests to work with new batch envelope response format |
| Multiple error handling files | Renames RPC error classes with "Rpc" suffix for consistency |
Comments suppressed due to low confidence (1)
services/catalog/src/simcore_service_catalog/service/catalog_services.py:1
- The assertion compares found items to all input ids, but this will fail when some services are missing. Should compare to deduplicated ids or remove assertion entirely since partial success is now supported.
"""Includes manifest (services in the registry) and function-service (front-end/back-end services)"""
Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.
packages/models-library/src/models_library/api_schemas_webserver/projects_nodes.py
Outdated
Show resolved
Hide resolved
services/web/server/src/simcore_service_webserver/projects/_controller/_rest_exceptions.py
Outdated
Show resolved
Hide resolved
9fb4957 to
cfd0ab0
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks
packages/models-library/src/models_library/api_schemas_webserver/projects_nodes.py
Outdated
Show resolved
Hide resolved
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
thx
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
👍
services/catalog/src/simcore_service_catalog/service/catalog_services.py
Show resolved
Hide resolved
5c53092 to
9a4b982
Compare
…mprove test assertions
…arity and consistency
9a4b982 to
5fd0e88
Compare
|
@mergify queue |
🟠 Waiting for conditions to match
|
|



What do these changes do?
The main motiviation to this PR was to fix the
503response fromget_project_servicesin the web-api when a service listed in one of the project nodes is not anymore part of the catalog.We took the opportunity to:
Highlights
GET /v0/projects/{project_id}/nodes/-/serviceswhen one of the services in the project was not found int he catalog:{ "error_code": "OEC:83e338830cbb-1759912388229", "context": { "request": "<Request GET /v0/projects/fe9494f6-e2ad-5970-b860-e6cb97dc25ce/nodes/-/services >", "request.method": "GET", "request.path": "/v0/projects/fe9494f6-e2ad-5970-b860-e6cb97dc25ce/nodes/-/services", "error_code": "OEC:83e338830cbb-1759912388229", "user_id": 6, "product_name": "osparc", "services_ids": [ [ "simcore/services/dynamic/cc-1d-viewer", "3.0.3" ], [ "simcore/services/frontend/file-picker", "1.0.0" ], [ "simcore/services/comp/rabbit-ss-0d-cardiac-model", "1.0.0" ], [ "simcore/services/comp/rabbit-ss-1d-cardiac-model", "1.0.0" ], [ "simcore/services/comp/rabbit-ss-2d-cardiac-model", "1.0.0" ], [ "simcore/services/dynamic/cc-0d-viewer", "3.0.3" <----------this was not in the catalog ] ], "message": "Catalog service failed unexpectedly", "code": "CatalogApiBaseError.CatalogNotAvailableError" }, "tip": null }This is because the underlying batch-get operations did not allow partial failure and one of the services in the project is not in the catalog.
Now that endpoints returns
missingservices in the response payload (see changes in OAS)missingfield in theGET /v0/projects/{project_id}/nodes/-/servicesresponse model that include a list of key/value tuples of missing servicesIn Detail (AI generated)
This pull request introduces a new generic batch-get envelope model for service retrieval operations, enabling partial success responses and improved error handling when some requested services are missing. It refactors the batch-get endpoints and service logic to use this new model, ensures deduplication and validation of input identifiers, and updates related error handling and tests to reflect these changes.
Batch-get envelope and validation improvements
BatchGetEnvelopemodel and acreate_batch_ids_validatorutility inbatch_operations.pyto support partial results and deduplicate input identifiers for batch operations.MyServicesRpcBatchGet, which returns both found and missing services, replacing the previous list-based response. [1] [2]Service logic and error handling
batch_get_user_servicesand related endpoints to use the new envelope model, handle missing services gracefully, and raise a newCatalogServiceNotFoundErrorwhen no services are found. [1] [2] [3] [4] [5]CatalogBadRequestErrorand improved error templates for catalog-related exceptions.API and test updates
Type aliasing and code organization
BatchGetUserServicesResulttype alias for future-proofing schema/domain separation.Related issue/s
How to test
models_library: new generic batch-operations models and utils-
packages/models-library/tests/test_batch_operations.pyfor new generic models and utilsweb-serverservicesservices/web/server/tests/unit/with_dbs/02/test_projects_nodes_handlers__services_access.pycatalogservicesservices/catalog/tests/unit/with_dbs/test_service_catalog_services.pyservices/catalog/tests/unit/with_dbs/test_api_rpc.pyDev-ops
None