SDX subsystems, services, runtime groups and connections#1313
SDX subsystems, services, runtime groups and connections#1313ikethecoder wants to merge 30 commits intodevfrom
Conversation
There was a problem hiding this comment.
Pull request overview
This pull request introduces SDX (Secure Data Exchange) service functionality to the API Services Portal. The changes add comprehensive support for managing organization subsystems and OpenAPI service specifications through a new API endpoint structure.
Key changes:
- New SDX v1 API with endpoints for catalog, subsystems, and OAS services
- Database schema additions for Subsystem and OpenAPISpec entities
- New "system-owner" role with System.Manage scope for organization-level permissions
- Refactored namespace creation into a reusable workflow service
Reviewed changes
Copilot reviewed 64 out of 66 changed files in this pull request and generated 43 comments.
Show a summary per file
| File | Description |
|---|---|
| src/tsoa-sdx-v1.json | Configuration for new SDX v1 API specification and routes |
| src/lists/Subsystem.js, OpenAPISpec.js | New Keystone list definitions for subsystems and OpenAPI specs |
| src/services/batch/subsystem.ts, oas-service.ts | Service layer for subsystem and OAS management |
| src/services/workflow/openapi-spec-loader.ts | Workflow for loading and validating OpenAPI specifications |
| src/services/workflow/create-namespace.ts | Refactored namespace creation into reusable service |
| src/controllers/sdx/v1/*.ts | New SDX API controllers for catalog, subsystems, and services |
| src/services/org-groups/roles.ts | Added system-owner role with System.Manage permissions |
| src/authz/matrix.csv | Updated authorization rules for new entities and roles |
| local/db/keystone-init.sql | Database schema for Subsystem and OpenAPISpec tables |
| e2e/cypress/tests/21-sdx-api/** | Comprehensive E2E tests for new SDX endpoints |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
|
|
||
| describe('Subsystem Happy Paths', () => { | ||
| it('PUT /organizations/{org}/subsystems', () => { | ||
| const { org, gateway, dataset, datasetId, product } = workingData |
There was a problem hiding this comment.
Unused variable gateway.
|
|
||
| describe('Subsystem Happy Paths', () => { | ||
| it('PUT /organizations/{org}/subsystems', () => { | ||
| const { org, gateway, dataset, datasetId, product } = workingData |
There was a problem hiding this comment.
Unused variable dataset.
|
|
||
| describe('Subsystem Happy Paths', () => { | ||
| it('PUT /organizations/{org}/subsystems', () => { | ||
| const { org, gateway, dataset, datasetId, product } = workingData |
There was a problem hiding this comment.
Unused variable product.
| }) | ||
|
|
||
| it('DELETE /organizations/{org}/subsystems/{name}', () => { | ||
| const { org, gateway, dataset, datasetId, product } = workingData |
There was a problem hiding this comment.
Unused variable gateway.
| const { org, gateway, dataset, datasetId, product } = workingData | |
| const { org, dataset, datasetId, product } = workingData |
| }) | ||
|
|
||
| it('DELETE /organizations/{org}/subsystems/{name}', () => { | ||
| const { org, gateway, dataset, datasetId, product } = workingData |
There was a problem hiding this comment.
Unused variable dataset.
| // node dist/test/integrated/gateway-patterns/catalog.js | ||
|
|
||
| import { logger } from '../../../logger'; | ||
| import YAML from 'js-yaml'; |
There was a problem hiding this comment.
Unused import YAML.
| } from '../../../services/keycloak'; | ||
| import { o } from '../util'; | ||
| import { GroupMembership } from '@/services/org-groups/types'; | ||
| import { remove } from 'lodash'; |
There was a problem hiding this comment.
Unused import remove.
| authentication: { item: identity }, | ||
| }); | ||
|
|
||
| if (false) { |
There was a problem hiding this comment.
This statement is unreachable.
| await kc.createOrUpdateGroupAccess(access, ['idir']); | ||
| } | ||
| if (true) { | ||
| if (false) { |
There was a problem hiding this comment.
This statement is unreachable.
| // const access = await kc.getGroupAccess('databc'); | ||
| // console.log(JSON.stringify(access, null, 4)); | ||
| // } | ||
| if (false) { |
There was a problem hiding this comment.
This statement is unreachable.
rustyjux
left a comment
rustyjux
left a comment
There was a problem hiding this comment.
Wow there's a lot in here!
Generally looks good. A few questions:
- Seems like subsystem and oas-service are just there for cataloging; will oas-service be linked up to actual (SDX) services at some stage? oas-service be a child of an SDX service?
- For catalog/discovery purposes, do we need a way of linking users/contacts to oas-services (or perhaps subsystems)? Or are we good with oas-service > subsystem > gateway > owners / access managers, which should be accessible with the current path?
- Forget about org units? too fickle?
I see mocks haven't been created for the new endpoints but we can add those later if they are deemed necessary for future frontend dev.
Request for changes: A few new test failures have arisen which need addressing - #1328. Only these 3 are expected/acceptable w/ Kong 3 - #1306
I also left one or two comments of significance.
phowells
left a comment
phowells
left a comment
There was a problem hiding this comment.
A lot to look at here. It would be helpful if there were more in-line comments on the code as well as in-line comments in the PR explaining why you made changes.
ikethecoder
changed the title
ikethecoder
changed the title
Closes the following issues:
🚀 Feature branch deployment: https://api-services-portal-feature-sdx-service.apps.silver.devops.gov.bc.ca
🚀 Feature branch deployment: https://api-services-portal-feature-sdx-service.apps.silver.devops.gov.bc.ca