Skip to content

Commit 42cf6bd

Browse files
rustyjuxPaul Howellsphowells
authored
OAS validation API (#249)
* APS-4031 Add OAS Validation endpoints - See microservices/csitOasValidationApi/openapi.json - Includes endpoint implementations - Unit tests - Instructions for manual integration tests in local environment * Updated based on PR Review feedback Fixes for Problem Detail Response and OpenAPI json generation Updated to include install instructions in the readme Adding health endpoints Update to use python 3.14 Updates from code review to correct spelling and formatting Fix for generated results class Updated list rulesets endpoint and tests for sorting by directory Completed TODOs in unit tests - Add body to requests Update Readme to fix the 'run test's command Updated API to use a preexisting local cache of the API Governance repository tags * More updates based on PR feedback * APS-4094 Validation API Production Readiness (#244) * Create working dockerfile for validation api * add CI for validation API * use FastAPIOffline to serve docs files locally instead of from CDN * Refactor spectral validation process to handle large outputs and improve error handling * Feature/fix build (#245) - fix CI workflows * upgrade docker/build-push-action to v5 * fix tag (slashes) * add set up buildx * update registry to ghcr.io * update master build action --------- Co-authored-by: Paul Howells <paul.howells@gov.bc.ca> Co-authored-by: phowells <paul.steven.howells@gmail.com>
1 parent 4e1ed51 commit 42cf6bd

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

56 files changed

+4779
-72
lines changed

‎.github/workflows/dev.yml‎

Lines changed: 148 additions & 36 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,9 @@
11
# This is a basic workflow to help you get started with Actions
22

33
name: Build
4+
permissions:
5+
contents: read
6+
packages: write
47

58
# Controls when the action will run. Triggers the workflow on push or pull request
69
# events but only for the master branch
@@ -68,6 +71,22 @@ jobs:
6871
poetry run coverage run --branch -m pytest -s -v
6972
poetry run coverage xml
7073
74+
- uses: actions/setup-python@v6
75+
with:
76+
python-version: "3.14"
77+
- name: Test coverage for CSIT OAS Validation API
78+
run: |
79+
export PATH=/root/.local/bin:$PATH
80+
sudo apt install -y nodejs npm
81+
sudo npm install -g @stoplight/spectral-cli@6.14.2
82+
cd microservices/csitOasValidationApi
83+
poetry env use python3.14
84+
poetry install --no-root
85+
./checkout-ruleset-tags.sh ruleset_tag_cache || true
86+
export GITHUB_TAG_CACHE_PATH="$(realpath -m ./ruleset_tag_cache)"
87+
poetry run coverage run --branch -m pytest -s -v
88+
poetry run coverage xml
89+
7190
- name: SonarCloud Scan
7291
uses: sonarsource/sonarqube-scan-action@master
7392
env:
@@ -93,19 +112,31 @@ jobs:
93112
- name: Skip message
94113
if: steps.check.outputs.build_needed == 'false'
95114
run: echo "No changes in gatewayApi, skipping build"
96-
- uses: docker/build-push-action@v1
115+
- name: Sanitize tag name
116+
if: steps.check.outputs.build_needed == 'true'
117+
id: tag
118+
run: echo "tag=$(echo '${{ github.ref_name }}' | tr '/' '-')" >> $GITHUB_OUTPUT
119+
- name: Set up Docker Buildx
120+
if: steps.check.outputs.build_needed == 'true'
121+
uses: docker/setup-buildx-action@v3
122+
- name: Log in to GitHub Container Registry
97123
if: steps.check.outputs.build_needed == 'true'
124+
uses: docker/login-action@v3
98125
with:
99-
registry: docker.pkg.github.com
100-
username: $GITHUB_ACTOR
126+
registry: ghcr.io
127+
username: ${{ github.actor }}
101128
password: ${{ secrets.GITHUB_TOKEN }}
102-
repository: bcgov/gwa-api/gwa-gateway-api
103-
path: microservices/gatewayApi
104-
dockerfile: microservices/gatewayApi/Dockerfile
105-
tag_with_ref: true
106-
tag_with_sha: false
107-
add_git_labels: true
129+
logout: false
130+
- uses: docker/build-push-action@v5
131+
if: steps.check.outputs.build_needed == 'true'
132+
with:
133+
context: microservices/gatewayApi
134+
file: microservices/gatewayApi/Dockerfile
108135
push: true
136+
tags: ghcr.io/bcgov/gwa-api/gwa-gateway-api:${{ steps.tag.outputs.tag }}
137+
labels: |
138+
org.opencontainers.image.source=${{ github.server_url }}/${{ github.repository }}
139+
org.opencontainers.image.revision=${{ github.sha }}
109140
110141
gwa-scheduler:
111142
needs: sonar-scan
@@ -126,19 +157,31 @@ jobs:
126157
- name: Skip message
127158
if: steps.check.outputs.build_needed == 'false'
128159
run: echo "No changes in gatewayJobScheduler, skipping build"
129-
- uses: docker/build-push-action@v1
160+
- name: Sanitize tag name
161+
if: steps.check.outputs.build_needed == 'true'
162+
id: tag
163+
run: echo "tag=$(echo '${{ github.ref_name }}' | tr '/' '-')" >> $GITHUB_OUTPUT
164+
- name: Set up Docker Buildx
130165
if: steps.check.outputs.build_needed == 'true'
166+
uses: docker/setup-buildx-action@v3
167+
- name: Log in to GitHub Container Registry
168+
if: steps.check.outputs.build_needed == 'true'
169+
uses: docker/login-action@v3
131170
with:
132-
registry: docker.pkg.github.com
133-
username: $GITHUB_ACTOR
171+
registry: ghcr.io
172+
username: ${{ github.actor }}
134173
password: ${{ secrets.GITHUB_TOKEN }}
135-
repository: bcgov/gwa-api/gwa-scheduler
136-
path: microservices/gatewayJobScheduler
137-
dockerfile: microservices/gatewayJobScheduler/Dockerfile
138-
tag_with_ref: true
139-
tag_with_sha: false
140-
add_git_labels: true
174+
logout: false
175+
- uses: docker/build-push-action@v5
176+
if: steps.check.outputs.build_needed == 'true'
177+
with:
178+
context: microservices/gatewayJobScheduler
179+
file: microservices/gatewayJobScheduler/Dockerfile
141180
push: true
181+
tags: ghcr.io/bcgov/gwa-api/gwa-scheduler:${{ steps.tag.outputs.tag }}
182+
labels: |
183+
org.opencontainers.image.source=${{ github.server_url }}/${{ github.repository }}
184+
org.opencontainers.image.revision=${{ github.sha }}
142185
143186
gwa-kube-api:
144187
needs: sonar-scan
@@ -159,19 +202,31 @@ jobs:
159202
- name: Skip message
160203
if: steps.check.outputs.build_needed == 'false'
161204
run: echo "No changes in kubeApi, skipping build"
162-
- uses: docker/build-push-action@v1
205+
- name: Sanitize tag name
163206
if: steps.check.outputs.build_needed == 'true'
207+
id: tag
208+
run: echo "tag=$(echo '${{ github.ref_name }}' | tr '/' '-')" >> $GITHUB_OUTPUT
209+
- name: Set up Docker Buildx
210+
if: steps.check.outputs.build_needed == 'true'
211+
uses: docker/setup-buildx-action@v3
212+
- name: Log in to GitHub Container Registry
213+
if: steps.check.outputs.build_needed == 'true'
214+
uses: docker/login-action@v3
164215
with:
165-
registry: docker.pkg.github.com
166-
username: $GITHUB_ACTOR
216+
registry: ghcr.io
217+
username: ${{ github.actor }}
167218
password: ${{ secrets.GITHUB_TOKEN }}
168-
repository: bcgov/gwa-api/gwa-kube-api
169-
path: microservices/kubeApi
170-
dockerfile: microservices/kubeApi/Dockerfile
171-
tag_with_ref: true
172-
tag_with_sha: false
173-
add_git_labels: true
219+
logout: false
220+
- uses: docker/build-push-action@v5
221+
if: steps.check.outputs.build_needed == 'true'
222+
with:
223+
context: microservices/kubeApi
224+
file: microservices/kubeApi/Dockerfile
174225
push: true
226+
tags: ghcr.io/bcgov/gwa-api/gwa-kube-api:${{ steps.tag.outputs.tag }}
227+
labels: |
228+
org.opencontainers.image.source=${{ github.server_url }}/${{ github.repository }}
229+
org.opencontainers.image.revision=${{ github.sha }}
175230
176231
gwa-compatibility-api:
177232
needs: sonar-scan
@@ -192,16 +247,73 @@ jobs:
192247
- name: Skip message
193248
if: steps.check.outputs.build_needed == 'false'
194249
run: echo "No changes in compatibilityApi, skipping build"
195-
- uses: docker/build-push-action@v1
250+
- name: Sanitize tag name
251+
if: steps.check.outputs.build_needed == 'true'
252+
id: tag
253+
run: echo "tag=$(echo '${{ github.ref_name }}' | tr '/' '-')" >> $GITHUB_OUTPUT
254+
- name: Set up Docker Buildx
255+
if: steps.check.outputs.build_needed == 'true'
256+
uses: docker/setup-buildx-action@v3
257+
- name: Log in to GitHub Container Registry
196258
if: steps.check.outputs.build_needed == 'true'
259+
uses: docker/login-action@v3
197260
with:
198-
registry: docker.pkg.github.com
199-
username: $GITHUB_ACTOR
261+
registry: ghcr.io
262+
username: ${{ github.actor }}
200263
password: ${{ secrets.GITHUB_TOKEN }}
201-
repository: bcgov/gwa-api/gwa-compatibility-api
202-
path: microservices/compatibilityApi
203-
dockerfile: microservices/compatibilityApi/Dockerfile
204-
tag_with_ref: true
205-
tag_with_sha: false
206-
add_git_labels: true
264+
logout: false
265+
- uses: docker/build-push-action@v5
266+
if: steps.check.outputs.build_needed == 'true'
267+
with:
268+
context: microservices/compatibilityApi
269+
file: microservices/compatibilityApi/Dockerfile
270+
push: true
271+
tags: ghcr.io/bcgov/gwa-api/gwa-compatibility-api:${{ steps.tag.outputs.tag }}
272+
labels: |
273+
org.opencontainers.image.source=${{ github.server_url }}/${{ github.repository }}
274+
org.opencontainers.image.revision=${{ github.sha }}
275+
276+
gwa-csit-oas-validation-api:
277+
needs: sonar-scan
278+
name: Docker Image for gwa-csit-oas-validation-api
279+
runs-on: ubuntu-latest
280+
steps:
281+
- uses: actions/checkout@v4
282+
with:
283+
fetch-depth: 2
284+
- name: Check if build needed
285+
id: check
286+
run: |
287+
if git diff --name-only HEAD^ HEAD | grep -q "^microservices/csitOasValidationApi/\|^.github/workflows/dev.yml"; then
288+
echo "build_needed=true" >> $GITHUB_OUTPUT
289+
else
290+
echo "build_needed=false" >> $GITHUB_OUTPUT
291+
fi
292+
- name: Skip message
293+
if: steps.check.outputs.build_needed == 'false'
294+
run: echo "No changes in csitOasValidationApi, skipping build"
295+
- name: Sanitize tag name
296+
if: steps.check.outputs.build_needed == 'true'
297+
id: tag
298+
run: echo "tag=$(echo '${{ github.ref_name }}' | tr '/' '-')" >> $GITHUB_OUTPUT
299+
- name: Set up Docker Buildx
300+
if: steps.check.outputs.build_needed == 'true'
301+
uses: docker/setup-buildx-action@v3
302+
- name: Log in to GitHub Container Registry
303+
if: steps.check.outputs.build_needed == 'true'
304+
uses: docker/login-action@v3
305+
with:
306+
registry: ghcr.io
307+
username: ${{ github.actor }}
308+
password: ${{ secrets.GITHUB_TOKEN }}
309+
logout: false
310+
- uses: docker/build-push-action@v5
311+
if: steps.check.outputs.build_needed == 'true'
312+
with:
313+
context: microservices/csitOasValidationApi
314+
file: microservices/csitOasValidationApi/Dockerfile
207315
push: true
316+
tags: ghcr.io/bcgov/gwa-api/gwa-csit-oas-validation-api:${{ steps.tag.outputs.tag }}
317+
labels: |
318+
org.opencontainers.image.source=${{ github.server_url }}/${{ github.repository }}
319+
org.opencontainers.image.revision=${{ github.sha }}

‎.github/workflows/master.yml‎

Lines changed: 64 additions & 36 deletions
Original file line numberDiff line numberDiff line change
@@ -65,6 +65,22 @@ jobs:
6565
poetry run coverage run --branch -m pytest -s -v
6666
poetry run coverage xml
6767
68+
- uses: actions/setup-python@v6
69+
with:
70+
python-version: "3.14"
71+
- name: Test coverage for CSIT OAS Validation API
72+
run: |
73+
export PATH=/root/.local/bin:$PATH
74+
sudo apt install -y nodejs npm
75+
sudo npm install -g @stoplight/spectral-cli@6.14.2
76+
cd microservices/csitOasValidationApi
77+
poetry env use python3.14
78+
poetry install --no-root
79+
./checkout-ruleset-tags.sh ruleset_tag_cache || true
80+
export GITHUB_TAG_CACHE_PATH="$(realpath -m ./ruleset_tag_cache)"
81+
poetry run coverage run --branch -m pytest -s -v
82+
poetry run coverage xml
83+
6884
- name: SonarCloud Scan
6985
uses: sonarsource/sonarcloud-github-action@master
7086
env:
@@ -81,54 +97,66 @@ jobs:
8197
echo "Got tag name ${{ steps.release.outputs.tag_name }}"
8298
echo "Got release version ${{ steps.release.outputs.version }}"
8399
84-
- name: Create gwa-api docker image related to the release
85-
uses: docker/build-push-action@v1
100+
- name: Set up Docker Buildx
101+
uses: docker/setup-buildx-action@v3
102+
- name: Log in to GitHub Container Registry
103+
uses: docker/login-action@v3
86104
with:
87-
registry: docker.pkg.github.com
88-
username: $GITHUB_ACTOR
105+
registry: ghcr.io
106+
username: ${{ github.actor }}
89107
password: ${{ secrets.GITHUB_TOKEN }}
90-
repository: bcgov/gwa-api/gwa-gateway-api
91-
path: microservices/gatewayApi
92-
dockerfile: microservices/gatewayApi/Dockerfile
108+
logout: false
109+
- name: Create gwa-api docker image related to the release
110+
uses: docker/build-push-action@v5
111+
with:
112+
context: microservices/gatewayApi
113+
file: microservices/gatewayApi/Dockerfile
93114
push: true
94-
tags: ${{ steps.release.outputs.tag_name }}
95-
tag_with_sha: false
115+
tags: ghcr.io/bcgov/gwa-api/gwa-gateway-api:${{ steps.release.outputs.tag_name }}
116+
labels: |
117+
org.opencontainers.image.source=${{ github.server_url }}/${{ github.repository }}
118+
org.opencontainers.image.revision=${{ github.sha }}
96119
97120
- name: Create gwa-kube-api docker image related to the release
98-
uses: docker/build-push-action@v1
121+
uses: docker/build-push-action@v5
99122
with:
100-
registry: docker.pkg.github.com
101-
username: $GITHUB_ACTOR
102-
password: ${{ secrets.GITHUB_TOKEN }}
103-
repository: bcgov/gwa-api/gwa-kube-api
104-
path: microservices/kubeApi
105-
dockerfile: microservices/kubeApi/Dockerfile
123+
context: microservices/kubeApi
124+
file: microservices/kubeApi/Dockerfile
106125
push: true
107-
tags: ${{ steps.release.outputs.tag_name }}
108-
tag_with_sha: false
126+
tags: ghcr.io/bcgov/gwa-api/gwa-kube-api:${{ steps.release.outputs.tag_name }}
127+
labels: |
128+
org.opencontainers.image.source=${{ github.server_url }}/${{ github.repository }}
129+
org.opencontainers.image.revision=${{ github.sha }}
109130
110131
- name: Create gwa-scheduler docker image related to the release
111-
uses: docker/build-push-action@v1
132+
uses: docker/build-push-action@v5
112133
with:
113-
registry: docker.pkg.github.com
114-
username: $GITHUB_ACTOR
115-
password: ${{ secrets.GITHUB_TOKEN }}
116-
repository: bcgov/gwa-api/gwa-scheduler
117-
path: microservices/gatewayJobScheduler
118-
dockerfile: microservices/gatewayJobScheduler/Dockerfile
134+
context: microservices/gatewayJobScheduler
135+
file: microservices/gatewayJobScheduler/Dockerfile
119136
push: true
120-
tags: ${{ steps.release.outputs.tag_name }}
121-
tag_with_sha: false
137+
tags: ghcr.io/bcgov/gwa-api/gwa-scheduler:${{ steps.release.outputs.tag_name }}
138+
labels: |
139+
org.opencontainers.image.source=${{ github.server_url }}/${{ github.repository }}
140+
org.opencontainers.image.revision=${{ github.sha }}
122141
123142
- name: Create gwa-compatibility-api docker image related to the release
124-
uses: docker/build-push-action@v1
143+
uses: docker/build-push-action@v5
125144
with:
126-
registry: docker.pkg.github.com
127-
username: $GITHUB_ACTOR
128-
password: ${{ secrets.GITHUB_TOKEN }}
129-
repository: bcgov/gwa-api/gwa-compatibility-api
130-
path: microservices/compatibilityApi
131-
dockerfile: microservices/compatibilityApi/Dockerfile
145+
context: microservices/compatibilityApi
146+
file: microservices/compatibilityApi/Dockerfile
147+
push: true
148+
tags: ghcr.io/bcgov/gwa-api/gwa-compatibility-api:${{ steps.release.outputs.tag_name }}
149+
labels: |
150+
org.opencontainers.image.source=${{ github.server_url }}/${{ github.repository }}
151+
org.opencontainers.image.revision=${{ github.sha }}
152+
153+
- name: Create gwa-csit-oas-validation-api docker image related to the release
154+
uses: docker/build-push-action@v5
155+
with:
156+
context: microservices/csitOasValidationApi
157+
file: microservices/csitOasValidationApi/Dockerfile
132158
push: true
133-
tags: ${{ steps.release.outputs.tag_name }}
134-
tag_with_sha: false
159+
tags: ghcr.io/bcgov/gwa-api/gwa-csit-oas-validation-api:${{ steps.release.outputs.tag_name }}
160+
labels: |
161+
org.opencontainers.image.source=${{ github.server_url }}/${{ github.repository }}
162+
org.opencontainers.image.revision=${{ github.sha }}

‎.python-version‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
3.14.0

0 commit comments

Comments
 (0)