CLOUDP-290416: Add workflow for releasing IPA metrics #386

lovisaberggren · 2025-01-23T12:28:46Z

Proposed changes

Adding workflow to release IPA metrics to s3.

Uses staging creds and s3 bucket for now
Not included in release spec workflow, this workflow will be triggered manually to verify it works as expected

Follow up:

Add workflow to release-spec.yml run when we're confident it works as expected
Update failure-handler.yml to also consider ipa release run result status
Later on: When stable update to push to production s3 instead

Jira ticket: CLOUDP-290416

lovisaberggren · 2025-01-23T12:29:53Z

.github/workflows/release-IPA-metrics.yml

+      - name: Checkout
+        uses: actions/checkout@v4
+
+      - name: Get Previous Run Date and Status


Only run the release steps if the latest run was not today or if the last run ended in failure

Could you clarify this decision? Adding a comment on the step would be beneficial to remember why we made this decision.

Why can't we run the workflow once a day automatically and file an issue on the repo in the event of a failure?

yelizhenden-mdb · 2025-01-23T13:52:29Z

.github/workflows/release-IPA-metrics.yml

+      - name: Run Metric Collection Job
+        if: ${{steps.get_previous_status.outputs.result == 'true' }}
+        working-directory: ./tools/spectral/ipa/metrics/scripts
+        run: node ./tools/spectral/ipa/metrics/scripts/runMetricCollection.js ../../../../../openapi-foas.json


Q1: Why don't we use the downloaded openapi-foas from the previous step?
Q2: If we set the working directory, do we need to give the full path for the script file?

Q1: It should use the downloaded foas, the downloaded one is named openapi-foas.yaml
Q2: Good catch, did a change and forgot to change the command, it should be node runMetricCollection.js ../../../../../openapi-foas.json, I'll update!

I assumed it would be named something like openapi-foas-dev. Thanks for the info 👍

yelizhenden-mdb · 2025-01-23T14:06:44Z

.github/workflows/release-IPA-metrics.yml

+        run: node runMetricCollection.js ../../../../../openapi-foas.json
+
+      - name: Dump Metric Collection Job Data to S3
+        if: ${{steps.get_previous_status.outputs.result == 'true' }}


Nit: we might use working-directory here as well

yelizhenden-mdb · 2025-01-23T14:14:36Z

tools/spectral/ipa/metrics/scripts/getShouldRunMetricsRelease.js

+
+  const lastRunDate = new Date(runs[1].created_at);
+  const today = new Date();
+  const lastRunWasToday =


Nit: you can check if two dates are equal with checking if their toDateString() equal to each other
if(lastRunDate.toDateString() === today.toDateString())

I think this will also include hours, minutes and seconds, which may not be the same, so opted for only checking the day

Nope, it only outputs the day, month, and year (link)

Ah nice, missed toDateString! I'll update

yelizhenden-mdb

LGTM! Exciting! 🚀

andreaangiolillo · 2025-01-27T10:52:10Z

.github/workflows/release-IPA-metrics.yml

@@ -0,0 +1,81 @@
+name: IPA Validation Metrics Release


Could you add a comment at the top of this file explaining what the workflow is doing? What metrics release mean? where are they stored? By looking at the file is seems that it pushing something to s3 maybe but it is not really clear 🤔

Sure! I can add some comments to clarify

andreaangiolillo · 2025-01-27T10:53:44Z

.github/workflows/release-IPA-metrics.yml

+#      aws_access_key:
+#        required: true
+#      aws_secret_key:
+#        required: true
+#      aws_s3_bucket_prefix:
+#        required: true


why do you have these comments? Can we remove them?

These have been removed in this PR: https://github.com/mongodb/openapi/pull/388/files Where it's also included in the oas release workflow

andreaangiolillo · 2025-01-27T10:54:42Z

.github/workflows/release-IPA-metrics.yml

+#    inputs:
+#      env:
+#        description: 'Environment for the FOAS to use for IPA metrics collection'
+#        required: true
+#        type: string


Removed in this PR: https://github.com/mongodb/openapi/pull/388/files

andreaangiolillo · 2025-01-27T10:55:46Z

.github/workflows/release-IPA-metrics.yml

+      - name: Checkout
+        uses: actions/checkout@v4
+
+      - name: Get Previous Run Date and Status


Could you clarify this decision? Adding a comment on the step would be beneficial to remember why we made this decision.

Why can't we run the workflow once a day automatically and file an issue on the repo in the event of a failure?

andreaangiolillo · 2025-01-27T10:56:07Z

.github/workflows/release-IPA-metrics.yml

+          AWS_ACCESS_KEY_ID: ${{ secrets.IPA_S3_BUCKET_DW_STAGING_USERNAME }}       # TODO: Change to passed secret
+          AWS_SECRET_ACCESS_KEY: ${{ secrets.IPA_S3_BUCKET_DW_STAGING_PASSWORD }}   # TODO: Change to passed secret
+          S3_BUCKET_PREFIX: ${{ secrets.IPA_S3_BUCKET_DW_STAGING_PREFIX }}          # TODO: Change to passed secret


Can the remove the todo comments?

Removed in this PR: https://github.com/mongodb/openapi/pull/388/files

lovisaberggren · 2025-01-27T11:23:50Z

@andreaangiolillo

Could you clarify this decision? Adding a comment on the step would be beneficial to remember why we made this decision.

This was a request on the scope, to add a check before uploading the IPA metrics so that:

If it already ran today, skip (the metrics don't need to be updated more than once a day)
If the last run failed, run the job

I can add some comments to clarify what it does

Why can't we run the workflow once a day automatically and file an issue on the repo in the event of a failure?

We wanted to run the workflow as part of the oas release, so that it's triggered only if there are changes detected (similarly to postman release), currently this is covered in the follow up PR which also includes IPA release in the failure-handler

CLOUDP-290416: Add workflow for releasing IPA metrics

fa602c3

lovisaberggren commented Jan 23, 2025

View reviewed changes

CLOUDP-290416: Fix pass secrets

ab5b6e0

lovisaberggren marked this pull request as ready for review January 23, 2025 12:45

lovisaberggren requested a review from a team as a code owner January 23, 2025 12:45

yelizhenden-mdb reviewed Jan 23, 2025

View reviewed changes

CLOUDP-290416: Fix metrics command

734376d

yelizhenden-mdb reviewed Jan 23, 2025

View reviewed changes

CLOUDP-290416: Fix

0029fd2

yelizhenden-mdb reviewed Jan 23, 2025

View reviewed changes

CLOUDP-290416: Fix

3bbc49c

yelizhenden-mdb approved these changes Jan 23, 2025

View reviewed changes

lovisaberggren merged commit 1118fdc into main Jan 23, 2025
13 checks passed

lovisaberggren deleted the CLOUDP-290416 branch January 23, 2025 14:36

andreaangiolillo reviewed Jan 27, 2025

View reviewed changes

lovisaberggren mentioned this pull request Jan 27, 2025

CLOUDP-296909: Add some comments to the IPA release workflow #390

Merged

CLOUDP-290416: Add workflow for releasing IPA metrics #386

CLOUDP-290416: Add workflow for releasing IPA metrics #386

Uh oh!

Conversation

lovisaberggren commented Jan 23, 2025

Proposed changes

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

yelizhenden-mdb left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

lovisaberggren commented Jan 27, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants