chore: Generates and checks PURLS on the CI #1364

oarbusi · 2025-06-11T13:56:31Z

Proposed changes

Generate PURLs for all the resources.
Adds check in the CI to avoid drift in PURLs when dependencies change
Removed serverless-privatelink-endpoint tests in CI (resource is deprecated)
Fixed mongoDB version in Online archive tests (was using deprecated version)

Majority of changes in this PR are due to building the resources and generating PURLS. Important changes are in Makefile, /scripts and .github/workflows

Will do a followup PR that will automatically update PURLS in dependabot PRs and SDK Updates

Link to any related issue(s): CLOUDP-324019

Type of change:

Bug fix (non-breaking change which fixes an issue)
New feature (non-breaking change which adds functionality)
Breaking change (fix or feature that would cause existing functionality to not work as
expected)
This change requires a documentation update
If changes include removal or addition of 3rd party GitHub actions, I updated our internal document. Reach out to the APIx Integration slack channel to get access to the internal document.

Manual QA performed:

cfn invoke for each of CRUDL/cfn test
Updated resource in example
Published to AWS private registry
Used the template in example to create and update a stack in AWS
Deleted stack to ensure resources are deleted
Created multiple resources in same stack
Validated in Atlas UI
Included screenshots

Required Checklist:

I have signed the MongoDB CLA
I have added tests that prove my fix is effective or that my feature works
I have checked that this change does not generate any credentials and that they are NOT accidentally logged anywhere.
I have added any necessary documentation (if appropriate)
I have run make fmt and formatted my code
For CFN Resources: I have released by changes in the private registry and proved by change
works in Atlas

Further comments

oarbusi · 2025-06-11T15:45:08Z

.github/workflows/contract-testing.yaml

   - uses: dorny/paths-filter@de90cc6fb38fc0963ad72b210f1f284cd68cea36
     id: filter
     with:
+      predicate-quantifier: 'every'


This avoids running contract test when only the files in /compliance have changed (see https://github.com/dorny/paths-filter/tree/de90cc6fb38fc0963ad72b210f1f284cd68cea36/ for more details)

oarbusi · 2025-06-11T16:05:55Z

.github/workflows/code-health.yaml

      run: | 
        cd cfn-resources
        go build -v ./...
+    - name: Install CloudFormation CLI


necessary, as gen-purls runs make build inside the resource directory, and that runs cfn generate

I think you need setup-python before, e.g.: https://github.com/mongodb/mongodbatlas-cloudformation-resources/blob/master/.github/workflows/publish.yaml#L56

lantoli · 2025-06-12T05:49:08Z

.github/workflows/contract-testing.yaml

+          - '!cfn-resources/resource-policy/compliance/**'
        search-deployment:
          - 'cfn-resources/search-deployment/**'
-        serverless-private-endpoint:


is serverless-private-endpoint not needed?

it is deprecated and tests can no longer be run (API does not work). I cleaned this up

lantoli · 2025-06-12T05:50:24Z

cfn-resources/datalakes/docs/datalakeawscloudproviderconfigview.md

 _Type_: String

-_Minimum_: <code>20</code>
+_Minimum Length_: <code>20</code>


it looks like some changes are no related to the main changes of the PR, consider doing them in a different PR

This are the changes of building the resources, which is needed to generate the PURLS. We need this if we want to generate the purls

lantoli · 2025-06-12T05:50:50Z

cfn-resources/datalakes/resource-role.yaml

                Action:
                - "secretsmanager:GetSecretValue"
-                - "ec2:CreateVpcEndpoint"
-                - "ec2:DeleteVpcEndpoints"


similar comment, these changes don't seem to be related to the main changes of the PR

This are the changes of building the resources, which is needed to generate the PURLS. We need this if we want to generate the purls

can't this built first in a different PR? and then do the pr for purls?

lantoli · 2025-06-12T05:54:34Z

cfn-resources/access-list-api-key/compliance/purls.txt

@@ -0,0 +1,19 @@
+pkg:golang/../vendor/go.mongodb.org/atlas-sdk/v20231115014@(devel)


if I understand correctly, purls.txt depends on go.mod and go.sum. We share the same go.mod files for all the resources, so all purls.txt will be always be the same.
Do we need to generate all of them, or can we just generate one purls.txt?

purl file is generated from a binary, and a binary is generated for each of the resources, so even if they share go.mod and go.sum, technically the binary is different(and that is what we are shipping) so we have to generate the purls for each

lantoli · 2025-06-12T06:05:01Z

scripts/generate-purls.sh

@@ -0,0 +1,30 @@
+#!/usr/bin/env bash


consider exposing check & generate purls GHAs from TF repo so all the repos can use it, and we don't need the scripts & make goals in all the repos

CFN and TF are built different, so it's not exactly the same and wouldn't be 100% reusable. For now will keep it as is but will keep an eye for next repos.

ok, that's fair

oarbusi · 2025-06-12T08:33:35Z

Will not put PURLS in the repo anymore, and will generate it only when needed for the release

oarbusi added 3 commits June 11, 2025 15:53

generate and check purls

c0b4282

dont run tests when compliance files are changed

c47d88e

predicate-quantifier

95466a6

oarbusi commented Jun 11, 2025

View reviewed changes

install cli in ci

21513a8

oarbusi marked this pull request as ready for review June 11, 2025 15:55

oarbusi requested a review from a team as a code owner June 11, 2025 15:55

oarbusi added 2 commits June 11, 2025 18:00

fix mongodb version in test of online archive

2c6b71d

remove serverless-privatelink-endpoint tests in CI

b8f6fb8

oarbusi commented Jun 11, 2025

View reviewed changes

lantoli reviewed Jun 12, 2025

View reviewed changes

oarbusi closed this Jun 12, 2025

		@@ -0,0 +1,19 @@
		pkg:golang/../vendor/go.mongodb.org/atlas-sdk/v20231115014@(devel)

chore: Generates and checks PURLS on the CI #1364

chore: Generates and checks PURLS on the CI #1364

Uh oh!

Conversation

oarbusi commented Jun 11, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Proposed changes

Type of change:

Manual QA performed:

Required Checklist:

Further comments

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

lantoli Jun 12, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

oarbusi commented Jun 12, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

oarbusi commented Jun 11, 2025 •

edited

Loading

lantoli Jun 12, 2025 •

edited

Loading