Skip to content

feat(ci): dependabot configuration + using helm/kind-action for e2e cluster setup #49

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Aug 17, 2025
Merged

feat(ci): dependabot configuration + using helm/kind-action for e2e cluster setup #49

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
b9ed174
feat(ci): dependabot configuration + using helm/kind-action for e2e c…
SequeI Aug 12, 2025
32e436c
fix: remove mod tidy where it is not needed
SequeI Aug 13, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
37 changes: 37 additions & 0 deletions .github/dependabot.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
version: 2
updates:
- package-ecosystem: "gomod"
directory: "/"
schedule:
interval: "weekly"
open-pull-requests-limit: 10
groups:
go-modules:
patterns:
- "*"
pull-request-title:
prefix: "chore(deps)"
separator: " "
labels:
- "dependencies"
- "go"

- package-ecosystem: "github-actions"
directory: "/"
schedule:
interval: "weekly"
open-pull-requests-limit: 10
groups:
actions-updates:
patterns:
- "*"
update-types:
- "patch"
- "minor"
- "major"
pull-request-title:
prefix: "chore(deps-dev)"
separator: " "
labels:
- "dependencies"
- "github-actions"
20 changes: 16 additions & 4 deletions .github/workflows/lint.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,23 +1,35 @@
name: Lint
name: Linter

on:
push:
branches: [ "main" ]
pull_request:
branches: [ "main" ]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just for clarification this will just run the CI whenever a PR is opened against the main branch, right?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes, that's right. It was previously push/PRs based on any branch


permissions:
contents: read

jobs:
lint:
name: Run on Ubuntu
name: Run Linting
runs-on: ubuntu-latest

steps:
- name: Clone the code
- name: Checkout code
uses: actions/checkout@v4

- name: Setup Go
uses: actions/setup-go@v5
with:
go-version-file: go.mod
cache: true

- name: Go mod tidy
run: go mod tidy
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seeing this I would still advocate for good practice in failing the build if the code checked into the repository is not up-to-date and can not be compiled.

Suggested change
- name: Go mod tidy
run: go mod tidy

Other context: #42 (comment)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

#44 will introduce a workflow to enforce that, if you get time please take a look

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I decided to still add it as I figured the purpose of the workflow is to just run a linter, not check if it is up to date or not, which I believe Kevin's PR on the new gh workflow would check by itself

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

From my point of view, it makes little sense to add a test to check whether the go.mod file is up-to-date like on #44 and update the go.mod file on other workflows.

What would be the goal? Running tests on a state that will not be merged due to the fact its not compilable? From my point of view this would burn valuable CI time of the Sigstore organisation.

Maybe I am missing something here, but for me it's really hard to see the value with this approach :)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Which is exactly why I removed it from all workflows assuming #44 will be merged, hence never running actions on a state of the project that will not actually be merged

I left a comment at the bottom about this too.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@miyunari I believe @SequeI has already removed this, it's not in the latest version of the PR


- name: Run linter
- name: Run golangci-lint
uses: golangci/golangci-lint-action@v8
with:
version: v2.3.0
args: --timeout=5m --issues-exit-code=0 --verbose
cache: true
18 changes: 10 additions & 8 deletions .github/workflows/sign-model.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
name: Sign Model Tensorflow Saved Model
name: Sign model with Sigstore

on:
workflow_dispatch:
Expand Down Expand Up @@ -27,18 +27,20 @@ jobs:
"$ACTIONS_ID_TOKEN_REQUEST_URL&audience=sigstore" | jq -r '.value')
echo "OIDC_TOKEN=$token" >> $GITHUB_ENV

- name: Set up environment and sign model
- name: Sign model
env:
OIDC_TOKEN: ${{ env.OIDC_TOKEN }}
run: |
docker run --rm -v $(pwd)/testdata/tensorflow_saved_model:/tensorflow_saved_model:z -w /tensorflow_saved_model ghcr.io/sigstore/model-transparency-cli:v1.0.1 sign sigstore --signature="/tensorflow_saved_model/model.sig" --identity_token "$OIDC_TOKEN" /tensorflow_saved_model
docker run --rm -v $(pwd)/testdata/tensorflow_saved_model:/tensorflow_saved_model:z -w \
/tensorflow_saved_model ghcr.io/sigstore/model-transparency-cli:v1.0.1 sign sigstore \
--signature="/tensorflow_saved_model/model.sig" --identity_token "$OIDC_TOKEN" /tensorflow_saved_model

- name: Create tar.gz of the signed model
- name: Create a tarball of the signed model and signature
run: |
tar -czvf signed_model.tar.gz -C $(pwd)/testdata/tensorflow_saved_model .
tar -czvf signed_model_bundle.tar.gz -C $(pwd)/testdata/tensorflow_saved_model .

- name: Upload signed model as artifact
- name: Upload signed model artifact
uses: actions/upload-artifact@v4
with:
name: signed-model
path: signed_model.tar.gz
name: signed-model-bundle
path: signed_model_bundle.tar.gz
37 changes: 13 additions & 24 deletions .github/workflows/test-e2e.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,50 +2,39 @@ name: E2E Tests

on:
push:
branches: [ "main" ]
pull_request:
branches: [ "main" ]

permissions:
contents: read

jobs:
test-e2e:
name: Run on Ubuntu
name: Run E2E Tests
runs-on: ubuntu-latest
steps:
- name: Clone the code
- name: Checkout code
uses: actions/checkout@v4

- name: Setup Go
uses: actions/setup-go@v5
with:
go-version-file: go.mod

- name: Install the latest version of kind
run: |
curl -Lo ./kind https://kind.sigs.k8s.io/dl/latest/kind-linux-amd64
chmod +x ./kind
sudo mv ./kind /usr/local/bin/kind

- name: Verify kind installation
run: kind version

- name: Install kubectl
run: |
curl -LO "https://dl.k8s.io/release/$(curl -L -s https://dl.k8s.io/release/stable.txt)/bin/linux/amd64/kubectl"
chmod +x kubectl
sudo mv kubectl /usr/local/bin/kubectl

- name: Verify kubectl installation
run: kubectl version --client
cache: true

- name: Create kind cluster
run: kind create cluster
uses: helm/kind-action@v1
with:
cluster_name: kind
Comment on lines +27 to +29
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That looks like a great improvement. But its only usable on the github workflow. To make it easier to run our e2e tests on local machines or other CI pipelines does it make sense to provide the general as Makefile rules?

Here we would then just need to run:

make tools kind-start

Where tools would install kind and whatever other tools we need for development into ${PWD}/bin/tools on this repository. We can pin the kind version in the Makefile.

A kind-start rule could look like this:

.PHONY: kind-start
kind-start: kind
	$(KIND) create cluster --name model-validation-testing --config kind-1-33-0.yaml

.PHONY: tools
tools:
	$(call go-get-tool,$(KIND),sigs.k8s.io/kind,$(KIND_VERSION))

It would make it much easier to reproduce CI issues locally.

Copy link
Contributor

@knrc knrc Aug 13, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note this doesn't change the existing behaviour, it replaces the SDK scaffolded bits with the helm kind action to install/run the cluster. I think this is cleaner than what was there previously.

The makefile already has options for running the e2e test against an existing kind cluster, just not for creating it.

I also find that act is good for running workflows locally, if we need to reproduce CI issues.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

^ Agreed, we can include in the docs how to run workflows locally soon instead


- name: Wait for cluster to be ready
- name: Wait for kind cluster to be ready
run: |
echo "Waiting for cluster to be ready..."
kubectl wait --for=condition=Ready nodes --all --timeout=300s
kubectl wait --for=condition=Ready --namespace=kube-system pod --all --timeout=300s
echo "Cluster is ready"

- name: Running Test e2e
- name: Run E2E tests
run: |
go mod tidy
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
go mod tidy

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FYI That's left over from a previous PR, scaffolded by the SDK.

make test-e2e
2 changes: 2 additions & 0 deletions .github/workflows/test.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,9 @@ name: Tests

on:
push:
branches: [ "main" ]
pull_request:
branches: [ "main" ]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess we can then get rid of this entire workflow?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we really want to remove this? I think we should keep unit tests and e2e tests

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed, we need to keep, this workflow is responsible for running the general unit tests, then the other workflow is specifically for the e2e tests we have so far


jobs:
test:
Expand Down