Skip to content

Adjust release process after quickstart changes #4104

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 3 commits into
base: develop
Choose a base branch
from
Open

Adjust release process after quickstart changes #4104

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
e30a3fe
Adjust release process after quickstart changes
schustmi Oct 27, 2025
831a33a
Remove more quickstart CI code
schustmi Oct 31, 2025
6fdc3f3
Merge branch 'develop' into misc/release-changes-after-new-quickstart
schustmi Oct 31, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 0 additions & 4 deletions .github/workflows/release_prepare.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -100,10 +100,6 @@ jobs:
fi
done
git add examples/quickstart
- name: Regenerate quickstart cloud requirements
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As far as I can see this used to generate examples/quickstart/requirements_{aws,azure,gcp}.txt files for the quickstart testing script. Since this is now disabled (which was the right call I think), wouldn't the qs_run_release_flow fail?

It has a check:

...
REQS="requirements_${CLOUD}.txt"
...
if [[ ! -f "${REQS}" ]]; then
  echo "Error: Requirements file not found: ${REQS}" >&2
  exit 1
fi
...
``

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same script also looks for config files that do not exist.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The installation seems a bit weird as well. The runner starts, checks out to the branch, already install ZenML once, passes the branch name to the test script, the test script manually replaces the hard-coded zenml requirement (which is the only requirement), pip installs zenml throught the requirements.txt file again and only then runs the example. This seems like a very complicated and redundant process.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep, this whole thing has been a mess. I removed as much of it as I could now.

run: |
scripts/generate_cloud_requirements.sh --zenml-version "${{ needs.fetch-versions.outputs.new_version }}"
git add examples/quickstart/requirements_*.txt
# Generate and append release notes
- name: Generate release notes
# Only run this part once when the release branch is created
Expand Down
1 change: 0 additions & 1 deletion docker/zenml-quickstart-dev.Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,6 @@ FROM $BASE_IMAGE AS base
# Set the working directory
WORKDIR /app

ARG ZENML_BRANCH
ARG CLOUD_PROVIDER

# Install the Python requirements
Expand Down
29 changes: 0 additions & 29 deletions docker/zenml-quickstart.Dockerfile
View file
Open in desktop

This file was deleted.

3 changes: 0 additions & 3 deletions release-cloudbuild-preparation.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -77,7 +77,6 @@ steps:
--platform linux/amd64 \
--build-arg BASE_IMAGE=$$USERNAME/prepare-release:base-${_ZENML_NEW_VERSION} \
--build-arg CLOUD_PROVIDER=gcp \
--build-arg ZENML_BRANCH=${_ZENML_BRANCH} \
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am confused about all this. Maybe you can clear it for me. I see now that we don't even build any quickstart images for the release (which I like). But if that's the case, why do we even need to build them now for testing the release? Isn't ok for us to use the base image to test?

Previously, different quickstarts required a different "pre-specified" list of requirements to run the quickstart. As far as I can see, this is no longer a concern of the new one.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These image still installed the stack requirements in there, but I don't even understand why that was necessary back then. I removed all of that now though.

-f docker/zenml-quickstart-dev.Dockerfile \
-t $$USERNAME/prepare-release:quickstart-gcp-${_ZENML_NEW_VERSION}

Expand All @@ -97,7 +96,6 @@ steps:
--platform linux/amd64 \
--build-arg BASE_IMAGE=$$USERNAME/prepare-release:base-${_ZENML_NEW_VERSION} \
--build-arg CLOUD_PROVIDER=aws \
--build-arg ZENML_BRANCH=${_ZENML_BRANCH} \
-f docker/zenml-quickstart-dev.Dockerfile \
-t $$USERNAME/prepare-release:quickstart-aws-${_ZENML_NEW_VERSION}
id: build-quickstart-aws
Expand All @@ -116,7 +114,6 @@ steps:
--platform linux/amd64 \
--build-arg BASE_IMAGE=$$USERNAME/prepare-release:base-${_ZENML_NEW_VERSION} \
--build-arg CLOUD_PROVIDER=azure \
--build-arg ZENML_BRANCH=${_ZENML_BRANCH} \
-f docker/zenml-quickstart-dev.Dockerfile \
-t $$USERNAME/prepare-release:quickstart-azure-${_ZENML_NEW_VERSION}
id: build-quickstart-azure
Expand Down
69 changes: 0 additions & 69 deletions release-cloudbuild.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -156,75 +156,6 @@ steps:
secretEnv:
- USERNAME

# build client quickstart gcp image - python 3.11
- name: gcr.io/cloud-builders/docker
args:
- '-c'
- |
docker build \
--platform linux/amd64 \
--build-arg ZENML_VERSION=$TAG_NAME \
--build-arg PYTHON_VERSION=3.11 \
--build-arg CLOUD_PROVIDER=gcp \
-f docker/zenml-quickstart.Dockerfile . \
-t $$USERNAME/zenml-public-pipelines:quickstart-$TAG_NAME-py3.11-gcp
id: build-quickstart-3.11-gcp
waitFor: [ 'push-base' ]
entrypoint: bash
secretEnv:
- USERNAME

# build client quickstart aws image - python 3.11
- name: gcr.io/cloud-builders/docker
args:
- '-c'
- |
docker build \
--platform linux/amd64 \
--build-arg ZENML_VERSION=$TAG_NAME \
--build-arg PYTHON_VERSION=3.11 \
--build-arg CLOUD_PROVIDER=aws \
-f docker/zenml-quickstart.Dockerfile . \
-t $$USERNAME/zenml-public-pipelines:quickstart-$TAG_NAME-py3.11-aws
id: build-quickstart-3.11-aws
waitFor: [ 'push-base' ]
entrypoint: bash
secretEnv:
- USERNAME

# build client quickstart azure image - python 3.11
- name: gcr.io/cloud-builders/docker
args:
- '-c'
- |
docker build \
--platform linux/amd64 \
--build-arg ZENML_VERSION=$TAG_NAME \
--build-arg PYTHON_VERSION=3.11 \
--build-arg CLOUD_PROVIDER=azure \
-f docker/zenml-quickstart.Dockerfile . \
-t $$USERNAME/zenml-public-pipelines:quickstart-$TAG_NAME-py3.11-azure
id: build-quickstart-3.11-azure
waitFor: [ 'push-base' ]
entrypoint: bash
secretEnv:
- USERNAME

# push quickstart images
- name: gcr.io/cloud-builders/docker
args:
- '-c'
- docker push --all-tags $$USERNAME/zenml-public-pipelines
id: push-quickstart
waitFor:
- build-quickstart-3.11-gcp
- build-quickstart-3.11-aws
- build-quickstart-3.11-azure
- docker-login
entrypoint: bash
secretEnv:
- USERNAME

timeout: 3600s
availableSecrets:
secretManager:
Expand Down
158 changes: 0 additions & 158 deletions scripts/generate_cloud_requirements.sh
View file
Open in desktop

This file was deleted.

Loading