Skip to content

RELEASE_PROCESS.md

Latest commit

 

History

History
345 lines (266 loc) · 14.4 KB

RELEASE_PROCESS.md

File metadata and controls

345 lines (266 loc) · 14.4 KB

Release Process

This guide is for OCM release managers.

Scope: CLI and Kubernetes Controller are released together in version lockstep (RC and final).

General Information

Context and scope

Development in the Open Component Model monorepo happens on main, while releases are prepared and promoted from release branches (releases/vX.Y). For the time being, we will only ship minor releases (e.g., v0.17.0, v0.18.0) and patch releases (e.g., v0.17.1), without any major releases. This repository follows a lockstep release model for multiple components — currently the CLI and the Kubernetes Controller.

Release cadence

The default cadence is sprint-based: one release every two weeks. At the start of each sprint, create a new release branch and a new Release Candidate (RC). After one sprint of testing, promote the previous RC to a final release. At the end of each sprint (typically Friday), assign the next release responsible.

Patch releases are handled out-of-band and created on demand for critical fixes. A scheduled release can be skipped if the branch is not ready or there are no meaningful changes.

Release Responsible

The release responsible coordinates and executes the release flow. This role rotates each sprint and involves:

  • Agreeing with the team, if a new release is needed for the past sprint
  • Triggering and supervising release branch creation
  • Creating release candidates for all lockstep components
  • Agreeing with the team, if the new release is ready for promotion
  • Promoting release candidates to final releases
  • Coordinating patch releases when needed
  • Keeping this guide accurate and up to date

Quick links: Create release branch · Create RC · Promote to Final · Patch release

Release Checklist

Copy this checklist to your "Sprint Responsible" issue:

- [ ] New release branch created (e.g `releases/v0.17`)
- [ ] CLI RC created and verified (`v0.17-rc.1`)
- [ ] Controller RC created and verified (`v0.17-rc.1`)
- [ ] CLI Final promoted from last RC (`v0.16-rc.1` --> `v0.16.0`)
- [ ] Controller Final promoted from last RC (`v0.16-rc.1` --> `v0.16.0`)
- [ ] Both releases visible on GitHub Releases page (`v0.16.0`)

Timeline

When Action
Sprint Start Create new release branch and RCs for both components
During sprint Issue patch releases or increment RCs if required (Security issues, bugs, ..)
Sprint End Assign next release responsible
Next Sprint Start Promote previous RCs to Final

Release Workflow Diagram

The following diagram illustrates a typical release cycle with an initial release (v0.17.0) and a subsequent patch release (v0.17.1):

gitGraph TB:
    commit id: "VERSION 0.17.0-dev"
    commit id: "feat: some feature"
    branch "releases/v0.17"
    commit tag: "v0.17.0-rc.1" type: REVERSE
    checkout main
    commit id: "fix: hotfix bug" type: HIGHLIGHT
    checkout releases/v0.17
    cherry-pick id: "fix: hotfix bug"
    commit tag: "v0.17.0-rc.2"
    branch "tag/v0.17.0"
    checkout "tag/v0.17.0"
    commit id: "VERSION 0.17.0" tag:"v0.17.0"
    checkout main
    commit id: "VERSION 0.18.0-dev"
    commit id: "fix: another hotfix" type: HIGHLIGHT
    checkout releases/v0.17
    cherry-pick id: "fix: another hotfix"
    commit tag: "v0.17.1-rc.1"
    branch "tag/v0.17.1"
    checkout "tag/v0.17.1"
    commit id: "VERSION 0.17.1" tag:"v0.17.1"
    checkout main
    commit id: "feat: another feature"
    branch "releases/v0.18"
    commit tag: "v0.18.0-rc.1"
Loading

What to do

1) Create the release branch

A new release branch marks the cut-off point for that minor release line. Once created, only bug fixes and documentation changes are allowed.

  1. Run workflow Release Branch Creation.
  2. Set target branch to releases/vX.Y.
  3. Confirm the branch was created successfully.
What happens in the background?
  • Validate target_branch against regex pattern releases/v0.[0-9]+ (for the time being, we only allow minor releases)
  • Check if target_branch already exists; if yes, skip creation and report success
  • Check out the repository and identify the latest commit on source_branch (default: main)
  • Create a new branch refs/heads/<target_branch> pointing to this commit
  • Publish workflow summary with source branch, target branch, and commit SHA

Caution

Cut-off policy for this release branch

  • No feature or breaking-change work is accepted.
  • Non-bugfix and non-documentation changes require release responsible approval.
  • Any change that is not a bug fix or a documentation change require release responsible approval.
  • Any bug fix that is not deemed critical must be approved by the release manager.
  • Fixes must be merged to main first, then cherry-picked to this branch.

2) Create a Release Candidate (CLI + Controller)

Release candidates are created for both components sequentially, in lock-step.

  1. Run workflow CLI Release with:
    • dry_run = true first to validate
    • dry_run = false for actual release
    • Branch to release from set to the release branch created in step 1 (e.g., releases/v0.17)
  2. Run workflow Controller Release with equivalent inputs.
  3. Verify both pre-releases were created successfully on the GitHub Releases page.
    • CLI assets: platform binaries (ocm-*) and OCI tarballs
    • Controller assets: Helm chart (.tgz)

Caution

Always do a dry-run first before the actual release. CLI and Controller are released together. Do not release only one of them.

Understanding the Dry-Run Summary

Dry-run mode validates the release workflow without creating any tags, releases, or artifacts. The workflow summary shows exactly what would happen:

Field Description
RC Version The RC version that would be created (e.g., 0.17.1-rc.1)
Base Version The final version after promotion (e.g., 0.17.1)
Set Latest Whether latest tag would be applied to OCI images
Highest Final Version The current highest released version

Important for Patch Releases:

When releasing a patch for an older version line (e.g., v0.16.1 when v0.17.0 already exists), the dry-run summary will show:

Set Latest: false
Highest Final Version: 0.17.0

This means the patch release will not be marked as the GitHub "Latest Release" and the latest OCI tag will not be updated. This is expected behavior to prevent users from accidentally downgrading when pulling latest.

What happens in the background?
  • CLI path (cli-release.yml)
    • prepare: compute next RC metadata + changelog.
    • tag_rc: create/push RC tag (skipped on dry-run).
    • build: call cli.yml to build binaries and OCI artifacts.
    • release_rc: publish GitHub pre-release with binary and OCI tarball assets.
  • Controller path (controller-release.yml)
    • prepare: compute next RC metadata + changelog (shared release-candidate-version.yml).
    • tag_rc: create/push RC tag via release environment gate (skipped on dry-run).
    • build: call kubernetes-controller.yml to build the controller image and Helm chart.
    • release_rc: publish GitHub pre-release with Helm chart .tgz as asset.
  • If one component fails, do not proceed to final promotion.

3) Promote RC to Final Release (CLI + Controller)

Final promotion takes an existing RC and promotes it to a stable release. The workflow verifies attestations before creating any final artifacts.

The final promotion is triggered automatically after the RC phase completes and the respective environment gate is approved.

  1. Wait for the RC to be tested (typically 1 sprint).
  2. Go to the CLI Release workflow run in GitHub Actions.
  3. Approve the release environment gate (requires reviewer approval + wait timer).
  4. The CLI workflow will automatically:
    • Verify binary and OCI image attestations
    • Create final tag from RC commit
    • Promote OCI image tags
    • Publish final GitHub release with binaries
  5. Go to the Controller Release workflow run.
  6. Approve the release environment gate (requires reviewer approval).
  7. The Controller workflow will automatically:
    • Verify image and Helm chart attestations
    • Create final tag from RC commit
    • Promote controller image tags
    • Re-package the Helm chart with the final version, push it to OCI, and attest it
    • Publish final GitHub release with the final Helm chart
  8. Verify both final releases are published on the GitHub Releases page.

Important

🔐 Security: Both workflows automatically verify all attestations from their respective RC releases before proceeding. If verification fails, the promotion is aborted.

What happens in the background?
  • CLI path (cli-release.yml)
    • verify_attestations: verify binary and OCI image attestations (gated by release environment).
    • promote_final: create final tag from RC commit, promote OCI image tags.
    • release_final: publish final GitHub release with assets from RC.
  • Controller path (controller-release.yml)
    • verify_attestations: verify controller image and Helm chart attestations (gated by release environment).
    • promote_and_release_final (single combined job):
      • Create final tag from RC commit.
      • Promote controller image tags via ORAS.
      • Re-package Helm chart with final version strings (Chart.yaml, values.yaml), push to OCI registry, and create build-provenance attestation.
      • Run helm/verify-promote to validate RC-to-Final chart consistency.
      • Publish final GitHub release with the re-packaged Helm chart.
    • Due to Helm specifics, the chart cannot simply be re-tagged from RC to Final — it must be re-packaged and re-attested because it embeds version strings.
  • Final is only valid when both components are promoted in the same cycle.

4) Create a patch release (if needed)

Patch releases address critical fixes for an already-released version. The fix must always land on main first, then be cherry-picked to the release branch.

Important

⚠️ Older version lines: When creating a patch for an older release line (e.g., v0.16.1 when v0.17.0 exists), the release will NOT be marked as "latest" on GitHub and the latest OCI tags will remain on the newer version. This is intentional — use dry-run first to verify this behavior in the summary.

  1. Ensure the fix was merged to main first.
  2. Cherry-pick the fix to the correct release branch, e.g. releases/v0.17 for a patch to v0.17.1.
  3. Create a PR with the cherry-picked commit to the release branch.
  4. Create and test RCs for both CLI and Controller.
  5. Promote both components to final.
Patch PR naming and cherry-pick flow 
# Check out the target release branch
git checkout releases/vX.Y

# Cherry-pick the commit from main
git cherry-pick <commit-hash>

# Create a PR to the release branch
gh pr create \
   --title "[releases/vX.Y] cherry-pick: <Original PR or Commit>" \
   --body "Cherry-pick of <Original PR or Commit> from main to releases/vX.Y" \
   --base releases/vX.Y \
   --draft

Merge the PR, then follow the RC and Final promotion steps as usual.

Release notes

Release notes are generated automatically by the release workflows using git-cliff. The release responsible does not need to manually compose notes for normal RC or final runs—the changelog is derived from conventional commit messages.

Troubleshooting

If something goes wrong during a release, check the following common issues:

RC was not created

  • Check if the workflow run failed before tag creation.
  • Rerun as dry-run first, then rerun with dry_run=false.

Final promotion fails with "no RC found"

  • Verify that the latest RC tag exists on the release branch.
  • Create a new RC for both components, then promote again.

CLI and Controller versions diverged

  • Ensure both workflows were run for the same release branch in the same cycle.
  • Stop promotion and align on a fresh RC cycle in lockstep.

Final release exists for one component but not the other

  • Check the status of both workflow runs and the release pages.
  • Complete or rerun the missing component release immediately.

Attestation verification failed

  • Verify that the RC release binaries have valid build provenance attestations.
  • Check that the OCI image was pushed to GHCR with attestation metadata.
  • Use gh attestation verify <file> --repo <repo> to manually verify CLI binaries.
  • Use gh attestation verify oci://<image>@<digest> --repo <repo> to verify OCI images.
  • For the controller, also verify the Helm chart OCI artifact: gh attestation verify oci://<chart-repo>@<chart-digest> --repo <repo>
  • If attestations are missing, create a new RC (attestations are generated during build).

Retracting a release

If a release needs to be retracted due to critical bugs or security issues:

  1. Edit the GitHub Release and mark it as "Pre-release" to hide it from the latest release view.
  2. Prepend a deprecation notice to the release notes: 
    > ⚠️ **RETRACTED**: This release has been retracted due to [reason].
> Please upgrade to vX.Y.Z instead.
  3. Create a new patch release with the fix as soon as possible.
  4. If security-related, consider filing a security advisory via GitHub's Security tab.

The OCI image tags remain available in GHCR for users who have pinned to the specific version, but the retraction notice guides new users to the replacement release.