forked from moby/buildkit
-
Notifications
You must be signed in to change notification settings - Fork 0
Add workflow automation for maintaining patched fork #14
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from 3 commits
Commits
Show all changes
9 commits
Select commit
Hold shift + click to select a range
9e3ef17
Consolidate documentation into single DEPLOYMENT.md
adityamaru 2da592c
Simplify to rebase-based workflow for patch management
adityamaru 24413f6
Remove FORK_WORKFLOW.md - consolidated into DEPLOYMENT.md
adityamaru 008d487
Fix yamllint errors: add newlines at end of workflow files
adityamaru 798bc75
Fix newline handling in patch messages
adityamaru 1ec4dcd
Fix build: Create bin directory before building binaries
adityamaru 78ccdff
Fix YAML syntax and git fetch issues in release workflow
adityamaru 485be4f
Add error checking for critical git push operations
adityamaru 8e1b7b0
Add required permissions to GitHub Actions workflows
adityamaru File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,102 @@ | ||
| name: Rebase on Upstream | ||
|
|
||
| on: | ||
| workflow_dispatch: | ||
| schedule: | ||
| # Run weekly on Mondays at 9am UTC | ||
| - cron: '0 9 * * 1' | ||
|
|
||
| jobs: | ||
| rebase: | ||
| runs-on: ubuntu-latest | ||
| steps: | ||
| - name: Checkout repository | ||
| uses: actions/checkout@v4 | ||
| with: | ||
| fetch-depth: 0 | ||
| token: ${{ secrets.GITHUB_TOKEN }} | ||
|
|
||
| - name: Setup git | ||
| run: | | ||
| git config user.name "GitHub Actions" | ||
| git config user.email "actions@github.com" | ||
|
|
||
| - name: Add upstream remote | ||
| run: | | ||
| git remote add upstream https://github.com/moby/buildkit.git || true | ||
| git fetch upstream master | ||
| git fetch origin master | ||
|
|
||
| - name: Rebase master on upstream | ||
| id: rebase | ||
| run: | | ||
| git checkout master | ||
|
|
||
| # Count our patches before rebase | ||
| BEFORE_COUNT=$(git rev-list upstream/master..HEAD --count) | ||
| echo "Patches before rebase: $BEFORE_COUNT" | ||
|
|
||
| # Perform rebase | ||
| if git rebase upstream/master; then | ||
| echo "✅ Rebase successful" | ||
|
|
||
| # Count our patches after rebase | ||
| AFTER_COUNT=$(git rev-list upstream/master..HEAD --count) | ||
| echo "Patches after rebase: $AFTER_COUNT" | ||
|
|
||
| # Check what changed | ||
| if [ "$BEFORE_COUNT" -ne "$AFTER_COUNT" ]; then | ||
| echo "" | ||
| echo "🎉 Patch count changed from $BEFORE_COUNT to $AFTER_COUNT" | ||
| echo "This likely means some of your patches were merged upstream!" | ||
| fi | ||
|
|
||
| # Push the rebased master | ||
| git push origin master --force-with-lease | ||
|
|
||
| echo "rebase_status=success" >> $GITHUB_OUTPUT | ||
| echo "patch_count=$AFTER_COUNT" >> $GITHUB_OUTPUT | ||
cursor[bot] marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| else | ||
| echo "❌ Rebase failed - manual intervention required" | ||
| echo "" | ||
| echo "To resolve manually:" | ||
| echo "1. git checkout master" | ||
| echo "2. git rebase upstream/master" | ||
| echo "3. Resolve conflicts" | ||
| echo "4. git push origin master --force-with-lease" | ||
|
|
||
| echo "rebase_status=failed" >> $GITHUB_OUTPUT | ||
| exit 1 | ||
| fi | ||
|
|
||
| - name: Create issue on failure | ||
| if: failure() | ||
| uses: actions/github-script@v7 | ||
| with: | ||
| script: | | ||
| await github.rest.issues.create({ | ||
| owner: context.repo.owner, | ||
| repo: context.repo.repo, | ||
| title: 'Manual rebase required: conflicts with upstream', | ||
| body: `The automated rebase on upstream has failed due to conflicts. | ||
|
|
||
| ## Action Required | ||
|
|
||
| Please manually rebase the master branch: | ||
|
|
||
| \`\`\`bash | ||
| git checkout master | ||
| git fetch upstream | ||
| git rebase upstream/master | ||
| # Resolve any conflicts | ||
| git push origin master --force-with-lease | ||
| \`\`\` | ||
|
|
||
| ## Workflow Run | ||
|
|
||
| [View failed workflow run](https://github.com/${{ github.repository }}/actions/runs/${{ github.run_id }}) | ||
|
|
||
| --- | ||
| *This issue was automatically created by the rebase workflow.*`, | ||
| labels: ['upstream-sync', 'needs-attention'] | ||
| }); | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,294 @@ | ||
| name: Release Patched Version | ||
|
|
||
| on: | ||
| workflow_dispatch: | ||
| inputs: | ||
| upstream_version: | ||
| description: 'Upstream version to patch (e.g., v0.17.0)' | ||
| required: true | ||
| type: string | ||
| release_suffix: | ||
| description: 'Suffix for release (default: blacksmith)' | ||
| required: false | ||
| type: string | ||
| default: 'blacksmith' | ||
|
|
||
| jobs: | ||
adityamaru marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| create-patched-release: | ||
| runs-on: ubuntu-latest | ||
| outputs: | ||
| release_tag: ${{ steps.create-release-branch.outputs.release_tag }} | ||
| patch_count: ${{ steps.create-release-branch.outputs.patch_count }} | ||
| patch_list: ${{ steps.create-release-branch.outputs.patch_list }} | ||
| steps: | ||
| - name: Checkout repository | ||
| uses: actions/checkout@v4 | ||
| with: | ||
| fetch-depth: 0 | ||
| token: ${{ secrets.GITHUB_TOKEN }} | ||
|
|
||
| - name: Setup git | ||
| run: | | ||
| git config user.name "GitHub Actions" | ||
| git config user.email "actions@github.com" | ||
|
|
||
| - name: Add and fetch upstream | ||
| run: | | ||
| git remote add upstream https://github.com/moby/buildkit.git || true | ||
| git fetch upstream --tags | ||
| git fetch origin master | ||
adityamaru marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
|
||
| - name: Verify upstream version exists | ||
| run: | | ||
| if ! git rev-parse "refs/tags/${{ github.event.inputs.upstream_version }}" >/dev/null 2>&1; then | ||
| echo "❌ Upstream version ${{ github.event.inputs.upstream_version }} not found" | ||
| echo "" | ||
| echo "Available recent versions:" | ||
| git tag -l 'v*.*.*' | grep -v '\-' | sort -rV | head -20 | ||
| exit 1 | ||
| fi | ||
| echo "✅ Found upstream version ${{ github.event.inputs.upstream_version }}" | ||
|
|
||
| - name: Create patched release branch | ||
| id: create-release-branch | ||
| run: | | ||
| VERSION="${{ github.event.inputs.upstream_version }}" | ||
| SUFFIX="${{ github.event.inputs.release_suffix }}" | ||
| RELEASE_TAG="${VERSION}-${SUFFIX}" | ||
| RELEASE_BRANCH="${VERSION}-${SUFFIX}" | ||
|
|
||
| # Check if already exists | ||
| if git rev-parse "refs/tags/$RELEASE_TAG" >/dev/null 2>&1; then | ||
| echo "❌ Release $RELEASE_TAG already exists" | ||
| echo "" | ||
| echo "To recreate it, first delete the existing tag:" | ||
| echo " git push origin --delete $RELEASE_TAG" | ||
| echo " git push origin --delete $RELEASE_BRANCH" | ||
| exit 1 | ||
| fi | ||
|
|
||
| # Checkout upstream version | ||
| echo "Creating release branch from $VERSION..." | ||
| git checkout -b "$RELEASE_BRANCH" "$VERSION" | ||
|
|
||
| # Find all commits that are in origin/master but not in upstream/master | ||
| echo "" | ||
| echo "Finding patches to apply..." | ||
| PATCHES=$(git rev-list upstream/master..origin/master --reverse --no-merges) | ||
|
|
||
| if [ -z "$PATCHES" ]; then | ||
| echo "⚠️ No patches found. Your master branch has no commits ahead of upstream." | ||
| echo "" | ||
| echo "Expected workflow:" | ||
| echo "1. Develop fixes on master branch" | ||
| echo "2. Keep master rebased on upstream: git rebase upstream/master" | ||
| echo "3. Run this workflow to apply your patches to releases" | ||
| exit 1 | ||
| fi | ||
|
|
||
| # Count patches | ||
| PATCH_COUNT=$(echo "$PATCHES" | wc -l | tr -d ' ') | ||
| echo "Found $PATCH_COUNT patches to apply" | ||
| echo "" | ||
|
|
||
| # Apply patches and collect results | ||
| APPLIED_PATCHES="" | ||
| FAILED_PATCHES="" | ||
| SUCCESS_COUNT=0 | ||
|
|
||
| for COMMIT in $PATCHES; do | ||
| COMMIT_MSG=$(git log -1 --pretty=format:"%h: %s" "$COMMIT") | ||
| FULL_MSG=$(git log -1 --pretty=format:"%s" "$COMMIT") | ||
|
|
||
| echo "Applying $COMMIT_MSG..." | ||
| if git cherry-pick "$COMMIT" >/dev/null 2>&1; then | ||
| echo " ✅ Success" | ||
| APPLIED_PATCHES="${APPLIED_PATCHES}\n- ${FULL_MSG}" | ||
| SUCCESS_COUNT=$((SUCCESS_COUNT + 1)) | ||
| else | ||
| echo " ❌ Failed (conflicts with this version)" | ||
| FAILED_PATCHES="${FAILED_PATCHES}\n- ${FULL_MSG}" | ||
| git cherry-pick --abort || true | ||
| fi | ||
| done | ||
|
|
||
| echo "" | ||
| echo "=========================================" | ||
| echo "Applied $SUCCESS_COUNT out of $PATCH_COUNT patches" | ||
|
|
||
| if [ -n "$APPLIED_PATCHES" ]; then | ||
| echo "" | ||
| echo "✅ Successfully applied:" | ||
| echo -e "$APPLIED_PATCHES" | ||
| fi | ||
|
|
||
| if [ -n "$FAILED_PATCHES" ]; then | ||
| echo "" | ||
| echo "⚠️ Failed to apply (incompatible with $VERSION):" | ||
| echo -e "$FAILED_PATCHES" | ||
| fi | ||
|
|
||
| if [ "$SUCCESS_COUNT" -eq 0 ]; then | ||
| echo "" | ||
| echo "❌ No patches could be applied to $VERSION" | ||
| exit 1 | ||
| fi | ||
|
|
||
| # Push branch and tag | ||
| echo "" | ||
| echo "Pushing branch and tag..." | ||
| git push origin "$RELEASE_BRANCH" | ||
|
|
||
| # Create annotated tag with patch list | ||
| TAG_MSG="Release $VERSION with $SUCCESS_COUNT patches | ||
|
|
||
| Patches applied:$APPLIED_PATCHES" | ||
|
|
||
| git tag -a "$RELEASE_TAG" -m "$TAG_MSG" | ||
| git push origin "$RELEASE_TAG" | ||
cursor[bot] marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
|
|
||
| echo "release_tag=$RELEASE_TAG" >> $GITHUB_OUTPUT | ||
| echo "patch_count=$SUCCESS_COUNT" >> $GITHUB_OUTPUT | ||
| echo "patch_list<<EOF" >> $GITHUB_OUTPUT | ||
| echo "$APPLIED_PATCHES" >> $GITHUB_OUTPUT | ||
adityamaru marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| echo "EOF" >> $GITHUB_OUTPUT | ||
|
|
||
| build-binaries: | ||
| needs: create-patched-release | ||
| runs-on: ${{ matrix.os }} | ||
| strategy: | ||
| fail-fast: false | ||
| matrix: | ||
| include: | ||
| - os: ubuntu-latest | ||
| goos: linux | ||
| goarch: amd64 | ||
| suffix: linux-amd64 | ||
| - os: ubuntu-latest | ||
| goos: linux | ||
| goarch: arm64 | ||
| suffix: linux-arm64 | ||
| - os: macos-latest | ||
| goos: darwin | ||
| goarch: amd64 | ||
| suffix: darwin-amd64 | ||
| - os: macos-latest | ||
| goos: darwin | ||
| goarch: arm64 | ||
| suffix: darwin-arm64 | ||
|
|
||
| steps: | ||
| - name: Checkout patched release | ||
| uses: actions/checkout@v4 | ||
| with: | ||
| ref: ${{ needs.create-patched-release.outputs.release_tag }} | ||
|
|
||
| - name: Setup Go | ||
| uses: actions/setup-go@v5 | ||
| with: | ||
| go-version-file: go.mod | ||
|
|
||
| - name: Build binaries | ||
| env: | ||
| GOOS: ${{ matrix.goos }} | ||
| GOARCH: ${{ matrix.goarch }} | ||
| CGO_ENABLED: 0 | ||
| run: | | ||
| echo "Building ${{ matrix.goos }}/${{ matrix.goarch }}..." | ||
|
|
||
| if [ "${{ matrix.goos }}" = "linux" ]; then | ||
| go build -o bin/buildkitd ./cmd/buildkitd | ||
| go build -o bin/buildctl ./cmd/buildctl | ||
| else | ||
| # macOS only has buildctl | ||
| go build -o bin/buildctl ./cmd/buildctl | ||
| fi | ||
cursor[bot] marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
|
||
| # Create tarball | ||
| tar -czf buildkit-${{ needs.create-patched-release.outputs.release_tag }}-${{ matrix.suffix }}.tar.gz -C bin . | ||
|
|
||
| - name: Upload binary artifact | ||
| uses: actions/upload-artifact@v4 | ||
| with: | ||
| name: binary-${{ matrix.suffix }} | ||
| path: buildkit-*.tar.gz | ||
|
|
||
| create-github-release: | ||
| needs: [create-patched-release, build-binaries] | ||
| runs-on: ubuntu-latest | ||
| steps: | ||
| - name: Download all binaries | ||
| uses: actions/download-artifact@v4 | ||
| with: | ||
| pattern: binary-* | ||
| merge-multiple: true | ||
| path: release-files | ||
|
|
||
| - name: Generate checksums | ||
| run: | | ||
| cd release-files | ||
| sha256sum *.tar.gz > SHA256SUMS | ||
| echo "Checksums:" | ||
| cat SHA256SUMS | ||
|
|
||
| - name: Create release | ||
| uses: softprops/action-gh-release@v2 | ||
| with: | ||
| tag_name: ${{ needs.create-patched-release.outputs.release_tag }} | ||
| name: BuildKit ${{ github.event.inputs.upstream_version }} (Patched) | ||
| body: | | ||
| # BuildKit ${{ github.event.inputs.upstream_version }} - Patched Release | ||
|
|
||
| This release includes upstream BuildKit ${{ github.event.inputs.upstream_version }} with ${{ needs.create-patched-release.outputs.patch_count }} production patches automatically applied from our master branch. | ||
|
|
||
| ## Patches Included | ||
| ${{ needs.create-patched-release.outputs.patch_list }} | ||
|
|
||
| ## Installation | ||
|
|
||
| ### Linux (AMD64) | ||
| ```bash | ||
| curl -L https://github.com/${{ github.repository }}/releases/download/${{ needs.create-patched-release.outputs.release_tag }}/buildkit-${{ needs.create-patched-release.outputs.release_tag }}-linux-amd64.tar.gz | tar xz | ||
| sudo mv buildkitd buildctl /usr/local/bin/ | ||
| ``` | ||
|
|
||
| ### Linux (ARM64) | ||
| ```bash | ||
| curl -L https://github.com/${{ github.repository }}/releases/download/${{ needs.create-patched-release.outputs.release_tag }}/buildkit-${{ needs.create-patched-release.outputs.release_tag }}-linux-arm64.tar.gz | tar xz | ||
| sudo mv buildkitd buildctl /usr/local/bin/ | ||
| ``` | ||
|
|
||
| ### macOS (Intel) | ||
| ```bash | ||
| curl -L https://github.com/${{ github.repository }}/releases/download/${{ needs.create-patched-release.outputs.release_tag }}/buildkit-${{ needs.create-patched-release.outputs.release_tag }}-darwin-amd64.tar.gz | tar xz | ||
| sudo mv buildctl /usr/local/bin/ | ||
| ``` | ||
|
|
||
| ### macOS (Apple Silicon) | ||
| ```bash | ||
| curl -L https://github.com/${{ github.repository }}/releases/download/${{ needs.create-patched-release.outputs.release_tag }}/buildkit-${{ needs.create-patched-release.outputs.release_tag }}-darwin-arm64.tar.gz | tar xz | ||
| sudo mv buildctl /usr/local/bin/ | ||
| ``` | ||
|
|
||
| ## Verification | ||
|
|
||
| ```bash | ||
| # Download checksums | ||
| curl -LO https://github.com/${{ github.repository }}/releases/download/${{ needs.create-patched-release.outputs.release_tag }}/SHA256SUMS | ||
|
|
||
| # Verify your download | ||
| sha256sum -c SHA256SUMS 2>/dev/null | grep OK | ||
| ``` | ||
|
|
||
| ## Source | ||
|
|
||
| - **Base version**: [BuildKit ${{ github.event.inputs.upstream_version }}](https://github.com/moby/buildkit/releases/tag/${{ github.event.inputs.upstream_version }}) | ||
| - **Patches from**: [`master`](https://github.com/${{ github.repository }}/tree/master) branch | ||
| - **Release tag**: [`${{ needs.create-patched-release.outputs.release_tag }}`](https://github.com/${{ github.repository }}/tree/${{ needs.create-patched-release.outputs.release_tag }}) | ||
|
|
||
| --- | ||
| *This release was automatically generated from commits in our master branch that are not yet in upstream BuildKit.* | ||
| files: | | ||
| release-files/* | ||
| draft: false | ||
| prerelease: false | ||
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.