-
-
Notifications
You must be signed in to change notification settings - Fork 638
Add PR comment command to trigger full CI suite and fix workflow issues #1974
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from 3 commits
Commits
Show all changes
6 commits
Select commit
Hold shift + click to select a range
c7fdb46
Add PR comment command to trigger full CI suite
justin808 21e9f1a
Fix CI comment trigger security and reliability issues
justin808 67ea4b4
Fix actionlint errors in workflow matrix conditionals
justin808 8ef43bc
Rename command from /run-full-ci to /run-skipped-ci
justin808 a23eff8
Fix actionlint errors with proper matrix filtering approach
justin808 1f5b84c
Fix workflow issues and improve CI reliability
justin808 File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,114 @@ | ||
| # GitHub Actions CI/CD Configuration | ||
|
|
||
| This directory contains GitHub Actions workflows for continuous integration and deployment. | ||
|
|
||
| ## PR Comment Commands | ||
|
|
||
| ### `/run-full-ci` - Run Full CI Suite | ||
|
|
||
| When you open a PR, CI automatically runs a subset of tests for faster feedback (latest Ruby/Node versions only). To run the **complete CI suite** including all dependency combinations, add a comment to your PR: | ||
|
|
||
| ``` | ||
| /run-full-ci | ||
| ``` | ||
|
|
||
| This command will trigger: | ||
|
|
||
| - ✅ Main test suite with both latest and minimum supported versions | ||
| - ✅ All example app generator tests | ||
| - ✅ React on Rails Pro integration tests | ||
| - ✅ React on Rails Pro package tests | ||
|
|
||
| The bot will: | ||
|
|
||
| 1. React with a 🚀 to your comment | ||
| 2. Post a confirmation message with links to the triggered workflows | ||
| 3. Start all CI jobs on your PR branch | ||
|
|
||
| ### Why This Exists | ||
|
|
||
| By default, PRs run a subset of CI jobs to provide fast feedback: | ||
|
|
||
| - Only latest dependency versions (Ruby 3.4, Node 22) | ||
| - Skips example generator tests | ||
| - Skips some Pro package tests | ||
|
|
||
| This is intentional to keep PR feedback loops fast. However, before merging, you should verify compatibility across all supported versions. The `/run-full-ci` command makes this easy without waiting for the PR to be merged to master. | ||
|
|
||
| ### Security & Access Control | ||
|
|
||
| **Only repository collaborators with write access can trigger full CI runs.** This prevents: | ||
|
|
||
| - Resource abuse from external contributors | ||
| - Unauthorized access to Pro package tests | ||
| - Potential DoS attacks via repeated CI runs | ||
|
|
||
| If an unauthorized user attempts to use `/run-full-ci`, they'll receive a message explaining the restriction. | ||
|
|
||
| ### Concurrency Protection | ||
|
|
||
| Multiple `/run-full-ci` comments on the same PR will cancel in-progress runs to prevent resource waste and duplicate results. | ||
|
|
||
| ## Testing Comment-Triggered Workflows | ||
|
|
||
| **Important**: Comment-triggered workflows (`issue_comment` event) only execute from the **default branch** (master). This creates a chicken-and-egg problem when developing workflow changes. | ||
|
|
||
| ### Recommended Testing Approach | ||
|
|
||
| 1. **Develop the workflow**: Create/modify the workflow in your feature branch | ||
| 2. **Test locally**: Validate YAML syntax and logic as much as possible | ||
| 3. **Merge to master**: The workflow must be in master to be triggered by comments | ||
| 4. **Test on a PR**: Create a test PR and use the comment command to verify | ||
|
|
||
| ### Why This Limitation Exists | ||
|
|
||
| GitHub Actions workflows triggered by `issue_comment` events always use the workflow definition from the default branch, not the PR branch. This is a security feature to prevent malicious actors from modifying workflows through PRs. | ||
|
|
||
| For more details, see [GitHub's documentation on issue_comment events](https://docs.github.com/en/actions/using-workflows/events-that-trigger-workflows#issue_comment). | ||
|
|
||
| ## Available Workflows | ||
|
|
||
| ### CI Workflows (Triggered on Push/PR) | ||
|
|
||
| - **`main.yml`** - Main test suite (dummy app integration tests) | ||
| - **`lint-js-and-ruby.yml`** - Linting for JavaScript and Ruby code | ||
| - **`package-js-tests.yml`** - JavaScript unit tests for the package | ||
| - **`rspec-package-specs.yml`** - RSpec tests for the Ruby package | ||
| - **`examples.yml`** - Generator tests for example apps | ||
| - **`playwright.yml`** - Playwright E2E tests | ||
| - **`pro-integration-tests.yml`** - Pro package integration tests | ||
| - **`pro-package-tests.yml`** - Pro package unit tests | ||
| - **`pro-lint.yml`** - Pro package linting | ||
|
|
||
| ### Utility Workflows | ||
|
|
||
| - **`run-full-ci.yml`** - Triggered by `/run-full-ci` comment on PRs | ||
| - **`pr-welcome-comment.yml`** - Auto-comments on new PRs with helpful info | ||
| - **`detect-changes.yml`** - Detects which parts of the codebase changed | ||
|
|
||
| ### Code Review Workflows | ||
|
|
||
| - **`claude.yml`** - Claude AI code review | ||
| - **`claude-code-review.yml`** - Additional Claude code review checks | ||
|
|
||
| ### Other Workflows | ||
|
|
||
| - **`check-markdown-links.yml`** - Validates markdown links | ||
|
|
||
| ## Workflow Permissions | ||
|
|
||
| Most workflows use minimal permissions. The comment-triggered workflows require: | ||
|
|
||
| - `contents: read` - To read the repository code | ||
| - `pull-requests: write` - To post comments and reactions | ||
| - `actions: write` - To trigger other workflows | ||
|
|
||
| ## Conditional Execution | ||
|
|
||
| Many workflows use change detection to skip unnecessary jobs: | ||
|
|
||
| - Runs all jobs on pushes to `master` | ||
| - Runs only relevant jobs on PRs based on changed files | ||
| - Can be overridden with `workflow_dispatch` or `/run-full-ci` command | ||
|
|
||
| See `script/ci-changes-detector` for the change detection logic. | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -38,15 +38,19 @@ | |
|
|
||
| examples: | ||
| needs: detect-changes | ||
| if: github.ref == 'refs/heads/master' || needs.detect-changes.outputs.run_generators == 'true' | ||
| # For regular PRs: only run latest versions | ||
| # For master/workflow_dispatch: run all versions | ||
| if: | | ||
| (github.ref == 'refs/heads/master' || github.event_name == 'workflow_dispatch' || needs.detect-changes.outputs.run_generators == 'true') && | ||
|
Check failure on line 44 in .github/workflows/examples.yml
|
||
| (matrix.dependency-level != 'minimum' || github.ref == 'refs/heads/master' || github.event_name == 'workflow_dispatch') | ||
| strategy: | ||
| fail-fast: false | ||
| matrix: | ||
| include: | ||
| # Always run: Latest versions (fast feedback on PRs) | ||
| - ruby-version: '3.4' | ||
| dependency-level: 'latest' | ||
| # Master only: Minimum supported versions (full coverage) | ||
| # Master and workflow_dispatch: Minimum supported versions (full coverage) | ||
| - ruby-version: '3.2' | ||
| dependency-level: 'minimum' | ||
| env: | ||
|
|
||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -38,17 +38,20 @@ | |
|
|
||
| build-dummy-app-webpack-test-bundles: | ||
| needs: detect-changes | ||
| # Run on master OR when tests needed on PR (but skip minimum deps on PR) | ||
| # Run on master, workflow_dispatch, OR when tests needed on PR | ||
| # For regular PRs: only run latest versions | ||
| # For master/workflow_dispatch: run all versions | ||
| if: | | ||
| (github.ref == 'refs/heads/master' || needs.detect-changes.outputs.run_dummy_tests == 'true') | ||
| (github.ref == 'refs/heads/master' || github.event_name == 'workflow_dispatch' || needs.detect-changes.outputs.run_dummy_tests == 'true') && | ||
|
Check failure on line 45 in .github/workflows/main.yml
|
||
| (matrix.dependency-level != 'minimum' || github.ref == 'refs/heads/master' || github.event_name == 'workflow_dispatch') | ||
| strategy: | ||
| matrix: | ||
| include: | ||
| # Always run: Latest versions (fast feedback on PRs) | ||
| - ruby-version: '3.4' | ||
| node-version: '22' | ||
| dependency-level: 'latest' | ||
| # Master only: Minimum supported versions (full coverage) | ||
| # Master and workflow_dispatch: Minimum supported versions (full coverage) | ||
| - ruby-version: '3.2' | ||
| node-version: '20' | ||
| dependency-level: 'minimum' | ||
|
|
@@ -122,9 +125,12 @@ | |
|
|
||
| dummy-app-integration-tests: | ||
| needs: [detect-changes, build-dummy-app-webpack-test-bundles] | ||
| # Run on master OR when tests needed on PR (but skip minimum deps on PR) | ||
| # Run on master, workflow_dispatch, OR when tests needed on PR | ||
| # For regular PRs: only run latest versions | ||
| # For master/workflow_dispatch: run all versions | ||
| if: | | ||
| (github.ref == 'refs/heads/master' || needs.detect-changes.outputs.run_dummy_tests == 'true') | ||
| (github.ref == 'refs/heads/master' || github.event_name == 'workflow_dispatch' || needs.detect-changes.outputs.run_dummy_tests == 'true') && | ||
|
Check failure on line 132 in .github/workflows/main.yml
|
||
| (matrix.dependency-level != 'minimum' || github.ref == 'refs/heads/master' || github.event_name == 'workflow_dispatch') | ||
| strategy: | ||
| fail-fast: false | ||
| matrix: | ||
|
|
@@ -133,7 +139,7 @@ | |
| - ruby-version: '3.4' | ||
| node-version: '22' | ||
| dependency-level: 'latest' | ||
| # Master only: Minimum supported versions (full coverage) | ||
| # Master and workflow_dispatch: Minimum supported versions (full coverage) | ||
| - ruby-version: '3.2' | ||
| node-version: '20' | ||
| dependency-level: 'minimum' | ||
|
|
||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,38 @@ | ||
| name: PR Welcome Comment | ||
|
|
||
| on: | ||
| pull_request: | ||
| types: [opened] | ||
|
|
||
| jobs: | ||
| welcome: | ||
| # Skip for bots (dependabot, renovate, etc.) | ||
| if: github.event.pull_request.user.type != 'Bot' | ||
| runs-on: ubuntu-22.04 | ||
| permissions: | ||
| pull-requests: write | ||
| steps: | ||
| - name: Add welcome comment with CI command info | ||
| uses: peter-evans/create-or-update-comment@v4 | ||
| with: | ||
| issue-number: ${{ github.event.pull_request.number }} | ||
| body: | | ||
| 👋 Thanks for opening this PR! | ||
|
|
||
| ### 🚀 Running Full CI Suite | ||
|
|
||
| By default, PRs run a subset of CI jobs for faster feedback (latest Ruby/Node versions only). | ||
|
|
||
| To run the **complete CI suite** including all dependency combinations and skipped jobs, comment: | ||
|
|
||
| ``` | ||
| /run-full-ci | ||
| ``` | ||
|
|
||
| This will trigger: | ||
| - ✅ Minimum supported versions (Ruby 3.2, Node 20) | ||
| - ✅ All example app tests | ||
| - ✅ Pro package integration tests | ||
| - ✅ All test matrices | ||
|
|
||
| The full CI suite takes longer but ensures compatibility across all supported versions before merging. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.