-
Notifications
You must be signed in to change notification settings - Fork 6
Open
Description
Problem
Some tests in our CI are expensive (long-running, resource-heavy, or
require special secrets).
Running them for every PR is inefficient and unnecessary.
Goal
Enable opt‑in expensive test runs by commenting a slash-command on a
PR, for example:
/run-expensive-tests
This should:
- Trigger a GitHub Actions workflow via
issue_comment - Detect that the comment is on a PR
- Verify the comment text starts with
/run-expensive-tests - Check out the PR merge ref
- Run the expensive test suite only on demand
Proposed Workflow (GitHub Actions)
Create .github/workflows/expensive-tests-on-comment.yml:
name: Expensive Tests (On Comment)
on:
issue_comment:
types: [created]
jobs:
expensive-tests:
# Must be a PR comment AND contain the trigger command
if: >
github.event.issue.pull_request &&
startsWith(github.event.comment.body, '/run-expensive-tests')
runs-on: ubuntu-latest
steps:
- name: Extract PR number
run: echo "PR_NUMBER=${{ github.event.issue.number }}" >> $GITHUB_ENV
- name: Checkout PR merge commit
uses: actions/checkout@v4
with:
ref: refs/pull/${{ env.PR_NUMBER }}/merge
- name: Run expensive tests
run: ./scripts/run_expensive_tests.shNotes / Best Practices
issue_commentdoes fire for PR comments (PRs are issues
internally)github.event.issue.pull_requestfilters out non‑PR comments- This avoids running the expensive suite on every PR automatically
- For repos needing extra safety:
-
restrict execution to team members:
if: github.event.comment.author_association == 'MEMBER'
-
- If the PR is from a fork:
- secrets will not be available (GitHub restriction)
- but compute-only tests still run fine
Acceptance Criteria
- Team members can trigger expensive CI by commenting
/run-expensive-tests - The workflow runs only when explicitly invoked
- Regular fast CI remains unchanged
- Documentation added to
CONTRIBUTING.md
Next Steps
- Add the workflow file
- Add label/documentation for contributors
- (Optional) Add bot reaction or status comment to PR
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
No labels