Skip to content

Commit b09fbcb

Browse files
authored
cicd: add a github workflow for xfails report script (flashinfer-ai#2273)
<!-- .github/pull_request_template.md --> ## 📌 Description Adds workflow that runs weekly on Mondays (and can be triggered manually) to execute scripts/xfails_tracker.py, which scans the test suite for xfail markers and outputs a comprehensive report to reports/xfails_report.txt. If the report has changed, it automatically creates a pull request to commit the updated report to the repository. ## 🔍 Related Issues cont. testing item from november roadmap ## 🚀 Pull Request Checklist Thank you for contributing to FlashInfer! Before we review your pull request, please make sure the following items are complete. ### ✅ Pre-commit Checks - [x] I have installed `pre-commit` by running `pip install pre-commit` (or used your preferred method). - [x] I have installed the hooks with `pre-commit install`. - [x] I have run the hooks manually with `pre-commit run --all-files` and fixed any reported issues. > If you are unsure about how to set up `pre-commit`, see [the pre-commit documentation](https://pre-commit.com/). ## 🧪 Tests - [x] Tests have been added or updated as needed. - [x] All tests are passing (`unittest`, etc.). ## Reviewer Notes <!-- Optional: anything you'd like reviewers to focus on, concerns, etc. --> <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Chores** * Added an automated weekly report generation that detects changes and opens pull requests to incorporate updates. * Updated automation to remove generated-attribution boilerplate from automated commit/PR messages. <sub>✏️ Tip: You can customize this high-level summary in your review settings.</sub> <!-- end of auto-generated comment: release notes by coderabbit.ai -->
1 parent 6f1624c commit b09fbcb

File tree

2 files changed

+102
-4
lines changed

2 files changed

+102
-4
lines changed

.github/workflows/update-codeowners.yml

100644100755
Lines changed: 0 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -73,10 +73,6 @@ jobs:
7373
chore: update CODEOWNERS based on git history
7474
7575
Auto-generated CODEOWNERS update based on commit activity over the last ${{ env.DAYS_BACK }} days.
76-
77-
🤖 Generated with [Claude Code](https://claude.com/claude-code)
78-
79-
Co-Authored-By: Claude <[email protected]>
8076
branch: auto-update-codeowners
8177
base: main
8278
delete-branch: true
Lines changed: 102 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,102 @@
1+
name: Update XFails Report
2+
3+
on:
4+
schedule:
5+
# Run weekly on Monday at 00:00 UTC
6+
- cron: '0 0 * * 1'
7+
workflow_dispatch: # Allow manual triggering
8+
9+
permissions:
10+
contents: write
11+
pull-requests: write
12+
13+
jobs:
14+
update-xfails-report:
15+
runs-on: ubuntu-latest
16+
timeout-minutes: 30
17+
steps:
18+
- name: Checkout repository
19+
uses: actions/[email protected]
20+
with:
21+
fetch-depth: 1 # Shallow clone is sufficient for scanning test files
22+
token: ${{ secrets.GITHUB_TOKEN }}
23+
24+
- name: Set up Python
25+
uses: actions/setup-python@v5
26+
with:
27+
python-version: '3.11'
28+
29+
- name: Create reports directory
30+
run: mkdir -p reports
31+
32+
- name: Run XFails tracker
33+
run: |
34+
python scripts/xfails_tracker.py > reports/xfails_report.txt 2>&1 || {
35+
echo "Error: XFails tracker script failed"
36+
cat reports/xfails_report.txt
37+
exit 1
38+
}
39+
40+
- name: Check for changes
41+
id: check_changes
42+
run: |
43+
# Check if xfails report file is new (unstaged) or has changes
44+
if git ls-files --error-unmatch reports/xfails_report.txt >/dev/null 2>&1; then
45+
# File is tracked, check for changes
46+
if git diff --quiet reports/xfails_report.txt; then
47+
echo "changed=false" >> $GITHUB_OUTPUT
48+
echo "No changes detected in xfails report"
49+
else
50+
echo "changed=true" >> $GITHUB_OUTPUT
51+
echo "Changes detected in xfails report"
52+
fi
53+
else
54+
# File is untracked (newly created)
55+
echo "changed=true" >> $GITHUB_OUTPUT
56+
echo "XFails report file is new"
57+
fi
58+
59+
- name: Create Pull Request
60+
if: steps.check_changes.outputs.changed == 'true'
61+
uses: peter-evans/create-pull-request@v7
62+
with:
63+
token: ${{ secrets.FLASHINFER_BOT_TOKEN }}
64+
committer: flashinfer-bot <[email protected]>
65+
author: flashinfer-bot <[email protected]>
66+
commit-message: |
67+
chore: update xfails report
68+
69+
Auto-generated xfails report based on current test suite markers.
70+
branch: auto-update-xfails-report
71+
base: main
72+
delete-branch: true
73+
title: 'chore: Update XFails Report'
74+
body: |
75+
## Summary
76+
77+
This PR updates the xfails report based on the current test suite.
78+
79+
## Changes
80+
81+
- Updated `reports/xfails_report.txt` with current xfail markers from the test suite
82+
83+
## How to Review
84+
85+
1. Review the changes to `reports/xfails_report.txt`
86+
2. Check if the number of xfails has increased or decreased
87+
3. Review the reasons for any new xfails
88+
4. Consider addressing xfails that may no longer be needed
89+
90+
## Notes
91+
92+
- This is an automated PR generated weekly
93+
- The report shows all `pytest.mark.xfail` markers in the test suite
94+
- Grouped by reason for easy analysis
95+
96+
---
97+
98+
🤖 This PR was automatically generated by the [update-xfails-report workflow](.github/workflows/update-xfails-report.yml)
99+
labels: |
100+
automated
101+
maintenance
102+
testing

0 commit comments

Comments
 (0)