Skip to content

Commit ecb43e5

Browse files
joseph-isaacsjoseph-isaacs
authored
feat: Add Stage 2 automated fix workflow for fuzzer crashes (#5319)
1 parent 5ee3983 commit ecb43e5

File tree

1 file changed

+330
-0
lines changed

1 file changed

+330
-0
lines changed

‎.github/workflows/fuzzer-fix-automation.yml‎

Lines changed: 330 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,330 @@
1+
name: Fuzzer Fix Automation
2+
3+
on:
4+
issues:
5+
types: [opened, labeled]
6+
7+
jobs:
8+
attempt-fix:
9+
name: "Attempt to Fix Fuzzer Crash"
10+
# Only run when:
11+
# 1. Issue is opened with 'fuzzer' label, OR
12+
# 2. 'fuzzer' label is added to existing issue
13+
if: |
14+
(github.event.action == 'opened' && contains(github.event.issue.labels.*.name, 'fuzzer')) ||
15+
(github.event.action == 'labeled' && github.event.label.name == 'fuzzer')
16+
17+
runs-on: ubuntu-latest
18+
timeout-minutes: 60
19+
20+
permissions:
21+
contents: write
22+
pull-requests: write
23+
issues: write
24+
id-token: write
25+
26+
steps:
27+
- name: Checkout repository
28+
uses: actions/checkout@v5
29+
30+
- name: Setup Rust
31+
uses: ./.github/actions/setup-rust
32+
with:
33+
repo-token: ${{ secrets.GITHUB_TOKEN }}
34+
toolchain: nightly
35+
36+
- name: Install llvm
37+
uses: aminya/setup-cpp@v1
38+
with:
39+
compiler: llvm
40+
41+
- name: Install cargo fuzz
42+
run: cargo install --locked cargo-fuzz
43+
44+
- name: Extract crash details from issue
45+
id: extract
46+
run: |
47+
# Extract target name from issue body
48+
TARGET=$(echo "${{ github.event.issue.body }}" | grep -oP '(?<=\*\*Target\*\*: `)[^`]+' || echo "file_io")
49+
echo "target=$TARGET" >> $GITHUB_OUTPUT
50+
51+
# Extract crash file name
52+
CRASH_FILE=$(echo "${{ github.event.issue.body }}" | grep -oP '(?<=\*\*Crash File\*\*: `)[^`]+' || echo "")
53+
echo "crash_file=$CRASH_FILE" >> $GITHUB_OUTPUT
54+
55+
# Extract artifact URL
56+
ARTIFACT_URL=$(echo "${{ github.event.issue.body }}" | grep -oP 'https://[^\s]+/artifacts/[0-9]+' | head -1 || echo "")
57+
echo "artifact_url=$ARTIFACT_URL" >> $GITHUB_OUTPUT
58+
59+
echo "Extracted: target=$TARGET, crash_file=$CRASH_FILE"
60+
61+
- name: Attempt to fix crash with Claude
62+
env:
63+
ISSUE_NUMBER: ${{ github.event.issue.number }}
64+
ISSUE_TITLE: ${{ github.event.issue.title }}
65+
ISSUE_BODY: ${{ github.event.issue.body }}
66+
TARGET: ${{ steps.extract.outputs.target }}
67+
CRASH_FILE: ${{ steps.extract.outputs.crash_file }}
68+
ARTIFACT_URL: ${{ steps.extract.outputs.artifact_url }}
69+
uses: anthropics/claude-code-action@v1
70+
with:
71+
claude_code_oauth_token: ${{ secrets.CLAUDE_CODE_OAUTH_TOKEN }}
72+
github_token: ${{ secrets.GITHUB_TOKEN }}
73+
show_full_output: true
74+
prompt: |
75+
# Fuzzer Crash Fix Automation
76+
77+
You are analyzing a fuzzer-detected crash to attempt an automated fix. This issue was created by our fuzzing automation.
78+
79+
## Your Mission
80+
81+
1. **Download and reproduce the crash**
82+
2. **Analyze the root cause** using the stack trace and source code
83+
3. **Create a fix** if the issue is straightforward
84+
4. **Write regression tests** that would fail without your fix
85+
5. **Verify the fix** by running the fuzzer and tests
86+
6. **Post your findings** as a comment on the issue
87+
88+
## Issue Details
89+
90+
- **Issue**: #${{ env.ISSUE_NUMBER }}
91+
- **Title**: ${{ env.ISSUE_TITLE }}
92+
- **Target**: ${{ env.TARGET }}
93+
- **Crash File**: ${{ env.CRASH_FILE }}
94+
95+
## Step 1: Download the Crash Artifact
96+
97+
The issue body contains an artifact URL. Extract it and download the crash file:
98+
99+
```bash
100+
# The issue body should contain the crash artifact URL
101+
# Download it using gh or curl
102+
# Example:
103+
gh run download RUN_ID --name ${{ env.TARGET }}-fuzzing-crash-artifacts
104+
```
105+
106+
Look for the artifact URL in the issue body. If you can't download it automatically, note this in your comment and provide manual instructions.
107+
108+
## Step 2: Reproduce the Crash
109+
110+
Once you have the crash file, reproduce it:
111+
112+
```bash
113+
cargo +nightly fuzz run --sanitizer=none ${{ env.TARGET }} <path-to-crash-file>
114+
```
115+
116+
Capture the output and verify you can reproduce the panic/crash.
117+
118+
## Step 3: Analyze the Root Cause
119+
120+
1. Read the **Stack Trace** from the issue body
121+
2. Identify the **Crash Location** (file and line)
122+
3. Read the source code at that location
123+
4. Understand what input caused the crash (check the Debug Output in the issue)
124+
5. Determine the root cause:
125+
- Bounds check missing?
126+
- Invalid assumption?
127+
- Edge case not handled?
128+
- Integer overflow?
129+
- etc.
130+
131+
## Step 4: Assess Fixability
132+
133+
Determine if this is something you can fix:
134+
135+
**CAN FIX** (straightforward):
136+
- Missing bounds check
137+
- Missing validation
138+
- Edge case handling
139+
- Simple panic that should be an error
140+
- Off-by-one error
141+
142+
**CANNOT FIX** (needs human):
143+
- Architectural issues
144+
- Complex logic errors
145+
- Requires domain knowledge
146+
- Multiple files/modules affected
147+
- Unclear requirements
148+
149+
## Step 5: If Fixable - Create the Fix
150+
151+
1. **Modify the source code** to fix the issue
152+
2. **Add validation** or bounds checks as needed
153+
3. **Handle the edge case** properly
154+
4. **Follow the project's code style** (see CLAUDE.md)
155+
5. **Keep changes minimal** - only fix the specific issue
156+
157+
## Step 6: Write Regression Tests
158+
159+
Create tests that:
160+
1. **Would fail before your fix** (reproduce the crash)
161+
2. **Pass after your fix** (verify it's solved)
162+
3. **Use the crash file as input** (the actual fuzzer input that triggered it)
163+
4. **Are placed in the right location** (near the code being tested)
164+
165+
Example structure:
166+
```rust
167+
#[test]
168+
fn test_fuzzer_crash_issue_${{ env.ISSUE_NUMBER }}() {
169+
// This test reproduces the crash from issue #${{ env.ISSUE_NUMBER }}
170+
// The fuzzer discovered this input that caused a panic
171+
172+
let input = /* minimal reproducing input */;
173+
174+
// This should not panic
175+
let result = function_that_crashed(input);
176+
177+
// Assert the expected behavior
178+
assert!(result.is_ok() || result.is_err()); // depending on expected outcome
179+
}
180+
```
181+
182+
## Step 7: Verify Your Fix
183+
184+
1. Run the new regression test:
185+
```bash
186+
cargo test test_fuzzer_crash_issue_${{ env.ISSUE_NUMBER }}
187+
```
188+
189+
2. Run the fuzzer with the crash file:
190+
```bash
191+
cargo +nightly fuzz run --sanitizer=none ${{ env.TARGET }} <path-to-crash-file> -- -runs=100
192+
```
193+
194+
3. Run related tests:
195+
```bash
196+
cargo test --package <affected-package>
197+
```
198+
199+
4. Check for lint issues:
200+
```bash
201+
cargo clippy --all-targets --all-features
202+
```
203+
204+
5. Format code:
205+
```bash
206+
cargo +nightly fmt --all
207+
```
208+
209+
## Step 8: Post Your Analysis
210+
211+
Comment on issue #${{ env.ISSUE_NUMBER }} with your findings:
212+
213+
### If You Created a Fix:
214+
215+
```markdown
216+
## 🤖 Automated Fix Attempt
217+
218+
I've analyzed this crash and created a potential fix.
219+
220+
### Root Cause Analysis
221+
222+
[Explain what caused the crash in 2-3 sentences]
223+
224+
### The Fix
225+
226+
**Modified files:**
227+
- `path/to/file.rs` - [brief description of changes]
228+
229+
**Key changes:**
230+
- [Bullet point summary of what you changed]
231+
232+
### Regression Tests
233+
234+
Created test(s):
235+
- `test_fuzzer_crash_issue_${{ env.ISSUE_NUMBER }}()` in `path/to/test.rs`
236+
237+
**Test verification:**
238+
```
239+
[Output from running the test]
240+
```
241+
242+
### Verification
243+
244+
✅ Regression test passes
245+
✅ Fuzzer no longer crashes on the input
246+
✅ Related tests pass
247+
✅ Clippy checks pass
248+
✅ Code formatted
249+
250+
### Next Steps
251+
252+
Please review the fix and:
253+
1. Verify the logic is correct
254+
2. Check if additional edge cases should be handled
255+
3. Consider if this fix should be applied elsewhere
256+
4. Merge if satisfactory or provide feedback
257+
258+
**Note**: This is an automated fix attempt. Please review carefully before merging.
259+
```
260+
261+
Use the `gh issue comment` command to post this.
262+
263+
### If You Cannot Fix It:
264+
265+
```markdown
266+
## 🤖 Automated Analysis
267+
268+
I've analyzed this crash but cannot create an automated fix.
269+
270+
### Root Cause Analysis
271+
272+
[Explain what caused the crash]
273+
274+
### Why I Can't Fix It
275+
276+
[Explain why this needs human intervention - e.g., architectural issue, requires domain knowledge, etc.]
277+
278+
### Suggested Approach
279+
280+
[Provide suggestions for how a human might fix this:
281+
- What code needs to change
282+
- What validation might be needed
283+
- Potential approaches to consider]
284+
285+
### Reproduction Verified
286+
287+
[If you were able to reproduce it, confirm here]
288+
289+
**Note**: This issue requires human analysis and fixing.
290+
```
291+
292+
## Important Guidelines
293+
294+
- **Be conservative**: Only create fixes for straightforward issues
295+
- **Minimal changes**: Don't refactor, just fix the specific bug
296+
- **Test thoroughly**: Your regression tests must actually catch the bug
297+
- **Follow CLAUDE.md**: Use project conventions
298+
- **Comment your reasoning**: Help reviewers understand the fix
299+
- **Don't commit yet**: Post your analysis first for review
300+
301+
## Available Tools
302+
303+
You have access to:
304+
- Full repository source code (Read/Write/Edit)
305+
- Cargo toolchain (build, test, clippy, fmt, fuzz)
306+
- Git operations (for creating branches if requested)
307+
- GitHub CLI (for commenting on issues)
308+
309+
## Issue Body
310+
311+
Here's the full issue body for reference:
312+
313+
```
314+
${{ env.ISSUE_BODY }}
315+
```
316+
317+
## Start Here
318+
319+
Begin by reading the issue body carefully to extract:
320+
1. The stack trace
321+
2. The crash location
322+
3. The error message
323+
4. The artifact download URL
324+
5. Any debug output
325+
326+
Then proceed with your analysis. Good luck! 🚀
327+
claude_args: |
328+
--model claude-opus-4-20250514
329+
--max-turns 40
330+
--allowedTools "Read,Write,Edit,Glob,Grep,Bash(cargo:*),Bash(gh issue comment:*),Bash(gh run download:*),Bash(curl:*),Bash(find:*),Bash(ls:*),Bash(cat:*),Bash(RUST_BACKTRACE=*:*)"

0 commit comments

Comments
 (0)