Skip to content

Commit dbda77c

Browse files
chore: add automated crash reporting for array_ops fuzzing (#5317)
## Summary Add Claude-powered crash analysis and GitHub issue creation for the `array_ops` fuzzing target, matching the existing `file_io` fuzzing setup. ## Changes - **Extract shared crash reporting logic**: Created reusable composite action at `.github/actions/report-fuzz-failure/action.yml` - **Add crash detection to `ops_fuzz` job**: - Added outputs for `crashes_found`, `first_crash_name`, and `artifact_url` - Added "Check for crashes" step to detect crash artifacts - Added log archiving for crash analysis - **Add `report-ops-fuzz-failures` job**: New job using shared action for `array_ops` target - **Refactor `report-io-fuzz-failures`**: Updated to use shared action, reducing duplication ## Benefits - Reduces code duplication (~180 lines removed from workflow) - All crash reporting logic maintained in single location - Easy to add fuzzing crash reporting for additional targets - Both `file_io` and `array_ops` now have automated issue creation on crashes ## Test Plan - [ ] Verify workflow syntax is valid - [ ] Confirm both fuzz jobs can run successfully - [ ] Test crash reporting with a known crash (if available) 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Signed-off-by: Joe Isaacs <[email protected]>
1 parent 6043a57 commit dbda77c

File tree

2 files changed

+331
-185
lines changed

2 files changed

+331
-185
lines changed

.github/workflows/fuzz.yml

Lines changed: 74 additions & 185 deletions
Original file line numberDiff line numberDiff line change
@@ -107,194 +107,23 @@ jobs:
107107
name: "Report IO Fuzz Failures"
108108
needs: io_fuzz
109109
if: always() && needs.io_fuzz.outputs.crashes_found == 'true'
110-
runs-on: ubuntu-latest
111110
permissions:
112111
issues: write
113112
contents: read
114113
id-token: write
115114
pull-requests: read
116-
steps:
117-
- name: Checkout repository
118-
uses: actions/checkout@v5
119-
120-
- name: Download fuzzer logs
121-
uses: actions/download-artifact@v4
122-
with:
123-
name: io-fuzzing-logs
124-
path: ./logs
125-
126-
- name: Analyze and report crash with Claude
127-
env:
128-
CRASH_FILE: ${{ needs.io_fuzz.outputs.first_crash_name }}
129-
ARTIFACT_URL: ${{ needs.io_fuzz.outputs.artifact_url }}
130-
BRANCH: ${{ github.ref_name }}
131-
COMMIT: ${{ github.sha }}
132-
uses: anthropics/claude-code-action@v1
133-
with:
134-
claude_code_oauth_token: ${{ secrets.CLAUDE_CODE_OAUTH_TOKEN }}
135-
github_token: ${{ secrets.GITHUB_TOKEN }}
136-
show_full_output: true
137-
prompt: |
138-
# Fuzzer Crash Analysis and Reporting
139-
140-
A fuzzing run for the `file_io` target has detected a crash. Analyze it and report by creating or updating a GitHub issue.
141-
142-
## Step 1: Analyze the Crash
143-
144-
1. Read the fuzzer log: `logs/fuzz_output.log`
145-
2. Extract:
146-
- Stack trace (lines with `#0`, `#1`, etc.)
147-
- Error message ("panicked at" or "ERROR:")
148-
- Crash location (top user code frame, not std/core/libfuzzer)
149-
- Debug output (look for "Output of `std::fmt::Debug`:" section before the crash)
150-
3. Read the source code at the crash location to understand root cause
151-
152-
## Step 2: Check for Duplicates
153-
154-
1. List existing fuzzer issues:
155-
```bash
156-
gh issue list --repo ${{ github.repository }} --label fuzzer --state open --json number,title,body --limit 50
157-
```
158-
159-
2. For each existing issue, compare:
160-
- **Crash location**: Same file + function? (line numbers can differ)
161-
- **Error pattern**: Same error after normalizing values?
162-
- "index 5 out of bounds" = "index 12 out of bounds" (SAME)
163-
- "len is 100" = "len is 5" (SAME)
164-
- Read source code if needed to verify same root cause
165-
166-
3. Determine duplication level:
167-
- **EXACT DUPLICATE**: Same crash location + same error pattern → Add reaction
168-
- **SIMILAR**: Same general area but unclear → Add comment
169-
- **NEW BUG**: Different location or different error type → Create new issue
170-
171-
## Step 3: Take Action
172-
173-
### If EXACT DUPLICATE (high confidence):
174-
Update or create a tracking comment to count occurrences:
175-
176-
1. First, check if there's already a tracking comment (look for a comment starting with "<!-- occurrences: ")
177-
2. If found, extract the count, increment it, and edit the comment
178-
3. If not found, create a new comment
179-
180-
Comment format:
181-
```markdown
182-
<!-- occurrences: N -->
183-
**Crash seen N time(s)**
184-
185-
Latest occurrence:
186-
- Crash file: $CRASH_FILE
187-
- Artifact: $ARTIFACT_URL
188-
- Branch: $BRANCH
189-
- Commit: $COMMIT
190-
```
191-
192-
Use:
193-
- `gh api repos/${{ github.repository }}/issues/ISSUE_NUM/comments` to list comments
194-
- `gh api repos/${{ github.repository }}/issues/comments/COMMENT_ID -X PATCH` to update
195-
- `gh issue comment ISSUE_NUM` to create new
196-
197-
### If SIMILAR (medium confidence):
198-
Comment on the existing issue with analysis:
199-
```bash
200-
gh issue comment ISSUE_NUM --repo ${{ github.repository }} --body "..."
201-
```
202-
203-
Include in comment:
204-
- Note that another crash was detected
205-
- Your confidence level and reasoning
206-
- Key differences if any
207-
- Crash file: $CRASH_FILE
208-
- Workflow: $WORKFLOW_RUN
209-
210-
### If NEW BUG (not a duplicate):
211-
Create a new issue with `gh issue create`:
212-
```bash
213-
gh issue create --repo ${{ github.repository }} \
214-
--title "Fuzzing Crash: [brief description]" \
215-
--label "bug,fuzzer" \
216-
--body "..."
217-
```
218-
219-
Issue body must include:
220-
```markdown
221-
## Fuzzing Crash Report
222-
223-
### Analysis
224-
225-
**Crash Location**: `file.rs:function_name`
226-
227-
**Error Message**:
228-
```
229-
[error message]
230-
```
231-
232-
**Stack Trace**:
233-
```
234-
[top 5-7 frames - keep in code block to prevent markdown rendering issues]
235-
```
236-
237-
Note: Keep stack traces in code blocks to prevent `#0`, `#1` from being interpreted as markdown headers.
238-
239-
**Root Cause**: [Your analysis]
240-
241-
<details>
242-
<summary>Debug Output</summary>
243-
244-
```
245-
[Include the complete "Output of std::fmt::Debug:" section from the fuzzer log]
246-
```
247-
</details>
248-
249-
### Summary
250-
251-
- **Target**: `file_io`
252-
- **Crash File**: `$CRASH_FILE`
253-
- **Branch**: $BRANCH
254-
- **Commit**: $COMMIT
255-
- **Crash Artifact**: $ARTIFACT_URL
256-
257-
### Reproduction
258-
259-
1. Download the crash artifact:
260-
- **Direct download**: $ARTIFACT_URL
261-
- Or find `io-fuzzing-crash-artifacts` at: $WORKFLOW_RUN
262-
- Extract the zip file
263-
264-
2. Reproduce locally:
265-
```bash
266-
# The artifact contains file_io/$CRASH_FILE
267-
cargo +nightly fuzz run --sanitizer=none file_io file_io/$CRASH_FILE
268-
```
269-
270-
3. Get full backtrace:
271-
```bash
272-
RUST_BACKTRACE=full cargo +nightly fuzz run --sanitizer=none file_io file_io/$CRASH_FILE
273-
```
274-
275-
---
276-
*Auto-created by fuzzing workflow with Claude analysis*
277-
```
278-
279-
## Important Guidelines
280-
281-
- Be conservative: when unsure, prefer creating a new issue over marking as duplicate
282-
- Focus on ROOT CAUSE, not specific values in error messages
283-
- You have full repo access - read source code to understand crashes
284-
- Only use +1 reaction for truly identical crashes
285-
286-
## Environment Variables
287-
288-
- CRASH_FILE: $CRASH_FILE
289-
- ARTIFACT_URL: $ARTIFACT_URL (direct link to crash artifact)
290-
- BRANCH: $BRANCH
291-
- COMMIT: $COMMIT
292-
293-
Start by reading `logs/fuzz_output.log`.
294-
claude_args: |
295-
--model claude-sonnet-4-5-20250929
296-
--max-turns 25
297-
--allowedTools "Read,Write,Bash(gh issue list:*),Bash(gh issue view:*),Bash(gh issue create:*),Bash(gh issue comment:*),Bash(gh api:*),Bash(echo:*),Bash(cat:*),Bash(jq:*),Bash(grep:*),Bash(cargo +nightly fuzz run:*),Bash(RUST_BACKTRACE=* cargo +nightly fuzz run:*)"
115+
uses: ./.github/workflows/report-fuzz-crash.yml
116+
with:
117+
fuzz_target: file_io
118+
crash_file: ${{ needs.io_fuzz.outputs.first_crash_name }}
119+
artifact_url: ${{ needs.io_fuzz.outputs.artifact_url }}
120+
artifact_name: io-fuzzing-crash-artifacts
121+
logs_artifact_name: io-fuzzing-logs
122+
branch: ${{ github.ref_name }}
123+
commit: ${{ github.sha }}
124+
secrets:
125+
claude_code_oauth_token: ${{ secrets.CLAUDE_CODE_OAUTH_TOKEN }}
126+
gh_token: ${{ secrets.GITHUB_TOKEN }}
298127

299128

300129
ops_fuzz:
@@ -307,6 +136,10 @@ jobs:
307136
- disk=large
308137
- extras=s3-cache
309138
- tag=ops-fuzz
139+
outputs:
140+
crashes_found: ${{ steps.check.outputs.crashes_found }}
141+
first_crash_name: ${{ steps.check.outputs.first_crash_name }}
142+
artifact_url: ${{ steps.upload_artifacts.outputs.artifact-url }}
310143
steps:
311144
- uses: runs-on/action@v2
312145
with:
@@ -335,13 +168,47 @@ jobs:
335168
AWS_ENDPOINT_URL: "https://01e9655179bbec953276890b183039bc.r2.cloudflarestorage.com"
336169
- name: Run fuzzing target
337170
id: fuzz
338-
run: RUST_BACKTRACE=1 cargo +nightly fuzz run --release --debug-assertions array_ops -- -max_total_time=7200
171+
run: |
172+
RUST_BACKTRACE=1 cargo +nightly fuzz run --release --debug-assertions array_ops -- -max_total_time=7200 2>&1 | tee fuzz_output.log
339173
continue-on-error: true
174+
- name: Check for crashes
175+
id: check
176+
run: |
177+
if [ -d "fuzz/artifacts" ] && [ "$(ls -A fuzz/artifacts 2>/dev/null)" ]; then
178+
echo "crashes_found=true" >> $GITHUB_OUTPUT
179+
180+
# Get the first crash file only
181+
FIRST_CRASH=$(find fuzz/artifacts -type f \( -name "crash-*" -o -name "leak-*" -o -name "timeout-*" -o -name "oom-*" \) | head -1)
182+
183+
if [ -n "$FIRST_CRASH" ]; then
184+
echo "first_crash=$FIRST_CRASH" >> $GITHUB_OUTPUT
185+
echo "first_crash_name=$(basename $FIRST_CRASH)" >> $GITHUB_OUTPUT
186+
187+
# Count all crashes for reporting
188+
CRASH_COUNT=$(find fuzz/artifacts -type f \( -name "crash-*" -o -name "leak-*" -o -name "timeout-*" -o -name "oom-*" \) | wc -l)
189+
echo "crash_count=$CRASH_COUNT" >> $GITHUB_OUTPUT
190+
echo "Found $CRASH_COUNT crash(es), will process first: $(basename $FIRST_CRASH)"
191+
fi
192+
else
193+
echo "crashes_found=false" >> $GITHUB_OUTPUT
194+
echo "crash_count=0" >> $GITHUB_OUTPUT
195+
echo "No crashes found"
196+
fi
340197
- name: Archive crash artifacts
198+
id: upload_artifacts
199+
if: steps.check.outputs.crashes_found == 'true'
341200
uses: actions/upload-artifact@v5
342201
with:
343202
name: operations-fuzzing-crash-artifacts
344203
path: fuzz/artifacts
204+
retention-days: 30
205+
- name: Archive fuzzer output log
206+
if: steps.check.outputs.crashes_found == 'true'
207+
uses: actions/upload-artifact@v4
208+
with:
209+
name: ops-fuzzing-logs
210+
path: fuzz_output.log
211+
retention-days: 30
345212
- name: Persist corpus
346213
shell: bash
347214
run: |
@@ -353,5 +220,27 @@ jobs:
353220
AWS_REGION: "us-east-1"
354221
AWS_ENDPOINT_URL: "https://01e9655179bbec953276890b183039bc.r2.cloudflarestorage.com"
355222
- name: Fail job if fuzz run found a bug
356-
if: steps.fuzz.outcome == 'failure'
223+
if: steps.check.outputs.crashes_found == 'true'
357224
run: exit 1
225+
226+
report-ops-fuzz-failures:
227+
name: "Report Array Operations Fuzz Failures"
228+
needs: ops_fuzz
229+
if: always() && needs.ops_fuzz.outputs.crashes_found == 'true'
230+
permissions:
231+
issues: write
232+
contents: read
233+
id-token: write
234+
pull-requests: read
235+
uses: ./.github/workflows/report-fuzz-crash.yml
236+
with:
237+
fuzz_target: array_ops
238+
crash_file: ${{ needs.ops_fuzz.outputs.first_crash_name }}
239+
artifact_url: ${{ needs.ops_fuzz.outputs.artifact_url }}
240+
artifact_name: operations-fuzzing-crash-artifacts
241+
logs_artifact_name: ops-fuzzing-logs
242+
branch: ${{ github.ref_name }}
243+
commit: ${{ github.sha }}
244+
secrets:
245+
claude_code_oauth_token: ${{ secrets.CLAUDE_CODE_OAUTH_TOKEN }}
246+
gh_token: ${{ secrets.GITHUB_TOKEN }}

0 commit comments

Comments
 (0)