Skip to content

Commit 36d3767

Browse files
Copilotjsturtevant
Copilot
and
jsturtevant
authored
Add automatic issue creation for scheduled fuzzing job failures with area/fuzzing label filtering (#823)
* Initial plan * Add fuzzing failure notification system - Created dev/notify-fuzzing-failure.sh script to handle GitHub issue creation/updates - Modified Fuzzing.yml and dep_fuzzing.yml workflows to add failure notifications - Added proper permissions for issue creation and comments - Script checks for existing open fuzzing failure issues and either creates new ones or adds comments - Includes robust error handling and test mode for validation Co-authored-by: jsturtevant <[email protected]> * Move fuzzing failure notification from reusable workflow to scheduled workflow - Removed notification step and issues permission from dep_fuzzing.yml (reusable workflow) - Added separate notify-failure job to Fuzzing.yml (scheduled workflow) - This ensures issues are only created for scheduled cron job failures, not PR runs Co-authored-by: jsturtevant <[email protected]> * Add --repo parameter to gh commands for testing support - Added --repo "$REPO" to gh issue comment command (line 103) - Added --repo "$REPO" to gh issue create command (line 148) - This enables testing against specific repositories by setting the REPO variable Co-authored-by: jsturtevant <[email protected]> * Rebase and revert the code change from copilot Signed-off-by: James Sturtevant <[email protected]> * Add kind/fuzzing label for better issue filtering and management Co-authored-by: jsturtevant <[email protected]> * Update fuzzing label from kind/fuzzing to area/fuzzing Co-authored-by: jsturtevant <[email protected]> * Rename variable names for clarity: testing_label and fuzzing_label Co-authored-by: jsturtevant <[email protected]> --------- Signed-off-by: James Sturtevant <[email protected]> Co-authored-by: copilot-swe-agent[bot] <[email protected]> Co-authored-by: jsturtevant <[email protected]> Co-authored-by: James Sturtevant <[email protected]>
1 parent c97feb6 commit 36d3767

File tree

2 files changed

+169
-1
lines changed

2 files changed

+169
-1
lines changed

‎.github/workflows/Fuzzing.yml‎

Lines changed: 16 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -15,4 +15,19 @@ jobs:
1515
with:
1616
targets: '["fuzz_host_print", "fuzz_guest_call", "fuzz_host_call"]' # Pass as a JSON array
1717
max_total_time: 18000 # 5 hours in seconds
18-
secrets: inherit
18+
secrets: inherit
19+
20+
notify-failure:
21+
runs-on: ubuntu-latest
22+
needs: fuzzing
23+
if: always() && needs.fuzzing.result == 'failure'
24+
permissions:
25+
issues: write
26+
steps:
27+
- name: Checkout code
28+
uses: actions/checkout@v5
29+
30+
- name: Notify Fuzzing Failure
31+
run: ./dev/notify-fuzzing-failure.sh "fuzz_host_print,fuzz_guest_call,fuzz_host_call"
32+
env:
33+
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}

‎dev/notify-fuzzing-failure.sh‎

Lines changed: 153 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,153 @@
1+
#!/bin/bash
2+
set -e
3+
set -u
4+
set -o pipefail
5+
6+
## DESCRIPTION:
7+
##
8+
## This script creates or updates GitHub issues when fuzzing jobs fail.
9+
## It checks for existing open fuzzing failure issues and either creates
10+
## a new one or adds a comment to an existing one.
11+
##
12+
## PRE-REQS:
13+
##
14+
## This script assumes that the gh cli is installed and in the PATH
15+
## and that there is a GitHub PAT in the GITHUB_TOKEN env var
16+
## with the following permissions:
17+
## - issues (read/write)
18+
## or that the user is logged into the gh cli with an account with those permissions
19+
##
20+
## Run this script locally like:
21+
## GITHUB_REPOSITORY="fork/hyperlight" GITHUB_RUN_ID=1 ./dev/notify-fuzzing-failure.sh "fuzz_host_print,fuzz_guest_call,fuzz_host_call"
22+
23+
REPO="${GITHUB_REPOSITORY:-hyperlight-dev/hyperlight}"
24+
WORKFLOW_RUN_URL="${GITHUB_SERVER_URL:-https://github.com}/${REPO}/actions/runs/${GITHUB_RUN_ID:-unknown}"
25+
FUZZING_TARGETS="${1:-unknown}"
26+
27+
# Check if running in test mode (handle both first and second arguments)
28+
if [ "${1:-}" = "--test" ] || [ "${2:-}" = "--test" ]; then
29+
echo "✅ Running in test mode - script structure is valid"
30+
echo "Would check for fuzzing failure issues in $REPO"
31+
echo "Would create issue or comment for fuzzing targets: ${1:-unknown}"
32+
echo "Workflow URL would be: $WORKFLOW_RUN_URL"
33+
exit 0
34+
fi
35+
36+
echo "Checking for existing fuzzing failure issues in $REPO..."
37+
38+
# Extract owner and repo name from the repository
39+
OWNER=$(echo "$REPO" | cut -d'/' -f1)
40+
REPO_NAME=$(echo "$REPO" | cut -d'/' -f2)
41+
42+
# Define the issue title and labels
43+
ISSUE_TITLE="Fuzzing Job Failure - $(date '+%Y-%m-%d')"
44+
TESTING_LABEL="area/testing"
45+
FAILURE_LABEL="kind/bug"
46+
FUZZING_LABEL="area/fuzzing"
47+
LIFECYCLE_LABEL="lifecycle/needs-review"
48+
49+
# Search for existing open fuzzing failure issues
50+
echo "Searching for existing open fuzzing failure issues..."
51+
EXISTING_ISSUES=$(gh api graphql -f query='
52+
query($owner: String!, $repo: String!) {
53+
repository(owner: $owner, name: $repo) {
54+
issues(first: 10, states: OPEN, labels: ["area/fuzzing"]) {
55+
totalCount
56+
nodes {
57+
number
58+
title
59+
url
60+
labels(first: 20) {
61+
nodes {
62+
name
63+
}
64+
}
65+
}
66+
}
67+
}
68+
}' -f owner="$OWNER" -f repo="$REPO_NAME" --jq '.data.repository.issues')
69+
70+
# Filter for fuzzing-related issues (now all results should be fuzzing issues due to label filter)
71+
FUZZING_ISSUES=$(echo "$EXISTING_ISSUES" | jq '.nodes[]' 2>/dev/null || echo "")
72+
FUZZING_ISSUE_COUNT=0
73+
if [ -n "$FUZZING_ISSUES" ]; then
74+
FUZZING_ISSUE_COUNT=$(echo "$FUZZING_ISSUES" | jq -s 'length' 2>/dev/null || echo "0")
75+
fi
76+
77+
echo "Found $FUZZING_ISSUE_COUNT existing fuzzing failure issue(s)"
78+
79+
if [ "$FUZZING_ISSUE_COUNT" -gt 0 ]; then
80+
# Get the most recent fuzzing failure issue
81+
ISSUE_NUMBER=$(echo "$FUZZING_ISSUES" | jq -r '.number' | head -1)
82+
ISSUE_URL=$(echo "$FUZZING_ISSUES" | jq -r '.url' | head -1)
83+
84+
if [ "$ISSUE_NUMBER" = "null" ] || [ -z "$ISSUE_NUMBER" ]; then
85+
echo "⚠️ Could not parse issue number from fuzzing issues, creating new issue instead"
86+
FUZZING_ISSUE_COUNT=0
87+
else
88+
echo "Adding comment to existing issue #$ISSUE_NUMBER"
89+
90+
# Create comment body
91+
COMMENT_BODY="## Fuzzing Job Failed Again
92+
93+
**Date:** $(date '+%Y-%m-%d %H:%M:%S UTC')
94+
**Workflow Run:** [$WORKFLOW_RUN_URL]($WORKFLOW_RUN_URL)
95+
**Fuzzing Targets:** $FUZZING_TARGETS
96+
97+
The scheduled fuzzing job has failed again. Please check the workflow logs and artifacts for details."
98+
99+
# Add comment to the existing issue
100+
if gh issue comment "$ISSUE_NUMBER" --body "$COMMENT_BODY" --repo "$REPO"; then
101+
echo "✅ Added comment to existing issue #$ISSUE_NUMBER: $ISSUE_URL"
102+
else
103+
echo "❌ Failed to add comment to existing issue. Creating new issue instead."
104+
FUZZING_ISSUE_COUNT=0
105+
fi
106+
fi
107+
fi
108+
109+
if [ "$FUZZING_ISSUE_COUNT" -eq 0 ]; then
110+
echo "No existing fuzzing failure issues found. Creating new issue..."
111+
112+
# Create issue body
113+
ISSUE_BODY="## Fuzzing Job Failure Report
114+
115+
**Date:** $(date '+%Y-%m-%d %H:%M:%S UTC')
116+
**Workflow Run:** [$WORKFLOW_RUN_URL]($WORKFLOW_RUN_URL)
117+
**Fuzzing Targets:** $FUZZING_TARGETS
118+
119+
The scheduled fuzzing job has failed. This issue was automatically created to track the failure.
120+
121+
### Details
122+
The fuzzing workflow failed during execution. Please check the workflow logs and any uploaded artifacts for more details.
123+
124+
### Next Steps
125+
- [ ] Review the workflow logs for error details
126+
- [ ] Download and analyze any crash artifacts if available
127+
- [ ] Determine the root cause of the failure
128+
- [ ] Fix the underlying issue
129+
130+
### Related Documentation
131+
- [Fuzzing README](https://github.com/$REPO/blob/main/fuzz/README.md)
132+
- [Security Guidance](https://github.com/$REPO/blob/main/docs/security-guidance-for-developers.md)
133+
134+
---
135+
*This issue was automatically created by the fuzzing failure notification system.*"
136+
137+
# Create the new issue
138+
if ISSUE_URL=$(gh issue create \
139+
--title "$ISSUE_TITLE" \
140+
--body "$ISSUE_BODY" \
141+
--label "$TESTING_LABEL" \
142+
--label "$FAILURE_LABEL" \
143+
--label "$FUZZING_LABEL" \
144+
--label "$LIFECYCLE_LABEL" \
145+
--repo "$REPO"); then
146+
echo "✅ Created new fuzzing failure issue: $ISSUE_URL"
147+
else
148+
echo "❌ Failed to create new fuzzing failure issue"
149+
exit 1
150+
fi
151+
fi
152+
153+
echo "Fuzzing failure notification completed successfully"

0 commit comments

Comments
 (0)