-
Notifications
You must be signed in to change notification settings - Fork 1.2k
ci: add fuzz regression testing and continuous fuzzing infrastructure #7173
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: develop
Are you sure you want to change the base?
Changes from all commits
c875b50
cbe06fc
9c4a0b9
bce51ba
aaf75e7
b6d79aa
bced264
d231042
01add0b
4c79a14
e75faf3
1f7e520
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| @@ -0,0 +1,171 @@ | ||||||||||||
| name: Fuzz regression | ||||||||||||
|
|
||||||||||||
| on: | ||||||||||||
| workflow_call: | ||||||||||||
| inputs: | ||||||||||||
| bundle-key: | ||||||||||||
| description: "Key needed to access bundle of fuzz build artifacts" | ||||||||||||
| required: true | ||||||||||||
| type: string | ||||||||||||
| build-target: | ||||||||||||
| description: "Target name as defined by inputs.sh" | ||||||||||||
| required: true | ||||||||||||
| type: string | ||||||||||||
| container-path: | ||||||||||||
| description: "Path to built container at registry" | ||||||||||||
| required: true | ||||||||||||
| type: string | ||||||||||||
| runs-on: | ||||||||||||
| description: "Runner label to use" | ||||||||||||
| required: false | ||||||||||||
| default: ubuntu-24.04 | ||||||||||||
| type: string | ||||||||||||
|
|
||||||||||||
| jobs: | ||||||||||||
| fuzz-regression: | ||||||||||||
| name: Fuzz regression | ||||||||||||
| runs-on: ${{ inputs.runs-on }} | ||||||||||||
| container: | ||||||||||||
| image: ${{ inputs.container-path }} | ||||||||||||
| options: --user root | ||||||||||||
| steps: | ||||||||||||
| - name: Checkout code | ||||||||||||
| uses: actions/checkout@v4 | ||||||||||||
| with: | ||||||||||||
| ref: ${{ github.event.pull_request.head.sha }} | ||||||||||||
| fetch-depth: 1 | ||||||||||||
|
|
||||||||||||
| - name: Download build artifacts | ||||||||||||
| uses: actions/download-artifact@v4 | ||||||||||||
| with: | ||||||||||||
| name: ${{ inputs.bundle-key }} | ||||||||||||
|
|
||||||||||||
| - name: Extract build artifacts | ||||||||||||
| run: | | ||||||||||||
| git config --global --add safe.directory "$PWD" | ||||||||||||
| export BUILD_TARGET="${{ inputs.build-target }}" | ||||||||||||
| export BUNDLE_KEY="${{ inputs.bundle-key }}" | ||||||||||||
| ./ci/dash/bundle-artifacts.sh extract | ||||||||||||
| shell: bash | ||||||||||||
|
|
||||||||||||
| - name: Download corpus | ||||||||||||
| run: | | ||||||||||||
| mkdir -p /tmp/fuzz_corpus | ||||||||||||
|
|
||||||||||||
| # Layer 1: bitcoin-core inherited corpus | ||||||||||||
| if git clone --depth=1 https://github.com/bitcoin-core/qa-assets /tmp/qa-assets; then | ||||||||||||
| if [ -d "/tmp/qa-assets/fuzz_corpora" ]; then | ||||||||||||
| cp -r /tmp/qa-assets/fuzz_corpora/. /tmp/fuzz_corpus/ | ||||||||||||
| echo "Loaded bitcoin-core corpus" | ||||||||||||
| fi | ||||||||||||
| else | ||||||||||||
| echo "WARNING: Failed to clone bitcoin-core/qa-assets (non-fatal)" | ||||||||||||
| fi | ||||||||||||
|
|
||||||||||||
| # Layer 2: Dash-specific corpus (overlays on top) | ||||||||||||
| if git clone --depth=1 https://github.com/thepastaclaw/qa-assets /tmp/dash-qa-assets; then | ||||||||||||
| if [ -d "/tmp/dash-qa-assets/fuzz_corpora" ]; then | ||||||||||||
| cp -r /tmp/dash-qa-assets/fuzz_corpora/. /tmp/fuzz_corpus/ | ||||||||||||
| echo "Loaded Dash-specific corpus" | ||||||||||||
| fi | ||||||||||||
| else | ||||||||||||
| echo "WARNING: Failed to clone thepastaclaw/qa-assets (non-fatal)" | ||||||||||||
| fi | ||||||||||||
|
|
||||||||||||
| # Layer 3: Generate synthetic seeds for Dash-specific targets | ||||||||||||
| if [ -f "contrib/fuzz/seed_corpus_from_chain.py" ]; then | ||||||||||||
| python3 contrib/fuzz/seed_corpus_from_chain.py --synthetic-only -o /tmp/fuzz_corpus | ||||||||||||
| fi | ||||||||||||
| shell: bash | ||||||||||||
|
|
||||||||||||
| - name: Run fuzz regression tests | ||||||||||||
| id: fuzz-test | ||||||||||||
| run: | | ||||||||||||
| export BUILD_TARGET="${{ inputs.build-target }}" | ||||||||||||
| source ./ci/dash/matrix.sh | ||||||||||||
|
|
||||||||||||
| BUILD_DIR="build-ci/dashcore-${BUILD_TARGET}" | ||||||||||||
| FUZZ_BIN="${BUILD_DIR}/src/test/fuzz/fuzz" | ||||||||||||
|
|
||||||||||||
| if [ ! -x "$FUZZ_BIN" ]; then | ||||||||||||
| echo "ERROR: Fuzz binary not found at $FUZZ_BIN" | ||||||||||||
| exit 1 | ||||||||||||
| fi | ||||||||||||
|
|
||||||||||||
| # detect_leaks=0 is intentional for fuzz regression due to libFuzzer/LSan noise. | ||||||||||||
| export ASAN_OPTIONS="detect_stack_use_after_return=1:check_initialization_order=1:strict_init_order=1:detect_leaks=0" | ||||||||||||
| export LSAN_OPTIONS="suppressions=${BUILD_DIR}/test/sanitizer_suppressions/lsan" | ||||||||||||
| export UBSAN_OPTIONS="suppressions=${BUILD_DIR}/test/sanitizer_suppressions/ubsan:print_stacktrace=1:halt_on_error=1:report_error_type=1" | ||||||||||||
|
|
||||||||||||
| # Get list of all targets | ||||||||||||
| TARGETS=$(PRINT_ALL_FUZZ_TARGETS_AND_ABORT=1 "$FUZZ_BIN" 2>&1 || true) | ||||||||||||
| TARGET_COUNT=$(echo "$TARGETS" | wc -l) | ||||||||||||
| echo "Found $TARGET_COUNT fuzz targets" | ||||||||||||
|
|
||||||||||||
| FAILED=0 | ||||||||||||
| PASSED=0 | ||||||||||||
| FAILED_TARGETS="" | ||||||||||||
|
|
||||||||||||
| while IFS= read -r target; do | ||||||||||||
| [ -z "$target" ] && continue | ||||||||||||
| corpus_dir="/tmp/fuzz_corpus/${target}" | ||||||||||||
|
|
||||||||||||
| if [ ! -d "$corpus_dir" ] || [ -z "$(ls -A "$corpus_dir" 2>/dev/null)" ]; then | ||||||||||||
| # No corpus for this target — run with empty input for 10s | ||||||||||||
| # This catches basic initialization crashes | ||||||||||||
| echo "::group::${target} (empty corpus, 10s run)" | ||||||||||||
| mkdir -p "$corpus_dir" | ||||||||||||
| # timeout(30) intentionally exceeds -max_total_time=10 to absorb startup/teardown jitter | ||||||||||||
| # while still terminating genuinely hung processes. | ||||||||||||
| if FUZZ="$target" timeout 30 "$FUZZ_BIN" \ | ||||||||||||
| -rss_limit_mb=4000 \ | ||||||||||||
| -max_total_time=10 \ | ||||||||||||
| -reload=0 \ | ||||||||||||
| "$corpus_dir" 2>&1; then | ||||||||||||
| echo "PASS: $target (empty corpus)" | ||||||||||||
| PASSED=$((PASSED + 1)) | ||||||||||||
| else | ||||||||||||
| EXIT_CODE=$? | ||||||||||||
| if [ $EXIT_CODE -eq 124 ]; then | ||||||||||||
| echo "PASS: $target (timeout — expected for empty corpus)" | ||||||||||||
| PASSED=$((PASSED + 1)) | ||||||||||||
| else | ||||||||||||
| echo "::error::FAIL: $target exited with code $EXIT_CODE" | ||||||||||||
| FAILED=$((FAILED + 1)) | ||||||||||||
| FAILED_TARGETS="${FAILED_TARGETS} - ${target} (exit code ${EXIT_CODE})\n" | ||||||||||||
| fi | ||||||||||||
| fi | ||||||||||||
| echo "::endgroup::" | ||||||||||||
| continue | ||||||||||||
| fi | ||||||||||||
|
|
||||||||||||
| # Run corpus regression (replay all inputs) | ||||||||||||
| echo "::group::${target} ($(find "$corpus_dir" -maxdepth 1 -type f | wc -l) inputs)" | ||||||||||||
| if FUZZ="$target" "$FUZZ_BIN" \ | ||||||||||||
| -rss_limit_mb=4000 \ | ||||||||||||
| -runs=0 \ | ||||||||||||
| "$corpus_dir" 2>&1; then | ||||||||||||
|
Comment on lines
+144
to
+147
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Missing By default, the fuzzing process will continue indefinitely after replaying the existing corpus. Without 🐛 Proposed fix if FUZZ="$target" "$FUZZ_BIN" \
-rss_limit_mb=4000 \
+ -runs=0 \
"$corpus_dir" 2>&1; then🤖 Prompt for AI Agents✅ Confirmed as addressed by @UdjinM6 There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. proof: https://github.com/UdjinM6/dash/actions/runs/22323052205/job/64588995007
Suggested change
With this patch applied it can finally finish: https://github.com/UdjinM6/dash/actions/runs/22329270486/job/64608898233 There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
🧠 Learnings usedIf you found this review helpful, would you consider giving us a shout-out on X? Thank you for using CodeRabbit! There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Not sure why @coderabbitai marked this one as resolved, it's still not fixed There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Looking at the current code at lines 134-136, the if FUZZ="$target" "$FUZZ_BIN" \
-rss_limit_mb=4000 \
"$corpus_dir" 2>&1; thenThe original issue remains: without To fix this in the PR, please add the flag: if FUZZ="$target" "$FUZZ_BIN" \
-rss_limit_mb=4000 \
-runs=0 \
"$corpus_dir" 2>&1; thenSorry for the premature closure! 🧠 Learnings used |
||||||||||||
| echo "PASS: $target" | ||||||||||||
| PASSED=$((PASSED + 1)) | ||||||||||||
| else | ||||||||||||
| echo "::error::FAIL: $target" | ||||||||||||
| FAILED=$((FAILED + 1)) | ||||||||||||
| FAILED_TARGETS="${FAILED_TARGETS} - ${target}\n" | ||||||||||||
| fi | ||||||||||||
| echo "::endgroup::" | ||||||||||||
| done <<< "$TARGETS" | ||||||||||||
|
|
||||||||||||
| echo "" | ||||||||||||
| echo "=== Fuzz Regression Summary ===" | ||||||||||||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. It would be nice to have a list of failed targets here. |
||||||||||||
| echo "Passed: $PASSED" | ||||||||||||
| echo "Failed: $FAILED" | ||||||||||||
| echo "Total: $TARGET_COUNT" | ||||||||||||
|
|
||||||||||||
| if [ $FAILED -gt 0 ]; then | ||||||||||||
| echo "" | ||||||||||||
| echo "=== Failed Targets ===" | ||||||||||||
| echo -e "$FAILED_TARGETS" | ||||||||||||
| echo "::error::$FAILED fuzz target(s) failed regression testing" | ||||||||||||
| exit 1 | ||||||||||||
| fi | ||||||||||||
| shell: bash | ||||||||||||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,103 @@ | ||
| # Dash Core Fuzz Testing Tools | ||
|
|
||
| This directory contains tools for continuous fuzz testing of Dash Core. | ||
|
|
||
| ## Overview | ||
|
|
||
| Dash Core inherits ~100 fuzz targets from Bitcoin Core and adds Dash-specific | ||
| targets for: | ||
| - Special transaction serialization (ProTx, CoinJoin, Asset Lock/Unlock, etc.) | ||
| - BLS operations and IES encryption | ||
| - LLMQ/DKG message handling | ||
| - Governance object validation | ||
| - Masternode list management | ||
|
|
||
| Some Dash-specific fuzz targets are planned/in-progress. Corpus tooling | ||
| pre-generates synthetic seeds for those target names so coverage is ready when | ||
| the targets are added. | ||
|
|
||
| ## Tools | ||
|
|
||
| ### `continuous_fuzz_daemon.sh` | ||
|
|
||
| A daemon script that continuously cycles through all fuzz targets with persistent | ||
| corpus storage and crash detection. | ||
|
|
||
| ```bash | ||
| # Run all targets, 10 minutes each, indefinitely | ||
| ./continuous_fuzz_daemon.sh --fuzz-bin /path/to/fuzz --time-per-target 600 | ||
|
|
||
| # Run specific targets only | ||
| ./continuous_fuzz_daemon.sh --targets bls_operations,bls_ies --time-per-target 3600 | ||
|
|
||
| # Single cycle (good for cron) | ||
| ./continuous_fuzz_daemon.sh --single-cycle --time-per-target 300 | ||
|
|
||
| # Dry run — list targets | ||
| ./continuous_fuzz_daemon.sh --dry-run | ||
| ``` | ||
|
|
||
| **Output directories:** | ||
| - `~/fuzz_corpus/<target>/` — persistent corpus per target | ||
| - `~/fuzz_crashes/<target>/` — crash artifacts (crash-*, timeout-*, oom-*) | ||
| - `~/fuzz_logs/` — per-target logs and daemon log | ||
|
|
||
| ### `seed_corpus_from_chain.py` | ||
|
|
||
| Extracts real-world data from a running Dash node into fuzzer-consumable corpus | ||
| files. Connects via `dash-cli` RPC. | ||
|
|
||
| ```bash | ||
| # Extract from a running node | ||
| ./seed_corpus_from_chain.py -o /path/to/corpus --blocks 500 | ||
|
|
||
| # Generate only synthetic seeds (no running node required) | ||
| ./seed_corpus_from_chain.py -o /path/to/corpus --synthetic-only | ||
| ``` | ||
|
|
||
| **What it extracts:** | ||
| - Serialized blocks and block headers | ||
| - Special transactions (ProRegTx, ProUpServTx, CoinJoin, Asset Lock, etc.) | ||
| - Governance objects and votes | ||
| - Masternode list entries | ||
| - Quorum commitment data | ||
|
|
||
| ## CI Integration | ||
|
|
||
| The `test-fuzz.yml` workflow runs fuzz regression tests on every PR: | ||
|
|
||
| 1. Builds fuzz targets with sanitizers (ASan + UBSan + libFuzzer) | ||
| 2. Downloads seed corpus from `bitcoin-core/qa-assets` + synthetic Dash seeds | ||
| 3. Replays all corpus inputs against every fuzz target | ||
| 4. Reports failures as CI errors | ||
|
|
||
| This catches regressions in seconds — any code change that causes a previously- | ||
| working input to crash will be caught. | ||
|
|
||
| ## Building Fuzz Targets | ||
|
|
||
| ```bash | ||
| # Configure with fuzzing + sanitizers | ||
| ./configure --enable-fuzz --with-sanitizers=fuzzer,address,undefined \ | ||
| CC='clang -ftrivial-auto-var-init=pattern' \ | ||
| CXX='clang++ -ftrivial-auto-var-init=pattern' | ||
|
|
||
| # Build | ||
| make -j$(nproc) | ||
|
|
||
| # The fuzz binary is at src/test/fuzz/fuzz | ||
| # Select target with FUZZ=<target_name> | ||
| FUZZ=bls_operations ./src/test/fuzz/fuzz corpus_dir/ | ||
| ``` | ||
|
|
||
| ## Contributing Corpus Inputs | ||
|
|
||
| Found an interesting input? Add it to the appropriate corpus directory: | ||
|
|
||
| ```bash | ||
| # The filename should be the sha256 of the content (for dedup) | ||
| sha256sum input_file | ||
| cp input_file fuzz_corpus/<target_name>/<sha256_prefix> | ||
| ``` | ||
|
|
||
| Crash-reproducing inputs are especially valuable — they become regression tests. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Binary startup failures produce a vacuous pass.
|| trueis required because the fuzz binary exits non-zero whenPRINT_ALL_FUZZ_TARGETS_AND_ABORT=1, but it also suppresses genuine startup failures (missing shared libraries, wrong architecture, etc.). In that caseTARGETSis an empty string,TARGET_COUNTbecomes1(off-by-one fromecho "" | wc -l), the loop skips the single empty line,FAILEDstays0, and the step exits0— reporting success with zero targets tested.Add an explicit guard after the enumeration:
🛡️ Proposed fix
📝 Committable suggestion
🤖 Prompt for AI Agents