Commit d337e1d
feat(skills): add Atheris fuzz harness with CI workflow integration (#1102)
## Description
Added a polyglot Atheris fuzz harness convention to the powerpoint
skill, established it as a required pattern for all Python skills with
tests, documented the convention across security and contributing
guides, and integrated coverage-guided fuzzing into the PR validation
pipeline. The fuzz harness immediately surfaced two crash bugs in
*pptx_colors.py* that were fixed in the same changeset. A shell script
bug in *generate.sh* was also discovered and fixed during the PR
workflow.
### Fuzz Harness Implementation
> OSSF Scorecard Fuzzing detection requires an `import atheris`
statement reachable by its Phase 3 regex scanner. The polyglot harness
pattern satisfies this while keeping CI-friendly pytest tests as the
default execution mode.
- Added **fuzz_harness.py** (222 lines) to the powerpoint skill with 4
fuzz targets covering `resolve_color`, `hex_brightness`, `max_severity`,
and `_has_formatting_variation`
- Dual-mode design: pytest tests (21 tests across 4 test classes) run
everywhere; Atheris fuzzing activates only when the package is available
- `fuzz_dispatch` router uses first byte modulo target count to
distribute fuzzer input across targets
- Added `fuzz = ["atheris>=3.0"]` dependency group to
*pyproject.toml*, intentionally separate from `dev` since Atheris only
ships manylinux wheels
- Added `python_files = ["test_*.py",
"fuzz_harness.py"]` to pytest discovery configuration
### CI Workflow Integration
- Added **fuzz-tests.yml** reusable workflow for coverage-guided Atheris
fuzzing in PR validation
- Discovers Python projects via existing `discover-python-projects` job
and gates on `fuzz_harness.py` presence
- Runs configurable fuzz iterations (default 10,000) with `uv sync
--locked --dev --group fuzz`
- Produces GitHub Actions step summary with run statistics (iterations,
edges, features, corpus size, crashes)
- Uploads crash artifacts and JSON results per fuzz project for
traceability
- Supports `soft-fail` and `changed-files-only` inputs for flexible
pipeline configuration
- Follows established workflow conventions (pwsh shell, three-dot diff,
artifact name sanitization)
- Integrated **fuzz-tests** job into **pr-validation.yml** with matrix
strategy across discovered Python projects
### Bug Fixes
- Fixed **short hex string crashes** in *pptx_colors.py* discovered by
the fuzz harness
- `resolve_color` now returns a safe default RGB when hex strings are
shorter than 6 characters
- `hex_brightness` now returns `0` for short hex strings instead of
silently producing incorrect values
- Fixed **empty pathspec bug** in *generate.sh* where `printf '%s\n'` on
an empty array produced a blank newline, causing `git diff` to fail with
`fatal: empty string is not a valid pathspec`
- Wrapped in an array length guard: `if [[ ${#specs[@]} -gt 0 ]]`
### Validation & Documentation
- Added **error-level validation** in *Validate-SkillStructure.ps1*
checking that Python skills with a `tests/` directory include
`tests/fuzz_harness.py`
- Updated *.github/copilot-instructions.md* with 3 convention entries
for the fuzz harness requirement
- Added *docs/security/fuzzing.md* covering detection strategy, running
instructions, and platform compatibility
- Updated *docs/contributing/skills.md* with a fuzz harness section,
directory tree update, and 2 submission checklist items
- Updated *docs/security/README.md* and *scripts/linting/README.md* with
index entries for the new content
## Related Issue(s)
Closes #1021
## Type of Change
Select all that apply:
**Code & Documentation:**
* [ ] Bug fix (non-breaking change fixing an issue)
* [x] New feature (non-breaking change adding functionality)
* [ ] Breaking change (fix or feature causing existing functionality to
change)
* [x] Documentation update
**Infrastructure & Configuration:**
* [x] GitHub Actions workflow
* [ ] Linting configuration (markdown, PowerShell, etc.)
* [ ] Security configuration
* [ ] DevContainer configuration
* [ ] Dependency update
**AI Artifacts:**
* [ ] Reviewed contribution with `prompt-builder` agent and addressed
all feedback
* [x] Copilot instructions (`.github/instructions/*.instructions.md`)
* [ ] Copilot prompt (`.github/prompts/*.prompt.md`)
* [ ] Copilot agent (`.github/agents/*.agent.md`)
* [x] Copilot skill (`.github/skills/*/SKILL.md`)
**Other:**
* [x] Script/automation (`.ps1`, `.sh`, `.py`)
* [ ] Other (please describe):
## Sample Prompts (for AI Artifact Contributions)
## Testing
- Ran `npm run validate:skills` — fuzz harness validation passes for the
powerpoint skill
- Ran `npm run lint:ps` — PowerShell analysis passes for
Validate-SkillStructure.ps1 changes
- Ran `npm run lint:md` — markdown linting passes for all modified
documentation files
- Ran `npm run spell-check` — spelling validation passes
- Ran `npm run lint:frontmatter` — frontmatter validation passes
- Ran `npm run lint:md-links` — markdown link checking passes
- Ran `actionlint` — workflow validation passes for both fuzz-tests.yml
and pr-validation.yml
- Ran `yaml-lint` — YAML validation passes for fuzz-tests.yml
- Fuzz harness pytest mode executed successfully with all 21 tests
passing
- Local fuzz run completed 10,000 iterations with 0 crashes (~66s
wall-clock)
## Checklist
### Required Checks
* [x] Documentation is updated (if applicable)
* [x] Files follow existing naming conventions
* [x] Changes are backwards compatible (if applicable)
* [ ] Tests added for new functionality (if applicable)
### AI Artifact Contributions
* [ ] Used `/prompt-analyze` to review contribution
* [ ] Addressed all feedback from `prompt-builder` review
* [ ] Verified contribution follows common standards and type-specific
requirements
### Required Automated Checks
The following validation commands must pass before merging:
* [x] Markdown linting: `npm run lint:md`
* [x] Spell checking: `npm run spell-check`
* [x] Frontmatter validation: `npm run lint:frontmatter`
* [x] Skill structure validation: `npm run validate:skills`
* [x] Link validation: `npm run lint:md-links`
* [x] PowerShell analysis: `npm run lint:ps`
## Security Considerations
No sensitive data, credentials, or privilege escalation paths
introduced. The Atheris dependency is isolated in a `fuzz` group that is
not installed by default. The `pptx_colors.py` fixes close
input-validation gaps that could have caused unexpected behavior with
malformed color strings. The fuzz-tests workflow runs with `contents:
read` permissions only.
## Additional Notes
- The `fuzz` dependency group is intentionally separate from `dev`
because Atheris only ships manylinux wheels (no macOS). This prevents
`uv sync` failures on non-Linux platforms.
- The *uv.lock* update includes a `cairosvg` wheel addition that appears
to be a pre-existing lock drift correction.
- The fuzz-tests workflow aligns with established conventions from
python-lint.yml and pip-audit.yml (pwsh shell defaults, three-dot diff
for changed-files detection, artifact name sanitization, GITHUB_ENV
gating).
---------
Co-authored-by: Bill Berry <wbery@microsoft.com>1 parent f79c272 commit d337e1d
File tree
30 files changed
+701
-12
lines changed- .github
- skills
- experimental/powerpoint
- scripts
- tests
- corpus
- workflows
- docs
- contributing
- security
- scripts/linting
30 files changed
+701
-12
lines changed| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
62 | 62 | | |
63 | 63 | | |
64 | 64 | | |
| 65 | + | |
65 | 66 | | |
66 | 67 | | |
| 68 | + | |
67 | 69 | | |
68 | 70 | | |
69 | 71 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
189 | 189 | | |
190 | 190 | | |
191 | 191 | | |
| 192 | + | |
| 193 | + | |
| 194 | + | |
192 | 195 | | |
193 | 196 | | |
194 | 197 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
18 | 18 | | |
19 | 19 | | |
20 | 20 | | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
21 | 25 | | |
22 | 26 | | |
23 | 27 | | |
24 | 28 | | |
| 29 | + | |
25 | 30 | | |
26 | 31 | | |
27 | 32 | | |
| |||
Lines changed: 4 additions & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
82 | 82 | | |
83 | 83 | | |
84 | 84 | | |
| 85 | + | |
| 86 | + | |
85 | 87 | | |
86 | 88 | | |
87 | 89 | | |
| |||
151 | 153 | | |
152 | 154 | | |
153 | 155 | | |
| 156 | + | |
| 157 | + | |
154 | 158 | | |
155 | 159 | | |
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Lines changed: 1 addition & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
Lines changed: 1 addition & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
0 commit comments