Skip to content

Commit d337e1d

Browse files
WilliamBerryiiiBill Berry
andauthored
feat(skills): add Atheris fuzz harness with CI workflow integration (#1102)
## Description Added a polyglot Atheris fuzz harness convention to the powerpoint skill, established it as a required pattern for all Python skills with tests, documented the convention across security and contributing guides, and integrated coverage-guided fuzzing into the PR validation pipeline. The fuzz harness immediately surfaced two crash bugs in *pptx_colors.py* that were fixed in the same changeset. A shell script bug in *generate.sh* was also discovered and fixed during the PR workflow. ### Fuzz Harness Implementation &gt; OSSF Scorecard Fuzzing detection requires an `import atheris` statement reachable by its Phase 3 regex scanner. The polyglot harness pattern satisfies this while keeping CI-friendly pytest tests as the default execution mode. - Added **fuzz_harness.py** (222 lines) to the powerpoint skill with 4 fuzz targets covering `resolve_color`, `hex_brightness`, `max_severity`, and `_has_formatting_variation` - Dual-mode design: pytest tests (21 tests across 4 test classes) run everywhere; Atheris fuzzing activates only when the package is available - `fuzz_dispatch` router uses first byte modulo target count to distribute fuzzer input across targets - Added `fuzz = [&quot;atheris&gt;=3.0&quot;]` dependency group to *pyproject.toml*, intentionally separate from `dev` since Atheris only ships manylinux wheels - Added `python_files = [&quot;test_*.py&quot;, &quot;fuzz_harness.py&quot;]` to pytest discovery configuration ### CI Workflow Integration - Added **fuzz-tests.yml** reusable workflow for coverage-guided Atheris fuzzing in PR validation - Discovers Python projects via existing `discover-python-projects` job and gates on `fuzz_harness.py` presence - Runs configurable fuzz iterations (default 10,000) with `uv sync --locked --dev --group fuzz` - Produces GitHub Actions step summary with run statistics (iterations, edges, features, corpus size, crashes) - Uploads crash artifacts and JSON results per fuzz project for traceability - Supports `soft-fail` and `changed-files-only` inputs for flexible pipeline configuration - Follows established workflow conventions (pwsh shell, three-dot diff, artifact name sanitization) - Integrated **fuzz-tests** job into **pr-validation.yml** with matrix strategy across discovered Python projects ### Bug Fixes - Fixed **short hex string crashes** in *pptx_colors.py* discovered by the fuzz harness - `resolve_color` now returns a safe default RGB when hex strings are shorter than 6 characters - `hex_brightness` now returns `0` for short hex strings instead of silently producing incorrect values - Fixed **empty pathspec bug** in *generate.sh* where `printf '%s\n'` on an empty array produced a blank newline, causing `git diff` to fail with `fatal: empty string is not a valid pathspec` - Wrapped in an array length guard: `if [[ ${#specs[@]} -gt 0 ]]` ### Validation &amp; Documentation - Added **error-level validation** in *Validate-SkillStructure.ps1* checking that Python skills with a `tests/` directory include `tests/fuzz_harness.py` - Updated *.github/copilot-instructions.md* with 3 convention entries for the fuzz harness requirement - Added *docs/security/fuzzing.md* covering detection strategy, running instructions, and platform compatibility - Updated *docs/contributing/skills.md* with a fuzz harness section, directory tree update, and 2 submission checklist items - Updated *docs/security/README.md* and *scripts/linting/README.md* with index entries for the new content ## Related Issue(s) Closes #1021 ## Type of Change Select all that apply: **Code &amp; Documentation:** * [ ] Bug fix (non-breaking change fixing an issue) * [x] New feature (non-breaking change adding functionality) * [ ] Breaking change (fix or feature causing existing functionality to change) * [x] Documentation update **Infrastructure &amp; Configuration:** * [x] GitHub Actions workflow * [ ] Linting configuration (markdown, PowerShell, etc.) * [ ] Security configuration * [ ] DevContainer configuration * [ ] Dependency update **AI Artifacts:** * [ ] Reviewed contribution with `prompt-builder` agent and addressed all feedback * [x] Copilot instructions (`.github/instructions/*.instructions.md`) * [ ] Copilot prompt (`.github/prompts/*.prompt.md`) * [ ] Copilot agent (`.github/agents/*.agent.md`) * [x] Copilot skill (`.github/skills/*/SKILL.md`) **Other:** * [x] Script/automation (`.ps1`, `.sh`, `.py`) * [ ] Other (please describe): ## Sample Prompts (for AI Artifact Contributions) ## Testing - Ran `npm run validate:skills` — fuzz harness validation passes for the powerpoint skill - Ran `npm run lint:ps` — PowerShell analysis passes for Validate-SkillStructure.ps1 changes - Ran `npm run lint:md` — markdown linting passes for all modified documentation files - Ran `npm run spell-check` — spelling validation passes - Ran `npm run lint:frontmatter` — frontmatter validation passes - Ran `npm run lint:md-links` — markdown link checking passes - Ran `actionlint` — workflow validation passes for both fuzz-tests.yml and pr-validation.yml - Ran `yaml-lint` — YAML validation passes for fuzz-tests.yml - Fuzz harness pytest mode executed successfully with all 21 tests passing - Local fuzz run completed 10,000 iterations with 0 crashes (~66s wall-clock) ## Checklist ### Required Checks * [x] Documentation is updated (if applicable) * [x] Files follow existing naming conventions * [x] Changes are backwards compatible (if applicable) * [ ] Tests added for new functionality (if applicable) ### AI Artifact Contributions * [ ] Used `/prompt-analyze` to review contribution * [ ] Addressed all feedback from `prompt-builder` review * [ ] Verified contribution follows common standards and type-specific requirements ### Required Automated Checks The following validation commands must pass before merging: * [x] Markdown linting: `npm run lint:md` * [x] Spell checking: `npm run spell-check` * [x] Frontmatter validation: `npm run lint:frontmatter` * [x] Skill structure validation: `npm run validate:skills` * [x] Link validation: `npm run lint:md-links` * [x] PowerShell analysis: `npm run lint:ps` ## Security Considerations No sensitive data, credentials, or privilege escalation paths introduced. The Atheris dependency is isolated in a `fuzz` group that is not installed by default. The `pptx_colors.py` fixes close input-validation gaps that could have caused unexpected behavior with malformed color strings. The fuzz-tests workflow runs with `contents: read` permissions only. ## Additional Notes - The `fuzz` dependency group is intentionally separate from `dev` because Atheris only ships manylinux wheels (no macOS). This prevents `uv sync` failures on non-Linux platforms. - The *uv.lock* update includes a `cairosvg` wheel addition that appears to be a pre-existing lock drift correction. - The fuzz-tests workflow aligns with established conventions from python-lint.yml and pip-audit.yml (pwsh shell defaults, three-dot diff for changed-files detection, artifact name sanitization, GITHUB_ENV gating). --------- Co-authored-by: Bill Berry <wbery@microsoft.com>
1 parent f79c272 commit d337e1d

30 files changed

+701
-12
lines changed

.cspell.json

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -62,8 +62,10 @@
6262
"general-technical"
6363
],
6464
"words": [
65+
"atheris",
6566
"behaviour",
6667
"brainwriting",
68+
"clusterfuzzlite",
6769
"easyops",
6870
"hideable",
6971
"learning",

.github/copilot-instructions.md

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -189,6 +189,9 @@ Python skills include a `pyproject.toml` validated by `validate:skills` via `Tes
189189
* `[tool.ruff]` - Required. Enables `lint:py` compatibility across all Python skills.
190190
* `[tool.ruff.lint]` - Recommended. Configures rule selection (e.g., `select = ["E", "F", "I", "W"]`).
191191
* `[tool.pytest.ini_options]` - Required when the skill contains a `tests/` directory.
192+
* `tests/fuzz_harness.py` - Required when the skill contains a `tests/` directory. Polyglot Atheris fuzz harness for OSSF Scorecard compliance.
193+
* `fuzz` dependency group with `atheris>=3.0` - Required alongside `fuzz_harness.py`. Kept separate from `dev` (no macOS wheels).
194+
* `python_files = ["test_*.py", "fuzz_harness.py"]` in `[tool.pytest.ini_options]` - Required alongside `fuzz_harness.py`. Enables pytest discovery.
192195
* `ruff` in dev dependencies - Recommended. Ensures the linter is available in the skill's virtual environment.
193196

194197
### Environment Synchronization

.github/skills/experimental/powerpoint/pyproject.toml

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -18,10 +18,15 @@ dev = [
1818
"ruff>=0.15",
1919
"hypothesis>=6.100",
2020
]
21+
# Atheris ships manylinux-only wheels; keep separate from dev so uv sync works on macOS.
22+
fuzz = [
23+
"atheris>=3.0",
24+
]
2125

2226
[tool.pytest.ini_options]
2327
testpaths = ["tests"]
2428
pythonpath = ["scripts"]
29+
python_files = ["test_*.py", "fuzz_harness.py"]
2530
markers = [
2631
"integration: roundtrip integration tests",
2732
"slow: tests that create full presentations",

.github/skills/experimental/powerpoint/scripts/pptx_colors.py

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -82,6 +82,8 @@ def resolve_color(
8282
return {"rgb": RGBColor(0, 0, 0)}
8383

8484
hex_str = value.lstrip("#")
85+
if len(hex_str) < 6:
86+
return {"rgb": RGBColor(0, 0, 0)}
8587
return {
8688
"rgb": RGBColor(
8789
int(hex_str[0:2], 16), int(hex_str[2:4], 16), int(hex_str[4:6], 16)
@@ -151,5 +153,7 @@ def rgb_to_hex(rgb_color) -> str | None:
151153
def hex_brightness(hex_color: str) -> int:
152154
"""Calculate perceived brightness (0-255) from a hex color string."""
153155
h = hex_color.lstrip("#")
156+
if len(h) < 6:
157+
return 0
154158
r, g, b = int(h[0:2], 16), int(h[2:4], 16), int(h[4:6], 16)
155159
return int(0.299 * r + 0.587 * g + 0.114 * b)
1 Byte
Binary file not shown.
3 Bytes
Binary file not shown.
52 Bytes
Binary file not shown.
67 Bytes
Binary file not shown.
Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
ZZZZZZZZZZ

0 commit comments

Comments
 (0)