Skip to content

Commit af35cde

Browse files
bohendodguido
andauthored
init mutation-testing skill (#140)
* init mutation-testing skill * fix metadata * init codex symlink * polish according to review feedback --------- Co-authored-by: Dan Guido <dan@trailofbits.com>
1 parent 6b79b5e commit af35cde

File tree

9 files changed

+794
-0
lines changed

9 files changed

+794
-0
lines changed

.claude-plugin/marketplace.json

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -145,6 +145,17 @@
145145
},
146146
"source": "./plugins/entry-point-analyzer"
147147
},
148+
{
149+
"name": "mutation-testing",
150+
"version": "1.0.0",
151+
"description": "Configures mewt or muton mutation testing campaigns — scopes targets, tunes timeouts, and optimizes long-running runs. Use when the user mentions mewt, muton, mutation testing, or wants to configure or optimize a mutation testing campaign.",
152+
"author": {
153+
"name": "Trail of Bits",
154+
"email": "opensource@trailofbits.com",
155+
"url": "https://github.com/trailofbits"
156+
},
157+
"source": "./plugins/mutation-testing"
158+
},
148159
{
149160
"name": "property-based-testing",
150161
"description": "Property-based testing guidance for multiple languages and smart contracts",

.codex/skills/mutation-testing

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
../../plugins/mutation-testing/skills/mutation-testing

CODEOWNERS

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -23,6 +23,7 @@
2323
/plugins/insecure-defaults/ @dariushoule @dguido
2424
/plugins/let-fate-decide/ @tob-scott-a @dguido
2525
/plugins/modern-python/ @Ninja3047 @dguido
26+
/plugins/mutation-testing/ @bohendo @dguido
2627
/plugins/property-based-testing/ @hbrodin @dguido
2728
/plugins/seatbelt-sandboxer/ @smichaels-tob @dguido
2829
/plugins/skill-improver/ @GrosQuildu @dguido

README.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -80,6 +80,7 @@ cd /path/to/parent # e.g., if repo is at ~/projects/skills, be in ~/projects
8080
| Plugin | Description |
8181
|--------|-------------|
8282
| [constant-time-analysis](plugins/constant-time-analysis/) | Detect compiler-induced timing side-channels in cryptographic code |
83+
| [mutation-testing](plugins/mutation-testing/) | Configure mewt/muton mutation testing campaigns — scope targets, tune timeouts, optimize long runs |
8384
| [property-based-testing](plugins/property-based-testing/) | Property-based testing guidance for multiple languages and smart contracts |
8485
| [spec-to-code-compliance](plugins/spec-to-code-compliance/) | Specification-to-code compliance checker for blockchain audits |
8586
| [zeroize-audit](plugins/zeroize-audit/) | Detect missing or compiler-eliminated zeroization of secrets in C/C++ and Rust |
Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,10 @@
1+
{
2+
"name": "mutation-testing",
3+
"version": "1.0.0",
4+
"description": "Configures mewt or muton mutation testing campaigns — scopes targets, tunes timeouts, and optimizes long-running runs. Use when the user mentions mewt, muton, mutation testing, or wants to configure or optimize a mutation testing campaign.",
5+
"author": {
6+
"name": "Trail of Bits",
7+
"email": "opensource@trailofbits.com",
8+
"url": "https://github.com/trailofbits"
9+
}
10+
}

plugins/mutation-testing/README.md

Lines changed: 43 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,43 @@
1+
# Mutation Testing — Campaign Configuration (mewt/muton)
2+
3+
Helps configure [mewt](https://github.com/trailofbits/mewt) or [muton](https://github.com/trailofbits/muton) mutation testing campaigns — scoping targets, tuning timeouts, and optimizing long-running runs so you can execute `mewt run` or `muton run` with confidence.
4+
5+
> **Note**: muton and mewt share identical interfaces but target different languages — mewt for general-purpose languages, muton for TON smart contracts (Tact, Tolk, FunC). All commands and configuration patterns in this plugin apply to both tools. File names change accordingly: `mewt.toml``muton.toml`, `mewt.sqlite``muton.sqlite`.
6+
7+
## What It Does
8+
9+
Walks through a 5-phase configuration workflow:
10+
11+
1. **Initialize and validate targets** — run `mewt init`, review auto-generated config, fix include/ignore patterns
12+
2. **Generate mutants and assess scope** — count mutants, time the test command, estimate campaign duration
13+
3. **Decide on optimization** — choose between full run, targeted components, high/medium severity only, or two-phase campaign
14+
4. **Validate test command and timeout** — verify the command works; set manual timeout for recompilation-heavy languages (Solidity/Foundry, heavy C++)
15+
5. **Final validation checklist** — confirm config, mutant count, target selection, and timeout before running
16+
17+
## When to Use
18+
19+
- Setting up a new mutation testing campaign
20+
- Optimizing a campaign that would take too long to run
21+
- Diagnosing why no mutants are generated or why the test command fails
22+
23+
## Prerequisites
24+
25+
- [mewt](https://github.com/trailofbits/mewt) v3.0.0+ or [muton](https://github.com/trailofbits/muton) v3.0.0+ installed
26+
- A test suite runnable from the command line
27+
- Source code in a supported language:
28+
- **mewt**: Rust, Solidity, Go, TypeScript, JavaScript
29+
- **muton**: Tact, Tolk, FunC (TON smart contract languages)
30+
31+
## Example Usage
32+
33+
```
34+
User: "Help me set up mewt for this Solidity project"
35+
User: "Configure muton for this FunC codebase"
36+
User: "My mewt campaign would take 30 hours — how do I optimize it?"
37+
→ Guides through configuration, scope assessment, and optimization
38+
```
39+
40+
## References
41+
42+
- [mewt GitHub Repository](https://github.com/trailofbits/mewt)
43+
- [Use mutation testing to find the bugs your tests don't catch](https://blog.trailofbits.com/2025/09/18/use-mutation-testing-to-find-the-bugs-your-tests-dont-catch/) — Trail of Bits blog post
Lines changed: 76 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,76 @@
1+
---
2+
name: mutation-testing
3+
description: "Configures mewt or muton mutation testing campaigns — scopes targets, tunes timeouts, and optimizes long-running runs. Use when the user mentions mewt, muton, mutation testing, or wants to configure or optimize a mutation testing campaign."
4+
allowed-tools:
5+
- Read
6+
- Write
7+
- Bash
8+
- Grep
9+
---
10+
11+
# Mutation Testing — Campaign Configuration (mewt/muton)
12+
13+
> **Note**: muton and mewt share identical interfaces but target different languages — mewt for general-purpose languages (Rust, Solidity, Go, TypeScript, JavaScript), muton for TON smart contracts (Tact, Tolk, FunC). All examples use `mewt` commands, but they work exactly the same with `muton`. File names change accordingly: `mewt.toml``muton.toml`, `mewt.sqlite``muton.sqlite`.
14+
15+
## When to Use
16+
17+
Use this skill when the user:
18+
- Mentions "mewt", "muton", or "mutation testing"
19+
- Needs to configure or optimize a mutation testing campaign
20+
- Wants to run `mewt run` and needs help getting set up first
21+
22+
## When NOT to Use
23+
24+
Do not use this skill when the user:
25+
- Wants to analyze or report on completed campaign results
26+
- Asks about tests or coverage without mentioning mutation testing
27+
28+
---
29+
30+
## Quick Start
31+
32+
Load [workflows/configuration.md](workflows/configuration.md) — a 5-phase guide from `mewt init` to a validated, ready-to-run campaign.
33+
34+
**General question or unfamiliar command?**
35+
Run `mewt --help` or `mewt <subcommand> --help`, then assist.
36+
37+
---
38+
39+
## Reference Index
40+
41+
| File | Content |
42+
|------|---------|
43+
| [workflows/configuration.md](workflows/configuration.md) | 5-phase guide: init, scope, optimize, validate, run |
44+
| [references/optimization-strategies.md](references/optimization-strategies.md) | Per-file targeting, two-phase campaigns, mutation type filtering |
45+
46+
---
47+
48+
## Essential Commands
49+
50+
```bash
51+
# Initialize and mutate
52+
mewt init # Create mewt.toml and mewt.sqlite
53+
mewt mutate [paths] # Generate mutants without running tests
54+
mewt run [paths] # Run the full campaign
55+
56+
# Inspect configuration and scope
57+
mewt print config # View effective configuration
58+
mewt print targets # Table of all targeted files
59+
mewt print mutations --language [lang] # Available mutation types
60+
mewt status # Mutant count and per-file breakdown
61+
62+
# Investigate specific mutants
63+
mewt print mutants --target [path] # All mutants for a file
64+
mewt print mutants --severity high # Filter by severity
65+
mewt print mutant --id [id] # View mutated code diff
66+
mewt test --ids [ids] # Re-test specific mutants
67+
```
68+
69+
---
70+
71+
## What Results Mean
72+
73+
- **Caught/TestFail**: Tests detected the mutation (good)
74+
- **Uncaught**: Mutation survived — indicates untested logic
75+
- **Timeout**: Tests took too long, inconclusive
76+
- **Skipped**: A more severe mutant already failed on the same line

0 commit comments

Comments
 (0)