Skip to content

Commit f4f00e7

Browse files
boneskullclaude
andauthored
feat(adapters): add test framework adapters for node:test, Mocha, and AVA (#157)
Co-authored-by: Claude <[email protected]>
1 parent 895805c commit f4f00e7

File tree

18 files changed

+2801
-20
lines changed

18 files changed

+2801
-20
lines changed

AGENTS.md

Lines changed: 24 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -8,12 +8,16 @@ In Markdown text, refer to the name of this project as `**modestbench**` (bold)
88

99
### Git Workflow
1010

11-
**Linear History Required:**
11+
**IF YOU ARE EXECUTING A PLAN, ALWAYS CREATE A NEW WORKTREE AND BRANCH**. Non-planned work can be done in the main branch / worktree. See [Feature Development with Worktrees](#feature-development-with-worktrees) for more details.
12+
13+
#### Linear History Required
1214

1315
- No merge commits - use `git rebase` or `git merge --ff-only`
1416
- If merge commits exist, use `git rebase -i` to eliminate them
1517

16-
**Feature Development with Worktrees:**
18+
#### Feature Development with Worktrees
19+
20+
**IF YOU ARE ALREADY IN A WORKTREE, YOU CAN SKIP THIS STEP**.
1721

1822
```bash
1923
# Create worktree INSIDE project directory
@@ -25,28 +29,28 @@ cd .worktrees/<feature-name>/
2529
# When done, see skills/collaboration/finishing-a-development-branch
2630
```
2731

28-
**Branch Naming:**
32+
#### Branch Naming
2933

3034
- Use `feature/<descriptive-name>` (not `feat/`)
31-
- Worktrees go in `../modestbench.worktree/` directory
35+
- Worktrees go in `.worktrees/` directory
3236

3337
### Code Style
3438

35-
**Testing:**
39+
#### Testing
3640

3741
- Follow TDD principles (search your user rules for "test-driven development")
3842
- Contract tests in `test/contract/`
3943
- Integration tests in `test/integration/`
4044
- Use the wallaby MCP tools for runtime debugging
4145

42-
**Architecture:**
46+
#### Architecture
4347

4448
- Core engine: `src/core/engine.ts` - benchmark execution
4549
- CLI commands: `src/cli/commands/` - yargs-based CLI
4650
- Reporters: `src/reporters/` - output formats (human, json, csv)
4751
- Config: `src/config/` - configuration management with Zod
4852

49-
**Key Principles:**
53+
#### Key Principles
5054

5155
- Reporters: `--quiet` suppresses progress (stderr), not data output (stdout)
5256
- When no `--output` specified, data reporters write to stdout
@@ -59,19 +63,19 @@ cd .worktrees/<feature-name>/
5963
- Write tests first (TDD)
6064
- Keep functions focused (see `skills/coding/keeping-routines-focused`)
6165
- Name by domain (see `skills/coding/naming-by-domain`)
62-
3. **Testing:** Run `npm test` - aim for high coverage
63-
4. **Building:** `npm run build` before committing
64-
5. **Verification:** Check linter with `eslint` (see `skills/debugging/verification-before-completion`)
66+
3. **Linting:** Run `npm run fix` to fix linting errors (and report remaining errors)
67+
4. **Testing:** Run `npm test` - aim for high coverage (see `skills/debugging/verification-before-completion`)
68+
5. **Building:** `npm run build` before committing
6569

6670
### Common Patterns
6771

68-
**CLI Options:**
72+
#### CLI Options
6973

7074
- Use yargs for argument parsing
7175
- Options in `src/cli/commands/*.ts`
7276
- Types in `src/types/cli.ts`
7377

74-
**Reporter Structure:**
78+
#### Reporter Structure
7579

7680
```typescript
7781
class MyReporter implements Reporter {
@@ -86,7 +90,7 @@ class MyReporter implements Reporter {
8690
}
8791
```
8892

89-
**Adding CLI Flags:**
93+
#### Adding CLI Flags
9094

9195
1. Add to command definition in `src/cli/commands/run.ts`
9296
2. Add type to `src/types/cli.ts`
@@ -97,9 +101,15 @@ class MyReporter implements Reporter {
97101

98102
- Source: `src/`
99103
- Tests: `test/` (contract and integration)
104+
- `tsd` (type) tests: `test-d/`
100105
- Built output: `dist/` (gitignored)
101106
- Examples: `examples/`
102-
- Docs: `README.md`, `ARCHITECTURE.md`, `CONTRIBUTING.md`
107+
- Documentation sources: `site/`, `public/`
108+
- Generated documentation: `docs/`
109+
- Important documentation files: `README.md`, `ARCHITECTURE.md`, `CONTRIBUTING.md`
110+
- Design reference: `assets/`
111+
- Reusable scripts: `scripts/`
112+
- Ad-hoc and temporary scripts: `.tmp/`
103113

104114
### External Dependencies
105115

README.md

Lines changed: 43 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -14,6 +14,7 @@
1414
- **Multiple Output Formats**: Human-readable, JSON, and CSV reports (at the same time!!)
1515
- **Historical Tracking**: Store and compare benchmark results over time
1616
- **Code Profiling & Analysis**: Identify hot code paths that need benchmarking using V8's built-in profiler
17+
- **Run Tests as Benchmarks**: Use your existing Mocha, AVA, or node:test tests as benchmarks
1718
- **Performance Budgets**: Enforce performance standards and prevent regressions
1819
- **CLI & API**: Command-line interface and programmatic API
1920
- **TypeScript Support**: Full type safety
@@ -403,6 +404,48 @@ The profile command uses Node.js's built-in V8 profiler to identify functions th
403404

404405
Functions that appear at the top of the profile report are good candidates for benchmarking, as optimizing them will have the most impact on overall performance.
405406

407+
### Run Tests as Benchmarks
408+
409+
Already have test files? Run them as benchmarks without writing any new code:
410+
411+
```bash
412+
# Run Mocha tests as benchmarks
413+
modestbench test mocha "test/*.spec.js"
414+
415+
# Run node:test files as benchmarks
416+
modestbench test node-test "test/*.test.js"
417+
418+
# Run AVA tests as benchmarks
419+
modestbench test ava "test/*.js"
420+
421+
# With options
422+
modestbench test mocha "test/unit/*.spec.js" --iterations 500 --json
423+
```
424+
425+
**Supported Frameworks:**
426+
427+
- `mocha` - Mocha test files with `describe`/`it` syntax
428+
- `node-test` - Node.js built-in test runner (`node:test`)
429+
- `ava` - AVA test files
430+
431+
**How It Works:**
432+
433+
The `test` command captures test definitions from your test files and runs each test as a benchmark task. Test suites map to benchmark suites, and individual tests become benchmark tasks. Setup/teardown hooks (`beforeEach`/`afterEach`) are preserved and run with each iteration.
434+
435+
This is useful for:
436+
437+
- **Quick performance checks** - See how fast your tests actually run
438+
- **Regression detection** - Identify tests that have become slower
439+
- **Optimization targets** - Find slow tests that might benefit from optimization
440+
441+
**Test Command Options:**
442+
443+
- `--iterations`, `-i` - Number of iterations per test (default: 100)
444+
- `--warmup`, `-w` - Number of warmup iterations (default: 5)
445+
- `--bail`, `-b` - Stop on first failure
446+
- `--json` - Output results as JSON
447+
- `--quiet`, `-q` - Minimal output
448+
406449
## Configuration
407450

408451
### Project Configuration

cspell.json

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -71,7 +71,8 @@
7171
"asciinema",
7272
"pasqal",
7373
"illionth",
74-
"gintacent"
74+
"gintacent",
75+
"nargs"
7576
],
7677
"words": ["bupkis", "deoptimization"]
7778
}

package.json

Lines changed: 27 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -26,11 +26,31 @@
2626
"import": "./dist/index.js",
2727
"require": "./dist/index.cjs"
2828
},
29+
"./ava": {
30+
"types": "./dist/adapters/ava-register.d.cts",
31+
"import": "./dist/adapters/ava-register.js",
32+
"require": "./dist/adapters/ava-register.cjs"
33+
},
34+
"./ava-hooks": {
35+
"types": "./dist/adapters/ava-hooks.d.cts",
36+
"import": "./dist/adapters/ava-hooks.js",
37+
"require": "./dist/adapters/ava-hooks.cjs"
38+
},
2939
"./cli": {
3040
"types": "./dist/cli/index.d.cts",
3141
"import": "./dist/cli/index.js",
3242
"require": "./dist/cli/index.cjs"
3343
},
44+
"./node-test": {
45+
"types": "./dist/adapters/node-test-register.d.cts",
46+
"import": "./dist/adapters/node-test-register.js",
47+
"require": "./dist/adapters/node-test-register.cjs"
48+
},
49+
"./node-test-hooks": {
50+
"types": "./dist/adapters/node-test-hooks.d.cts",
51+
"import": "./dist/adapters/node-test-hooks.js",
52+
"require": "./dist/adapters/node-test-hooks.cjs"
53+
},
3454
"./package.json": "./package.json"
3555
},
3656
"files": [
@@ -171,7 +191,9 @@
171191
"src/index.ts",
172192
"src/cli/index.ts",
173193
"examples/**/*.ts",
174-
"examples/**/*.js"
194+
"examples/**/*.js",
195+
"src/adapters/*-hooks.ts",
196+
"src/adapters/*-register.ts"
175197
]
176198
}
177199
},
@@ -211,7 +233,11 @@
211233
"bin": "./src/cli/index.ts",
212234
"exports": {
213235
".": "./src/index.ts",
236+
"./ava": "./src/adapters/ava-register.ts",
237+
"./ava-hooks": "./src/adapters/ava-hooks.ts",
214238
"./cli": "./src/cli/index.ts",
239+
"./node-test": "./src/adapters/node-test-register.ts",
240+
"./node-test-hooks": "./src/adapters/node-test-hooks.ts",
215241
"./package.json": "./package.json"
216242
}
217243
}

site/src/content/docs/guides/cli.md

Lines changed: 120 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -448,6 +448,126 @@ The profiler displays results with color-coded percentages:
448448
See the [Profiling Guide](/guides/profiling/) for detailed workflows, best practices, and programmatic usage.
449449
:::
450450

451+
### `modestbench test`
452+
453+
Run existing test files as benchmarks. Captures test definitions from Mocha, node:test, or AVA test files and executes them as benchmark tasks.
454+
455+
```bash
456+
# Run Mocha tests as benchmarks
457+
modestbench test mocha "test/*.spec.js"
458+
459+
# Run node:test files as benchmarks
460+
modestbench test node-test "test/*.test.js"
461+
462+
# Run AVA tests as benchmarks
463+
modestbench test ava "test/*.js"
464+
465+
# Multiple patterns
466+
modestbench test mocha "test/unit/*.spec.js" "test/integration/*.spec.js"
467+
```
468+
469+
#### Supported Frameworks
470+
471+
- **`mocha`** - Mocha test files using `describe`/`it` syntax
472+
- **`node-test`** - Node.js built-in test runner (`node:test` module)
473+
- **`ava`** - AVA test files
474+
475+
#### Test Command Options
476+
477+
##### `<framework>` (required)
478+
479+
The test framework to use. Must be one of: `mocha`, `node-test`, `ava`.
480+
481+
##### `[files..]`
482+
483+
Test file paths or glob patterns.
484+
485+
```bash
486+
modestbench test mocha "test/**/*.spec.js"
487+
```
488+
489+
##### `--iterations <number>`, `-i <number>`
490+
491+
Number of iterations per test (default: 100).
492+
493+
```bash
494+
modestbench test mocha test/*.spec.js --iterations 500
495+
```
496+
497+
##### `--warmup <number>`, `-w <number>`
498+
499+
Number of warmup iterations before measurement (default: 5).
500+
501+
```bash
502+
modestbench test mocha test/*.spec.js --warmup 10
503+
```
504+
505+
##### `--bail`, `-b`
506+
507+
Stop on first failure.
508+
509+
```bash
510+
modestbench test mocha test/*.spec.js --bail
511+
```
512+
513+
##### `--json`
514+
515+
Output results in JSON format.
516+
517+
```bash
518+
modestbench test mocha test/*.spec.js --json
519+
```
520+
521+
##### `--quiet`, `-q`
522+
523+
Minimal output mode.
524+
525+
```bash
526+
modestbench test mocha test/*.spec.js --quiet
527+
```
528+
529+
#### How It Works
530+
531+
The `test` command:
532+
533+
1. **Captures** test definitions by intercepting the test framework's API
534+
2. **Converts** test suites to benchmark suites and tests to benchmark tasks
535+
3. **Preserves** setup/teardown hooks (`beforeEach`/`afterEach`)
536+
4. **Executes** each test multiple times to collect timing statistics
537+
538+
Test hierarchy is preserved: `describe` blocks become benchmark suites, `it`/`test` blocks become benchmark tasks.
539+
540+
#### Use Cases
541+
542+
- **Quick performance checks** - See how fast your tests actually run
543+
- **Regression detection** - Identify tests that have become slower over time
544+
- **Optimization targets** - Find slow tests that might benefit from optimization
545+
- **CI integration** - Add performance metrics to your test pipeline
546+
547+
#### Test Command Examples
548+
549+
```bash
550+
# Basic Mocha benchmarking
551+
modestbench test mocha "test/*.spec.js"
552+
553+
# node:test with more iterations
554+
modestbench test node-test "test/*.test.js" --iterations 500
555+
556+
# AVA with JSON output for CI
557+
modestbench test ava "test/*.js" --json --quiet
558+
559+
# Stop on first slow test
560+
modestbench test mocha "test/unit/*.spec.js" --bail
561+
```
562+
563+
:::tip[Framework Loader]
564+
For the test command to intercept imports correctly, modestbench uses ES module loader hooks. This happens automatically when you use the `test` command.
565+
:::
566+
567+
:::note[Learn More]
568+
See the [Running Tests as Benchmarks](/guides/test-adapters/) guide for detailed workflows, framework-specific examples, and best practices.
569+
:::
570+
451571
### `modestbench history`
452572

453573
Manage benchmark history and compare results over time.

0 commit comments

Comments
 (0)