Skip to content

marian2js/skill-doctor

Use this GitHub action with your project
Add this Action to an existing workflow or create a new one
View on Marketplace

Repository files navigation

Skill Doctor

CI license

Let coding agents diagnose agent skills before they break routing, onboarding, or evals.

skill-doctor scans local skill packages for frontmatter issues, broken resource references, weak trigger descriptions, missing workflow guidance, and malformed evals/evals.json, then turns the result into a 0-100 score with actionable findings.

How it works

skill-doctor walks a single skill root or a whole workspace, discovers every SKILL.md, and runs a static analysis pass over:

  1. Spec and metadata: frontmatter presence, YAML validity, name, description, and basic compatibility with common skill conventions.
  2. Bundle integrity: broken local links, references that escape the skill root, empty helper files, and missing resource files.
  3. Trigger quality: whether the description clearly says what the skill does and when it should trigger.
  4. Instruction quality: whether the body provides enough workflow or usage guidance to be actionable.
  5. Eval hygiene: optional validation for evals/evals.json, including schema shape, duplicate IDs, missing input files, and mismatched skill names.

The scoring model is intentionally conservative in default mode. Strong real-world skills should score cleanly or near-cleanly. Stricter guidance is available through --strictness strict and --strictness pedantic.

Install

Run this at the root of a skill or skill workspace:

npx -y skill-doctor@latest .

Show affected files and line numbers:

npx -y skill-doctor@latest . --verbose

Get just the numeric score:

npx -y skill-doctor@latest . --score

Machine-readable output:

npx -y skill-doctor@latest . --format json

Example output

skill doctor                                        99 excellent
static diagnostics for agent skills
metadata • bundle integrity • trigger quality • eval hygiene

17 skills scanned • 15 healthy • 0 errors • 2 warnings • 28ms
████████████████████████

workspace overview
name                   score   findings
theme-factory           94   1 warn
webapp-testing          94   1 warn
algorithmic-art        100   clean

GitHub Actions

- uses: actions/checkout@v5
- uses: marian2js/skill-doctor@main
  with:
    directory: .
    strictness: default
    fail-on: error
Input Default Description
directory . Skill directory or workspace to scan
verbose true Show file details per finding
fail-on error Exit with error code on error, warning, or none
strictness default Analysis strictness: default, strict, pedantic
node-version 20 Node.js version to use

The action outputs a score value you can use in later workflow steps.

Options

Usage: skill-doctor [directory] [options]

Options:
  -v, --version              display the version number
  --format <format>          output format: text or json
  --fail-on <level>          exit with error code on diagnostics: error, warning, none
  --strictness <level>       analysis strictness: default, strict, pedantic
  --verbose                  show file details per rule
  --score                    output only the score
  -h, --help                 display help for command

Node.js API

You can also use skill-doctor programmatically:

import { diagnose } from "skill-doctor";

const result = await diagnose("/path/to/skills");

console.log(result.score); // { score: 99, label: "Excellent" }
console.log(result.skills); // per-skill breakdown
console.log(result.diagnostics); // flattened findings across the workspace

Calibration

The analyzer was designed with a mix of sources:

The default heuristics were calibrated against Anthropic's public skill corpus so high-quality skills do not get buried in noisy style warnings.

Development

pnpm install
pnpm build
pnpm test
pnpm typecheck
pnpm lint

Run the CLI locally:

node packages/skill-doctor/dist/cli.js /path/to/skills

More contribution details live in CONTRIBUTING.md.

License

MIT, see LICENSE.

About

Help your agents create better skills

Topics

Resources

License

Contributing

Stars

Watchers

Forks

Packages

 
 
 

Contributors