Skip to content

Skill Doctor

Actions
Scan agent skills for static quality issues
v0.1.0
Latest
Star (26)
Skill Doctor

CI license

Let coding agents diagnose agent skills before they break routing, onboarding, or evals.

skill-doctor scans local skill packages for frontmatter issues, broken resource references, weak trigger descriptions, missing workflow guidance, and malformed evals/evals.json, then turns the result into a 0-100 score with actionable findings.

How it works

skill-doctor walks a single skill root or a whole workspace, discovers every SKILL.md, and runs a static analysis pass over:

  1. Spec and metadata: frontmatter presence, YAML validity, name, description, and basic compatibility with common skill conventions.
  2. Bundle integrity: broken local links, references that escape the skill root, empty helper files, and missing resource files.
  3. Trigger quality: whether the description clearly says what the skill does and when it should trigger.
  4. Instruction quality: whether the body provides enough workflow or usage guidance to be actionable.
  5. Eval hygiene: optional validation for evals/evals.json, including schema shape, duplicate IDs, missing input files, and mismatched skill names.

The scoring model is intentionally conservative in default mode. Strong real-world skills should score cleanly or near-cleanly. Stricter guidance is available through --strictness strict and --strictness pedantic.

Install

Run this at the root of a skill or skill workspace:

npx -y skill-doctor@latest .

Show affected files and line numbers:

npx -y skill-doctor@latest . --verbose

Get just the numeric score:

npx -y skill-doctor@latest . --score

Machine-readable output:

npx -y skill-doctor@latest . --format json

Example output

skill doctor                                        99 excellent
static diagnostics for agent skills
metadata • bundle integrity • trigger quality • eval hygiene

17 skills scanned • 15 healthy • 0 errors • 2 warnings • 28ms
████████████████████████

workspace overview
name                   score   findings
theme-factory           94   1 warn
webapp-testing          94   1 warn
algorithmic-art        100   clean

GitHub Actions

- uses: actions/checkout@v5
- uses: marian2js/skill-doctor@main
  with:
    directory: .
    strictness: default
    fail-on: error
Input Default Description
directory . Skill directory or workspace to scan
verbose true Show file details per finding
fail-on error Exit with error code on error, warning, or none
strictness default Analysis strictness: default, strict, pedantic
node-version 20 Node.js version to use

The action outputs a score value you can use in later workflow steps.

Options

Usage: skill-doctor [directory] [options]

Options:
  -v, --version              display the version number
  --format <format>          output format: text or json
  --fail-on <level>          exit with error code on diagnostics: error, warning, none
  --strictness <level>       analysis strictness: default, strict, pedantic
  --verbose                  show file details per rule
  --score                    output only the score
  -h, --help                 display help for command

Node.js API

You can also use skill-doctor programmatically:

import { diagnose } from "skill-doctor";

const result = await diagnose("/path/to/skills");

console.log(result.score); // { score: 99, label: "Excellent" }
console.log(result.skills); // per-skill breakdown
console.log(result.diagnostics); // flattened findings across the workspace

Calibration

The analyzer was designed with a mix of sources:

The default heuristics were calibrated against Anthropic's public skill corpus so high-quality skills do not get buried in noisy style warnings.

Development

pnpm install
pnpm build
pnpm test
pnpm typecheck
pnpm lint

Run the CLI locally:

node packages/skill-doctor/dist/cli.js /path/to/skills

More contribution details live in CONTRIBUTING.md.

License

MIT, see LICENSE.

Skill Doctor is not certified by GitHub. It is provided by a third-party and is governed by separate terms of service, privacy policy, and support documentation.

About

Scan agent skills for static quality issues
v0.1.0
Latest

Skill Doctor is not certified by GitHub. It is provided by a third-party and is governed by separate terms of service, privacy policy, and support documentation.