/harness-audit is not usable for consumer projects

[ymdvsymd]

Problem 1: /harness-audit only audits the ECC repo itself

In v1.9.0, /harness-audit was made deterministic via scripts/harness-audit.js (#524). However, REPO_ROOT was hardcoded to path.join(__dirname, '..'), meaning the script always audited the ECC repository regardless of where it was invoked.

When a consumer project user runs /harness-audit, it scores the ECC plugin installation — not their project.

Problem 2: Even with AUDIT_ROOT fix, it's useless for consumer projects

We submitted PR #978 to fix Problem 1 by changing REPO_ROOT to process.cwd() with an AUDIT_ROOT env var override. After applying this fix locally and running the audit on a consumer project, the result was 0/70 — all 26 checks failed.

The only way to reach 70/70 was to copy ECC repo files into the project's .claude/ directory. We did this and achieved 70/70 in 2 iterations, but then discovered:

1. All copied files duplicate the plugin. Agents, skills, commands, hooks, and scripts already exist at ~/.claude/ via the ECC plugin installation. Copying them into .claude/ creates redundant duplicates.
2. Hooks fire twice. Claude Code merges hooks from both user-level (~/.claude/) and project-level (.claude/) locations. Identical hook definitions in both locations cause double execution.
3. Some files serve no purpose. For example, package.json exists only to pass the quality-ci-validations check (which looks for validate-commands.js and tests/run-all.js in the test script). No one runs cd .claude && npm test in a consumer project.
4. The score is meaningless. 70/70 just means "you copied ECC's files" — it says nothing about the project's actual harness quality.

We closed PR #978 as a result.

Root cause

All 26 checks in scripts/harness-audit.js are file-existence checks against ECC's own repo structure:

{ pass: countFiles('agents', '.md') >= 10 }        // consumer gets these from plugin
{ pass: countFiles('skills', 'SKILL.md') >= 20 }   // consumer gets these from plugin  
{ pass: fileExists('scripts/doctor.js') }            // ECC internal tooling
{ pass: packageJson.scripts.test.includes('validate-commands.js') } // ECC CI chain

The script measures "does this directory look like the ECC repo?" — not "is this project's harness well-configured?"

What consumer projects actually need

A useful audit for consumer projects would check:

• Are project-specific hooks and agents correctly configured?
• Is CI/CD integrated with harness quality gates?
• Are security guardrails appropriate for the project's requirements?
• Are there custom skills or overrides that extend the plugin defaults?

None of these can be measured by checking whether ECC's own files exist.

Proposed solutions

Option A: Consumer-aware scoring

Detect whether the target is the ECC repo or a consumer project (e.g., check package.json name). Use different check sets for each.

Option B: Separate commands

• /harness-audit — deterministic ECC self-diagnostic (current behavior)
• /harness-review — LLM-driven analysis adapted to the target project (restore v1.8.0's adaptive capability as a distinct command)

Option C: Hybrid approach

Run the deterministic script for baseline scoring, then allow the LLM to perform additional project-specific analysis and recommendations.

Context

• v1.8.0's prompt-based audit could adapt to individual repos (e.g., creating custom GitHub Actions CI), but this was lost in the v1.9.0 deterministic rewrite
• The harness-optimizer agent calls /harness-audit as its first step, so this limitation cascades to that agent as well