Skip to content

Latest commit

 

History

History
247 lines (182 loc) · 11.4 KB

spec.md

File metadata and controls

247 lines (182 loc) · 11.4 KB

cli-validate Specification

Purpose

TBD - created by archiving change improve-validate-error-messages. Update Purpose after archive.

Requirements

Requirement: Validation SHALL provide actionable remediation steps

Validation output SHALL include specific guidance to fix each error, including expected structure, example headers, and suggested commands to verify fixes.

Scenario: No deltas found in change

  • WHEN validating a change with zero parsed deltas
  • THEN show error "No deltas found" with guidance:
    • Explain that change specs must include ## ADDED Requirements, ## MODIFIED Requirements, ## REMOVED Requirements, or ## RENAMED Requirements
    • Remind authors that files must live under openspec/changes/{id}/specs/<capability>/spec.md
    • Include an explicit note: "Spec delta files cannot start with titles before the operation headers"
    • Suggest running openspec change show {id} --json --deltas-only for debugging

Scenario: Missing required sections

  • WHEN a required section is missing
  • THEN include expected header names and a minimal skeleton:
    • For Spec: ## Purpose, ## Requirements
    • For Change: ## Why, ## What Changes
    • Provide an example snippet of the missing section with placeholder prose ready to copy
    • Mention the quick-reference section in openspec/AGENTS.md as the authoritative template

Scenario: Missing requirement descriptive text

  • WHEN a requirement header lacks descriptive text before scenarios
  • THEN emit an error explaining that ### Requirement: lines must be followed by narrative text before any #### Scenario: headers
    • Show compliant example: "### Requirement: Foo" followed by "The system SHALL ..."
    • Suggest adding 1-2 sentences describing the normative behavior prior to listing scenarios
    • Reference the pre-validation checklist in openspec/AGENTS.md

Requirement: Validator SHALL detect likely misformatted scenarios and warn with a fix

The validator SHALL recognize bulleted lines that look like scenarios (e.g., lines beginning with WHEN/THEN/AND) and emit a targeted warning with a conversion example to #### Scenario:.

Scenario: Bulleted WHEN/THEN under a Requirement

  • WHEN bullets that start with WHEN/THEN/AND are found under a requirement without any #### Scenario: headers
  • THEN emit warning: "Scenarios must use '#### Scenario:' headers", and show a conversion template:
#### Scenario: Short name
- **WHEN** ...
- **THEN** ...
- **AND** ...

Requirement: All issues SHALL include file paths and structured locations

Error, warning, and info messages SHALL include:

  • Source file path (openspec/changes/{id}/proposal.md, .../specs/{cap}/spec.md)
  • Structured path (e.g., deltas[0].requirements[0].scenarios)

Scenario: Zod validation error

  • WHEN a schema validation fails
  • THEN the message SHALL include file, path, and a remediation hint if applicable

Requirement: Invalid results SHALL include a Next steps footer in human-readable output

The CLI SHALL append a Next steps footer when the item is invalid and not using --json, including:

  • Summary line with counts
  • Top-3 guidance bullets (contextual to the most frequent or blocking errors)
  • A suggestion to re-run with --json and/or the debug command

Scenario: Change invalid summary

  • WHEN a change validation fails
  • THEN print "Next steps" with 2-3 targeted bullets and suggest openspec change show <id> --json --deltas-only

Requirement: Top-level validate command

The CLI SHALL provide a top-level validate command for validating changes and specs with flexible selection options.

Scenario: Interactive validation selection

  • WHEN executing openspec validate without arguments
  • THEN prompt user to select what to validate (all, changes, specs, or specific item)
  • AND perform validation based on selection
  • AND display results with appropriate formatting

Scenario: Non-interactive environments do not prompt

  • GIVEN stdin is not a TTY or --no-interactive is provided or environment variable OPEN_SPEC_INTERACTIVE=0
  • WHEN executing openspec validate without arguments
  • THEN do not prompt interactively
  • AND print a helpful hint listing available commands/flags and exit with code 1

Scenario: Direct item validation

  • WHEN executing openspec validate <item-name>
  • THEN automatically detect if item is a change or spec
  • AND validate the specified item
  • AND display validation results

Requirement: Bulk and filtered validation

The validate command SHALL support flags for bulk validation (--all) and filtered validation by type (--changes, --specs).

Scenario: Validate everything

  • WHEN executing openspec validate --all
  • THEN validate all changes in openspec/changes/ (excluding archive)
  • AND validate all specs in openspec/specs/
  • AND display a summary showing passed/failed items
  • AND exit with code 1 if any validation fails

Scenario: Scope of bulk validation

  • WHEN validating with --all or --changes

  • THEN include all change proposals under openspec/changes/

  • AND exclude the openspec/changes/archive/ directory

  • WHEN validating with --specs

  • THEN include all specs that have a spec.md under openspec/specs/<id>/spec.md

Scenario: Validate all changes

  • WHEN executing openspec validate --changes
  • THEN validate all changes in openspec/changes/ (excluding archive)
  • AND display results for each change
  • AND show summary statistics

Scenario: Validate all specs

  • WHEN executing openspec validate --specs
  • THEN validate all specs in openspec/specs/
  • AND display results for each spec
  • AND show summary statistics

Requirement: Validation options and progress indication

The validate command SHALL support standard validation options (--strict, --json) and display progress during bulk operations.

Scenario: Strict validation

  • WHEN executing openspec validate --all --strict
  • THEN apply strict validation to all items
  • AND treat warnings as errors
  • AND fail if any item has warnings or errors

Scenario: JSON output

  • WHEN executing openspec validate --all --json
  • THEN output validation results as JSON
  • AND include detailed issues for each item
  • AND include summary statistics

Scenario: JSON output schema for bulk validation

  • WHEN executing openspec validate --all --json (or --changes / --specs)
  • THEN output a JSON object with the following shape:
    • items: Array of objects with fields { id: string, type: "change"|"spec", valid: boolean, issues: Issue[], durationMs: number }
    • summary: Object { totals: { items: number, passed: number, failed: number }, byType: { change?: { items: number, passed: number, failed: number }, spec?: { items: number, passed: number, failed: number } } }
    • version: String identifier for the schema (e.g., "1.0")
  • AND exit with code 1 if any items[].valid === false

Where Issue follows the existing per-item validation report shape { level: "ERROR"|"WARNING"|"INFO", path: string, message: string }.

Scenario: Show validation progress

  • WHEN validating multiple items (--all, --changes, or --specs)
  • THEN show progress indicator or status updates
  • AND indicate which item is currently being validated
  • AND display running count of passed/failed items

Scenario: Concurrency limits for performance

  • WHEN validating multiple items
  • THEN run validations with a bounded concurrency (e.g., 4–8 in parallel)
  • AND ensure progress indicators remain responsive

Requirement: Item type detection and ambiguity handling

The validate command SHALL handle ambiguous names and explicit type overrides to ensure clear, deterministic behavior.

Scenario: Direct item validation with automatic type detection

  • WHEN executing openspec validate <item-name>
  • THEN if <item-name> uniquely matches a change or a spec, validate that item

Scenario: Ambiguity between change and spec names

  • GIVEN <item-name> exists both as a change and as a spec
  • WHEN executing openspec validate <item-name>
  • THEN print an ambiguity error explaining both matches
  • AND suggest passing --type change or --type spec, or using openspec change validate / openspec spec validate
  • AND exit with code 1 without performing validation

Scenario: Unknown item name

  • WHEN the <item-name> matches neither a change nor a spec
  • THEN print a not-found error
  • AND show nearest-match suggestions when available
  • AND exit with code 1

Scenario: Explicit type override

  • WHEN executing openspec validate --type change <item>

  • THEN treat <item> as a change ID and validate it (skipping auto-detection)

  • WHEN executing openspec validate --type spec <item>

  • THEN treat <item> as a spec ID and validate it (skipping auto-detection)

Requirement: Interactivity controls

  • The CLI SHALL respect --no-interactive to disable prompts.
  • The CLI SHALL respect OPEN_SPEC_INTERACTIVE=0 to disable prompts globally.
  • Interactive prompts SHALL only be shown when stdin is a TTY and interactivity is not disabled.

Scenario: Disabling prompts via flags or environment

  • WHEN openspec validate is executed with --no-interactive or with environment OPEN_SPEC_INTERACTIVE=0
  • THEN the CLI SHALL not display interactive prompts
  • AND SHALL print non-interactive hints or chosen outputs as appropriate

Requirement: Parser SHALL handle cross-platform line endings

The markdown parser SHALL correctly identify sections regardless of line ending format (LF, CRLF, CR).

Scenario: Required sections parsed with CRLF line endings

  • GIVEN a change proposal markdown saved with CRLF line endings
  • AND the document contains ## Why and ## What Changes
  • WHEN running openspec validate <change-id>
  • THEN validation SHALL recognize the sections and NOT raise parsing errors

Requirement: SHALL/MUST validation as WARNING for internationalization

The validator SHALL check for SHALL/MUST keywords in requirement text and emit a WARNING (not ERROR) to recommend RFC 2119 compliance while allowing non-English documentation.

Scenario: Requirement without SHALL/MUST in normal mode

  • WHEN validating a requirement that lacks SHALL or MUST keywords
  • THEN emit a WARNING-level issue with message "Requirement '[name]' should contain SHALL or MUST keyword (RFC 2119 best practice)"
  • AND validation SHALL pass (not fail)
  • AND the requirement SHALL be considered valid

Scenario: Requirement without SHALL/MUST in strict mode

  • WHEN validating with --strict flag
  • AND a requirement lacks SHALL or MUST keywords
  • THEN emit a WARNING-level issue
  • AND validation SHALL fail because --strict treats warnings as errors
  • AND exit code SHALL be 1

Scenario: Non-English requirement text

  • GIVEN a requirement written in a non-English language (e.g., Chinese, Japanese, Spanish)
  • AND the requirement does not contain SHALL or MUST keywords
  • WHEN validating the requirement
  • THEN emit a WARNING recommending SHALL/MUST for English specs
  • AND validation SHALL pass
  • AND users can proceed without blockers

Scenario: English requirement with SHALL/MUST

  • WHEN validating a requirement that contains SHALL or MUST
  • THEN no warning SHALL be emitted
  • AND validation passes cleanly