proposal: add artifact graph core query system (#400)

* proposal: add artifact graph core query system

Add OpenSpec change proposal for Slice 1 of the artifact POC - the core
"What's Ready?" query system. This implements:

- ArtifactGraph class for DAG-based dependency modeling
- Filesystem-based state detection (file existence = completion)
- Topological sort for build order calculation
- Ready/blocked artifact queries

This is a parallel module that will coexist with the current system.

* docs: specify Zod for schema validation in artifact graph proposal

- Add decision section for Zod schema validation in design.md
- Update data structures to show Zod schemas with z.infer<> types
- Update tasks to specify Zod usage for type definitions and parsing

* docs: add 2-level schema resolution and built-in schemas

- Add decision for global → built-in schema resolution pattern
- Add resolver.ts for schema lookup logic
- Add built-in schemas directory (spec-driven.yaml, tdd.yaml)
- Add schema resolution tests
- Follows ESLint/Prettier/Git patterns (defaults baked in package)

* experiment: add vertical slice version of artifact graph change

Creates add-artifact-graph-core-v2 with requirements organized as
vertical slices - each requirement file contains its spec, design
decisions, and tasks bundled together for comparison.

* feat(core): add getGlobalDataDir for XDG-compliant data directory

Add getGlobalDataDir() function following XDG Base Directory Specification
for storing user data like schema overrides:
- XDG_DATA_HOME takes precedence on all platforms
- Unix/macOS fallback: ~/.local/share/openspec/
- Windows fallback: %LOCALAPPDATA%/openspec/

* feat(artifact-graph): add core dependency graph module

Implement Slice 1 ("What's Ready?") of the artifact graph system:

- types.ts: Zod schemas for artifact definitions with derived TypeScript types
- schema.ts: YAML parsing with validation for duplicates, invalid refs, cycles
- graph.ts: ArtifactGraph class with Kahn's algorithm for topological sort
- state.ts: Filesystem-based completion detection with glob pattern support
- resolver.ts: Two-level schema resolution (global override → built-in)
- builtin-schemas.ts: spec-driven and tdd workflow definitions

Key design decisions:
- Filesystem as database (stateless, git-friendly)
- Cycle errors show full path (e.g., "A → B → C → A")
- Deterministic ordering via sorted queues

* test(artifact-graph): add comprehensive test suite

52 tests covering all artifact-graph functionality:

- schema.test.ts: Parsing, validation errors, cycle detection
- graph.test.ts: Build order, ready artifacts, blocked queries
- state.test.ts: File existence, glob patterns, missing directories
- resolver.test.ts: Schema resolution with global overrides

* docs(openspec): archive add-artifact-graph-core change

Archive completed change proposal and create artifact-graph spec with
6 requirements covering schema loading, build order, state detection,
ready queries, completion checks, and blocked queries.

* chore: remove experimental artifact-graph-core-v2 folder

Clean up experimental vertical slice proposal that is no longer needed.

* feat(artifact-graph): validate global schema overrides

Global schema overrides are now validated through the same pipeline as
built-in schemas, catching invalid schemas, cyclic dependencies, and
invalid requires references at load time. Added SchemaLoadError for
better error context with file paths.

* test(artifact-graph): add workflow integration tests

Add end-to-end integration tests that exercise the full artifact-graph
pipeline: resolveSchema → ArtifactGraph → detectCompleted → queries.

Tests cover:
- Complete spec-driven and tdd workflow progressions
- Out-of-order file creation handling
- Glob pattern matching with multiple files
- Build order consistency
- Edge cases (empty/missing directories, non-matching files)

* refactor(artifact-graph): adopt zod v4 error message format

Update custom error messages from string format to zod v4 object format
using `{ error: 'message' }` convention.

* fix(test): prevent hanging vitest threads after test runs

- Add teardownTimeout (3s) to vitest config for forced cleanup
- Add global teardown function to vitest.setup.ts
- Call child.unref() to prevent child processes from blocking event loop
- Explicitly destroy stdio streams on process close/error

Author: TabishB

openspec/changes/archive/2025-12-24-add-artifact-graph-core/design.md

@@ -0,0 +1,197 @@
+## Context
+
+This implements "Slice 1: What's Ready?" from the artifact POC analysis. The core insight is using the filesystem as a database - artifact completion is detected by file existence, making the system stateless and version-control friendly.
+
+This module will coexist with the current OpenSpec system as a parallel capability, potentially enabling future migration or integration.
+
+## Goals / Non-Goals
+
+**Goals:**
+- Pure dependency graph logic with no side effects
+- Stateless state detection (rescan filesystem each query)
+- Support glob patterns for multi-file artifacts (e.g., `specs/*.md`)
+- Load artifact definitions from YAML schemas
+- Calculate topological build order
+- Determine "ready" artifacts based on dependency completion
+
+**Non-Goals:**
+- CLI commands (Slice 4)
+- Multi-change management (Slice 2)
+- Template resolution and enrichment (Slice 3)
+- Agent integration or Claude commands
+- Replacing existing OpenSpec functionality
+
+## Decisions
+
+### Decision: Filesystem as Database
+Use file existence for state detection rather than a separate state file.
+
+**Rationale:**
+- Stateless - no state corruption possible
+- Git-friendly - state derived from committed files
+- Simple - no sync issues between state file and actual files
+
+**Alternatives considered:**
+- JSON/SQLite state file: More complex, sync issues, not git-friendly
+- Git metadata: Too coupled to git, complex implementation
+
+### Decision: Kahn's Algorithm for Topological Sort
+Use Kahn's algorithm for computing build order.
+
+**Rationale:**
+- Well-understood, O(V+E) complexity
+- Naturally detects cycles during execution
+- Produces a stable, deterministic order
+
+### Decision: Glob Pattern Support
+Support glob patterns like `specs/*.md` in artifact `generates` field.
+
+**Rationale:**
+- Allows multiple files to satisfy a single artifact requirement
+- Common pattern for spec directories with multiple files
+- Uses standard glob syntax
+
+### Decision: Immutable Completed Set
+Represent completion state as an immutable Set of completed artifact IDs.
+
+**Rationale:**
+- Functional style, easier to reason about
+- State derived fresh each query, no mutation needed
+- Clear separation between graph structure and runtime state
+- Filesystem can only detect binary existence (complete vs not complete)
+
+**Note:** `inProgress` and `failed` states are deferred to future slices. They would require external state tracking (e.g., a status file) since file existence alone cannot distinguish these states.
+
+### Decision: Zod for Schema Validation
+Use Zod for validating YAML schema structure and deriving TypeScript types.
+
+**Rationale:**
+- Already a project dependency (v4.0.17) used in `src/core/schemas/`
+- Type inference via `z.infer<>` - single source of truth for types
+- Runtime validation with detailed error messages
+- Consistent with existing project patterns (`base.schema.ts`, `config-schema.ts`)
+
+**Alternatives considered:**
+- Manual validation: More code, error-prone, no type inference
+- JSON Schema: Would require additional dependency, less TypeScript integration
+- io-ts: Not already in project, steeper learning curve
+
+### Decision: Two-Level Schema Resolution
+Schemas resolve from global user data directory, falling back to package built-ins.
+
+**Resolution order:**
+1. `${XDG_DATA_HOME:-~/.local/share}/openspec/schemas/<name>.yaml` - Global user override
+2. `<package>/schemas/<name>.yaml` - Built-in defaults
+
+**Rationale:**
+- Follows XDG Base Directory Specification (schemas are data, not config)
+- Mirrors existing `getGlobalConfigDir()` pattern in `src/core/global-paths.ts`
+- Built-ins baked into package, never auto-copied
+- Users customize by creating files in global data dir
+- Simple - no project-level overrides (can add later if needed)
+
+**XDG compliance:**
+- Uses `XDG_DATA_HOME` env var when set (all platforms)
+- Unix/macOS fallback: `~/.local/share/openspec/`
+- Windows fallback: `%LOCALAPPDATA%/openspec/`
+
+**Alternatives considered:**
+- Project-level overrides: Added complexity, not needed initially
+- Auto-copy to user space: Creates drift, harder to update defaults
+- Config directory (`XDG_CONFIG_HOME`): Schemas are workflow definitions (data), not user preferences (config)
+
+### Decision: Template Field Parsed But Not Resolved
+The `template` field is required in schema YAML for completeness, but template resolution is deferred to Slice 3.
+
+**Rationale:**
+- Slice 1 focuses on "What's Ready?" - dependency and completion queries only
+- Template paths are validated syntactically (non-empty string) but not resolved
+- Keeps Slice 1 focused and independently testable
+
+### Decision: Cycle Error Format
+Cycle errors list all artifact IDs in the cycle for easy debugging.
+
+**Format:** `"Cyclic dependency detected: A → B → C → A"`
+
+**Rationale:**
+- Shows the full cycle path, not just that a cycle exists
+- Actionable - developer can see exactly which artifacts to fix
+- Consistent with Kahn's algorithm which naturally identifies cycle participants
+
+## Data Structures
+
+**Zod Schemas (source of truth):**
+
+```typescript
+import { z } from 'zod';
+
+// Artifact definition schema
+export const ArtifactSchema = z.object({
+  id: z.string().min(1, 'Artifact ID is required'),
+  generates: z.string().min(1),      // e.g., "proposal.md" or "specs/*.md"
+  description: z.string(),
+  template: z.string(),              // path to template file
+  requires: z.array(z.string()).default([]),
+});
+
+// Full schema YAML structure
+export const SchemaYamlSchema = z.object({
+  name: z.string().min(1, 'Schema name is required'),
+  version: z.number().int().positive(),
+  description: z.string().optional(),
+  artifacts: z.array(ArtifactSchema).min(1, 'At least one artifact required'),
+});
+
+// Derived TypeScript types
+export type Artifact = z.infer<typeof ArtifactSchema>;
+export type SchemaYaml = z.infer<typeof SchemaYamlSchema>;
+```
+
+**Runtime State (not Zod - internal only):**
+
+```typescript
+// Slice 1: Simple completion tracking via filesystem
+type CompletedSet = Set<string>;
+
+// Return type for blocked query
+interface BlockedArtifacts {
+  [artifactId: string]: string[];  // artifact → list of unmet dependencies
+}
+
+interface ArtifactGraphResult {
+  completed: string[];
+  ready: string[];
+  blocked: BlockedArtifacts;
+  buildOrder: string[];
+}
+```
+
+## File Structure
+
+```
+src/core/artifact-graph/
+├── index.ts           # Public exports
+├── types.ts           # Zod schemas and type definitions
+├── graph.ts           # ArtifactGraph class
+├── state.ts           # State detection logic
+├── resolver.ts        # Schema resolution (global → built-in)
+└── schemas/           # Built-in schema definitions (package level)
+    ├── spec-driven.yaml   # Default: proposal → specs → design → tasks
+    └── tdd.yaml           # Alternative: tests → implementation → docs
+```
+
+**Schema Resolution Paths:**
+- Global user override: `${XDG_DATA_HOME:-~/.local/share}/openspec/schemas/<name>.yaml`
+- Package built-in: `src/core/artifact-graph/schemas/<name>.yaml` (bundled with package)
+
+## Risks / Trade-offs
+
+| Risk | Mitigation |
+|------|------------|
+| Glob pattern edge cases | Use well-tested glob library (fast-glob or similar) |
+| Cycle detection | Kahn's algorithm naturally fails on cycles; provide clear error |
+| Schema evolution | Version field in schema, validate on load |
+
+## Open Questions
+
+None - all questions resolved in Decisions section.

openspec/changes/archive/2025-12-24-add-artifact-graph-core/proposal.md

@@ -0,0 +1,18 @@
+## Why
+
+The current OpenSpec system relies on conventions and AI inference for artifact ordering. A formal artifact graph with dependency awareness would enable deterministic "what's ready?" queries, making the system more predictable and enabling future features like automated pipeline execution.
+
+## What Changes
+
+- Add `ArtifactGraph` class to model artifacts as a DAG with dependency relationships
+- Add `ArtifactState` type to track completion status (completed, in_progress, failed)
+- Add filesystem-based state detection using file existence and glob patterns
+- Add schema YAML parser to load artifact definitions
+- Implement topological sort (Kahn's algorithm) for build order calculation
+- Add `getNextArtifacts()` to find artifacts ready for creation
+
+## Impact
+
+- Affected specs: New `artifact-graph` capability
+- Affected code: `src/core/artifact-graph/` (new directory)
+- No changes to existing functionality - this is a parallel module

openspec/changes/archive/2025-12-24-add-artifact-graph-core/specs/artifact-graph/spec.md

@@ -0,0 +1,103 @@
+## ADDED Requirements
+
+### Requirement: Schema Loading
+The system SHALL load artifact graph definitions from YAML schema files.
+
+#### Scenario: Valid schema loaded
+- **WHEN** a valid schema YAML file is provided
+- **THEN** the system returns an ArtifactGraph with all artifacts and dependencies
+
+#### Scenario: Invalid schema rejected
+- **WHEN** a schema YAML file is missing required fields
+- **THEN** the system throws an error with a descriptive message
+
+#### Scenario: Cyclic dependencies detected
+- **WHEN** a schema contains cyclic artifact dependencies
+- **THEN** the system throws an error listing the artifact IDs in the cycle
+
+#### Scenario: Invalid dependency reference
+- **WHEN** an artifact's `requires` array references a non-existent artifact ID
+- **THEN** the system throws an error identifying the invalid reference
+
+#### Scenario: Duplicate artifact IDs rejected
+- **WHEN** a schema contains multiple artifacts with the same ID
+- **THEN** the system throws an error identifying the duplicate
+
+### Requirement: Build Order Calculation
+The system SHALL compute a valid topological build order for artifacts.
+
+#### Scenario: Linear dependency chain
+- **WHEN** artifacts form a linear chain (A → B → C)
+- **THEN** getBuildOrder() returns [A, B, C]
+
+#### Scenario: Diamond dependency
+- **WHEN** artifacts form a diamond (A → B, A → C, B → D, C → D)
+- **THEN** getBuildOrder() returns A before B and C, and D last
+
+#### Scenario: Independent artifacts
+- **WHEN** artifacts have no dependencies
+- **THEN** getBuildOrder() returns them in a stable order
+
+### Requirement: State Detection
+The system SHALL detect artifact completion state by scanning the filesystem.
+
+#### Scenario: Simple file exists
+- **WHEN** an artifact generates "proposal.md" and the file exists
+- **THEN** the artifact is marked as completed
+
+#### Scenario: Simple file missing
+- **WHEN** an artifact generates "proposal.md" and the file does not exist
+- **THEN** the artifact is not marked as completed
+
+#### Scenario: Glob pattern with files
+- **WHEN** an artifact generates "specs/*.md" and the specs/ directory contains .md files
+- **THEN** the artifact is marked as completed
+
+#### Scenario: Glob pattern empty
+- **WHEN** an artifact generates "specs/*.md" and the specs/ directory is empty or missing
+- **THEN** the artifact is not marked as completed
+
+#### Scenario: Missing change directory
+- **WHEN** the change directory does not exist
+- **THEN** all artifacts are marked as not completed (empty state)
+
+### Requirement: Ready Artifact Query
+The system SHALL identify which artifacts are ready to be created based on dependency completion.
+
+#### Scenario: Root artifacts ready initially
+- **WHEN** no artifacts are completed
+- **THEN** getNextArtifacts() returns artifacts with no dependencies
+
+#### Scenario: Dependent artifact becomes ready
+- **WHEN** an artifact's dependencies are all completed
+- **THEN** getNextArtifacts() includes that artifact
+
+#### Scenario: Blocked artifacts excluded
+- **WHEN** an artifact has uncompleted dependencies
+- **THEN** getNextArtifacts() does not include that artifact
+
+### Requirement: Completion Check
+The system SHALL determine when all artifacts in a graph are complete.
+
+#### Scenario: All complete
+- **WHEN** all artifacts in the graph are in the completed set
+- **THEN** isComplete() returns true
+
+#### Scenario: Partially complete
+- **WHEN** some artifacts in the graph are not completed
+- **THEN** isComplete() returns false
+
+### Requirement: Blocked Query
+The system SHALL identify which artifacts are blocked and return all their unmet dependencies.
+
+#### Scenario: Artifact blocked by single dependency
+- **WHEN** artifact B requires artifact A and A is not complete
+- **THEN** getBlocked() returns `{ B: ['A'] }`
+
+#### Scenario: Artifact blocked by multiple dependencies
+- **WHEN** artifact C requires A and B, and only A is complete
+- **THEN** getBlocked() returns `{ C: ['B'] }`
+
+#### Scenario: Artifact blocked by all dependencies
+- **WHEN** artifact C requires A and B, and neither is complete
+- **THEN** getBlocked() returns `{ C: ['A', 'B'] }`

openspec/changes/archive/2025-12-24-add-artifact-graph-core/tasks.md

@@ -0,0 +1,61 @@
+## 1. Type Definitions
+- [x] 1.1 Create `src/core/artifact-graph/types.ts` with Zod schemas (`ArtifactSchema`, `SchemaYamlSchema`) and inferred types via `z.infer<>`
+- [x] 1.2 Define `CompletedSet` (Set<string>), `BlockedArtifacts`, and `ArtifactGraphResult` types for runtime state
+
+## 2. Schema Parser
+- [x] 2.1 Create `src/core/artifact-graph/schema.ts` with YAML loading and Zod validation via `.safeParse()`
+- [x] 2.2 Implement dependency reference validation (ensure `requires` references valid artifact IDs)
+- [x] 2.3 Implement duplicate artifact ID detection
+- [x] 2.4 Add cycle detection during schema load (error format: "Cyclic dependency detected: A → B → C → A")
+
+## 3. Artifact Graph Core
+- [x] 3.1 Create `src/core/artifact-graph/graph.ts` with ArtifactGraph class
+- [x] 3.2 Implement `fromYaml(path)` - load graph from schema file
+- [x] 3.3 Implement `getBuildOrder()` - topological sort via Kahn's algorithm
+- [x] 3.4 Implement `getArtifact(id)` - retrieve single artifact definition
+- [x] 3.5 Implement `getAllArtifacts()` - list all artifacts
+
+## 4. State Detection
+- [x] 4.1 Create `src/core/artifact-graph/state.ts` with state detection logic
+- [x] 4.2 Implement file existence checking for simple paths
+- [x] 4.3 Implement glob pattern matching for multi-file artifacts
+- [x] 4.4 Implement `detectCompleted(graph, changeDir)` - scan filesystem and return CompletedSet
+- [x] 4.5 Handle missing changeDir gracefully (return empty CompletedSet)
+
+## 5. Ready Calculation
+- [x] 5.1 Implement `getNextArtifacts(graph, completed)` - find artifacts with all deps completed
+- [x] 5.2 Implement `isComplete(graph, completed)` - check if all artifacts done
+- [x] 5.3 Implement `getBlocked(graph, completed)` - return BlockedArtifacts map (artifact → unmet deps)
+
+## 6. Schema Resolution
+- [x] 6.1 Create `src/core/artifact-graph/resolver.ts` with schema resolution logic
+- [x] 6.2 Add `getGlobalDataDir()` to `src/core/global-config.ts` (XDG_DATA_HOME with platform fallbacks)
+- [x] 6.3 Implement `resolveSchema(name)` - global (`${XDG_DATA_HOME}/openspec/schemas/`) → built-in fallback
+
+## 7. Built-in Schemas
+- [x] 7.1 Create `src/core/artifact-graph/schemas/spec-driven.yaml` (default: proposal → specs → design → tasks)
+- [x] 7.2 Create `src/core/artifact-graph/schemas/tdd.yaml` (alternative: tests → implementation → docs)
+
+## 8. Integration
+- [x] 8.1 Create `src/core/artifact-graph/index.ts` with public exports
+
+## 9. Testing
+- [x] 9.1 Test: Parse valid schema YAML returns correct artifact graph
+- [x] 9.2 Test: Parse invalid schema (missing fields) throws descriptive error
+- [x] 9.3 Test: Duplicate artifact IDs throws error
+- [x] 9.4 Test: Invalid `requires` reference throws error identifying the invalid ID
+- [x] 9.5 Test: Cycle in schema throws error listing cycle path (e.g., "A → B → C → A")
+- [x] 9.6 Test: Compute build order returns correct topological ordering (linear chain)
+- [x] 9.7 Test: Compute build order handles diamond dependencies correctly
+- [x] 9.8 Test: Independent artifacts return in stable order
+- [x] 9.9 Test: Empty/missing changeDir returns empty CompletedSet
+- [x] 9.10 Test: File existence marks artifact as completed
+- [x] 9.11 Test: Glob pattern specs/*.md detected as complete when files exist
+- [x] 9.12 Test: Glob pattern with empty directory not marked complete
+- [x] 9.13 Test: getNextArtifacts returns only root artifacts when nothing completed
+- [x] 9.14 Test: getNextArtifacts includes artifact when all deps completed
+- [x] 9.15 Test: getBlocked returns artifact with all unmet dependencies listed
+- [x] 9.16 Test: isComplete() returns true when all artifacts completed
+- [x] 9.17 Test: isComplete() returns false when some artifacts incomplete
+- [x] 9.18 Test: Schema resolution finds global override before built-in
+- [x] 9.19 Test: Schema resolution falls back to built-in when no global