Merge branch 'main' into copilot/suggest-onboarding-experience-designs

Author: pmcelhaney

.github/issue-proposals/agent-context-reset-endpoint.md

@@ -0,0 +1,43 @@
+---
+title: "Add context reset endpoint to Admin API for agent-friendly state recovery"
+parentIssue: 1720
+labels:
+  - enhancement
+  - agent-ux
+assignees: []
+milestone:
+---
+
+AI agents frequently make state-mutating requests while exploring an API (creating resources, updating fields, etc.). When an agent realises it has reached an inconsistent state, it currently has no way to recover short of restarting the Counterfact process. That restart destroys any session, breaks the agent's tooling loop, and wastes time.
+
+A single HTTP call to reset every context back to its initial state would let agents recover instantly and retry from a known baseline, without restarting.
+
+## Proposed change
+
+Add a new admin API endpoint:
+
+```
+POST /_counterfact/api/contexts/reset
+```
+
+The endpoint should:
+
+1. Remove every path-level context entry that was created or modified since startup (i.e. anything beyond the root `/` context that the `ContextRegistry` adds in its constructor).
+2. Reset the root `/` context to an empty object `{}`.
+3. Return `{ success: true, message: "All contexts reset" }`.
+
+Optionally accept a JSON body `{ "path": "/pets" }` to reset only a single subtree rather than everything.
+
+## Motivation
+
+- Agents need fast, zero-friction feedback loops. A `POST /_counterfact/api/contexts/reset` call is trivially scriptable.
+- The existing `POST /_counterfact/api/contexts/:path` endpoint can **update** a context, but it cannot erase keys or remove path-level entries; a dedicated reset endpoint is required.
+- Human developers working in automated test loops also benefit: each test can start with a clean slate without restarting the server.
+
+## Acceptance criteria
+
+- [ ] `POST /_counterfact/api/contexts/reset` returns `200 { success: true }` and all contexts are restored to their initial state
+- [ ] After a reset, `GET /_counterfact/api/contexts` returns only the root `/` context with an empty object
+- [ ] `POST /_counterfact/api/contexts/reset` with body `{ "path": "/pets" }` resets only the `/pets` subtree and leaves other paths unchanged
+- [ ] The endpoint is protected by the same bearer-token / loopback guard as all other admin API endpoints
+- [ ] Unit tests cover the happy path, the scoped reset, and the auth-guard behaviour

.github/issue-proposals/agent-openapi-json-endpoint.md

@@ -0,0 +1,48 @@
+---
+title: "Serve parsed OpenAPI spec as JSON via Admin API for programmatic route discovery"
+parentIssue: 1720
+labels:
+  - enhancement
+  - agent-ux
+assignees: []
+milestone:
+---
+
+Counterfact already serves the OpenAPI document at `/counterfact/openapi` as YAML so Swagger UI can display it. However, AI agents need to consume the spec programmatically — as a JSON object — without knowing the original file path or performing a YAML-to-JSON conversion themselves.
+
+The Admin API should expose the fully-resolved (bundle-dereferenced) spec as JSON at a stable, well-known endpoint so agents can discover all available routes, schemas, and response codes in a single HTTP call.
+
+## Proposed change
+
+Add a new read-only endpoint to the Admin API:
+
+```
+GET /_counterfact/api/openapi
+```
+
+Response: `Content-Type: application/json`, body is the bundled OpenAPI document as a plain JSON object (identical to what `openapiMiddleware` currently produces, but serialised as JSON instead of YAML).
+
+Optionally, also add:
+
+```
+GET /_counterfact/api/openapi/paths              → just the `paths` object
+GET /_counterfact/api/openapi/paths?route=<path> → schema for a single path item
+                                                    (route is URL-encoded, e.g. route=%2Fpet%2F%7BpetId%7D)
+```
+
+The `route` query parameter avoids the routing ambiguity that would arise from embedding a slash-containing OpenAPI path into URL segments.
+
+## Motivation
+
+- The existing `/counterfact/openapi` endpoint is designed for Swagger UI, which accepts YAML. Agents prefer JSON and parse it without additional dependencies.
+- The Admin API already acts as the programmatic interface; consolidating spec access there keeps the surface area consistent and auth-guarded.
+- Agents building or testing against the mock server can dynamically enumerate valid paths, required parameters, and response schemas — enabling them to generate correct requests on the first attempt rather than guessing.
+
+## Acceptance criteria
+
+- [ ] `GET /_counterfact/api/openapi` returns `200 application/json` with the fully-resolved OpenAPI document
+- [ ] The returned JSON matches the structure produced by `@apidevtools/json-schema-ref-parser`'s `bundle()` call
+- [ ] `GET /_counterfact/api/openapi/paths` returns only the `paths` section of the spec
+- [ ] The endpoint is protected by the same bearer-token / loopback guard as other admin API endpoints
+- [ ] Returns `503` (or a clear error) when Counterfact was started without a spec (`openApiPath === "_"`)
+- [ ] Unit tests cover the happy path, the paths sub-endpoint, and the no-spec case

.github/issue-proposals/agent-prompt-generator-command.md

@@ -0,0 +1,73 @@
+---
+title: "Add `counterfact agent-prompt` command to generate LLM system prompts"
+parentIssue: 1720
+labels:
+  - enhancement
+  - agent-ux
+  - dx
+assignees: []
+milestone:
+---
+
+AI coding agents work best when given a precise system prompt that tells them the capabilities and conventions of the tools they are using. Today there is no built-in way to describe Counterfact — its Admin API endpoints, context model, route conventions, and OpenAPI-driven type system — in a format suitable for injection into an LLM context window.
+
+A `counterfact agent-prompt` CLI command would auto-generate a tailored system prompt from the running server, giving agents everything they need to generate correct route handlers and Admin API calls on the first attempt.
+
+## Proposed change
+
+Add a new sub-command to the CLI:
+
+```bash
+npx counterfact agent-prompt [--url <server-url>] [--format <markdown|json>]
+```
+
+When run, the command:
+
+1. Connects to the Counterfact Admin API at `<server-url>` (default `http://localhost:3100`).
+2. Fetches `/_counterfact/api/openapi`, `/_counterfact/api/routes`, and `/_counterfact/api/config`.
+3. Generates a structured prompt containing:
+   - A short description of Counterfact's purpose and conventions.
+   - The list of registered routes (path + method + summary from the spec).
+   - The Admin API surface (endpoints, request/response shapes).
+   - The context model (how to read and write context via the Admin API).
+   - TypeScript route handler conventions (file structure, `$.context`, response helpers).
+4. Writes the prompt to stdout (for piping into other tools) or to a file with `--output <path>`.
+
+### Example output (abbreviated)
+
+```markdown
+# Counterfact Mock Server — Agent Instructions
+
+## Server info
+- Base URL: http://localhost:3100
+- OpenAPI spec: petstore3
+
+## Available routes
+- GET  /pet/{petId}  → Find pet by ID
+- POST /pet          → Add a new pet
+
+## Admin API
+- GET  /_counterfact/api/health         → Server health
+- POST /_counterfact/api/contexts/reset → Reset all context state
+…
+
+## Writing route handlers
+Route handlers live in `./api/routes/`. Each file exports GET, POST, etc.
+Use `$.context` to read/write shared state. Return `{ status: 200, body: … }`.
+```
+
+## Motivation
+
+- Reduces the number of LLM iterations needed to produce a working route handler from scratch.
+- Makes the Admin API self-describing — an agent can call `agent-prompt`, inject the output into its context window, and immediately start using the Admin API correctly.
+- Keeps the prompt up-to-date automatically: because it is generated from the live server, it always reflects the actual spec and config.
+
+## Acceptance criteria
+
+- [ ] `npx counterfact agent-prompt` prints a Markdown prompt to stdout when a Counterfact server is running locally on the default port
+- [ ] The prompt includes the list of registered routes with HTTP methods and summaries
+- [ ] The prompt includes a description of every Admin API endpoint
+- [ ] `--format json` outputs a machine-readable JSON object with the same information
+- [ ] `--output <path>` writes the prompt to a file instead of stdout
+- [ ] The command exits with a non-zero code and a clear error message when the server is not reachable
+- [ ] The command is documented in `docs/usage.md` under a new "Using with AI agents" section

.github/issue-proposals/agent-request-history-endpoint.md

@@ -0,0 +1,65 @@
+---
+title: "Expose request/response history via Admin API for agent introspection"
+parentIssue: 1720
+labels:
+  - enhancement
+  - agent-ux
+assignees: []
+milestone:
+---
+
+AI agents need to understand what happened during a sequence of calls in order to reason about correctness and debug failures. Currently the only way to see request/response pairs is to tail the debug log, which is not machine-readable and is not available via HTTP.
+
+Adding a request-history endpoint to the Admin API gives agents a reliable, structured view of recent interactions without requiring external tooling.
+
+## Proposed change
+
+1. Add an in-memory ring buffer (e.g. last 100 entries, configurable via a `--history-size` flag or a field in `Config`) to the Koa dispatcher. After each request is handled, append a record:
+
+```ts
+interface ValidationError {
+  location: "query" | "header" | "path" | "body";
+  name?: string;   // parameter name (for query/header/path)
+  path?: string;   // JSON Pointer (for body fields, e.g. "/price")
+  message: string; // human-readable description
+}
+
+interface HistoryEntry {
+  id: number;                  // monotonically increasing
+  timestamp: string;           // ISO-8601
+  method: string;
+  path: string;
+  query: Record<string, string>;
+  requestHeaders: Record<string, string>;
+  requestBody: unknown;
+  statusCode: number;
+  responseHeaders: Record<string, string>;
+  responseBody: unknown;
+  durationMs: number;
+  validationErrors: ValidationError[];  // structured errors from the request-validator
+}
+```
+
+2. Expose it via the Admin API:
+
+```
+GET /_counterfact/api/requests            → last N entries (newest first)
+GET /_counterfact/api/requests?limit=10   → last 10 entries
+DELETE /_counterfact/api/requests         → clear the history
+```
+
+## Motivation
+
+- Agents can replay or compare request/response pairs to verify that their code is producing correct calls.
+- Validation errors are surfaced alongside the request that triggered them, giving agents immediate, structured feedback.
+- Human developers troubleshooting auto-generated routes also benefit from a time-ordered log of all interactions.
+
+## Acceptance criteria
+
+- [ ] After making several requests, `GET /_counterfact/api/requests` returns a JSON array of history entries in reverse-chronological order
+- [ ] Each entry includes method, path, status code, request body, response body, duration, and any validation errors
+- [ ] The `?limit=N` query parameter limits the number of returned entries
+- [ ] `DELETE /_counterfact/api/requests` empties the history and returns `{ success: true }`
+- [ ] Admin auth guard (bearer token / loopback) is applied to all history endpoints
+- [ ] The ring buffer is capped (default 100 entries) to avoid unbounded memory growth
+- [ ] Unit tests cover normal recording, the limit parameter, and the clear endpoint

.github/issue-proposals/agent-scenario-save-load.md

@@ -0,0 +1,71 @@
+---
+title: "Add named scenario save/load to Admin API for rapid agent state switching"
+parentIssue: 1720
+labels:
+  - enhancement
+  - agent-ux
+assignees: []
+milestone:
+---
+
+AI agents often need to test the same application code against multiple server states (e.g. "empty inventory", "one item in cart", "checkout in progress"). Currently the only way to set up a known state is to call a series of `POST /_counterfact/api/contexts/:path` requests. There is no way to save that state, give it a name, and restore it later in one operation.
+
+A scenario system lets agents define and switch between named server states with a single HTTP call.
+
+## Proposed change
+
+### Endpoints
+
+```
+POST   /_counterfact/api/scenarios          → create a scenario from current context state
+GET    /_counterfact/api/scenarios          → list all saved scenarios
+GET    /_counterfact/api/scenarios/:name    → get a specific scenario
+PUT    /_counterfact/api/scenarios/:name    → create or overwrite a named scenario
+POST   /_counterfact/api/scenarios/:name/load → restore server state from this scenario
+DELETE /_counterfact/api/scenarios/:name    → delete a scenario
+```
+
+### Schema
+
+```ts
+interface Scenario {
+  name: string;
+  description?: string;
+  createdAt: string;   // ISO-8601
+  contexts: Record<string, unknown>;  // path → context object snapshot
+}
+```
+
+### Behaviour
+
+- Scenarios are stored **in memory** by default (cleared on restart). When `--persist-scenarios` is passed, scenarios are read from `.counterfact-scenarios.json` in `basePath` at startup (if the file exists), and every create/update/delete operation is immediately written back to that file. This means a server restarted with `--persist-scenarios` automatically restores all previously saved scenarios.
+- `POST /_counterfact/api/scenarios/:name/load` calls the existing context reset logic, then replays every entry in the scenario's `contexts` map.
+
+### Example workflow
+
+```http
+# 1. Set up a "one pet in store" state
+POST /_counterfact/api/contexts/pets  { "pets": [{ "id": 1, "name": "Fido" }] }
+
+# 2. Save it
+PUT /_counterfact/api/scenarios/one-pet  { "description": "Single pet in store" }
+
+# 3. Later, restore it instantly
+POST /_counterfact/api/scenarios/one-pet/load
+```
+
+## Motivation
+
+- Agents switching between test scenarios today must issue a reset followed by N context-update calls. A single `POST /load` cuts that to one round-trip.
+- Named scenarios are self-describing, making it easy for a human reviewing the agent's work to understand what each test case represents.
+- Scenarios can be committed alongside application code (via `--persist-scenarios`) to give teams a reproducible set of server states.
+
+## Acceptance criteria
+
+- [ ] `PUT /_counterfact/api/scenarios/empty-cart` saves the current context state under the name `empty-cart`
+- [ ] `POST /_counterfact/api/scenarios/empty-cart/load` restores all contexts to the saved state
+- [ ] `GET /_counterfact/api/scenarios` lists all saved scenarios with their names and descriptions
+- [ ] `DELETE /_counterfact/api/scenarios/empty-cart` removes the scenario
+- [ ] All endpoints respect the bearer-token / loopback auth guard
+- [ ] Scenarios survive a context reset (reset only affects live contexts, not the saved scenario definitions)
+- [ ] Unit tests cover CRUD operations and the load/restore round-trip

.github/issue-proposals/agent-seeded-random-responses.md

@@ -0,0 +1,47 @@
+---
+title: "Support seeded/deterministic random responses for reproducible agent testing"
+parentIssue: 1720
+labels:
+  - enhancement
+  - agent-ux
+assignees: []
+milestone:
+---
+
+When Counterfact generates random example responses (via `@faker-js/faker` or similar), the values change on every request. For human developers iterating interactively this is fine, but for AI agents it causes two problems:
+
+1. **Non-deterministic test failures** – an agent writing assertions against a response can't predict the value it will receive.
+2. **Irreproducible debugging sessions** – when an agent records a failing scenario and tries to reproduce it, the server produces different data.
+
+Allowing consumers to fix the random seed makes responses repeatable across runs without requiring every route handler to be rewritten.
+
+## Proposed change
+
+1. Add an optional `--seed <number>` CLI flag (and a corresponding `seed` field in `Config`, default `undefined`).
+2. When a seed is provided, initialise `faker` (and any other PRNG used in response generation) with that seed before the server starts.
+3. Expose the seed via the Admin API:
+
+```
+GET  /_counterfact/api/config/seed      → { seed: 42 | null }
+POST /_counterfact/api/config/seed      → body { "seed": 42 } sets a new seed at runtime
+DELETE /_counterfact/api/config/seed    → removes the seed, restoring random behaviour
+```
+
+Setting the seed at runtime resets the PRNG state, so the sequence of random values produced by subsequent requests is identical to what would be produced by starting the server with that seed.
+
+The seed must be a non-negative integer (0–2³²−1). Providing a non-integer, a negative number, or a value outside this range returns `400 { "error": "seed must be a non-negative integer ≤ 4294967295" }`.
+
+## Motivation
+
+- Agents running regression tests need stable, assertable values.
+- CI pipelines benefit from deterministic snapshots they can diff against a known baseline.
+- Human developers can share a seed when filing a bug report, letting others reproduce the exact same generated data.
+
+## Acceptance criteria
+
+- [ ] `counterfact --seed 42 …` produces the same sequence of random responses across multiple server restarts
+- [ ] `POST /_counterfact/api/config/seed` with `{ "seed": 42 }` takes effect immediately; the next generated response uses the seeded sequence
+- [ ] `DELETE /_counterfact/api/config/seed` restores non-deterministic (truly random) behaviour
+- [ ] Without `--seed`, behaviour is unchanged (fully random as today)
+- [ ] The `seed` value is included in `GET /_counterfact/api/config` response
+- [ ] Unit tests assert that the same seed yields the same output, and that different seeds yield different outputs