docs: add a Markdown rendering of the design summary

Converts 04-design-summary.html to Markdown so the design is readable in the
repo, in diffs and in review, rather than only as a rendered artifact.

Not a mechanical tag-for-tag translation. The workflow graph was a hand-authored
inline SVG with absolute coordinates, which carries nothing into Markdown — it
is now a mermaid flowchart with the same topology, which GitHub renders and
docs/wiki.md already uses. The two retrieval guards become blockquotes so they
stay set apart from the prose around them, and the CSS-counter stage lists
become numbered headings so they appear in the outline and can be linked to.

The HTML is kept as the artifact source. The two will drift; if that becomes a
problem, one of them should be deleted rather than both maintained.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

Author: NinjaRocks

docs/implementation/04-design-summary.md

@@ -0,0 +1,299 @@
+# Suitability QA — Design Summary
+
+> **Abacus.Run.Service · Workflow design summary**
+
+A suitability report asserts facts about a client. The supporting case file either corroborates those
+facts or it doesn't. This workflow runs ten regulatory checks across that gap and returns findings
+anchored to the exact lines of the report that make each claim.
+
+| | |
+| --- | --- |
+| Host | Microsoft Agent Framework via Abacus.Run |
+| Checks | 10 |
+| Categories | A–I |
+
+---
+
+## Deployables — two services, one boundary
+
+Reference data and results outlive any particular workflow. They are owned by a separate service, so
+a revised check prompt or a corrected finding does not require redeploying the thing that runs the
+checks — and the workflow host keeps only what it alone knows.
+
+### `Abacus.Data.Service` — system of record
+
+Owns the catalog — document categories and assessment checks, managed by API and seeded from the
+shipped CSVs — and the assessment tree of findings, anchors and evidence. Renders the report from
+that data on demand. This is what reviewers and downstream systems read.
+
+### `Abacus.Run.Service` — workflow host and audit trail
+
+Runs the MAF workflow: ingest, retrieve, judge. Reads the catalog over HTTP and publishes each
+finding back. It produces the data a report is made of; it does not render one. Its own store holds
+the ingestion ledger, and its audit record goes to the framework's generic audit store.
+
+---
+
+## The shape of the problem — two corpora, deliberately kept apart
+
+Everything downstream follows from one distinction: the suitability report is the thing being
+audited, and the rest of the case file is what it's audited against. Conflating them would let the
+report corroborate itself, so the separation is enforced structurally rather than by convention — the
+report's category is never ingested, and never offered to retrieval.
+
+### Source of facts — the suitability report, category I
+
+Located in the excluded-category folder, parsed to a line-numbered document with a page map. Never
+embedded, never searched. Passages are attributed to checks by keyword so each check sees the part of
+the report it's about, not the whole file.
+
+### Source of evidence — the supporting case file, categories A–H
+
+Fact finds, meeting notes, risk profiles, policy information, cashflow modelling, research,
+illustrations. Converted, chunked and embedded with case and category metadata, then retrieved per
+check under a mandatory case filter.
+
+---
+
+## Composition — the workflow graph
+
+Ingestion and parsing run once per case. The ten checks then fan out as independent nodes and rejoin
+at a barrier, and the terminal node settles the assessment in the data service. Nothing here renders
+a report — that is the data service's job, on request.
+
+```mermaid
+flowchart LR
+  ingest["ingest-supporting"] --> parse["parse-suitability"]
+  parse --> open["open-assessment"]
+
+  open -->|fan-out| c1["check:CHK-001"]
+  open -->|fan-out| c2["check:CHK-002"]
+  open -->|fan-out| cn["check:CHK-###"]
+  open -->|fan-out| c10["check:CHK-010"]
+
+  c1 -->|fan-in barrier| agg["aggregate-outcomes"]
+  c2 -->|fan-in barrier| agg
+  cn -->|fan-in barrier| agg
+  c10 -->|fan-in barrier| agg
+
+  agg --> complete["complete-assessment"]
+```
+
+Fan-out broadcasts the same assessment state to every check; the fan-in barrier holds until all ten
+emit.
+
+Each check is its own node with its own executor id, rather than a loop inside one node. That buys
+three things: the runtime can execute them in parallel, a single check can fail or retry without
+disturbing the other nine, and tenants can gate an individual check through the existing
+gate-configuration API without any change to this graph.
+
+The node set is built from the catalog at `BuildAsync` time — one binding per check row, with the
+barrier's expected count taken from that same list — so adding an eleventh check is a catalog edit,
+not a code change.
+
+---
+
+## Per-check anatomy — what each check actually does
+
+A check is not one prompt. It is a reasoning step that decides what evidence would settle the
+question, a deterministic retrieval step that goes and gets it under filters the model cannot
+influence, and a judgement step over what came back.
+
+### 1. Plan — LLM reasoning
+
+The model receives the check's prompt, what-to-look-for, decision logic and regulatory basis, the
+suitability report's own assertions for that check, and the document-category catalog. It returns an
+`EvidencePlan`: a few targeted queries, each tagged with the categories that check implies, plus what
+each query is seeking.
+
+The alternative — one fixed query built from the check's title — asks the corpus a question nobody
+wrote. Letting the model derive "what would corroborate this" first is the difference between
+searching for the check and searching for the evidence the check needs.
+
+### 2. Search — deterministic and filtered
+
+Each planned query runs against the vector store. Planning is advisory; filtering is not.
+
+> **`case_reference` — mandatory by construction**
+>
+> `EvidenceQuery` throws if built without a case reference, category codes, or query text. There is
+> no "search everything" fallback to regress into. The retriever applies the case reference and the
+> model is never shown it, so it cannot widen or forget the scope.
+
+> **`document_category` — validated, not trusted**
+>
+> Codes the model proposes are checked against the seeded catalog and dropped if unknown. The catalog
+> offered to the planner already excludes the suitability report's own category, so `I` cannot be
+> chosen. If nothing valid survives, it falls back to the check's catalogued categories.
+
+### 3. Merge
+
+Hits are de-duplicated across queries by `(source, page, snippet)`, keeping the best score, then
+ranked and capped. The verdict call sees the strongest targeted evidence rather than everything the
+corpus could return.
+
+The prompt also lists the searches that ran and what each sought, so the model can reason about a gap
+as "I looked in these categories and found nothing" rather than inferring it from absence.
+
+### 4. Judge and persist
+
+A structured `CheckVerdict` comes back as a decision, a rationale, the report lines it anchors to,
+and the evidence it cites. It is written as one finding per check per assessment, upserted on
+`(AssessmentId, CheckId)` so a re-run replaces cleanly instead of accumulating.
+
+Decisions are `NoIssue`, `PotentialConcern`, or `NotApplicable`.
+
+---
+
+## Auditing — the record is declared, not hard-coded
+
+A regulated assessment has to be defensible months later: not just what was concluded, but what the
+model was asked, what it was shown, and what it actually said. That record belongs to the workflow
+that produced it — so the framework supplies the hook and the storage, and the workflow definition
+declares the shape.
+
+### 1. The definition declares its record
+
+`SuitabilityQaWorkflowDefinition` implements `IAuditedWorkflowDefinition` and returns an
+`AuditRecordDefinition`: a root kind — the assessment instance — plus the child constructs it may
+contain. For this workflow that is the submission, the ingestion summary, the parsed source document,
+and per check the retrieval plan, the searches executed, the input, the output, and the publication
+result.
+
+The declaration is the contract. A section the definition never declared is refused rather than
+silently stored, and the shape is discoverable at `GET /qa/audit/definition`.
+
+### 2. The runtime hands every node a recorder
+
+When a definition declares a record, the runner builds an `IWorkflowAuditRecorder` bound to it and
+attaches it to each node's `Runtime.Audit`, alongside the gate evaluator and middleware pipeline it
+already carried. A definition that declares nothing gets `null` and pays nothing.
+
+Executors then write as they go: the first node opens the root so a run that dies during ingestion
+still leaves a record of what it was asked to do, and the last closes it.
+
+### 3. Storage stays generic
+
+`IAuditRecordStore` holds a root and a stream of entries typed by string with JSON payloads. A new
+workflow declares new sections and needs no schema change. The framework ships an in-memory
+implementation; the host replaces it with SQLite.
+
+Entries are keyed by `(instance, kind, key)`, so a retried executor corrects its record rather than
+appending a contradictory second one.
+
+### 4. The record reads back through the framework
+
+`GET /workflows/{name}/instances/{id}/state` returns an instance's lifecycle status together with
+whatever record its workflow declared. The route holds no workflow-specific knowledge: the
+declaration drives the response, so every declared section appears — the empty ones included, which
+is what makes a run in progress readable. A workflow that declares nothing returns `audit: null`.
+
+The workflow name in the path is part of the identity, so reading an instance through another
+workflow's route is a 404 rather than a different view of the same record. Records grow large, so
+`?section=` narrows the response to named sections without changing its shape. The QA-specific
+`/qa/runs/…` endpoints remain for the presentation an investigator wants — the per-check exchange,
+the case index — and read the same store.
+
+One rule governs the whole mechanism: **an audit write must never fail the work it describes.**
+Storage errors are logged and swallowed — filing the explanation is not worth losing the result. The
+one place ordering matters is publication: the failure is recorded before it propagates, because a
+finding that could not be published is exactly what an investigator needs to find.
+
+---
+
+## Components — what each part is responsible for
+
+| Component | Responsibility | Why this way |
+| --- | --- | --- |
+| **Entry point** | `POST /workflows/suitability-qa/instances` accepts the `QAContext` — a document path and a case reference. Results are read from the data service; `GET /workflows/suitability-qa/instances/{id}/state` returns the run's status with its audit record, and `/qa/runs/…` presents the same record the way an investigation reads it. | Submission rides the framework's existing instance API, so the workflow inherits status, retry and gate handling rather than reimplementing them. |
+| **Catalog** | Nine document categories and ten checks, owned by the data service. Managed through `/catalog/checks` and `/catalog/categories`, seeded from the two shipped CSVs on startup or on demand via `POST /catalog/seed`. | Checks are content, not code. Seeding is idempotent and keyed by `Code` / `CheckId`, so a revised prompt lands by editing the catalog — nothing is redeployed and no workflow restart is needed. |
+| **Catalog guards** | A check cannot reference an unknown category; a category in use cannot be deleted; a check with findings cannot be deleted. | The failure these prevent is silent: a check pointed at a category that does not exist retrieves nothing and reports a gap that looks like missing evidence rather than a bad edit. |
+| **Ingestion** | Scan category folders → convert → chunk → embed → record. PdfPig, OpenXML, ClosedXML, Tesseract OCR, with an optional Docling sidecar per format. | Per-format converter choice is config, so a hard document type can be routed to Docling without the pipeline knowing. Failures downgrade one file to a warning instead of failing the case. |
+| **Chunker** | Paragraph-first splitting at ~700 tokens with 100 tokens of overlap, falling back to sentence and then hard splits. | Cutting mid-sentence damages both the embedding and the model's reading of the returned snippet. Overlap keeps a fact that straddles a boundary retrievable. |
+| **Vector store** | Qdrant in production, in-memory for dev and tests, behind one `IEvidenceStore` interface. | The access pattern is filtered ANN — always by case, always by category. Qdrant indexes those payload fields, so the filter is pushed down rather than applied after ranking. |
+| **Anchor extraction** | Splits the report on detected headings and attributes each passage to checks by keyword. | Deliberately zero-LLM: cheap, deterministic, nothing to keep in sync. Its only job is narrowing what each check reads; the LLM still quotes the exact fact. No headings found means one whole-document passage — less targeted, still correct. |
+| **Evidence retriever** | Plans queries with the LLM, executes them under mandatory case and category filters, merges the hits. | Splits the decision from the permission. The model chooses what to look for; the retriever decides what it is allowed to look in. |
+| **Aggregator** | Accumulates check results behind a fan-in barrier and emits once the last one arrives. | The runtime delivers one message per source per step, so the barrier target has to hold state and stay silent until the set is complete. |
+| **Report renderer** | Lives in the data service. Renders the Markdown anchor table — line, page, check, provided fact, evidence or gap, decision — plus rationale, and a JSON view, on demand from the stored assessment. | A report is a view over findings, and it must stay renderable long after the run that produced them. Rendering on demand means one source of truth: a corrected finding cannot leave a stale report behind it. Anchors carry their page number because only the workflow holds the parsed report that maps a line to a page — that is the workflow producing data for reporting, not reporting. |
+| **Audit hook** | Framework contract in `Abacus.Run`: a definition declares an `AuditRecordDefinition`, and every executor reaches an `IWorkflowAuditRecorder` through `Runtime.Audit`. | Auditing is opt-in per workflow and costs nothing to one that declines it. The framework supplies the hook and the storage; the shape and the meaning stay with the definition, because only it knows what is audit-significant about its own run. |
+| **Audit store** | Generic root-plus-entries storage with JSON payloads, backed by SQLite in the host and in-memory in the framework default. | Workflow-agnostic on purpose: a new workflow declares new sections and needs no schema change. Writes are best-effort — losing a record must never fail the work it describes. |
+| **Reconciler** | Background service that settles assessments left running when a workflow dead-stops, cancels or exhausts retries. | The happy path already completes the assessment. This exists so the sad paths don't leave a row stuck in `Running` forever. |
+
+---
+
+## Data model — findings anchored to evidence
+
+The split runs through the schema. One assessment per workflow instance and one finding per check
+live in the data service, each finding carrying the two halves of the audit: where in the report the
+claim was made, and what in the case file supports it. The workflow host's own tables record how it
+got there.
+
+| Entity | Owner | Key | Holds |
+| --- | --- | --- | --- |
+| `Assessment` | Data service | `Id` · unique `WorkflowInstanceId` | Case reference, status and timestamps — no stored report, since the report is rendered from the findings on request. Unique on instance id, so a resumed workflow reuses its assessment. |
+| `Finding` | Data service | unique `(AssessmentId, CheckId)` | Decision and rationale for one check. The uniqueness constraint is what makes a re-run an update. |
+| `Anchor` | Data service | `FindingId` | Line range and the verbatim fact quoted from the suitability report. |
+| `EvidenceHit` | Data service | `FindingId` | Source path, page, snippet and score from the supporting documents. |
+| `AuditRecordRoot` | Audit store | `InstanceId` | The root aggregate: root kind and key (here, the assessment keyed by case reference), status, and attributes including the assessment id it describes. |
+| `AuditRecordEntry` | Audit store | unique `(InstanceId, SectionKind, Key)` | One declared construct with a JSON payload. Keyed so a retried executor corrects its record instead of appending a contradictory second one. |
+| `IngestedDocument` | Workflow host | `(CaseReference, FileHash)` | SHA-256 ledger of embedded files. Re-ingesting an unchanged file is a no-op; changed content replaces its prior chunks. |
+
+---
+
+## Fault tolerance — degrade the finding, not the assessment
+
+The governing rule: a check that cannot be answered well should still be answered honestly. An
+assessment that stops halfway is worth less to a reviewer than one that completes and says where it
+was blind.
+
+| Failure | Response |
+| --- | --- |
+| **Transient LLM error** | Exponential backoff, three attempts, through the shared `LlmRetry` policy now used by both the planning and verdict calls. |
+| **Retrieval plan won't parse** | Falls back to a single query built from the check itself. A check is still assessed against real evidence, never silently against none. |
+| **A search fails** | That query is skipped and the rest still run. The finding records `retrieval-unavailable` so the gap is visible rather than implied. |
+| **Verdict won't parse** | Recorded as `PotentialConcern` with a note that manual review is required. Retrying malformed JSON just burns tokens. |
+| **Model invents a category** | Dropped by validation against the seeded catalog before the query is built. |
+| **Document won't convert** | Warned and skipped; the remaining documents ingest. Ten checks against a partial corpus beats no assessment. |
+| **SQLite busy or locked** | Classified as retryable — ten check executors write concurrently, so contention is expected rather than exceptional. |
+| **Data service unreachable** | Classified as retryable. It owns the catalog and the findings, so it is a hard dependency — but the assessment is resumable and the work already done is not lost. |
+| **Finding won't publish** | The audit row is written first, recording the error alongside the prompt and response, then the failure propagates. A finding that could not be published is exactly what an investigator needs to see. |
+| **Audit write fails** | Logged and swallowed. The trail is for investigation, not correctness — losing a row must not fail a check whose finding was already published. |
+| **No report, or path missing** | Dead-stop. There is nothing to audit, and retrying will not conjure a file. |
+
+---
+
+## Configuration — what's tunable without a rebuild
+
+```text
+— Abacus.Data.Service —
+QaData:Database               connection string for the catalog + assessments
+QaData:Seed                   startup seeding toggle and the two CSV paths
+
+— Abacus.Run.Service —
+Abacus:AuditRecords           connection string for the framework's generic audit-record store
+SuitabilityQa:Database        connection string for the workflow's ingestion ledger
+SuitabilityQa:DataService     base URL and timeout for the data service
+SuitabilityQa:Ingestion       chunk size, overlap, excluded categories, per-format converters
+SuitabilityQa:Ocr             Tesseract toggle, language, scanned-page threshold
+SuitabilityQa:Docling         sidecar URL and timeout — only used if a converter selects it
+SuitabilityQa:VectorStore     InMemory or Qdrant, collection, embedding dimensions
+SuitabilityQa:Retrieval       MaxQueriesPerCheck 4 · TopKPerQuery 4 · MaxEvidencePerCheck 10
+SuitabilityQa:Llm             model, temperature, max tokens
+```
+
+The excluded-category list is the single source of truth for the source/evidence split: the scanner
+skips it, the locator looks in it for the report, and the retriever removes it from the catalog
+offered to the planner. Change it in one place and all three follow.
+
+---
+
+## Known tradeoff — planned retrieval doubles the LLM calls
+
+Ten checks now make ten planning calls and ten verdict calls. The planning call is small and the
+retrieval options cap the work it can generate, but the cost is real. If a deployment values cost
+over retrieval precision, the planner is the component to make optional — the fallback path it
+already uses on a parse failure is exactly the old behaviour.
+
+---
+
+*Companion to [01 implementation plan](01-implementation-plan.md) · [02 tech stack](02-tech-stack.md) · [03 workflow realization](03-qa-workflow-realization.md)*