Skip to content

Commit 716df8e

Browse files
fix(134): threat-report attack tree baseline, MAESTRO layer rendering, filename convention (#135)
Three related bugs in the tachi threat-report generation pipeline: 1. Attack tree baseline handling lost Mermaid content on UNCHANGED findings. Replaced the old "skip UNCHANGED" rule with a three-rule approach that copies baseline Mermaid when nothing changed, regenerates all trees when architecture shifted, and reconciles structurally-equivalent trees against the baseline to avoid diff noise. 2. MAESTRO layer context was dropped from Section 3 narratives and the threats.md Section 7 summary table despite the data being parsed into findings. Promoted MAESTRO layer references to mandatory on first mention of every finding and added a MAESTRO Layer column to the summary table. 3. Attack tree filename convention was stated once as a parenthetical and not enforced, leading to files like AG-1-no-hitl-stdio.md that the spec 112 parser silently dropped. Added explicit convention block, dedicated skill section, and a validation checklist item. Schema plumbing: schemas/report.yaml bumps 1.0 -> 1.1 with baseline_source, baseline_date, delta_counts frontmatter fields and a conditional Section 8 Delta Summary. schemas/output.yaml adds status and maestro_layer columns. Closes #134 Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
1 parent 54c0f6a commit 716df8e

File tree

7 files changed

+133
-30
lines changed

7 files changed

+133
-30
lines changed

.claude/agents/tachi/threat-report.md

Lines changed: 32 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -118,7 +118,8 @@ Before finalizing the report, run the following checklist. Every check must pass
118118
#### Section Completeness
119119

120120
- [ ] All 7 report sections are present with non-empty content
121-
- [ ] YAML frontmatter contains all 6 required fields (schema_version, date, source_file, finding_count, risk_distribution, attack_tree_count)
121+
- [ ] YAML frontmatter is the FIRST content in the report (before Section 1), enclosed in a fenced `yaml` code block between `---` delimiters
122+
- [ ] YAML frontmatter contains ALL required fields: schema_version, date, source_file, finding_count, risk_distribution, attack_tree_count, baseline_source, baseline_date, delta_counts (see template for full structure)
122123
- [ ] Section headings match `../../../schemas/report.yaml` exactly (## 1. Executive Summary through ## 7. Appendix: Finding Reference)
123124

124125
#### Finding Traceability (Zero Loss Rule)
@@ -133,7 +134,7 @@ Before finalizing the report, run the following checklist. Every check must pass
133134
- [ ] Every High finding has an attack tree with minimum 2 levels of decomposition
134135
- [ ] No attack trees generated for Medium, Low, Note, or RESOLVED findings
135136
- [ ] Attack trees appear inline in Section 5 AND as standalone files in `attack-trees/`
136-
- [ ] Standalone file naming follows `{finding-id}-attack-tree.md` convention (lowercase, e.g., `ag-1-attack-tree.md`)
137+
- [ ] Standalone file naming follows `{finding-id}-attack-tree.md` convention — finding ID lowercased, `-attack-tree.md` suffix (e.g., `ag-1-attack-tree.md`, NOT `AG-1-attack-tree.md` or `AG-1-description-slug.md`)
137138

138139
#### Mermaid Syntax Integrity
139140

@@ -162,6 +163,10 @@ Before finalizing the report, run the following checklist. Every check must pass
162163

163164
## Report Generation Workflow
164165

166+
### Step 0: YAML Frontmatter (MANDATORY — generate FIRST)
167+
168+
**Before writing any section**, generate the YAML frontmatter block at the top of the report. Read `../../../templates/tachi/output-schemas/threat-report.md` for the exact field structure. The frontmatter MUST be the first content after the H1 heading, enclosed in a fenced `yaml` code block between `---` delimiters. Populate all fields from `threats.md`: schema_version (`"1.1"`), date, source_file, finding_count, risk_distribution (Critical/High/Medium/Low counts), attack_tree_count, baseline_source, baseline_date, and delta_counts. When no baseline exists, set baseline_source, baseline_date, and all delta_counts fields to `null`.
169+
165170
### Section 1: Executive Summary
166171

167172
**MANDATORY**: Read `.claude/skills/tachi-threat-reporting/references/narrative-templates.md` for the 5 required elements, language rules, and remediation timeline tiers.
@@ -178,7 +183,9 @@ Generate the Architecture Overview deriving system context from `threats.md` Sec
178183

179184
**MANDATORY**: Read `.claude/skills/tachi-threat-reporting/references/narrative-templates.md` for per-category subsection headers, per-finding narrative pattern, progressive depth rules, and large threat model handling.
180185

181-
Generate the Threat Analysis with agent-by-agent narrative covering all 8 categories. When findings include a `maestro_layer` field, reference the architectural layer in finding narratives for additional context (e.g., "This threat targets the Agent Framework layer (L3)"). MAESTRO layer references are informational -- they do not change narrative structure, severity assessments, or attack tree construction.
186+
Generate the Threat Analysis with agent-by-agent narrative covering all 8 categories.
187+
188+
**MAESTRO Layer References (MANDATORY when present)**: When findings include a `maestro_layer` field, you MUST reference the architectural layer in each finding's narrative. Include the layer designation on first mention of each finding — for example: "**S-1** targets the Agent Framework layer (L3), where..." or "Operating at the Data Operations layer (L2), **T-3** exploits...". Every finding narrative must include its MAESTRO layer context. These references are informational — they do not change narrative structure, severity assessments, or attack tree construction.
182189

183190
### Section 4: Cross-Cutting Themes
184191

@@ -192,7 +199,20 @@ Scan all findings for emergent patterns across categories that reveal systemic i
192199

193200
**MANDATORY**: Read `.claude/skills/tachi-threat-reporting/references/attack-tree-examples.md` before generating the first tree -- load once as reference patterns.
194201

195-
Generate Mermaid attack trees for every Critical and High finding following Bruce Schneier's attack tree methodology. **Skip findings with delta_status RESOLVED** -- resolved threats are no longer active and must not receive attack trees. **ALWAYS generate fresh attack trees for UNCHANGED findings** -- do NOT produce placeholder text like "carried forward from baseline." An unchanged threat description does not mean the attack paths are static; adjacent components and techniques may have changed. Every active Critical/High finding gets a fully constructed Mermaid attack tree, regardless of delta_status.
202+
Generate Mermaid attack trees for every Critical and High finding following Bruce Schneier's attack tree methodology.
203+
204+
**Attack tree delta handling (three rules):**
205+
206+
First, check `delta_counts` from the `threats.md` frontmatter to determine which rule applies:
207+
208+
**Rule 1 — All UNCHANGED (delta_counts: new=0, updated=0, resolved=0):** Architecture has not changed. For each UNCHANGED Critical/High finding, read and copy the full Mermaid content from the baseline at `{baseline_dir}/attack-trees/{finding-id}-attack-tree.md`. Derive `baseline_dir` from the `baseline.source` frontmatter path by dropping `threats.md`. Include the complete Mermaid code block in both inline (Section 5) and standalone file output. Do NOT output placeholder text without the diagram. If the baseline file is missing, generate fresh as fallback.
209+
210+
**Rule 2 — Any NEW/UPDATED/RESOLVED (any delta_counts > 0):** Architecture shifted -- attack paths to all threats may have changed. Generate fresh attack trees for ALL Critical/High findings, including UNCHANGED ones. For UPDATED findings, add a note: _"Context changed since baseline -- attack tree regenerated."_
211+
212+
**Rule 3 — Reconciliation (after Rule 2 only):** After generating all fresh trees, compare each UNCHANGED finding's fresh tree against its baseline version. If structurally similar (same nodes, same paths, minor wording only), use the baseline version for consistency. If materially different (new paths, removed nodes, structural changes), use the fresh version.
213+
214+
**RESOLVED**: Skip entirely -- no attack tree.
215+
**No baseline**: Generate all trees fresh.
196216

197217
### Section 6: Remediation Roadmap
198218

@@ -252,7 +272,14 @@ Embed each attack tree directly in the Attack Trees section using Mermaid code b
252272

253273
### Location 2: Standalone Files in attack-trees/
254274

255-
Save each attack tree as an independent Markdown file in the `attack-trees/` directory within the output directory. File naming: `{finding-id}-attack-tree.md` (lowercase, e.g., `ag-1-attack-tree.md`).
275+
Save each attack tree as an independent Markdown file in the `attack-trees/` directory within the output directory.
276+
277+
**File naming convention** (MUST follow exactly):
278+
- Pattern: `{finding-id}-attack-tree.md`
279+
- Case: **always lowercase** — the finding ID is lowercased in the filename
280+
- Suffix: **always `-attack-tree.md`** — never use a description slug
281+
- Examples: `AG-1` → `ag-1-attack-tree.md`, `LLM-2` → `llm-2-attack-tree.md`, `S-1` → `s-1-attack-tree.md`
282+
- **Wrong**: `AG-1-no-hitl-stdio.md`, `AG-1-attack-tree.md` (uppercase)
256283

257284
Each standalone file contains: H1 heading with finding ID and threat description, a metadata table (Finding ID, Component, Risk Level, Threat, Correlation), and the Mermaid code block **identical** to the inline version in `threat-report.md`.
258285

.claude/skills/tachi-threat-reporting/references/attack-tree-construction.md

Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -137,6 +137,20 @@ Target a maximum of approximately **20 nodes** per tree for readability. If a tr
137137

138138
---
139139

140+
## Standalone File Naming
141+
142+
Each attack tree is saved as a standalone file in the `attack-trees/` directory. The filename MUST follow this convention exactly:
143+
144+
- **Pattern**: `{finding-id}-attack-tree.md`
145+
- **Case**: The finding ID MUST be **lowercased** in the filename
146+
- **Suffix**: Always `-attack-tree.md` — never use a description slug or other suffix
147+
- **Examples**: Finding `AG-1` → filename `ag-1-attack-tree.md`, Finding `LLM-2` → filename `llm-2-attack-tree.md`, Finding `S-1` → filename `s-1-attack-tree.md`
148+
- **Wrong examples**: `AG-1-attack-tree.md` (uppercase), `ag-1-no-hitl-stdio.md` (description slug), `AG-1-no-hitl-stdio.md` (both wrong)
149+
150+
To produce the filename: take the finding ID (e.g., `AG-1`), convert to lowercase (e.g., `ag-1`), append `-attack-tree.md`.
151+
152+
---
153+
140154
## Validation Checklist
141155

142156
Before including any Mermaid attack tree in the report or standalone file, run every check below. A tree that fails any check must be corrected before output.
@@ -178,6 +192,12 @@ Before including any Mermaid attack tree in the report or standalone file, run e
178192
- [ ] Leaf nodes assigned `leaf` class
179193
- [ ] Color values match the standard palette: goal=`#ff6b6b`, andGate=`#ffa500`, orGate=`#4ecdc4`, leaf=`#95e1d3`
180194

195+
### Standalone File Naming
196+
197+
- [ ] Filename is the finding ID lowercased plus `-attack-tree.md` (e.g., AG-1 → `ag-1-attack-tree.md`)
198+
- [ ] No uppercase letters in filename
199+
- [ ] No description slugs — suffix is always `-attack-tree.md`
200+
181201
### Readability
182202

183203
- [ ] Total node count does not exceed ~20 nodes

.claude/skills/tachi-threat-reporting/references/narrative-templates.md

Lines changed: 5 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -75,10 +75,11 @@ For each category, include all findings from the corresponding STRIDE or AI tabl
7575

7676
For each finding, provide:
7777
1. **Finding reference**: State the finding ID (e.g., "**S-1**") as a bold reference
78-
2. **Component annotation**: Name the affected component
79-
3. **Threat description**: Explain the threat in context -- what could happen, how, and why it matters
80-
4. **Risk context**: State the likelihood, impact, and computed risk level
81-
5. **Mitigation summary**: Reference the recommended mitigation (the full mitigation text appears in the Remediation Roadmap)
78+
2. **MAESTRO layer context**: On first mention of each finding, include its MAESTRO architectural layer from the `maestro_layer` field. Integrate naturally into the sentence — for example: "**S-1** targets the Agent Framework layer (L3), where..." or "Operating at the Data Operations layer (L2), **T-3** exploits...". Every finding MUST include its MAESTRO layer on first reference.
79+
3. **Component annotation**: Name the affected component
80+
4. **Threat description**: Explain the threat in context -- what could happen, how, and why it matters
81+
5. **Risk context**: State the likelihood, impact, and computed risk level
82+
6. **Mitigation summary**: Reference the recommended mitigation (the full mitigation text appears in the Remediation Roadmap)
8283

8384
### Progressive Technical Depth Rules
8485

schemas/output.yaml

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -143,7 +143,9 @@ output:
143143
sort_order: "risk_level descending"
144144
fields:
145145
- finding_id
146+
- status
146147
- component
148+
- maestro_layer
147149
- threat
148150
- risk_level
149151
- mitigation

schemas/report.yaml

Lines changed: 50 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -9,7 +9,7 @@
99
#
1010
# Version: 1.0
1111

12-
schema_version: "1.0"
12+
schema_version: "1.1"
1313

1414
report:
1515
output_file: threat-report.md
@@ -19,7 +19,7 @@ report:
1919
fields:
2020
schema_version:
2121
type: string
22-
value: "1.0"
22+
value: "1.1"
2323
date:
2424
type: string
2525
format: "YYYY-MM-DD"
@@ -51,6 +51,37 @@ report:
5151
description: >
5252
Number of generated attack trees (one per
5353
Critical/High finding).
54+
baseline_source:
55+
type: string
56+
nullable: true
57+
description: >
58+
File path of the baseline threats.md used for delta
59+
comparison. Null when no baseline (first run).
60+
baseline_date:
61+
type: string
62+
nullable: true
63+
format: "YYYY-MM-DD"
64+
description: >
65+
ISO date of the baseline run. Null when no baseline.
66+
delta_counts:
67+
type: object
68+
nullable: true
69+
description: >
70+
Lifecycle breakdown counts. All values null when
71+
no baseline (first run).
72+
fields:
73+
new:
74+
type: integer
75+
nullable: true
76+
unchanged:
77+
type: integer
78+
nullable: true
79+
updated:
80+
type: integer
81+
nullable: true
82+
resolved:
83+
type: integer
84+
nullable: true
5485

5586
required_sections:
5687
- name: "Executive Summary"
@@ -100,9 +131,26 @@ report:
100131
requirement: >
101132
Complete mapping — zero finding loss.
102133
134+
- name: "Delta Summary"
135+
heading: "## 8. Delta Summary"
136+
conditional: true
137+
condition: >
138+
Present only when baseline_source is non-null in
139+
frontmatter. Omit entirely on first run.
140+
required_elements:
141+
- finding_lifecycle_breakdown
142+
- remediation_progress
143+
- baseline_reference
144+
103145
attack_tree_files:
104146
directory: "attack-trees/"
105147
naming: "{finding-id}-attack-tree.md"
148+
naming_rules:
149+
case: lowercase
150+
suffix: "-attack-tree.md"
151+
description: >
152+
Finding ID is lowercased for the filename. Example: finding AG-1
153+
becomes ag-1-attack-tree.md, finding LLM-2 becomes llm-2-attack-tree.md.
106154
severity_filter:
107155
- Critical
108156
- High

templates/tachi/output-schemas/threat-report.md

Lines changed: 11 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -196,12 +196,17 @@ flowchart TD
196196
>
197197
> **Correlated findings**: Each correlated finding receives its own individual tree with a cross-reference note to related finding IDs — not a single unified tree for the correlation group.
198198
>
199-
> **Baseline handling** (delta_status branching):
200-
> - **NEW**: Generate a fresh attack tree following all standard conventions above.
201-
> - **UPDATED**: Generate a fresh attack tree. Add a note below the tree: _"Context changed since baseline ({baseline_date}) — attack tree regenerated."_
202-
> - **UNCHANGED**: Do NOT generate an attack tree. Instead, note: _"Attack tree carried forward from baseline ({baseline_date}) — finding unchanged since last assessment."_
203-
> - **RESOLVED**: Not applicable — RESOLVED findings do not appear in Section 5. They appear only in Section 4b of the input threats.md.
204-
> - **No baseline**: When `baseline_source` is null in the input frontmatter, generate all attack trees fresh with no delta annotations. This is standard first-run behavior.
199+
> **Baseline handling** (three-rule approach):
200+
>
201+
> **Rule 1 — All UNCHANGED (no architecture change):** When `delta_counts` shows zero NEW, zero UPDATED, and zero RESOLVED findings, the architecture has not changed. For each UNCHANGED Critical/High finding, copy the full Mermaid attack tree content from the baseline's `attack-trees/{finding-id}-attack-tree.md` file. Derive the baseline directory from the `baseline.source` frontmatter field by dropping the `threats.md` filename. Include the complete Mermaid code block — do NOT output placeholder text without the actual diagram. If the baseline file is missing, generate a fresh tree as fallback.
202+
>
203+
> **Rule 2 — Any NEW/UPDATED/RESOLVED (architecture changed):** When `delta_counts` shows any non-zero value for `new`, `updated`, or `resolved`, the architecture has shifted and attack paths to all threats may have changed. Generate fresh attack trees for ALL Critical/High findings — including UNCHANGED ones — following all standard conventions above. For NEW findings, generate normally. For UPDATED findings, add a note below the tree: _"Context changed since baseline ({baseline_date}) — attack tree regenerated."_
204+
>
205+
> **Rule 3 — Post-generation reconciliation (applies only when Rule 2 fires):** After generating all fresh trees, compare each UNCHANGED finding's fresh tree against its baseline counterpart from `attack-trees/{finding-id}-attack-tree.md`. If they are structurally similar (same nodes, same paths, minor wording differences only), use the baseline version for consistency — this avoids noisy churn on trees that didn't actually change. If they are materially different (new attack paths, removed nodes, structural changes), use the fresh version. This ensures diffs between runs only show meaningful changes.
206+
>
207+
> **RESOLVED**: Not applicable — RESOLVED findings do not appear in Section 5. They appear only in Section 4b of the input threats.md.
208+
>
209+
> **No baseline**: When `baseline_source` is null in the input frontmatter, generate all attack trees fresh with no delta annotations. This is standard first-run behavior.
205210
206211
_When no Critical or High findings exist:_
207212

0 commit comments

Comments
 (0)