|
| 1 | +# Evidence Freshness |
| 2 | + |
| 3 | +Evidence has an expiration date. This guide explains why that matters and what to do about it. |
| 4 | + |
| 5 | +## Why Evidence Expires |
| 6 | + |
| 7 | +Imagine you benchmarked Redis vs Memcached six months ago. Redis won. You made the decision, recorded the DRR, moved on. |
| 8 | + |
| 9 | +Now it's six months later. The Memcached team shipped a major performance update. Your Node.js version changed. The benchmark numbers you relied on? They might not be accurate anymore. |
| 10 | + |
| 11 | +**The decision isn't necessarily wrong — it's just unverified.** |
| 12 | + |
| 13 | +This is what FPF calls **Evidence Decay**. Every piece of evidence has a `valid_until` date. When that date passes, the evidence is "stale" and the decisions built on it become questionable. |
| 14 | + |
| 15 | +## The Problem with Stale Evidence |
| 16 | + |
| 17 | +Stale evidence creates hidden risk. You're operating on assumptions that haven't been re-checked. Maybe they're still true. Maybe they're not. You don't know. |
| 18 | + |
| 19 | +Quint Code makes this visible instead of hiding it. |
| 20 | + |
| 21 | +## Checking Your Evidence |
| 22 | + |
| 23 | +Run `/q-decay` to see what's stale: |
| 24 | + |
| 25 | +``` |
| 26 | +/q-decay |
| 27 | +``` |
| 28 | + |
| 29 | +You'll get a freshness report showing which holons have expired evidence: |
| 30 | + |
| 31 | +``` |
| 32 | +## Evidence Freshness Report |
| 33 | +
|
| 34 | +### STALE (1 holon requires action) |
| 35 | +
|
| 36 | +#### Use Redis for Caching (L2) |
| 37 | +| Evidence ID | Type | Status | Details | |
| 38 | +|-------------|------|--------|---------| |
| 39 | +| ev-benchmark-2024-06-15 | validation | EXPIRED | 180 days overdue | |
| 40 | +| ev-load-test-2024-06-20 | validation | EXPIRED | 175 days overdue | |
| 41 | +
|
| 42 | +### FRESH (3 holons) |
| 43 | +- Use TypeScript for API (L2) |
| 44 | +- PostgreSQL for primary storage (L2) |
| 45 | +- JWT for auth tokens (L2) |
| 46 | +``` |
| 47 | + |
| 48 | +## Three Ways to Handle Stale Evidence |
| 49 | + |
| 50 | +When you see stale evidence, you have three options: |
| 51 | + |
| 52 | +### 1. Refresh — Re-run the test |
| 53 | + |
| 54 | +**When:** The decision is still correct, you just need fresh proof. |
| 55 | + |
| 56 | +**How:** Run `/q3-validate` on the hypothesis to gather new evidence. |
| 57 | + |
| 58 | +``` |
| 59 | +User: The Redis benchmark is old but Redis is still the right choice. |
| 60 | + Refresh the evidence. |
| 61 | +
|
| 62 | +Agent: Running validation for hypothesis-redis-caching... |
| 63 | + [executes benchmark] |
| 64 | + Evidence refreshed. Valid until 2025-06-21. |
| 65 | +``` |
| 66 | + |
| 67 | +### 2. Deprecate — Reconsider the decision |
| 68 | + |
| 69 | +**When:** The world has changed. The decision itself is questionable. |
| 70 | + |
| 71 | +**How:** Tell the agent to deprecate the hypothesis. It gets downgraded one level (L2→L1 or L1→L0). |
| 72 | + |
| 73 | +``` |
| 74 | +User: AWS announced they're deprecating this service. |
| 75 | + Deprecate that decision. |
| 76 | +
|
| 77 | +Agent: Deprecated: hypothesis-aws-service L2 → L1 |
| 78 | + This decision now needs re-evaluation. |
| 79 | + Run /q1-hypothesize to explore alternatives. |
| 80 | +``` |
| 81 | + |
| 82 | +### 3. Waive — Accept the risk temporarily |
| 83 | + |
| 84 | +**When:** You know it's stale, but you're shipping Tuesday and can't re-test everything right now. |
| 85 | + |
| 86 | +**How:** Tell the agent to waive the evidence with a deadline and reason. |
| 87 | + |
| 88 | +``` |
| 89 | +User: We're launching next week. Waive the benchmark until January 15th, |
| 90 | + we have a full test cycle scheduled for the 20th. |
| 91 | +
|
| 92 | +Agent: Waiver recorded: |
| 93 | + - Evidence: ev-benchmark-2024-06-15 |
| 94 | + - Waived until: 2025-01-15 |
| 95 | + - Rationale: Launch deadline. Full test cycle Jan 20. |
| 96 | +
|
| 97 | + Warning: Returns to EXPIRED after 2025-01-15. |
| 98 | +``` |
| 99 | + |
| 100 | +**A waiver is not ignoring the problem.** It's explicitly documenting that you know about the risk and accept it until a specific date. The waiver goes in the audit log — who waived what, why, and until when. |
| 101 | + |
| 102 | +## Natural Language Usage |
| 103 | + |
| 104 | +You don't need to memorize evidence IDs or parameters. Just describe what you want. |
| 105 | + |
| 106 | +The agent sees the freshness report and understands context. When you say "waive the benchmark until February," it finds the right evidence ID and calls the tool for you. |
| 107 | + |
| 108 | +**These all work:** |
| 109 | + |
| 110 | +``` |
| 111 | +"Waive everything until January 15th, we're launching" |
| 112 | +
|
| 113 | +"The load test is only 2 weeks overdue, refresh it" |
| 114 | +
|
| 115 | +"That API is being deprecated, deprecate our decision to use it" |
| 116 | +
|
| 117 | +"Waive the security audit until the 15th with rationale: re-audit scheduled" |
| 118 | +``` |
| 119 | + |
| 120 | +If you want to be explicit, you can: |
| 121 | + |
| 122 | +``` |
| 123 | +/q-decay --waive ev-benchmark-2024-06-15 --until 2025-02-01 --rationale "Migration pending" |
| 124 | +``` |
| 125 | + |
| 126 | +But natural language works fine. |
| 127 | + |
| 128 | +## The WLNK Principle |
| 129 | + |
| 130 | +A holon is **STALE** if *any* of its evidence is expired (and not waived). |
| 131 | + |
| 132 | +This is the Weakest Link (WLNK) principle. If you have three pieces of evidence and one is stale, the whole decision is questionable. You don't get to average it out. |
| 133 | + |
| 134 | +Think of it like a chain. Three strong links and one rusted link? The chain breaks at the rust. |
| 135 | + |
| 136 | +## Practical Workflows |
| 137 | + |
| 138 | +### Weekly Maintenance |
| 139 | + |
| 140 | +``` |
| 141 | +/q-decay # What's stale? |
| 142 | +# For each item: refresh, deprecate, or waive |
| 143 | +``` |
| 144 | + |
| 145 | +### Before a Release |
| 146 | + |
| 147 | +``` |
| 148 | +/q-decay # Check for stale decisions |
| 149 | +# Either refresh evidence or explicitly waive with rationale |
| 150 | +# Waivers become part of release documentation |
| 151 | +``` |
| 152 | + |
| 153 | +### After Major Changes |
| 154 | + |
| 155 | +Dependency update? API change? Security advisory? |
| 156 | + |
| 157 | +``` |
| 158 | +/q-decay # What's affected? |
| 159 | +# Deprecate obsolete decisions |
| 160 | +# Start new hypothesis cycle for replacements |
| 161 | +``` |
| 162 | + |
| 163 | +## Audit Trail |
| 164 | + |
| 165 | +All actions are logged: |
| 166 | + |
| 167 | +| Action | What's Recorded | |
| 168 | +|--------|----------------| |
| 169 | +| Deprecate | from_layer, to_layer, who, when | |
| 170 | +| Waive | evidence_id, until_date, rationale, who, when | |
| 171 | + |
| 172 | +You can always answer: "Who waived what and why?" |
| 173 | + |
| 174 | +## Summary |
| 175 | + |
| 176 | +- Evidence expires. This is normal. |
| 177 | +- `/q-decay` shows you what's stale. |
| 178 | +- **Refresh** if the decision is still right, you just need new proof. |
| 179 | +- **Deprecate** if the decision needs rethinking. |
| 180 | +- **Waive** if you accept the risk temporarily (with documented rationale). |
| 181 | +- Talk naturally — the agent handles the details. |
0 commit comments