Skip to content

Commit b14e063

Browse files
committed
feat(persistence): add HITL gate integration and comprehensive test suite (#262, #263)
HITL Gates: - Recovery escalation gate (GATE-AP-RECOVERY) for failed recovery attempts - False positive override gate (GATE-AP-FALSE-POSITIVE) for detection corrections - Destructive action gate (GATE-AP-DESTRUCTIVE) for pre-action approval - Gate integration module with trigger evaluation, display formatting, audit logging Test Suite (64 tests): - Pattern detection: confidence scoring, false positive reduction, edge cases - Recovery protocol: PDARE state machine, diagnosis, adaptation strategies - Best output selection: quality tracking, peak selection, degradation detection - Meta-tests: framework self-validation against laziness patterns - 8 diff fixture files for pattern detection testing Also adds path aliases to vitest.config.js for cleaner imports.
1 parent 918b8bd commit b14e063

20 files changed

+3652
-1
lines changed

agentic/code/addons/agent-persistence/docs/hitl-integration.md

Lines changed: 503 additions & 0 deletions
Large diffs are not rendered by default.
Lines changed: 297 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,297 @@
1+
# Destructive Action Gate
2+
# Triggered before allowing destructive actions (test deletion, feature removal)
3+
# Based on REF-071 METR Reward Hacking and REF-073 Microsoft Taxonomy
4+
# Issue: #262
5+
6+
$schema: "https://aiwg.io/schemas/hitl-gate/v1"
7+
8+
id: "GATE-AP-DESTRUCTIVE"
9+
name: "Destructive Action Approval Gate"
10+
description: |
11+
Pre-action gate requiring explicit human approval before allowing destructive
12+
operations like test deletion, feature removal, or coverage regression.
13+
Implements preventive control per ADR-AP-002 Rule Enforcement Strategy.
14+
15+
gate_type: approval
16+
17+
trigger:
18+
type: error_pattern
19+
pattern: "destructive_action_requested"
20+
description: |
21+
Triggered when agent attempts:
22+
- Delete test files/cases
23+
- Add skip patterns (.skip(), @Ignore)
24+
- Remove features or functionality
25+
- Weaken assertions significantly
26+
- Actions causing >2% coverage regression
27+
28+
behavior:
29+
mode: TERMINATE # MUST block until human approves
30+
timeout: 3600 # 1 hour for urgent decisions
31+
timeout_action: block # Do not proceed on timeout
32+
33+
auto_approve_conditions:
34+
- condition: "action_type == 'test_deletion' AND replacement_tests >= deleted_tests AND coverage_maintained"
35+
reason: "Legitimate test refactoring - replacing with better tests"
36+
- condition: "action_type == 'feature_removal' AND documented_in_requirements AND approved_by_pm"
37+
reason: "Intentional scope reduction with proper approval"
38+
39+
notification:
40+
channels:
41+
- cli
42+
- issue_comment
43+
urgency: high
44+
message_template: |
45+
## Destructive Action Requires Approval
46+
47+
**Gate**: GATE-AP-DESTRUCTIVE
48+
**Action**: {{action_type}}
49+
**Impact**: {{impact_summary}}
50+
51+
Human approval required before proceeding.
52+
53+
presentation:
54+
summary_template: |
55+
╭─────────────────────────────────────────────────────────────╮
56+
│ DESTRUCTIVE ACTION APPROVAL REQUIRED │
57+
│ Gate: GATE-AP-DESTRUCTIVE │
58+
├─────────────────────────────────────────────────────────────┤
59+
│ Context: │
60+
│ • Task: {{task_description}} │
61+
│ • Agent: {{agent_name}} │
62+
│ • Action Type: {{action_type}} │
63+
│ • Severity: {{severity}} │
64+
│ │
65+
│ Requested Action: │
66+
│ {{action_description}} │
67+
│ │
68+
│ Impact Analysis: │
69+
│ • Files Affected: {{file_count}} │
70+
│ • Tests Removed: {{tests_removed}} │
71+
│ • Coverage Impact: {{coverage_delta}} │
72+
│ • Features Affected: {{features_list}} │
73+
│ │
74+
│ Agent's Justification: │
75+
│ {{agent_justification}} │
76+
│ │
77+
│ Risk Assessment: │
78+
│ • Risk Level: {{risk_level}} │
79+
│ • Reversibility: {{reversible}} │
80+
│ • Production Impact: {{prod_impact}} │
81+
├─────────────────────────────────────────────────────────────┤
82+
│ Options: │
83+
│ [a] Approve - This action is intentional │
84+
│ [r] Reject - Find alternative approach │
85+
│ [v] View - Show detailed changes │
86+
│ [d] Diff - Compare before/after │
87+
│ [s] Suggest - Propose alternative solution │
88+
│ [q] Abort - Stop task entirely │
89+
╰─────────────────────────────────────────────────────────────╯
90+
91+
artifacts_to_show:
92+
- "{{affected_files}}"
93+
- ".aiwg/requirements/{{related_requirements}}"
94+
95+
questions:
96+
- id: "decision"
97+
question: "Approve this destructive action?"
98+
options:
99+
- "Approve - This is intentional and justified"
100+
- "Reject - Agent must find non-destructive alternative"
101+
- "Conditional - Approve with specific constraints"
102+
- "Abort - Stop this task"
103+
required: true
104+
105+
- id: "justification"
106+
question: "Why is this destructive action acceptable?"
107+
options:
108+
- "Refactoring - Replacing with better implementation"
109+
- "Scope Change - Requirements changed"
110+
- "Technical Debt - Removing obsolete code"
111+
- "Security - Removing vulnerable feature"
112+
- "Other (specify)"
113+
required: true
114+
115+
- id: "risk_mitigation"
116+
question: "What risk mitigation is in place?"
117+
options:
118+
- "Replacement tests provide better coverage"
119+
- "Feature documented as deprecated"
120+
- "Stakeholders approved removal"
121+
- "Can be reverted if needed"
122+
- "None - I accept the risk"
123+
required: true
124+
125+
- id: "documentation"
126+
question: "Is this change documented?"
127+
options:
128+
- "Yes - In requirements"
129+
- "Yes - In ADR"
130+
- "Yes - In commit message"
131+
- "No - I will document it now"
132+
- "Not needed"
133+
required: false
134+
135+
context_window: 200
136+
137+
cost_tracking:
138+
track_enabled: true
139+
metrics:
140+
- destructive_action_rate
141+
- approval_rate
142+
- rejection_rate
143+
- alternative_found_rate
144+
- regret_rate # Actions later reverted
145+
baseline_comparison: autonomous
146+
147+
audit:
148+
log_decision: true
149+
log_rationale: true
150+
require_signature: true # Critical for destructive actions
151+
retention_days: 365 # Keep long-term for auditing
152+
additional_fields:
153+
- action_type
154+
- files_affected
155+
- tests_removed
156+
- coverage_delta
157+
- risk_level
158+
- justification
159+
- risk_mitigation
160+
- documentation_location
161+
162+
# Integration with Anti-Laziness Framework
163+
integration:
164+
triggered_by:
165+
- laziness_detection_agent
166+
- file_write_hook
167+
- coverage_monitor
168+
169+
triggers_on:
170+
- test_file_deletion
171+
- test_skip_addition
172+
- feature_removal_detected
173+
- assertion_weakening
174+
- coverage_regression > 2%
175+
- validation_bypass_detected
176+
177+
on_approval:
178+
action: allow_destructive_action
179+
log_approval: true
180+
document_justification: true
181+
add_to_changelog: true
182+
notify_team: true
183+
set_review_reminder: 30_days # Re-evaluate decision
184+
185+
on_rejection:
186+
action: block_and_suggest_alternatives
187+
provide_guidance: true
188+
examples:
189+
test_deletion: "Instead of deleting tests, fix the code or refactor tests"
190+
feature_removal: "Instead of removing feature, disable with feature flag"
191+
coverage_regression: "Add tests to maintain coverage, don't delete existing"
192+
enforce_recovery_protocol: true
193+
194+
on_conditional:
195+
action: allow_with_constraints
196+
constraints:
197+
- must_add_replacement_tests
198+
- must_document_in_adr
199+
- must_notify_stakeholders
200+
- must_add_deprecation_notice
201+
verify_constraints_met: true
202+
203+
# Destructive Action Categorization
204+
action_types:
205+
test_deletion:
206+
severity: HIGH
207+
requires_approval: true
208+
acceptable_when:
209+
- "Replacing with better tests"
210+
- "Removing obsolete tests for removed features"
211+
- "Coverage maintained or improved"
212+
213+
test_skip:
214+
severity: HIGH
215+
requires_approval: true
216+
acceptable_when:
217+
- "Temporarily for debugging (must remove before commit)"
218+
- "Test framework issue documented"
219+
- "Never acceptable in committed code"
220+
221+
feature_removal:
222+
severity: CRITICAL
223+
requires_approval: true
224+
acceptable_when:
225+
- "Documented in requirements"
226+
- "Stakeholders approved"
227+
- "Deprecated first, then removed"
228+
- "Migration path provided"
229+
230+
assertion_weakening:
231+
severity: HIGH
232+
requires_approval: true
233+
acceptable_when:
234+
- "Original assertion was incorrect"
235+
- "Requirements changed to be less strict"
236+
- "Replacing with better assertion"
237+
238+
coverage_regression:
239+
severity: MEDIUM
240+
requires_approval: true
241+
acceptable_when:
242+
- "Adding untested code (tests planned)"
243+
- "Refactoring (temporary, will restore)"
244+
- "Removing dead code (coverage % increases)"
245+
246+
# Alternative Suggestions
247+
alternatives:
248+
test_deletion:
249+
- "Fix failing tests instead of deleting"
250+
- "Refactor tests to be more maintainable"
251+
- "Add better tests, then remove old ones"
252+
- "Use test.todo() to document planned improvements"
253+
254+
test_skip:
255+
- "Mock external dependencies causing flakiness"
256+
- "Fix timing issues with proper waits"
257+
- "Isolate test environment better"
258+
- "Debug and fix root cause"
259+
260+
feature_removal:
261+
- "Disable with feature flag"
262+
- "Mark as deprecated first"
263+
- "Provide migration guide for users"
264+
- "Keep code but remove from UI/API"
265+
266+
assertion_weakening:
267+
- "Fix implementation to meet assertion"
268+
- "Add defensive programming"
269+
- "Clarify requirements if assertion is wrong"
270+
- "Add multiple specific assertions"
271+
272+
# References
273+
references:
274+
requirements:
275+
- "@.aiwg/requirements/use-cases/UC-AP-001-detect-test-deletion.md"
276+
- "@.aiwg/requirements/use-cases/UC-AP-002-detect-feature-removal.md"
277+
architecture:
278+
- "@.aiwg/architecture/agent-persistence-sad.md"
279+
- "@.aiwg/architecture/decisions/ADR-AP-001-detection-hook-architecture.md"
280+
- "@.aiwg/architecture/decisions/ADR-AP-002-rule-enforcement-strategy.md"
281+
rules:
282+
- "@.claude/rules/hitl-gates.md"
283+
- "@.claude/rules/hitl-patterns.md"
284+
- "@.claude/rules/anti-laziness.md"
285+
agents:
286+
- "@.claude/agents/laziness-detector.md"
287+
- "@.claude/agents/recovery-orchestrator.md"
288+
research:
289+
- "@.aiwg/research/findings/REF-071-metr-reward-hacking.md"
290+
- "@.aiwg/research/findings/REF-073-microsoft-taxonomy.md"
291+
- "@.aiwg/research/findings/agentic-laziness-research.md"
292+
293+
# Issue tracking
294+
issues:
295+
- "#262" # HITL Gate Integration
296+
- "#96" # HITL Gates Implementation
297+
- "#264" # Anti-Laziness Rules

0 commit comments

Comments
 (0)