Skip to content

Commit 4d5bbeb

Browse files
rysweetclaude
andauthored
fix: Add keyword-based session type detection for investigation/troubleshooting (#1604) (#1606)
fix: Add keyword-based session type detection for investigation/troubleshooting (#1604) This PR fixes #1604 where power-steering incorrectly blocked investigation and troubleshooting sessions with development-specific checks. **Changes:** - Add INVESTIGATION_KEYWORDS constant (16 keywords) - Add _has_investigation_keywords() helper method - Update detect_session_type() to check keywords FIRST before tool heuristics - Add 19 comprehensive tests for keyword detection - Document fix in DISCOVERIES.md **The core insight:** User intent (expressed through keywords like "investigate", "troubleshoot", "debug") is more reliable than tool usage patterns for classifying session types. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
1 parent 3c35744 commit 4d5bbeb

File tree

3 files changed

+542
-4
lines changed

3 files changed

+542
-4
lines changed

.claude/context/DISCOVERIES.md

Lines changed: 38 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -4,6 +4,44 @@ This file documents non-obvious problems, solutions, and patterns discovered
44
during development. It serves as a living knowledge base that grows with the
55
project.
66

7+
## Power-Steering Session Type Detection Fix (2025-11-25)
8+
9+
### Problem
10+
11+
Power-steering was incorrectly blocking investigation/troubleshooting sessions with development-specific checks (PR, CI/CD, tests). Sessions like "Investigate SSH connection issues" were being classified as DEVELOPMENT or MAINTENANCE based on tool usage patterns (Bash commands, doc updates), even though they should be INVESTIGATION.
12+
13+
### Root Cause
14+
15+
The `detect_session_type()` method relied solely on tool-based heuristics:
16+
17+
- Bash commands → could indicate DEVELOPMENT (tests) or MAINTENANCE (git)
18+
- Doc file edits → MAINTENANCE
19+
- Read/Grep operations → INVESTIGATION (but only if no writes)
20+
21+
Troubleshooting sessions often involve:
22+
23+
- Bash commands to diagnose/fix issues (SSH, docker, etc.)
24+
- Doc updates (DISCOVERIES.md) to record findings
25+
- These patterns were misclassified
26+
27+
### Solution (PR #1606)
28+
29+
Added **keyword-based session type detection** with priority over tool heuristics:
30+
31+
1. Check first 5 user messages for investigation keywords
32+
2. Keywords include: investigate, troubleshoot, diagnose, debug, analyze, research, explore, understand, figure out, why does, how does, what causes, root cause, explain
33+
3. If found → immediately classify as INVESTIGATION
34+
4. Fall back to tool-based heuristics only if no keywords
35+
36+
### Key Insight
37+
38+
**User intent (expressed through keywords) is more reliable than tool usage patterns** for classifying session types. A user saying "troubleshoot the SSH issue" clearly wants an investigation, regardless of what tools get used.
39+
40+
### Files Changed
41+
42+
- `.claude/tools/amplihack/hooks/power_steering_checker.py`: Added `INVESTIGATION_KEYWORDS`, `_has_investigation_keywords()`, updated `detect_session_type()`
43+
- `.claude/tools/amplihack/hooks/tests/test_session_classification.py`: Added 18 tests for keyword detection
44+
745
## Transcripts System Investigation - Architecture Validated, Microsoft Amplifier Comparison Complete (2025-11-22)
846

947
### Investigation Summary

.claude/tools/amplihack/hooks/power_steering_checker.py

Lines changed: 79 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -196,6 +196,35 @@ class PowerSteeringChecker:
196196
"python -m unittest",
197197
]
198198

199+
# Keywords that indicate investigation/troubleshooting sessions
200+
# When found in early user messages, session is classified as INVESTIGATION
201+
# regardless of tool usage patterns (fixes #1604)
202+
#
203+
# Note: Using substring matching, so shorter forms match longer variants:
204+
# - "troubleshoot" matches "troubleshooting"
205+
# - "diagnos" matches "diagnose", "diagnosis", "diagnosing"
206+
# - "debug" matches "debugging"
207+
INVESTIGATION_KEYWORDS = [
208+
"investigate",
209+
"troubleshoot",
210+
"diagnos", # matches diagnose, diagnosis, diagnosing
211+
"analyze",
212+
"analyse",
213+
"research",
214+
"explore",
215+
"understand",
216+
"figure out",
217+
"why does",
218+
"why is",
219+
"how does",
220+
"how is",
221+
"what causes",
222+
"what's causing",
223+
"root cause",
224+
"debug",
225+
"explain",
226+
]
227+
199228
# Phase 1 fallback: Hardcoded considerations (top 5 critical)
200229
# Used when YAML file is missing or invalid
201230
PHASE1_CONSIDERATIONS = [
@@ -981,14 +1010,57 @@ def _has_investigation_indicators(
9811010
# Multiple Read/Grep without modifications
9821011
return read_grep_operations >= 2 and write_edit_operations == 0
9831012

1013+
def _has_investigation_keywords(self, transcript: List[Dict]) -> bool:
1014+
"""Check early user messages for investigation/troubleshooting keywords.
1015+
1016+
This check takes PRIORITY over tool-based heuristics. If investigation
1017+
keywords are found, the session is classified as INVESTIGATION regardless
1018+
of what tools were used. This fixes #1604 where troubleshooting sessions
1019+
were incorrectly blocked by development-specific checks.
1020+
1021+
Args:
1022+
transcript: List of message dictionaries
1023+
1024+
Returns:
1025+
True if investigation keywords found in early user messages
1026+
"""
1027+
# Check first 5 user messages for investigation keywords
1028+
user_messages = [m for m in transcript if m.get("type") == "user"][:5]
1029+
1030+
if not user_messages:
1031+
return False
1032+
1033+
for msg in user_messages:
1034+
content = str(msg.get("message", {}).get("content", "")).lower()
1035+
1036+
# Check for investigation keywords
1037+
for keyword in self.INVESTIGATION_KEYWORDS:
1038+
if keyword in content:
1039+
self._log(
1040+
f"Investigation keyword '{keyword}' found in user message",
1041+
"DEBUG",
1042+
)
1043+
return True
1044+
1045+
return False
1046+
9841047
def detect_session_type(self, transcript: List[Dict]) -> str:
9851048
"""Detect session type for selective consideration application.
9861049
9871050
Session Types:
9881051
- DEVELOPMENT: Code changes, tests, PR operations
9891052
- INFORMATIONAL: Q&A, help queries, capability questions
9901053
- MAINTENANCE: Documentation and configuration updates only
991-
- INVESTIGATION: Read-only exploration and analysis
1054+
- INVESTIGATION: Exploration, analysis, troubleshooting, and debugging
1055+
1056+
Detection Priority (fixes #1604):
1057+
1. Environment override (AMPLIHACK_SESSION_TYPE)
1058+
2. Investigation keywords in user messages (highest priority heuristic)
1059+
3. Tool usage patterns (code changes, tests, etc.)
1060+
1061+
The keyword detection takes priority over tool-based heuristics because
1062+
troubleshooting sessions often involve Bash commands and doc updates,
1063+
which can be misclassified as DEVELOPMENT or MAINTENANCE.
9921064
9931065
Args:
9941066
transcript: List of message dictionaries
@@ -1006,6 +1078,12 @@ def detect_session_type(self, transcript: List[Dict]) -> str:
10061078
if not transcript:
10071079
return "INFORMATIONAL"
10081080

1081+
# PRIORITY CHECK: Investigation keywords in user messages
1082+
# This takes precedence over tool-based heuristics (fixes #1604)
1083+
if self._has_investigation_keywords(transcript):
1084+
self._log("Session classified as INVESTIGATION via keyword detection", "INFO")
1085+
return "INVESTIGATION"
1086+
10091087
# Collect indicators from transcript
10101088
code_files_modified = False
10111089
doc_files_only = True

0 commit comments

Comments
 (0)