feat(r14): unify error context formatting with PromptComposer, ring buffer logs, and summary-first prompts

Author: rookiestar28

ROADMAP.md

@@ -292,14 +292,24 @@ graph TD
 
 *Sorted by priority (High → Low):*
 
-- [ ] **R14**: Error context extraction & prompt packaging optimization - 🔴 High ⚠️ *Use dev branch*
+- [x] **R14**: Error context extraction & prompt packaging optimization - 🔴 High ✅ *Completed (2026-01-14)*
   - **Problem**: LLM context is often dominated by raw tracebacks; log context capture is unreliable; env/pip list can waste tokens.
   - **Approach**:
     - Add `error_summary` before full traceback (exception line + optional pattern/category)
     - Reliable execution log context via a dedicated ring buffer (not dependent on ComfyUI logger handlers)
     - Expand non-traceback error triggers (single-line fatals, validation failures) with strict dedup/noise filtering
     - Build structured LLM context via pipeline (`llm_builder.py`) + token budgets (R12) instead of ad-hoc string concatenation
   - **Plan**: `.planning/260113-R14_ERROR_CONTEXT_EXTRACTION_OPTIMIZATION_PLAN.md`
+  - **Implementation Record**: `.planning/260113-R14_ERROR_CONTEXT_EXTRACTION_IMPLEMENTATION_RECORD.md`
+- [ ] **R15**: Canonicalize `system_info` + populate pipeline `execution_logs` - 🟡 Medium 🧩 *Follow-up*
+  - **Context**: R14 core is complete; remaining improvements are quality/consistency enhancements (not gating R14 acceptance).
+  - **Scope**:
+    - Normalize `get_system_environment()` output into a PromptComposer-friendly canonical shape (single schema across endpoints)
+    - Implement smart package selection for `system_info.packages` (keyword-based filtering; strict cap by default)
+    - Populate pipeline context `execution_logs` from `LogRingBuffer` (so pipeline and endpoints share the same log source)
+  - **Acceptance**:
+    - Prompts show consistent `system_info` formatting regardless of endpoint
+    - Pipeline-produced LLM context includes recent execution logs without relying on ComfyUI handler `.buffer`
 - [x] **R12**: Smart Token Budget Management - 🟡 Medium ✅ *Completed (2026-01-10)*
   - **Core Strategy**: Progressive trimming system with token estimation for LLM context management
   - **Implemented Features**:
@@ -315,8 +325,8 @@ graph TD
     - **Enhanced Metadata**: R12Metadata v1.0 schema for observability
     - **A/B Validation Harness**: Quality metrics tracking (`scripts/r12_ab_harness.py`)
   - **Deferred Features** (not implemented):
-    - ⏸️ Smart pip list filtering (keyword extraction from errors) → planned in **R14**
-    - ⏸️ Stack frame collapsing (first 5 + last 5 frames) → planned in **R14**
+    - ⏸️ Smart pip list filtering (keyword extraction from errors) → follow-up in **R15**
+    - ✅ Stack frame collapsing (first N + last M frames) → delivered in **R14**
   - **Cost Impact**: 40-60% token reduction estimated, saving $24-36 per 1000 analyses (GPT-4)
   - **Implementation**: `.planning/260110-R12_SMART_TOKEN_BUDGET_IMPLEMENTATION_RECORD.md`
   - **Integration**: `services/token_estimator.py`, `services/workflow_pruner.py`, `services/token_budget.py`

__init__.py

@@ -67,6 +67,10 @@ class DiagnosticsConfig:
     r12_prune_default_nodes: int = 40
     r12_overhead_fixed: int = 1000  # Fixed overhead (reserved tokens for structure)
 
+    # R14: Error Context Extraction & Prompt Optimization
+    r14_use_prompt_composer: bool = True  # Use PromptComposer for unified context formatting
+    r14_use_legacy_format: bool = False   # Fallback to legacy format (traceback-first)
+    
     def to_dict(self) -> dict:
         """Convert config to dictionary."""
         return asdict(self)

config.py

@@ -28,11 +28,21 @@
     from .analyzer import ErrorAnalyzer
     from .config import CONFIG
     from .history_store import HistoryStore, HistoryEntry
+    from services.log_ring_buffer import get_ring_buffer
+    from services.context_extractor import detect_fatal_pattern
 except ImportError:
     # Fallback for direct execution (tests)
     from analyzer import ErrorAnalyzer
     from config import CONFIG
     from history_store import HistoryStore, HistoryEntry
+    try:
+        from services.log_ring_buffer import get_ring_buffer
+    except ImportError:
+        get_ring_buffer = None
+    try:
+        from services.context_extractor import detect_fatal_pattern
+    except ImportError:
+        detect_fatal_pattern = None
 
 
 # ==============================================================================
@@ -281,6 +291,14 @@ def write(self, data):
             self._queue.put_nowait(data, priority=priority)
         except Exception:
             pass
+        
+        # R14: Add to ring buffer for reliable log context capture
+        try:
+            if get_ring_buffer:
+                ring_buffer = get_ring_buffer()
+                ring_buffer.add_line(data)
+        except Exception:
+            pass  # Never fail on ring buffer operations
 
     def flush(self):
         """Flush original stream."""
@@ -370,7 +388,17 @@ def _process_message(self, message):
                 return  # Skip Doctor's own output to prevent recursion
 
         # P3: Urgent single-line warnings (immediate analysis)
-        if "❌ CRITICAL" in message or "⚠️ Meta Tensor" in message:
+        # R14: Enhanced with detect_fatal_pattern for non-traceback errors
+        is_urgent = "❌ CRITICAL" in message or "⚠️ Meta Tensor" in message
+        
+        # R14: Check for fatal patterns (CUDA OOM, CRITICAL, etc.)
+        if not is_urgent and detect_fatal_pattern:
+            fatal_marker = detect_fatal_pattern(message)
+            if fatal_marker:
+                is_urgent = True
+                logging.debug(f"[Doctor] R14 fatal pattern detected: {fatal_marker}")
+        
+        if is_urgent:
             result = ErrorAnalyzer.analyze(message)
             suggestion, metadata = result if result else (None, None)
             if suggestion:

logger.py

@@ -57,6 +57,10 @@ class AnalysisContext:
 
     # R12 / LLM Data
     llm_context: Optional[Dict[str, Any]] = None
+    
+    # R14: Error Context Extraction
+    error_summary: Optional[str] = None  # Short summary (exception type + message)
+    execution_logs: List[str] = field(default_factory=list)  # Recent log lines from ring buffer
 
     def __post_init__(self) -> None:
         # Ensure metadata contract version is always present

pipeline/context.py

@@ -4,22 +4,33 @@
 from ..context import AnalysisContext
 try:
     from services.workflow_pruner import WorkflowPruner
+    from services.context_extractor import (
+        extract_error_summary,
+        collapse_stack_frames,
+        build_context_manifest,
+    )
 except ImportError:
     # Use relative import for flexibility
     import sys
     import os
     sys.path.append(os.path.abspath(os.path.join(os.path.dirname(__file__), "../..")))
     from services.workflow_pruner import WorkflowPruner
+    from services.context_extractor import (
+        extract_error_summary,
+        collapse_stack_frames,
+        build_context_manifest,
+    )
 
 logger = logging.getLogger(__name__)
 
 class LLMContextBuilderStage(PipelineStage):
     """
     Stage 4: LLM Context Builder.
-    Prepares the context for LLM analysis (R12).
-    - Prunes workflow JSON to relevant subgraph.
-    - Estimates token usage.
-    - Structures data for the LLM prompt.
+    Prepares the context for LLM analysis (R12 + R14).
+    - Extracts error summary (R14: summary-first prompt)
+    - Collapses stack frames (R14: semantic truncation)
+    - Prunes workflow JSON to relevant subgraph (R12)
+    - Builds context manifest for observability (R14)
     """
 
     def __init__(self, workflow_pruner: WorkflowPruner):
@@ -32,8 +43,8 @@ def __init__(self, workflow_pruner: WorkflowPruner):
         self._name = "LLMContextBuilderStage"
         self.stage_id = "llm_context_builder"
         self.requires = ["sanitized_traceback"]
-        self.provides = ["llm_context", "metadata.estimated_tokens"]
-        self.version = "1.0"
+        self.provides = ["llm_context", "metadata.estimated_tokens", "metadata.context_manifest"]
+        self.version = "2.0"  # R14 upgrade
         self.pruner = workflow_pruner
 
     @property
@@ -42,12 +53,36 @@ def name(self) -> str:
 
     def process(self, context: AnalysisContext) -> None:
         """
-        Builds the LLM context.
+        Builds the LLM context with R14 optimizations.
+        
+        Order of sections in llm_context (summary-first):
+        1. error_summary - Short exception type + message
+        2. node_info - Failed node details
+        3. traceback - Collapsed if long
+        4. execution_logs - Recent log lines
+        5. workflow_subset - Pruned workflow
+        6. system_info - Environment info
         """
         if not context.sanitized_traceback:
             return
 
-        # 1. Prune Workflow if available
+        # R14 Step 1: Extract error summary
+        pattern_category = context.metadata.get("pattern_match", {}).get("category")
+        error_summary = extract_error_summary(
+            context.sanitized_traceback,
+            pattern_category=pattern_category
+        )
+        if error_summary:
+            context.error_summary = error_summary.to_string()
+
+        # R14 Step 2: Collapse stack frames for token efficiency
+        collapsed_traceback = collapse_stack_frames(
+            context.sanitized_traceback,
+            head_frames=3,
+            tail_frames=5
+        )
+
+        # Step 3: Prune Workflow if available
         pruned_workflow = None
         if context.workflow_json and context.node_context and context.node_context.node_id:
             try:
@@ -57,23 +92,34 @@ def process(self, context: AnalysisContext) -> None:
                 )
             except Exception as e:
                 logger.warning(f"Workflow pruning failed: {e}")
-                # Fallback to original or summary?
-                # For now, maybe just keep original or nothing
                 pruned_workflow = context.workflow_json
 
-        # 2. Build LLM Context Dict
+        # Step 4: Build LLM Context Dict (summary-first order)
         llm_data = {
-            "traceback": context.sanitized_traceback,
+            "error_summary": context.error_summary,  # R14: First
             "node_info": context.node_context.to_dict() if context.node_context else {},
+            "traceback": collapsed_traceback,  # R14: Collapsed
+            "execution_logs": context.execution_logs,  # R14: From ring buffer
             "workflow_subset": pruned_workflow,
             "system_info": context.system_info
         }
 
         context.llm_context = llm_data
 
-        # 3. Estimate Tokens (Optional Metadata)
+        # R14 Step 5: Build context manifest for observability
+        manifest = build_context_manifest(
+            traceback_text=context.sanitized_traceback,
+            execution_logs=context.execution_logs,
+            workflow_json=context.workflow_json,
+            system_info=context.system_info,
+            error_summary=error_summary
+        )
+        context.add_metadata("context_manifest", manifest.to_dict())
+        
+        # Step 6: Estimate Tokens (Optional Metadata)
         try:
             tokens = self.pruner.estimate_tokens(llm_data)
             context.add_metadata("estimated_tokens", tokens)
         except Exception:
             pass
+