[Phase 4] Evaluate Subagent Context Isolation per Task

[frankbria]

Summary

Evaluate whether worker agents should receive fresh, isolated context per task rather than maintaining persistent context across multiple tasks, to prevent context pollution.

Background: State of the Art

From Philipp Schmid's "The Rise of Subagents":

"This addresses a key limitation of monolithic AI agents: context pollution. When a single, big and complex agent handles many tasks, its context window, number of tools, can become cluttered and less reliable."

"A subagent... has its own isolated context window."

The subagent pattern from Claude Code and Poke.com emphasizes:

• Fresh context per task: Subagent spawns with clean slate
• Task-specific tools: Only tools relevant to this specific task
• Synthesized output: Returns summary to orchestrator, not raw context

This prevents Task 3's debugging context from polluting Task 10's feature implementation.

Current State in CodeFRAME

CodeFRAME's architecture:

Lead Agent → Backend Worker / Frontend Worker / Test Worker / Review Worker

Workers appear to be persistent entities that handle multiple tasks over a session. Questions:

1. Context accumulation: Does the Backend Worker carry context from Task 1 into Task 5?
2. Tool consistency: Are worker tool sets static or task-adapted?
3. Context handoff: When Lead assigns a new task, is worker context reset or appended?

If workers accumulate context, a long session could see:

• Irrelevant error messages from early tasks
• Stale code snippets that have since changed
• Conflicting patterns from different features

Investigation Tasks

1. Audit current worker lifecycle

   • Trace context window contents across multiple task assignments
   • Measure context size growth per worker over a session
   • Identify what persists vs. what's task-specific

2. Measure context pollution impact

   • Compare task success rates: early tasks vs. late tasks in long sessions
   • Look for hallucination patterns referencing old task context
   • Check if workers reference code/files that are no longer relevant

3. Design isolation options

   Option A: Fresh context per task

   • Spawn new worker instance for each task
   • Pass only task-specific context from Lead
   • Worker returns synthesized result, then terminates
   • Pro: Maximum isolation. Con: No cross-task learning.

   Option B: Selective context reset

   • Clear task-specific context between assignments
   • Retain project-level context (architecture, patterns)
   • Pro: Balanced. Con: Harder to implement clean boundaries.

   Option C: Context summarization

   • After each task, summarize learnings into compact form
   • Clear detailed context, keep summaries
   • Pro: Preserves learning. Con: Summarization cost/accuracy.

4. Prototype and measure

   • Implement Option A for one worker type
   • Compare task success rate and token usage vs. current approach

Success Criteria

• Documented current worker context lifecycle
• Measured context size growth and pollution indicators
• Implemented isolation mechanism (if beneficial)
• A/B comparison showing impact on task success rate

Design Considerations

Orchestrator synthesis: The Lead Agent should receive only synthesized results from workers, not raw context. This keeps the orchestrator's context clean.

Learning preservation: If workers gain valuable insights ("this codebase uses async patterns everywhere"), that should persist at the project level, not pollute individual task contexts.

Cost tradeoff: Fresh contexts mean no accumulated cache benefits. Need to measure token cost of isolation vs. current approach.

References

• The Rise of Subagents - Philipp Schmid
• Agents 2.0: Deep Agents - Pillar 2: Hierarchical Delegation
• Claude Code's explicit subagent pattern

[traycerai]

Plan

Observations

CodeFRAME currently implements persistent worker agents that accumulate context across multiple task assignments. The AgentPoolManager explicitly reuses idle agents (get_or_create_agent), and context items are stored per agent_id in the database without automatic clearing between tasks. The flash_save mechanism only archives COLD tier items while retaining HOT/WARM context. Workers track tasks_completed counters, confirming they handle multiple tasks sequentially. This architecture matches the "context pollution" concern described in the ticket—Task 3's debugging context can indeed pollute Task 10's feature implementation.

Approach

The plan follows a measurement-first, then isolation strategy aligned with the ticket's investigation tasks. First, audit the current worker lifecycle to quantify context accumulation patterns and measure pollution impact on task success rates. Then design and prototype isolation mechanisms (fresh context per task, selective reset, or summarization) based on empirical data. The approach prioritizes Option A (fresh context per task) for prototyping as it provides maximum isolation and aligns with the subagent pattern from Claude Code/Poke.com, while measuring token cost tradeoffs against the current persistent approach.

Implementation Steps

Phase 1: Audit Current Worker Lifecycle

1.1 Instrument Context Tracking

Add telemetry to file:codeframe/agents/worker_agent.py and file:codeframe/lib/context_manager.py:

• In WorkerAgent.execute_task(), log context size before/after task execution
• Track context item counts by tier (HOT/WARM/COLD) per task
• Record which context items from previous tasks are loaded during current task
• Add metrics to file:codeframe/lib/metrics_tracker.py for context growth per agent

1.2 Create Context Lifecycle Audit Tool

Build diagnostic script file:scripts/audit_context_lifecycle.py:

• Query database for all agents and their task history
• Calculate context size growth over task sequence (tasks 1→5→10)
• Identify "stale" context items (created >N tasks ago but still in HOT/WARM)
• Generate report showing: agent_id, tasks_completed, total_context_tokens, stale_item_percentage

1.3 Add Session-Level Context Tracking

Extend file:codeframe/core/session_manager.py:

• Track context window size at session start/end
• Record context item age distribution (how many items from Task 1 remain at Task 10)
• Store metrics in session state JSON for longitudinal analysis

Phase 2: Measure Context Pollution Impact

2.1 Implement Task Success Rate Tracking

Modify file:codeframe/agents/lead_agent.py multi-agent execution loop:

• Track task success rate by position in sequence (early vs late tasks)
• Record self-correction attempts per task (indicator of confusion)
• Log blocker creation frequency by task position
• Store metrics in file:codeframe/persistence/database.py (new task_metrics table)

2.2 Detect Context Pollution Patterns

Create analyzer file:codeframe/lib/context_pollution_detector.py:

• Scan task execution logs for hallucinations referencing old context
• Identify cases where agents reference files/code from previous tasks
• Detect "context leakage" where Task N's error messages appear in Task N+5
• Generate pollution score: (irrelevant_context_items / total_context_items) * 100

2.3 Build Comparison Dashboard

Add UI component file:web-ui/src/components/ContextPollutionMetrics.tsx:

• Display context size growth chart per agent over task sequence
• Show task success rate: early tasks (1-3) vs late tasks (8-10)
• Visualize stale context percentage over time
• Compare agents with high vs low context pollution scores

Phase 3: Design Isolation Options

3.1 Option A - Fresh Context Per Task (Prototype)

Implement in file:codeframe/agents/agent_pool_manager.py:

New Method: create_ephemeral_agent()

• Spawn new worker instance for single task
• Pass only task-specific context from Lead Agent (task description, relevant files)
• Agent returns synthesized result (summary, file changes, test results)
• Terminate agent after task completion, clearing all context

Modify LeadAgent._assign_and_execute_task():

• Add flag use_ephemeral_workers (default: False for backward compatibility)
• When enabled, call create_ephemeral_agent() instead of get_or_create_agent()
• Collect synthesized results, store in Lead Agent's context (not worker's)

Context Handoff Protocol:

• Lead Agent extracts task-specific context: PRD section, related issues, codebase index results
• Worker receives clean slate + task context bundle
• Worker returns: {status, summary, changes, learnings} (no raw context)
• Lead Agent stores learnings at project level for future tasks

3.2 Option B - Selective Context Reset

Implement in file:codeframe/lib/context_manager.py:

New Method: reset_task_context()

• Clear all context items with item_type in [TASK, ERROR, TEST_RESULT]
• Retain CODE items (reusable patterns) and PRD_SECTION (project knowledge)
• Reassign retained items to WARM tier (age them down from HOT)

Modify WorkerAgent.execute_task():

• Call context_manager.reset_task_context() before starting new task
• Load only project-level context (architecture patterns, coding standards)
• Task-specific context starts fresh

Boundary Definition:

• Task-specific: errors, test failures, task descriptions, temporary code snippets
• Project-level: PRD sections, architecture decisions, reusable code patterns, style guides

3.3 Option C - Context Summarization

Implement in file:codeframe/lib/context_summarizer.py:

New Class: ContextSummarizer

• After task completion, call LLM to summarize learnings
• Input: all context items from completed task
• Output: 200-token summary of key insights
• Store summary as new context item with item_type=SUMMARY

Modify WorkerAgent.complete_task():

• Trigger summarization after quality gates pass
• Clear detailed task context (errors, test results)
• Retain summary in WARM tier
• Cost: ~500 tokens per summarization call

Phase 4: Prototype and Measure

4.1 Implement Option A for Backend Worker

Modify file:codeframe/agents/backend_worker_agent.py:

• Add ephemeral_mode flag to constructor
• When ephemeral, skip context loading from database
• Accept task_context_bundle parameter with pre-filtered context
• Return structured result with synthesized_output field

4.2 Create A/B Testing Framework

Build file:tests/integration/test_context_isolation_comparison.py:

Test Scenario: 10-task sequence (5 backend, 5 frontend)

• Control group: persistent workers (current behavior)
• Treatment group: ephemeral workers (Option A)

Metrics to Compare:

• Task success rate (% completed without blockers)
• Self-correction attempts per task
• Token usage (input + output tokens per task)
• Context pollution score (irrelevant items loaded)
• Time to completion per task

4.3 Run Longitudinal Experiment

Execute test suite with both modes:

• Run 5 sessions with persistent workers
• Run 5 sessions with ephemeral workers
• Collect metrics in file:experiments/context_isolation_results.json
• Generate comparison report with statistical significance tests

Phase 5: Implement Winning Strategy

5.1 Productionize Selected Option

Based on experiment results, implement winner:

If Option A wins:

• Make ephemeral mode default for all worker types
• Update file:codeframe/agents/lead_agent.py to use ephemeral workers
• Implement learning preservation at project level (Lead Agent stores patterns)

If Option B wins:

• Add reset_task_context() to worker lifecycle
• Define clear boundaries for task vs project context
• Update context tier assignment logic

If Option C wins:

• Integrate summarization into task completion flow
• Optimize summarization prompts for cost/quality
• Add summarization metrics to dashboard

5.2 Update Agent Pool Manager

Modify file:codeframe/agents/agent_pool_manager.py:

• Add configuration option: context_isolation_mode (persistent/ephemeral/selective/summarized)
• Implement mode-specific agent creation logic
• Update mark_agent_idle() to handle context cleanup based on mode

5.3 Migrate Existing Workers

Create migration path:

• Add feature flag ENABLE_CONTEXT_ISOLATION (default: False)
• Gradual rollout: enable for new projects first
• Monitor metrics for regressions
• Document migration guide in file:docs/CONTEXT_ISOLATION_MIGRATION.md

Phase 6: Documentation and Validation

6.1 Document Context Lifecycle

Update file:CLAUDE.md and file:specs/007-context-management/spec.md:

• Add section on context isolation strategy
• Document when context is reset vs retained
• Explain project-level vs task-level context boundaries
• Provide examples of context handoff protocol

6.2 Update Architecture Diagrams

Modify file:specs/CODEFRAME_SPEC.md:

sequenceDiagram
    participant Lead as Lead Agent
    participant Pool as Agent Pool
    participant Worker as Worker Agent
    participant Context as Context Manager
    
    Lead->>Pool: get_or_create_agent(type, ephemeral=True)
    Pool->>Worker: create with clean context
    Lead->>Worker: execute_task(task, context_bundle)
    Worker->>Context: load_task_context(bundle)
    Worker->>Worker: execute task
    Worker->>Lead: return synthesized_result
    Pool->>Worker: terminate (clear context)

Loading

6.3 Add Success Criteria Validation

Create validation suite file:tests/e2e/test_context_isolation_e2e.py:

Architecture Diagram

graph TD
    A[Lead Agent] -->|Task + Context Bundle| B{Isolation Mode}
    B -->|Persistent| C[Reuse Idle Worker]
    B -->|Ephemeral| D[Create Fresh Worker]
    
    C -->|Load All Context| E[Worker Executes]
    D -->|Load Bundle Only| E
    
    E -->|Complete| F{Quality Gates}
    F -->|Pass| G[Return Synthesized Result]
    F -->|Fail| H[Self-Correct]
    
    G -->|Persistent| I[Mark Idle, Keep Context]
    G -->|Ephemeral| J[Terminate, Clear Context]
    
    I -->|Next Task| C
    J -->|Next Task| D
    
    G -->|Store Learnings| K[Lead Agent Project Context]
    K -->|Next Task| A

Loading

Key Files to Modify/Create

File	Purpose
file:codeframe/agents/worker_agent.py	Add ephemeral mode, context telemetry
file:codeframe/agents/agent_pool_manager.py	Implement create_ephemeral_worker()
file:codeframe/agents/backend_worker_agent.py	Implement ephemeral mode, get_synthesized_result()
file:codeframe/agents/lead_agent.py	Add context bundle creation, learnings storage
file:codeframe/lib/context_manager.py	Add get_context_stats(), calculate_pollution_score()
file:codeframe/lib/context_bundle_builder.py	NEW: Build task-specific context bundles
file:codeframe/lib/experiment_metrics.py	NEW: Collect and export experiment metrics
file:tests/integration/test_context_isolation_ab.py	NEW: A/B testing framework
file:scripts/collect_baseline_metrics.py	NEW: Run baseline experiments
file:scripts/run_ephemeral_experiment.py	NEW: Run ephemeral experiments
file:scripts/compare_isolation_results.py	NEW: Generate comparison report
file:experiments/baseline_metrics.json	Output: Baseline results
file:experiments/ephemeral_metrics.json	Output: Ephemeral results
file:experiments/ISOLATION_COMPARISON_REPORT.md	Output: Comparison analysis
file:experiments/PHASE_4_ANALYSIS.md	Output: Findings and recommendations
file:docs/CONTEXT_ISOLATION_PHASE_4_LEARNINGS.md	Output: Learnings for Phase 5

Metrics to Collect and Compare

Metric	Baseline (Persistent)	Treatment (Ephemeral)	Success Indicator
Context tokens at Task 10	Measure	<50% of baseline	Lower is better
Task success rate (early)	Measure	≥ baseline	Maintain or improve
Task success rate (late)	Measure	≥ baseline	Improve (reduce degradation)
Self-corrections per task	Measure	<80% of baseline	Fewer retries
Pollution score (avg)	Measure	<10%	Much lower
Input tokens per task	Measure	Compare	May increase (fresh context)
Total tokens per task	Measure	Compare	Key cost metric
Task duration (seconds)	Measure	Compare	Should be similar

Success Criteria for Phase 4

• Baseline metrics collected from 5 persistent worker sessions
• Ephemeral mode implemented for Backend Worker
• Ephemeral metrics collected from 5 ephemeral worker sessions
• Comparison report generated with statistical analysis
• Phase 4 analysis document completed with recommendations
• Decision made on whether to proceed with productionization (Phase 5)

Risk Mitigation

Risk: Ephemeral mode has unacceptable token cost overhead

• Mitigation: Measure in Phase 4.4, compare against success rate gains; if cost too high, explore Option B/C

Risk: Fresh context loses critical learning, success rate drops

• Mitigation: Implement project-level learnings storage in Lead Agent; measure if sufficient

Risk: Experiment results are inconclusive

• Mitigation: Run additional sessions, increase sample size, or prototype Option B/C in parallel

Risk: Context bundle creation is expensive

• Mitigation: Optimize bundle builder, use caching, measure bundle creation cost separately

Import In IDE

🤖 Prompt for AI Agents

## Observations

CodeFRAME currently implements **persistent worker agents** that accumulate context across multiple task assignments. The `AgentPoolManager` explicitly reuses idle agents (`get_or_create_agent`), and context items are stored per `agent_id` in the database without automatic clearing between tasks. The `flash_save` mechanism only archives COLD tier items while retaining HOT/WARM context. Workers track `tasks_completed` counters, confirming they handle multiple tasks sequentially. This architecture matches the "context pollution" concern described in the ticket—Task 3's debugging context can indeed pollute Task 10's feature implementation.

## Approach

The plan follows a **measurement-first, then isolation** strategy aligned with the ticket's investigation tasks. First, audit the current worker lifecycle to quantify context accumulation patterns and measure pollution impact on task success rates. Then design and prototype isolation mechanisms (fresh context per task, selective reset, or summarization) based on empirical data. The approach prioritizes Option A (fresh context per task) for prototyping as it provides maximum isolation and aligns with the subagent pattern from Claude Code/Poke.com, while measuring token cost tradeoffs against the current persistent approach.

## Implementation Steps

### Phase 1: Audit Current Worker Lifecycle

#### 1.1 Instrument Context Tracking

Add telemetry to `file:codeframe/agents/worker_agent.py` and `file:codeframe/lib/context_manager.py`:

- In `WorkerAgent.execute_task()`, log context size before/after task execution
- Track context item counts by tier (HOT/WARM/COLD) per task
- Record which context items from previous tasks are loaded during current task
- Add metrics to `file:codeframe/lib/metrics_tracker.py` for context growth per agent

#### 1.2 Create Context Lifecycle Audit Tool

Build diagnostic script `file:scripts/audit_context_lifecycle.py`:

- Query database for all agents and their task history
- Calculate context size growth over task sequence (tasks 1→5→10)
- Identify "stale" context items (created >N tasks ago but still in HOT/WARM)
- Generate report showing: agent_id, tasks_completed, total_context_tokens, stale_item_percentage

#### 1.3 Add Session-Level Context Tracking

Extend `file:codeframe/core/session_manager.py`:

- Track context window size at session start/end
- Record context item age distribution (how many items from Task 1 remain at Task 10)
- Store metrics in session state JSON for longitudinal analysis

### Phase 2: Measure Context Pollution Impact

#### 2.1 Implement Task Success Rate Tracking

Modify `file:codeframe/agents/lead_agent.py` multi-agent execution loop:

- Track task success rate by position in sequence (early vs late tasks)
- Record self-correction attempts per task (indicator of confusion)
- Log blocker creation frequency by task position
- Store metrics in `file:codeframe/persistence/database.py` (new `task_metrics` table)

#### 2.2 Detect Context Pollution Patterns

Create analyzer `file:codeframe/lib/context_pollution_detector.py`:

- Scan task execution logs for hallucinations referencing old context
- Identify cases where agents reference files/code from previous tasks
- Detect "context leakage" where Task N's error messages appear in Task N+5
- Generate pollution score: `(irrelevant_context_items / total_context_items) * 100`

#### 2.3 Build Comparison Dashboard

Add UI component `file:web-ui/src/components/ContextPollutionMetrics.tsx`:

- Display context size growth chart per agent over task sequence
- Show task success rate: early tasks (1-3) vs late tasks (8-10)
- Visualize stale context percentage over time
- Compare agents with high vs low context pollution scores

### Phase 3: Design Isolation Options

#### 3.1 Option A - Fresh Context Per Task (Prototype)

Implement in `file:codeframe/agents/agent_pool_manager.py`:

**New Method: `create_ephemeral_agent()`**
- Spawn new worker instance for single task
- Pass only task-specific context from Lead Agent (task description, relevant files)
- Agent returns synthesized result (summary, file changes, test results)
- Terminate agent after task completion, clearing all context

**Modify `LeadAgent._assign_and_execute_task()`**:
- Add flag `use_ephemeral_workers` (default: False for backward compatibility)
- When enabled, call `create_ephemeral_agent()` instead of `get_or_create_agent()`
- Collect synthesized results, store in Lead Agent's context (not worker's)

**Context Handoff Protocol**:
- Lead Agent extracts task-specific context: PRD section, related issues, codebase index results
- Worker receives clean slate + task context bundle
- Worker returns: `{status, summary, changes, learnings}` (no raw context)
- Lead Agent stores `learnings` at project level for future tasks

#### 3.2 Option B - Selective Context Reset

Implement in `file:codeframe/lib/context_manager.py`:

**New Method: `reset_task_context()`**
- Clear all context items with `item_type` in [TASK, ERROR, TEST_RESULT]
- Retain CODE items (reusable patterns) and PRD_SECTION (project knowledge)
- Reassign retained items to WARM tier (age them down from HOT)

**Modify `WorkerAgent.execute_task()`**:
- Call `context_manager.reset_task_context()` before starting new task
- Load only project-level context (architecture patterns, coding standards)
- Task-specific context starts fresh

**Boundary Definition**:
- Task-specific: errors, test failures, task descriptions, temporary code snippets
- Project-level: PRD sections, architecture decisions, reusable code patterns, style guides

#### 3.3 Option C - Context Summarization

Implement in `file:codeframe/lib/context_summarizer.py`:

**New Class: `ContextSummarizer`**
- After task completion, call LLM to summarize learnings
- Input: all context items from completed task
- Output: 200-token summary of key insights
- Store summary as new context item with `item_type=SUMMARY`

**Modify `WorkerAgent.complete_task()`**:
- Trigger summarization after quality gates pass
- Clear detailed task context (errors, test results)
- Retain summary in WARM tier
- Cost: ~500 tokens per summarization call

### Phase 4: Prototype and Measure

#### 4.1 Implement Option A for Backend Worker

Modify `file:codeframe/agents/backend_worker_agent.py`:

- Add `ephemeral_mode` flag to constructor
- When ephemeral, skip context loading from database
- Accept `task_context_bundle` parameter with pre-filtered context
- Return structured result with `synthesized_output` field

#### 4.2 Create A/B Testing Framework

Build `file:tests/integration/test_context_isolation_comparison.py`:

**Test Scenario**: 10-task sequence (5 backend, 5 frontend)
- Control group: persistent workers (current behavior)
- Treatment group: ephemeral workers (Option A)

**Metrics to Compare**:
- Task success rate (% completed without blockers)
- Self-correction attempts per task
- Token usage (input + output tokens per task)
- Context pollution score (irrelevant items loaded)
- Time to completion per task

#### 4.3 Run Longitudinal Experiment

Execute test suite with both modes:

- Run 5 sessions with persistent workers
- Run 5 sessions with ephemeral workers
- Collect metrics in `file:experiments/context_isolation_results.json`
- Generate comparison report with statistical significance tests

### Phase 5: Implement Winning Strategy

#### 5.1 Productionize Selected Option

Based on experiment results, implement winner:

**If Option A wins**:
- Make ephemeral mode default for all worker types
- Update `file:codeframe/agents/lead_agent.py` to use ephemeral workers
- Implement learning preservation at project level (Lead Agent stores patterns)

**If Option B wins**:
- Add `reset_task_context()` to worker lifecycle
- Define clear boundaries for task vs project context
- Update context tier assignment logic

**If Option C wins**:
- Integrate summarization into task completion flow
- Optimize summarization prompts for cost/quality
- Add summarization metrics to dashboard

#### 5.2 Update Agent Pool Manager

Modify `file:codeframe/agents/agent_pool_manager.py`:

- Add configuration option: `context_isolation_mode` (persistent/ephemeral/selective/summarized)
- Implement mode-specific agent creation logic
- Update `mark_agent_idle()` to handle context cleanup based on mode

#### 5.3 Migrate Existing Workers

Create migration path:

- Add feature flag `ENABLE_CONTEXT_ISOLATION` (default: False)
- Gradual rollout: enable for new projects first
- Monitor metrics for regressions
- Document migration guide in `file:docs/CONTEXT_ISOLATION_MIGRATION.md`

### Phase 6: Documentation and Validation

#### 6.1 Document Context Lifecycle

Update `file:CLAUDE.md` and `file:specs/007-context-management/spec.md`:

- Add section on context isolation strategy
- Document when context is reset vs retained
- Explain project-level vs task-level context boundaries
- Provide examples of context handoff protocol

#### 6.2 Update Architecture Diagrams

Modify `file:specs/CODEFRAME_SPEC.md`:

```mermaid
sequenceDiagram
    participant Lead as Lead Agent
    participant Pool as Agent Pool
    participant Worker as Worker Agent
    participant Context as Context Manager
    
    Lead->>Pool: get_or_create_agent(type, ephemeral=True)
    Pool->>Worker: create with clean context
    Lead->>Worker: execute_task(task, context_bundle)
    Worker->>Context: load_task_context(bundle)
    Worker->>Worker: execute task
    Worker->>Lead: return synthesized_result
    Pool->>Worker: terminate (clear context)
```

#### 6.3 Add Success Criteria Validation

Create validation suite `file:tests/e2e/test_context_isolation_e2e.py`:

## Architecture Diagram

```mermaid
graph TD
    A[Lead Agent] -->|Task + Context Bundle| B{Isolation Mode}
    B -->|Persistent| C[Reuse Idle Worker]
    B -->|Ephemeral| D[Create Fresh Worker]
    
    C -->|Load All Context| E[Worker Executes]
    D -->|Load Bundle Only| E
    
    E -->|Complete| F{Quality Gates}
    F -->|Pass| G[Return Synthesized Result]
    F -->|Fail| H[Self-Correct]
    
    G -->|Persistent| I[Mark Idle, Keep Context]
    G -->|Ephemeral| J[Terminate, Clear Context]
    
    I -->|Next Task| C
    J -->|Next Task| D
    
    G -->|Store Learnings| K[Lead Agent Project Context]
    K -->|Next Task| A
```

## Key Files to Modify/Create

| File | Purpose |
|------|---------|
| `file:codeframe/agents/worker_agent.py` | Add ephemeral mode, context telemetry |
| `file:codeframe/agents/agent_pool_manager.py` | Implement `create_ephemeral_worker()` |
| `file:codeframe/agents/backend_worker_agent.py` | Implement ephemeral mode, `get_synthesized_result()` |
| `file:codeframe/agents/lead_agent.py` | Add context bundle creation, learnings storage |
| `file:codeframe/lib/context_manager.py` | Add `get_context_stats()`, `calculate_pollution_score()` |
| `file:codeframe/lib/context_bundle_builder.py` | NEW: Build task-specific context bundles |
| `file:codeframe/lib/experiment_metrics.py` | NEW: Collect and export experiment metrics |
| `file:tests/integration/test_context_isolation_ab.py` | NEW: A/B testing framework |
| `file:scripts/collect_baseline_metrics.py` | NEW: Run baseline experiments |
| `file:scripts/run_ephemeral_experiment.py` | NEW: Run ephemeral experiments |
| `file:scripts/compare_isolation_results.py` | NEW: Generate comparison report |
| `file:experiments/baseline_metrics.json` | Output: Baseline results |
| `file:experiments/ephemeral_metrics.json` | Output: Ephemeral results |
| `file:experiments/ISOLATION_COMPARISON_REPORT.md` | Output: Comparison analysis |
| `file:experiments/PHASE_4_ANALYSIS.md` | Output: Findings and recommendations |
| `file:docs/CONTEXT_ISOLATION_PHASE_4_LEARNINGS.md` | Output: Learnings for Phase 5 |

## Metrics to Collect and Compare

| Metric | Baseline (Persistent) | Treatment (Ephemeral) | Success Indicator |
|--------|----------------------|----------------------|-------------------|
| Context tokens at Task 10 | Measure | <50% of baseline | Lower is better |
| Task success rate (early) | Measure | ≥ baseline | Maintain or improve |
| Task success rate (late) | Measure | ≥ baseline | Improve (reduce degradation) |
| Self-corrections per task | Measure | <80% of baseline | Fewer retries |
| Pollution score (avg) | Measure | <10% | Much lower |
| Input tokens per task | Measure | Compare | May increase (fresh context) |
| Total tokens per task | Measure | Compare | Key cost metric |
| Task duration (seconds) | Measure | Compare | Should be similar |

## Success Criteria for Phase 4

- [x] Baseline metrics collected from 5 persistent worker sessions
- [x] Ephemeral mode implemented for Backend Worker
- [x] Ephemeral metrics collected from 5 ephemeral worker sessions
- [x] Comparison report generated with statistical analysis
- [x] Phase 4 analysis document completed with recommendations
- [x] Decision made on whether to proceed with productionization (Phase 5)

## Risk Mitigation

**Risk**: Ephemeral mode has unacceptable token cost overhead
- **Mitigation**: Measure in Phase 4.4, compare against success rate gains; if cost too high, explore Option B/C

**Risk**: Fresh context loses critical learning, success rate drops
- **Mitigation**: Implement project-level learnings storage in Lead Agent; measure if sufficient

**Risk**: Experiment results are inconclusive
- **Mitigation**: Run additional sessions, increase sample size, or prototype Option B/C in parallel

**Risk**: Context bundle creation is expensive
- **Mitigation**: Optimize bundle builder, use caching, measure bundle creation cost separately

---

Execution Information

Branch: main
Commit: 4d894a6

---

💡 Tips

Supported Commands (Inside Comments)

• Use @traycerai generate to iterate on the previous version of the implementation plan.

Supported Commands (Inside Description)

• Add @traycerai ignore anywhere in the ticket description to prevent this ticket from being processed.
• Add @traycerai branch:<branch-name> anywhere in the ticket description to specify the target branch for the implementation plan.

Community

• Join our Discord Community to get help, request features, and share feedback.
• Follow us on X/Twitter for updates and announcements.