[EPIC] Agentic enhancements: intelligent search, orchestration, and evaluation #123
Description
Overview
Enhance CodeWeaver with agentic capabilities for intelligent code search, orchestrated reasoning, and evaluation. This builds on the scaffolded agent and data provider infrastructure.
The Vision
Transform CodeWeaver from a semantic search system into an intelligent code understanding platform with:
- Agent-enhanced search: LLM agents that reason about search strategy
- Data source integration: External context (web search, documentation, etc.)
- Context agents: Internal orchestration for multi-step reasoning
- Graph-based pipelines: Structured orchestration with pydantic-graph
- Evaluation framework: Continuous quality improvement with pydantic-eval
Scaffolded Infrastructure
Already in place:
codeweaver.providers.agent- Thin wrapper around pydantic-aicodeweaver.providers.data- Data provider scaffolding- Registry integration in
codeweaver.common.registry.providerand.models
Implementation Phases
Phase 1: Agent Integration in find_code Pipeline - See #124
- Integrate pydantic-ai agents into search pipeline
- Agent-driven query refinement and strategy selection
- Tool integration for code-aware reasoning
Phase 2: Data Provider Integration - See #125
- Integrate external data sources (Tavily, DuckDuckGo, etc.)
- Context enrichment from documentation and web
- Data source orchestration
Phase 3: Context Agent Tooling - See #126
- Internal "context agent" for orchestrated search/response
- Multi-step reasoning over code context
- Intelligent result synthesis
Phase 4: pydantic-graph Pipeline Orchestration - See #127
- Structured pipeline orchestration with pydantic-graph
- Strategy-based execution paths
- Complex workflow composition
Phase 5: Evaluation Framework with pydantic-eval - See #128
- Agent performance evaluation
- Pipeline quality metrics
- Continuous improvement infrastructure
Dependencies
- DI System ([EPIC] DI + Monorepo: Unified architecture transformation #116): Phase 3 ([DI Phase 4] pydantic-ai integration (agents + data sources) #119) provides DI for agents and data sources
- Provider System (Implement provider system for reranker models (mirror embedding system) #109): Reranker provider system for quality improvements
- May require additional provider capabilities (Expand vector provider support and fix capabilities definition #104)
Success Criteria
- Agents successfully enhance search quality
- External data sources integrated seamlessly
- Context agents provide intelligent multi-step reasoning
- Pipelines are composable and maintainable via pydantic-graph
- Evaluation framework enables continuous improvement
- Performance remains acceptable (< 2x baseline latency)
Constitutional Alignment
✅ Empirical Approach (Principle III): Evaluation framework enables evidence-based improvements
✅ Proven Patterns (Principle II): Leverages pydantic-ai, pydantic-graph, pydantic-eval
✅ AI-First Context (Principle I): Agents enhance code understanding
✅ Simplicity (Principle V): Graph-based orchestration clarifies complex workflows
Related Work
This work complements but is distinct from:
- [EPIC] DI + Monorepo: Unified architecture transformation #116 (DI System): Infrastructure that enables agent injection
- Implement provider system for reranker models (mirror embedding system) #109 (Reranker System): Quality improvements work together
- Add monitoring: track current file, provider health checks, latency tracking #112 (Monitoring): Telemetry for agent performance
Source
- Scaffolded code:
src/codeweaver/providers/agent/,src/codeweaver/providers/data/ - Registry integration:
src/codeweaver/common/registry/ - Branch:
003-our-aim-to