diff --git a/.changeset/context-quality-v03.md b/.changeset/context-quality-v03.md new file mode 100644 index 0000000..2966090 --- /dev/null +++ b/.changeset/context-quality-v03.md @@ -0,0 +1,51 @@ +--- +"@lytics/dev-agent": minor +"@lytics/dev-agent-core": minor +"@lytics/dev-agent-mcp": minor +"@lytics/dev-agent-subagents": minor +--- + +feat: Context Quality release (v0.3.0) + +This release significantly enhances dev-agent's ability to provide rich, actionable context to AI assistants. + +## New Tools + +### `dev_refs` - Relationship Queries +Query code relationships to understand what calls what: +- Find all callers of a function +- Find all callees (what a function calls) +- Includes file paths, line numbers, and snippets + +### `dev_map` - Codebase Overview +Get a high-level view of repository structure: +- Directory tree with component counts +- **Hot Paths**: Most referenced files in the codebase +- **Smart Depth**: Adaptive expansion based on information density +- **Signatures**: Function/class signatures in export listings +- Configurable depth and focus directory + +## Enhanced Tools + +### `dev_plan` - Context Assembler (Breaking Change) +Completely refactored from heuristic task breakdown to context assembly: +- Returns rich context package instead of task lists +- Includes issue details with comments +- Includes relevant code snippets from semantic search +- Includes detected codebase patterns +- Let LLMs do the reasoning with better data + +**Migration:** The old task breakdown output is removed. The new output provides strictly more information for LLMs to create their own plans. + +### `dev_search` - Richer Results (from v0.2.0) +- Code snippets included in results +- Import statements for context +- Caller/callee hints +- Progressive disclosure based on token budget + +## Philosophy + +This release embraces the principle: **Provide structured data, let LLMs reason.** + +Instead of trying to be smart with heuristics, dev-agent now focuses on assembling comprehensive context that AI assistants can use effectively. + diff --git a/README.md b/README.md index c624246..52a612b 100644 --- a/README.md +++ b/README.md @@ -4,424 +4,260 @@ [![pnpm](https://img.shields.io/badge/pnpm-8.15.4-orange.svg)](https://pnpm.io/) [![License: MIT](https://img.shields.io/badge/License-MIT-blue.svg)](LICENSE) -**Deep code intelligence + specialized AI subagents, exposed via MCP protocol.** +**Local-first repository context provider for AI tools. Semantic code search, relationship queries, and codebase mapping via MCP.** ## What is dev-agent? -dev-agent combines two powerful capabilities: +dev-agent provides **rich, structured context** to AI assistants like Claude and Cursor. Instead of AI tools reading files one at a time, dev-agent gives them: -1. **🧠 Deep Code Intelligence** - TypeScript/JavaScript AST analysis, semantic search, type-aware understanding -2. **πŸ€– Specialized Subagents** - Action-capable AI agents for planning, exploring, and automating code workflows +- πŸ” **Semantic search** with code snippets and relationships +- πŸ—ΊοΈ **Codebase maps** showing structure and hot paths +- πŸ”— **Relationship queries** (what calls what) +- πŸ“‹ **Issue context** assembled for planning -Unlike generic code search tools or agent platforms, dev-agent specializes in **understanding and acting on codebases**. +**Philosophy:** Provide data, let LLMs reason. We don't try to be smart with heuristicsβ€”we provide comprehensive context so AI assistants can be smart. -### Key Features +## Quick Start -**Intelligence Layer:** -- 🌍 **TypeScript & JavaScript analysis** - Deep AST analysis using ts-morph and TypeScript Compiler API -- πŸ“ **Markdown parsing** - Documentation extraction with remark -- πŸ” **Semantic + structural search** - Combine vector embeddings with AST relationships -- πŸ“Š **Type-aware** - Deep integration with TypeScript types and inference -- πŸ“¦ **Local-first** - Works 100% offline with local embeddings +```bash +# Install globally +npm install -g dev-agent -**Subagent Layer:** -- πŸ“‹ **Planner** - Breaks down GitHub issues into actionable tasks using code context -- πŸ”Ž **Explorer** - Discovers patterns and relationships across your codebase -- πŸ”§ **PR Manager** - Automates PR creation with AI-generated, context-aware descriptions -- 🎯 **Coordinator** - Orchestrates multi-agent workflows with message passing +# Index your repository +cd /path/to/your/repo +dev index . -**Infrastructure:** -- πŸ“‘ **Event Bus** - Async pub/sub communication between components -- πŸ“ˆ **Observability** - Request tracking, structured logging, metrics (p50/p95/p99) -- πŸ’Ύ **Pluggable Storage** - StorageAdapter pattern for session and persistent state -- πŸ”„ **Request Correlation** - Track requests across the system with request IDs +# Install MCP integration +dev mcp install --cursor # For Cursor IDE +dev mcp install # For Claude Code -**Integration:** -- πŸ”Œ **MCP-native** - Full protocol support with tools, prompts, and resources -- 🎯 **8 Guided Prompts** - Workflow templates (analyze-issue, find-pattern, repo-overview, etc.) -- πŸͺ™ **Token Cost Visibility** - Real-time cost tracking with accurate estimation (<1% error) -- πŸ™ **GitHub-integrated** - Native issue/PR search with offline caching via GitHub CLI +# That's it! AI tools now have access to dev-agent capabilities. +``` -## Project Structure +## MCP Tools + +When integrated with Cursor or Claude Code, dev-agent provides 8 powerful tools: + +### `dev_search` - Semantic Code Search +Natural language search with rich results including code snippets, imports, and relationships. ``` -dev-agent/ -β”œβ”€β”€ packages/ -β”‚ β”œβ”€β”€ core/ # Core context provider -β”‚ β”‚ β”œβ”€β”€ src/ -β”‚ β”‚ β”‚ β”œβ”€β”€ scanner/ # Repository scanning -β”‚ β”‚ β”‚ β”œβ”€β”€ vector/ # Vector storage (LanceDB) -β”‚ β”‚ β”‚ β”œβ”€β”€ github/ # GitHub integration -β”‚ β”‚ β”‚ β”œβ”€β”€ context/ # Context provider -β”‚ β”‚ β”‚ β”œβ”€β”€ events/ # Event bus (pub/sub) -β”‚ β”‚ β”‚ β”œβ”€β”€ observability/# Logging, metrics, request tracking -β”‚ β”‚ β”‚ β”œβ”€β”€ utils/ # Retry logic, helpers -β”‚ β”‚ β”‚ └── api/ # HTTP API server -β”‚ β”‚ -β”‚ β”œβ”€β”€ cli/ # Command-line interface -β”‚ β”‚ β”œβ”€β”€ src/ -β”‚ β”‚ β”‚ β”œβ”€β”€ commands/ # CLI commands -β”‚ β”‚ β”‚ β”œβ”€β”€ ui/ # Terminal UI -β”‚ β”‚ β”‚ └── index.ts # Main CLI entry -β”‚ β”‚ -β”‚ β”œβ”€β”€ subagents/ # Subagent system -β”‚ β”‚ β”œβ”€β”€ src/ -β”‚ β”‚ β”‚ β”œβ”€β”€ coordinator/ # Coordinator + context + storage + circular buffers -β”‚ β”‚ β”‚ β”œβ”€β”€ planner/ # Planner subagent -β”‚ β”‚ β”‚ β”œβ”€β”€ explorer/ # Explorer subagent -β”‚ β”‚ β”‚ β”œβ”€β”€ github/ # GitHub indexer -β”‚ β”‚ β”‚ └── pr/ # PR subagent -β”‚ β”‚ -β”‚ β”œβ”€β”€ mcp-server/ # MCP protocol server -β”‚ β”‚ β”œβ”€β”€ src/ -β”‚ β”‚ β”‚ β”œβ”€β”€ adapters/ # Tool adapters (search, status, plan, explore, github, health) -β”‚ β”‚ β”‚ β”œβ”€β”€ server/ # MCP server, prompts, protocol, rate limiting -β”‚ β”‚ β”‚ └── formatters/ # Output formatting with token estimation -β”‚ β”‚ -β”‚ └── integrations/ # Tool integrations -β”‚ β”œβ”€β”€ claude/ # Claude Code integration -β”‚ └── vscode/ # VS Code extension -β”‚ -β”œβ”€β”€ examples/ # Example projects and usage -β”œβ”€β”€ docs/ # Documentation -└── PLAN.md # Implementation plan +Find authentication middleware that handles JWT tokens ``` -## Getting Started +**Features:** +- Code snippets included (not just file paths) +- Import statements for context +- Caller/callee hints +- Progressive disclosure based on token budget -### Prerequisites +### `dev_refs` - Relationship Queries ✨ New in v0.3 +Query what calls what and what is called by what. -- [Node.js](https://nodejs.org/) (v22 LTS or higher) -- [PNPM](https://pnpm.io/) (v8 or higher) +``` +Find all callers of the authenticate function +Find what functions validateToken calls +``` -## Technology Stack +**Features:** +- Bidirectional queries (callers/callees) +- File paths and line numbers +- Relevance scoring -**Core:** -- TypeScript (strict mode) -- Node.js >= 22 LTS -- pnpm 8.15.4 (package management) -- Turborepo (monorepo build orchestration) +### `dev_map` - Codebase Overview ✨ New in v0.3 +Get a high-level view of repository structure. -**Intelligence Layer:** -- ts-morph (TypeScript AST analysis) -- TypeScript Compiler API (type inference and relationships) -- remark (Markdown documentation parsing) -- @xenova/transformers (local embeddings with all-MiniLM-L6-v2) -- LanceDB (embedded vector storage) +``` +Show me the codebase structure with depth 3 +Focus on the packages/core directory +``` -**Subagent Layer:** -- Message-based agent coordination (inspired by claude-flow patterns) -- GitHub CLI (native GitHub integration) +**Features:** +- Directory tree with component counts +- **Hot Paths:** Most referenced files +- **Smart Depth:** Adaptive expansion based on density +- **Signatures:** Function/class signatures in exports -**Production Features:** -- Token bucket rate limiting (100 req/min burst) -- Exponential backoff retry logic with jitter -- Circular buffers for memory leak prevention -- Component health monitoring -- Event listener cleanup +**Example output:** +```markdown +# Codebase Map -**Tooling:** -- Biome (linting & formatting) -- Vitest (testing - 1100+ tests) -- GitHub Actions (CI/CD) +## Hot Paths (most referenced) +1. `packages/core/src/indexer/index.ts` (RepositoryIndexer) - 47 refs +2. `packages/core/src/vector/store.ts` (LanceDBVectorStore) - 32 refs -## Philosophy +## Directory Structure -**Built for personal productivity first.** Not chasing GitHub stars or trying to be everything to everyone. +└── packages/ (195 components) + β”œβ”€β”€ core/ (45 components) + β”‚ └── exports: function search(query): Promise, class RepositoryIndexer + β”œβ”€β”€ mcp-server/ (28 components) + β”‚ └── exports: class MCPServer, function createAdapter(config): Adapter +``` -### Why CLI-First? -- ⚑ **Fast**: Instant feedback, no waiting for UIs to load -- 🎨 **Beautiful**: Terminal output can be elegant (see examples below) -- πŸ”§ **Scriptable**: JSON mode for automation and CI/CD -- 🌍 **Universal**: Works anywhere, integrations come later +### `dev_plan` - Context Assembly ✨ Refactored in v0.3 +Assemble rich context for implementing GitHub issues. -### Why This Exists? -Tired of: -- ❌ Grepping through codebases to understand patterns -- ❌ Breaking down GitHub issues manually -- ❌ Writing PR descriptions from scratch -- ❌ Tools that don't understand multi-language repos +``` +Assemble context for issue #42 +``` -Want: -- βœ… Semantic search that actually understands code -- βœ… AI agents that can plan and explore codebases -- βœ… Something that works locally (no API keys required) -- βœ… A tool I'll actually use daily +**Returns:** +- Full issue with comments +- Relevant code snippets from semantic search +- Detected codebase patterns (test naming, locations) +- Metadata (tokens, timing) -### What Makes It Different? +**Note:** This tool no longer generates task breakdowns. It provides comprehensive context so the AI assistant can create better plans. -**vs Code Search Tools:** Adds action-capable subagents (planner, explorer) -**vs Agent Platforms:** Specializes in deep code understanding -**vs IDE Extensions:** CLI-first, works anywhere, add integrations later +### `dev_explore` - Code Exploration +Discover patterns, find similar code, analyze relationships. -See [ARCHITECTURE.md](./ARCHITECTURE.md) for detailed design decisions. +``` +Find code similar to src/auth/middleware.ts +Search for error handling patterns +``` ---- +### `dev_status` - Repository Status +View indexing status, component health, and repository information. -## Getting Started +### `dev_gh` - GitHub Search +Search issues and PRs with semantic understanding. -### Prerequisites +``` +Find authentication-related bugs +Search for performance issues in closed PRs +``` -- [Node.js](https://nodejs.org/) (v22 LTS or higher) -- [PNPM](https://pnpm.io/) (v8.15.4 or higher) -- [GitHub CLI](https://cli.github.com/) (for GitHub integration features) +### `dev_health` - Health Monitoring +Check MCP server and component health. -### Installation +## Key Features -**Option 1: Global Install (Recommended for Users)** +### Local-First +- πŸ“¦ Works 100% offline +- πŸ” Your code never leaves your machine +- ⚑ Fast local embeddings with all-MiniLM-L6-v2 -```bash -# Install globally via npm -npm install -g dev-agent +### Production Ready +- πŸ›‘οΈ Rate limiting (100 req/min burst) +- πŸ”„ Retry logic with exponential backoff +- πŸ’š Health monitoring +- 🧹 Memory-safe (circular buffers) +- βœ… 1300+ tests -# Verify installation -dev --version +### Token Efficient +- πŸͺ™ Progressive disclosure based on budget +- πŸ“Š Token estimation in results +- 🎯 Smart depth for codebase maps -# Index your repository -cd /path/to/your/repository -dev index . +## Installation -# Install MCP integration -dev mcp install --cursor # For Cursor IDE -dev mcp install # For Claude Code +### Prerequisites + +- [Node.js](https://nodejs.org/) v22 LTS or higher +- [pnpm](https://pnpm.io/) v8.15.4 or higher +- [GitHub CLI](https://cli.github.com/) (for GitHub features) + +### Global Install (Recommended) + +```bash +npm install -g dev-agent ``` -**Option 2: From Source (For Development)** +### From Source ```bash -# Clone the repository git clone https://github.com/lytics/dev-agent.git cd dev-agent - -# Install dependencies pnpm install - -# Build all packages pnpm build - -# Link CLI globally -cd packages/cli +cd packages/dev-agent npm link - -# Now you can use 'dev' anywhere -cd ~/your-project -dev index . ``` -**Basic Commands:** +## CLI Commands ```bash -# Index your repository +# Index repository dev index . -# Semantic search (natural language queries work!) +# Semantic search dev search "how do agents communicate" -dev search "vector embeddings" dev search "error handling" --threshold 0.3 -# Explore code patterns -dev explore pattern "test coverage utilities" --limit 5 -dev explore similar path/to/file.ts - # GitHub integration dev gh index # Index issues and PRs -dev gh search "authentication bug" # Search with semantic understanding +dev gh search "authentication bug" # Semantic search -# View statistics (now includes GitHub!) +# View statistics dev stats -``` - -**Real Example Output:** - -```bash -$ dev search "vector embeddings" --threshold 0.3 --limit 3 -1. EmbeddingProvider (58.5% match) - File: packages/core/src/vector/types.ts:36-54 - Signature: interface EmbeddingProvider - Doc: Generates vector embeddings from text - -2. EmbeddingDocument (51.0% match) - File: packages/core/src/vector/types.ts:8-12 - Signature: interface EmbeddingDocument - -3. VectorStore (47.9% match) - File: packages/core/src/vector/types.ts:60-97 - Signature: interface VectorStore - Doc: Stores and retrieves vector embeddings - -βœ” Found 3 result(s) -``` - -**MCP Integration (via Cursor/Claude):** - -``` -$ dev_search with query "token estimation" -1. [71%] function: estimateTokensForText (packages/mcp-server/src/formatters/utils.ts:15) -2. [31%] method: VerboseFormatter.estimateTokens (...) -3. [31%] function: estimateTokensForJSON (...) - -πŸͺ™ ~109 tokens -``` - -Notice the πŸͺ™ token footer - helps you track AI costs in real-time! - -## MCP Tools - -When integrated with Cursor or Claude Code, dev-agent provides 6 powerful tools: - -### 1. `dev_search` - Semantic Code Search -Natural language search across your indexed repository. -``` -Find authentication middleware that handles JWT tokens -``` - -### 2. `dev_status` - Repository Status -View indexing status, component health, and repository information. -``` -Show repository status +# MCP management +dev mcp install --cursor # Install for Cursor +dev mcp install # Install for Claude Code +dev mcp list # List configured servers ``` -### 3. `dev_explore` - Code Exploration -Discover patterns, find similar code, analyze relationships. -``` -Find code similar to src/auth/middleware.ts -``` +## Project Structure -### 4. `dev_plan` - Implementation Planning -Generate actionable plans from GitHub issues with code context. ``` -Create a plan for issue #42 +dev-agent/ +β”œβ”€β”€ packages/ +β”‚ β”œβ”€β”€ core/ # Scanner, vector storage, indexer +β”‚ β”œβ”€β”€ cli/ # Command-line interface +β”‚ β”œβ”€β”€ subagents/ # Planner, explorer, PR agents +β”‚ β”œβ”€β”€ mcp-server/ # MCP protocol server + adapters +β”‚ └── integrations/ # Claude Code, VS Code +β”œβ”€β”€ docs/ # Documentation +└── website/ # Documentation website ``` -### 5. `dev_gh` - GitHub Search -Search issues and PRs with semantic understanding. Auto-reloads when you run `dev gh index`. -``` -Find authentication-related bugs -``` +## Technology Stack -### 6. `dev_health` - Health Monitoring -Check MCP server and component health (vector storage, repository, GitHub index). -``` -Check server health with verbose details -``` +- **TypeScript** (strict mode) +- **ts-morph** / TypeScript Compiler API (AST analysis) +- **LanceDB** (embedded vector storage) +- **@xenova/transformers** (local embeddings) +- **MCP** (Model Context Protocol) +- **Turborepo** (monorepo builds) +- **Vitest** (1300+ tests) -**Production Features:** -- πŸ›‘οΈ **Rate Limiting:** 100 requests/minute burst per tool (token bucket algorithm) -- πŸ”„ **Retry Logic:** Automatic retry with exponential backoff for transient failures -- πŸ’š **Health Checks:** Proactive monitoring of all components -- 🧹 **Memory Safe:** Circular buffers prevent unbounded growth -- πŸ”Œ **Graceful Shutdown:** Proper cleanup, no zombie processes - -**Tips for Better Results:** -- **Use natural language**: "how do agents communicate" works better than "agent message" -- **Adjust thresholds**: Default is 0.7 (precise), use 0.25-0.4 for exploration -- **Exact matches score 70-90%**: Semantic matches score 25-60% - -### Current Status - -**βœ… Production Ready:** -- Core intelligence layer (scanner, vectors, indexer) - Complete -- MCP server with 6 adapters (search, status, plan, explore, github, health) - Production ready -- Prompts system with 8 guided workflows - Shipped -- Token cost visibility and accurate estimation - Validated (<1% error) -- **1100+ tests passing** across all packages -- CLI and MCP integrations - Fully functional (Cursor + Claude Code) -- Production hardening complete (rate limiting, retry, health, memory safety) - -**πŸš€ Available Now:** -- Semantic code search with type-aware understanding -- GitHub issue/PR indexing and search (auto-reload, works offline) -- Implementation planning from GitHub issues -- Code pattern exploration and relationship mapping -- Repository and component health monitoring -- Cursor IDE integration with dynamic workspace detection -- Claude Code integration with ~/.claude.json support - -**πŸ”§ Infrastructure:** -- **Event Bus** - Async pub/sub with typed events + cleanup -- **Observability** - Request tracking, structured logging, metrics -- **StorageAdapter** - Pluggable persistence for session/persistent state -- **SubagentCoordinator** - Integrated with MCP server + circular buffers -- **Rate Limiting** - Token bucket algorithm (100 req/min burst) -- **Retry Logic** - Exponential backoff with jitter -- **Health Monitoring** - Component health checks (`dev_health` tool) - -See [ARCHITECTURE.md](./ARCHITECTURE.md) for technical decisions and [packages/mcp-server/README.md](./packages/mcp-server/README.md) for MCP integration details. - -### Development - -#### Running Tests +## Development ```bash -# Run all tests -pnpm test +# Install dependencies +pnpm install -# Run tests in watch mode -pnpm -F "@monorepo/core" test:watch -``` +# Build all packages +pnpm build -#### Linting and Formatting +# Run tests +pnpm test -```bash -# Lint all packages +# Lint and format pnpm lint - -# Format all packages pnpm format -``` - -#### Building -```bash -# Build all packages -pnpm build - -# Build a specific package -pnpm -F "@monorepo/core" build +# Type check +pnpm typecheck ``` -## Implementation Plan +## Version History -See [PLAN.md](./PLAN.md) for the detailed implementation plan. +- **v0.3.0** - Context Quality release + - New `dev_refs` tool for relationship queries + - Enhanced `dev_map` with hot paths, smart depth, signatures + - Refactored `dev_plan` to context assembly +- **v0.2.0** - Richer search results with snippets and imports +- **v0.1.0** - Initial release ## Contributing -Contributions are welcome! Please follow the conventional commit format and include tests for any new features. - -## Working with this Monorepo - -### Adding a New Package - -1. Create a new folder in the `packages` directory -2. Add a `package.json` with appropriate dependencies -3. Add a `tsconfig.json` that extends from the root -4. Update root `tsconfig.json` with path mappings for the new package - -### Package Interdependencies - -Packages can depend on each other using the workspace protocol: - -```json -"dependencies": { - "@monorepo/core": "workspace:*" -} -``` - -## Versioning - -This template follows [Semantic Versioning](https://semver.org/) at the repository level: - -- **Git tags**: `v1.0.0`, `v1.1.0`, `v2.0.0` (for template releases) -- **Package versions**: Remain at `0.1.0` by default (customize after cloning) - -**Version bumps:** -- **MAJOR**: Breaking changes to template structure or tooling -- **MINOR**: New features, examples, or improvements -- **PATCH**: Bug fixes, documentation updates +Contributions welcome! Please follow conventional commit format and include tests. -See [AGENTS.md](AGENTS.md) for detailed versioning strategy. +See [CONTRIBUTING.md](CONTRIBUTING.md) for guidelines. ## License -[MIT](LICENSE) \ No newline at end of file +[MIT](LICENSE) diff --git a/website/content/docs/tools/_meta.js b/website/content/docs/tools/_meta.js index de5e6b8..5192685 100644 --- a/website/content/docs/tools/_meta.js +++ b/website/content/docs/tools/_meta.js @@ -1,6 +1,8 @@ export default { index: 'Overview', 'dev-search': 'dev_search', + 'dev-refs': 'dev_refs', + 'dev-map': 'dev_map', 'dev-plan': 'dev_plan', 'dev-explore': 'dev_explore', 'dev-gh': 'dev_gh', diff --git a/website/content/docs/tools/dev-map.mdx b/website/content/docs/tools/dev-map.mdx new file mode 100644 index 0000000..ff0832f --- /dev/null +++ b/website/content/docs/tools/dev-map.mdx @@ -0,0 +1,135 @@ +# dev_map + +Get a high-level overview of your codebase structure. + +## Overview + +`dev_map` generates a hierarchical view of your repository showing: +- Directory structure with component counts +- **Hot Paths** β€” Most referenced files in the codebase +- **Exports** β€” Public API with function signatures +- **Smart Depth** β€” Adaptive expansion based on density + +This helps AI assistants quickly understand codebase organization. + +## Parameters + +| Parameter | Type | Default | Description | +|-----------|------|---------|-------------| +| `depth` | number | `2` | Maximum depth (1-5) | +| `focus` | string | `""` | Focus on specific directory | +| `includeExports` | boolean | `true` | Show exported symbols | +| `includeHotPaths` | boolean | `true` | Show most referenced files | +| `smartDepth` | boolean | `false` | Adaptive depth based on density | +| `tokenBudget` | number | `2000` | Max tokens for output | + +## Example Output + +```markdown +# Codebase Map + +## Hot Paths (most referenced) +1. `packages/core/src/indexer/index.ts` (RepositoryIndexer) - 47 refs +2. `packages/core/src/vector/store.ts` (LanceDBVectorStore) - 32 refs +3. `packages/mcp-server/src/server/mcp-server.ts` (MCPServer) - 28 refs + +## Directory Structure + +└── packages/ (195 components) + β”œβ”€β”€ core/ (45 components) + β”‚ β”œβ”€β”€ src/ (42 components) + β”‚ β”‚ └── exports: function search(query): Promise, class RepositoryIndexer + β”‚ └── dist/ + β”œβ”€β”€ mcp-server/ (28 components) + β”‚ └── exports: class MCPServer, class SearchAdapter + β”œβ”€β”€ cli/ (15 components) + └── subagents/ (35 components) + +**Total:** 195 indexed components across 24 directories +``` + +## Features + +### Hot Paths + +Shows the most referenced files in your codebase. These are typically: +- Core utilities used everywhere +- Base classes/interfaces +- Central orchestrators + +Useful for understanding which code is most critical. + +### Smart Depth + +When enabled, expands directories with many components and collapses sparse ones: + +```json +{ + "smartDepth": true, + "smartDepthThreshold": 10 +} +``` + +- Directories with β‰₯10 components get expanded +- Sparse directories get collapsed +- First 2 levels always shown + +### Focus Directory + +Zoom into a specific part of the codebase: + +```json +{ + "focus": "packages/core/src", + "depth": 4 +} +``` + +### Signatures in Exports + +Exports show function/class signatures (truncated to 60 chars): + +``` +└── exports: function search(query, opts): Promise, class Indexer +``` + +## Use Cases + +### Onboarding + +``` +Show me the overall structure of this codebase +``` + +### Finding Entry Points + +``` +What are the main components in packages/mcp-server? +``` + +### Understanding Architecture + +``` +Show the codebase map with hot paths to understand the core modules +``` + +### Scoping Work + +``` +Focus on packages/core/src with depth 4 to see all scanner components +``` + +## Tips + +- Start with default depth (2) for overview +- Use `focus` to drill into specific areas +- Enable `smartDepth` for large codebases +- Hot paths reveal the most important code +- Signatures help understand APIs without reading files + +## Related Tools + +- [`dev_refs`](/docs/tools/dev-refs) β€” Query relationships for hot path files +- [`dev_search`](/docs/tools/dev-search) β€” Search within focused area +- [`dev_status`](/docs/tools/dev-status) β€” Check indexing completeness + diff --git a/website/content/docs/tools/dev-plan.mdx b/website/content/docs/tools/dev-plan.mdx index c9dda3b..e0aaad0 100644 --- a/website/content/docs/tools/dev-plan.mdx +++ b/website/content/docs/tools/dev-plan.mdx @@ -1,106 +1,173 @@ # dev_plan -Generate implementation plans from GitHub issues. Creates structured task breakdowns with estimates and dependencies. +Assemble rich context for implementing GitHub issues. -## Usage +## Overview -``` -dev_plan(issue, detailLevel?, format?, useExplorer?) -``` +`dev_plan` fetches a GitHub issue and assembles comprehensive context for AI assistants to create implementation plans. Instead of generating heuristic task breakdowns, it provides: + +- Full issue content with comments +- Relevant code snippets from semantic search +- Detected codebase patterns +- Metadata for token budgeting + +**Philosophy:** Provide data, let LLMs reason. AI assistants create better plans when given rich context. ## Parameters | Parameter | Type | Default | Description | |-----------|------|---------|-------------| | `issue` | number | required | GitHub issue number | -| `detailLevel` | `"simple"` \| `"detailed"` | `"detailed"` | Task granularity | | `format` | `"compact"` \| `"verbose"` | `"compact"` | Output format | -| `useExplorer` | boolean | true | Find relevant code via semantic search | +| `includeCode` | boolean | `true` | Include relevant code snippets | +| `includePatterns` | boolean | `true` | Include codebase patterns | +| `tokenBudget` | number | `4000` | Max tokens for output | + +## Example Output + +### Compact Format + +```markdown +# Issue #42: Add user preferences API + +**Author:** alice | **State:** open | **Labels:** enhancement, api + +## Description + +We need an API endpoint for user preferences. Should support: +- GET /preferences - retrieve current preferences +- PUT /preferences - update preferences -## Examples +## Comments -### Basic Planning +**bob** (2024-01-15): +Consider using the existing validation middleware from src/middleware/validation.ts -> "Use dev_plan to create a plan for issue 42" +## Relevant Code +### UserController (class) +**File:** `src/api/controllers/user.ts` | **Relevance:** 85% +**Reason:** Similar pattern + +```typescript +export class UserController { + async getUser(req: Request, res: Response) { + // ... + } +} ``` -## Implementation Plan: Add user preferences API (#42) - -### Tasks -- [ ] 1. Create preferences schema (2h) -- [ ] 2. Add database migration (1h) -- [ ] 3. Implement GET /preferences endpoint (2h) -- [ ] 4. Implement PUT /preferences endpoint (2h) -- [ ] 5. Add validation middleware (1h) -- [ ] 6. Write unit tests (2h) -- [ ] 7. Update API documentation (1h) - -### Relevant Code -- src/api/routes/user.ts (existing user routes) -- src/db/migrations/ (migration patterns) -- src/middleware/validation.ts (validation examples) - -### Estimated Total: 11 hours + +### validateBody (function) +**File:** `src/middleware/validation.ts` | **Relevance:** 72% +**Reason:** Relevant type definition + +```typescript +export function validateBody(schema: Schema) { + // ... +} ``` -### Simple Plan +## Codebase Patterns -> "Use dev_plan for issue 42, detailLevel simple" +- **Test naming:** *.test.ts +- **Test location:** __tests__/ +--- +*Context assembled at 2024-01-20T10:30:00Z | ~1250 tokens* ``` -## Implementation Plan: Add user preferences API (#42) -- [ ] Create preferences model and migration -- [ ] Implement API endpoints -- [ ] Add tests and documentation +### Verbose Format (JSON) -### Estimated Total: 4-6 hours +```json +{ + "issue": { + "number": 42, + "title": "Add user preferences API", + "body": "We need an API endpoint...", + "labels": ["enhancement", "api"], + "author": "alice", + "comments": [...] + }, + "relevantCode": [ + { + "file": "src/api/controllers/user.ts", + "name": "UserController", + "type": "class", + "snippet": "export class UserController {...}", + "relevanceScore": 0.85, + "reason": "Similar pattern" + } + ], + "codebasePatterns": { + "testPattern": "*.test.ts", + "testLocation": "__tests__/" + }, + "metadata": { + "generatedAt": "2024-01-20T10:30:00Z", + "tokensUsed": 1250 + } +} ``` ## How It Works -1. **Fetch issue** β€” Retrieves issue title, body, labels, and comments from GitHub -2. **Analyze context** β€” Understands the feature request or bug report -3. **Explore code** β€” Uses semantic search to find relevant existing code -4. **Generate plan** β€” Creates structured tasks with estimates -5. **Map dependencies** β€” Identifies task ordering and blockers +1. **Fetch issue** β€” Retrieves issue with comments via GitHub CLI +2. **Search code** β€” Finds relevant code using semantic search +3. **Detect patterns** β€” Identifies test naming and location patterns +4. **Assemble context** β€” Packages everything for LLM consumption ## Requirements -> ⚠️ **GitHub CLI required.** dev_plan uses `gh` to fetch issue data. Install it from [cli.github.com](https://cli.github.com). - -Make sure you're authenticated: +> ⚠️ **GitHub CLI required.** `dev_plan` uses `gh` to fetch issue data. ```bash -gh auth status +# Install GitHub CLI +brew install gh # macOS +# or visit https://cli.github.com + +# Authenticate +gh auth login ``` -## Index GitHub First +## Use Cases -For best results, index your GitHub issues: +### Implementation Planning -```bash -dev gh index ``` +Assemble context for issue #42 so I can create an implementation plan +``` + +### Understanding Requirements + +``` +Get the full context for issue #123 including comments +``` + +### Code Discovery -This enables semantic search across your project's issues and PRs. +``` +What existing code is relevant to implementing issue #55? +``` ## Tips -> **Use detailed plans for complex features.** The default `detailed` level produces 10-15 actionable tasks. +- **Comments are valuable** β€” They often contain design decisions and constraints +- **Relevant code shows patterns** β€” Use it to understand existing conventions +- **Codebase patterns help** β€” Know where to put tests and how to name them +- **Token budget matters** β€” Reduce if context is too large -> **Plans improve with codebase context.** The `useExplorer` option (default: true) finds relevant code patterns to inform the plan. +## Migration from v0.2 -## Response Metadata +In v0.2, `dev_plan` generated heuristic task breakdowns with estimates. In v0.3+, it returns raw context instead. -```json -{ - "metadata": { - "tokens": 892, - "duration_ms": 1234, - "issue_number": 42, - "tasks_generated": 7 - } -} -``` +**Why the change?** AI assistants create better plans when given comprehensive context rather than our heuristic guesses. + +**Old parameters removed:** +- `detailLevel` β€” No longer applicable +- `useExplorer` β€” Now always uses code search (controlled by `includeCode`) + +## Related Tools +- [`dev_search`](/docs/tools/dev-search) β€” Search for more relevant code +- [`dev_refs`](/docs/tools/dev-refs) β€” Understand relationships in relevant code +- [`dev_gh`](/docs/tools/dev-gh) β€” Search related issues/PRs diff --git a/website/content/docs/tools/dev-refs.mdx b/website/content/docs/tools/dev-refs.mdx new file mode 100644 index 0000000..53e32de --- /dev/null +++ b/website/content/docs/tools/dev-refs.mdx @@ -0,0 +1,126 @@ +# dev_refs + +Query code relationships to understand what calls what. + +## Overview + +`dev_refs` helps you understand code flow by finding: +- **Callers** β€” What functions/methods call a given symbol +- **Callees** β€” What functions/methods a given symbol calls + +This is invaluable for understanding impact of changes and navigating unfamiliar codebases. + +## Parameters + +| Parameter | Type | Default | Description | +|-----------|------|---------|-------------| +| `name` | string | required | Symbol name to query | +| `direction` | string | `"both"` | `"callers"`, `"callees"`, or `"both"` | +| `limit` | number | `10` | Maximum results | +| `tokenBudget` | number | `2000` | Max tokens for output | + +## Examples + +### Find Callers + +``` +Find all functions that call authenticate +``` + +```json +{ + "name": "authenticate", + "direction": "callers" +} +``` + +**Output:** +``` +## Callers of `authenticate` + +1. **loginHandler** (src/auth/handlers.ts:45) + Calls authenticate at line 52 + +2. **middleware** (src/auth/middleware.ts:23) + Calls authenticate at line 28 + +3. **apiRoute** (src/routes/api.ts:102) + Calls authenticate at line 110 +``` + +### Find Callees + +``` +What does the validateToken function call? +``` + +```json +{ + "name": "validateToken", + "direction": "callees" +} +``` + +**Output:** +``` +## Callees of `validateToken` + +1. **decodeJWT** (src/utils/jwt.ts:15) + Called at line 8 + +2. **checkExpiry** (src/utils/jwt.ts:45) + Called at line 12 + +3. **verifySignature** (src/crypto/verify.ts:22) + Called at line 15 +``` + +### Both Directions + +```json +{ + "name": "authenticate", + "direction": "both" +} +``` + +## Use Cases + +### Impact Analysis +Before modifying a function, find all callers to understand impact: +``` +Find all callers of the deprecated legacyAuth function +``` + +### Understanding Flow +Trace execution flow through the codebase: +``` +What does the main function call? +``` + +### Refactoring +Identify all usages before renaming or moving code: +``` +Find everything that calls the old validateUser function +``` + +## How It Works + +1. Searches indexed documents for the symbol name +2. Extracts `callers` and `callees` from document metadata +3. Filters based on `direction` parameter +4. Formats results with file paths and line numbers + +## Tips + +- Use exact function/method names for best results +- Combine with `dev_search` to first find the symbol, then query refs +- Use `direction: "callers"` for impact analysis +- Use `direction: "callees"` for understanding implementation + +## Related Tools + +- [`dev_search`](/docs/tools/dev-search) β€” Find symbols first +- [`dev_map`](/docs/tools/dev-map) β€” See hot paths (most referenced files) +- [`dev_explore`](/docs/tools/dev-explore) β€” Find similar patterns + diff --git a/website/content/docs/tools/index.mdx b/website/content/docs/tools/index.mdx index 1e73fe7..658c619 100644 --- a/website/content/docs/tools/index.mdx +++ b/website/content/docs/tools/index.mdx @@ -1,18 +1,26 @@ # MCP Tools Overview -dev-agent provides six tools through the Model Context Protocol (MCP). These tools give AI assistants deep understanding of your codebase. +dev-agent provides eight tools through the Model Context Protocol (MCP). These tools give AI assistants deep understanding of your codebase. ## Available Tools | Tool | Purpose | |------|---------| -| [`dev_search`](/docs/tools/dev-search) | Semantic code search | -| [`dev_plan`](/docs/tools/dev-plan) | Generate implementation plans from GitHub issues | +| [`dev_search`](/docs/tools/dev-search) | Semantic code search with snippets | +| [`dev_refs`](/docs/tools/dev-refs) | Query code relationships (callers/callees) | +| [`dev_map`](/docs/tools/dev-map) | Codebase structure overview | +| [`dev_plan`](/docs/tools/dev-plan) | Assemble context for GitHub issues | | [`dev_explore`](/docs/tools/dev-explore) | Explore code patterns and relationships | | [`dev_gh`](/docs/tools/dev-gh) | Search GitHub issues and PRs | | [`dev_status`](/docs/tools/dev-status) | Check repository indexing status | | [`dev_health`](/docs/tools/dev-health) | Monitor MCP server health | +## New in v0.3.0 + +- **`dev_refs`** β€” Query what calls what and what is called by what +- **`dev_map`** β€” Get codebase overview with hot paths and smart depth +- **`dev_plan`** β€” Refactored to assemble context (no more heuristic task breakdown) + ## How Tools Are Exposed Tools are exposed via the Model Context Protocol (MCP), a standard for AI tool integration: @@ -25,6 +33,8 @@ AI Assistant (Cursor/Claude) dev-agent MCP Server β”‚ β”œβ”€β”€ SearchAdapter β†’ dev_search + β”œβ”€β”€ RefsAdapter β†’ dev_refs + β”œβ”€β”€ MapAdapter β†’ dev_map β”œβ”€β”€ PlanAdapter β†’ dev_plan β”œβ”€β”€ ExploreAdapter β†’ dev_explore β”œβ”€β”€ GitHubAdapter β†’ dev_gh @@ -61,22 +71,11 @@ The `metadata` field helps AI assistants understand: Most tools support two output formats: -- **`compact`** (default) β€” Token-efficient, summaries only -- **`verbose`** β€” Full details including signatures and metadata - -``` -// compact format -1. [89%] class: AuthMiddleware (src/auth/middleware.ts:15) - -// verbose format -1. [Score: 89.0%] class: AuthMiddleware - Location: src/auth/middleware.ts:15 - Signature: export class AuthMiddleware implements Middleware {...} - Metadata: language: typescript, exported: true, lines: 28 -``` +- **`compact`** (default) β€” Token-efficient, markdown output +- **`verbose`** β€” Full JSON with all details ## Next Steps - [dev_search β†’](/docs/tools/dev-search) β€” Semantic code search -- [dev_plan β†’](/docs/tools/dev-plan) β€” Implementation planning - +- [dev_refs β†’](/docs/tools/dev-refs) β€” Relationship queries +- [dev_map β†’](/docs/tools/dev-map) β€” Codebase overview