diff --git a/.changeset/arrow-function-extraction.md b/.changeset/arrow-function-extraction.md new file mode 100644 index 0000000..5073078 --- /dev/null +++ b/.changeset/arrow-function-extraction.md @@ -0,0 +1,48 @@ +--- +"@lytics/dev-agent-core": minor +"@lytics/dev-agent": minor +--- + +feat(scanner): Extract arrow functions, function expressions, and exported constants + +### New Features + +**Arrow Function Extraction** +- Extract arrow functions assigned to `const`/`let` variables +- Extract function expressions assigned to variables +- Detect React hooks automatically (`use*` naming pattern) +- Detect async arrow functions + +**Exported Constant Extraction** +- Extract exported `const` with object literal initializers (config objects) +- Extract exported `const` with array literal initializers (static lists) +- Extract exported `const` with call expression initializers (factories like `createContext()`) + +### API Changes + +**New DocumentType value:** +- Added `'variable'` to `DocumentType` union + +**New metadata fields:** +- `isArrowFunction?: boolean` - true for arrow functions (vs function expressions) +- `isHook?: boolean` - true if name matches `/^use[A-Z]/` (React convention) +- `isAsync?: boolean` - true for async functions +- `isConstant?: boolean` - true for exported constants +- `constantKind?: 'object' | 'array' | 'value'` - kind of constant initializer + +### Examples + +Now extracts: +```typescript +export const useAuth = () => { ... } // Hook (isHook: true) +export const fetchData = async (url) => { ... } // Async (isAsync: true) +const validateEmail = (email: string) => ... // Utility function +export const API_CONFIG = { baseUrl: '...' } // Object constant +export const LANGUAGES = ['ts', 'js'] // Array constant +export const AppContext = createContext({}) // Factory constant +``` + +### Migration + +No breaking changes. The new `'variable'` DocumentType is additive. Existing queries for `'function'`, `'class'`, etc. continue to work unchanged. + diff --git a/packages/core/src/scanner/README.md b/packages/core/src/scanner/README.md index cd2a7e1..8cd0258 100644 --- a/packages/core/src/scanner/README.md +++ b/packages/core/src/scanner/README.md @@ -56,7 +56,7 @@ Represents a single extracted code element or documentation section: interface Document { id: string; // Unique identifier: "file:name:line" text: string; // Text to embed (for vector search) - type: DocumentType; // 'function' | 'class' | 'interface' | 'type' | 'method' | 'documentation' + type: DocumentType; // 'function' | 'class' | 'interface' | 'type' | 'method' | 'documentation' | 'variable' language: string; // 'typescript' | 'javascript' | 'markdown' metadata: { @@ -67,6 +67,13 @@ interface Document { signature?: string; // Full signature exported: boolean; // Is it a public API? docstring?: string; // Documentation comment + + // Variable/function metadata (for type: 'variable') + isArrowFunction?: boolean; // True for arrow functions + isHook?: boolean; // True for React hooks (use* pattern) + isAsync?: boolean; // True for async functions + isConstant?: boolean; // True for exported constants + constantKind?: 'object' | 'array' | 'value'; // Kind of constant }; } ``` @@ -364,8 +371,8 @@ const utilDocs = result.documents.filter( | Language | Scanner | Extracts | Status | |----------|---------|----------|--------| -| TypeScript | `TypeScriptScanner` | Functions, classes, methods, interfaces, types, JSDoc | ✅ Implemented | -| JavaScript | `TypeScriptScanner` | Functions, classes, methods, JSDoc | ✅ Implemented (via .ts scanner) | +| TypeScript | `TypeScriptScanner` | Functions, classes, methods, interfaces, types, arrow functions, exported constants, JSDoc | ✅ Implemented | +| JavaScript | `TypeScriptScanner` | Functions, classes, methods, arrow functions, exported constants, JSDoc | ✅ Implemented (via .ts scanner) | | Markdown | `MarkdownScanner` | Documentation sections, code blocks | ✅ Implemented | | Go | - | Functions, structs, interfaces | 🔄 Planned (tree-sitter) | | Python | - | Functions, classes, docstrings | 🔄 Planned (tree-sitter) | diff --git a/packages/core/src/scanner/__tests__/fixtures/arrow-functions.ts b/packages/core/src/scanner/__tests__/fixtures/arrow-functions.ts new file mode 100644 index 0000000..57a8931 --- /dev/null +++ b/packages/core/src/scanner/__tests__/fixtures/arrow-functions.ts @@ -0,0 +1,109 @@ +/** + * Test fixtures for arrow function and function expression extraction. + * This file contains various patterns that should be extracted by the scanner. + */ + +// Simple arrow function +const simpleArrow = () => { + return 'hello'; +}; + +// Arrow function with parameters +const typedArrow = (name: string, age: number): string => { + return `${name} is ${age} years old`; +}; + +// Exported arrow function +export const exportedArrow = (value: number) => value * 2; + +// Non-exported (private) helper +const privateHelper = (x: number) => x + 1; + +// React-style hook (name starts with 'use') +export const useCustomHook = (initialValue: string) => { + const value = initialValue; + const setValue = (newValue: string) => newValue; + return { value, setValue }; +}; + +// Async arrow function +export const fetchData = async (url: string) => { + const response = await fetch(url); + return response.json(); +}; + +// Function expression (legacy style) +// biome-ignore lint/complexity/useArrowFunction: Testing function expression extraction +const legacyFunction = function (a: number, b: number) { + return a + b; +}; + +// Arrow function that calls other functions (for callee extraction) +const composedFunction = (input: string) => { + const trimmed = input.trim(); + const upper = trimmed.toUpperCase(); + return privateHelper(upper.length); +}; + +/** + * A well-documented arrow function. + * This should have its JSDoc extracted. + */ +const documentedArrow = (param: string) => { + return param.toLowerCase(); +}; + +// ============================================ +// EXPORTED CONSTANTS - Should be extracted +// ============================================ + +/** + * API configuration object. + * This should be extracted as an exported constant. + */ +export const API_CONFIG = { + baseUrl: '/api', + timeout: 5000, + retries: 3, +}; + +// Exported array constant +export const SUPPORTED_LANGUAGES = ['typescript', 'javascript', 'python', 'go']; + +// Exported call expression (factory pattern) +// biome-ignore lint/suspicious/noEmptyBlockStatements: Test fixture +export const AppContext = (() => ({ value: null }))(); + +// Typed exported constant +export const THEME_CONFIG: { dark: boolean; primary: string } = { + dark: false, + primary: '#007bff', +}; + +// ============================================ +// NON-EXPORTED - Should NOT be extracted +// ============================================ +// biome-ignore lint/correctness/noUnusedVariables: Test fixtures for non-extraction + +// Plain constant (primitive) - never extracted +const plainConstant = 42; + +// Non-exported object - not extracted (only exported objects are extracted) +const configObject = { + apiUrl: '/api', + timeout: 5000, +}; + +// Non-exported array - not extracted +const colorList = ['red', 'green', 'blue']; + +// Suppress unused warnings - these are test fixtures +void plainConstant; +void configObject; +void colorList; + +// Exported primitive - NOT extracted (primitives have low semantic value) +export const API_ENDPOINT = 'https://api.example.com'; + +// Re-exported for testing +export { simpleArrow, typedArrow, composedFunction, documentedArrow, legacyFunction }; diff --git a/packages/core/src/scanner/__tests__/scanner.test.ts b/packages/core/src/scanner/__tests__/scanner.test.ts index bc92a8f..4a611ab 100644 --- a/packages/core/src/scanner/__tests__/scanner.test.ts +++ b/packages/core/src/scanner/__tests__/scanner.test.ts @@ -707,4 +707,274 @@ describe('Scanner', () => { expect(calleeNames.some((n) => n.includes('register'))).toBe(true); }); }); + + describe('Arrow Function Extraction', () => { + // Note: We override exclude to allow fixtures directory (excluded by default) + const fixtureExcludes = ['**/node_modules/**', '**/dist/**']; + + it('should extract arrow functions assigned to variables', async () => { + const result = await scanRepository({ + repoRoot, + include: ['packages/core/src/scanner/__tests__/fixtures/arrow-functions.ts'], + exclude: fixtureExcludes, + }); + + // Should find arrow function variables + const variables = result.documents.filter((d) => d.type === 'variable'); + expect(variables.length).toBeGreaterThan(0); + }); + + it('should mark arrow functions with isArrowFunction metadata', async () => { + const result = await scanRepository({ + repoRoot, + include: ['packages/core/src/scanner/__tests__/fixtures/arrow-functions.ts'], + exclude: fixtureExcludes, + }); + + const arrowFn = result.documents.find( + (d) => d.type === 'variable' && d.metadata.name === 'simpleArrow' + ); + expect(arrowFn).toBeDefined(); + expect(arrowFn?.metadata.isArrowFunction).toBe(true); + }); + + it('should detect React hooks by naming convention', async () => { + const result = await scanRepository({ + repoRoot, + include: ['packages/core/src/scanner/__tests__/fixtures/arrow-functions.ts'], + exclude: fixtureExcludes, + }); + + const hook = result.documents.find( + (d) => d.type === 'variable' && d.metadata.name === 'useCustomHook' + ); + expect(hook).toBeDefined(); + expect(hook?.metadata.isHook).toBe(true); + expect(hook?.metadata.isArrowFunction).toBe(true); + }); + + it('should detect async arrow functions', async () => { + const result = await scanRepository({ + repoRoot, + include: ['packages/core/src/scanner/__tests__/fixtures/arrow-functions.ts'], + exclude: fixtureExcludes, + }); + + const asyncFn = result.documents.find( + (d) => d.type === 'variable' && d.metadata.name === 'fetchData' + ); + expect(asyncFn).toBeDefined(); + expect(asyncFn?.metadata.isAsync).toBe(true); + }); + + it('should extract exported arrow functions', async () => { + const result = await scanRepository({ + repoRoot, + include: ['packages/core/src/scanner/__tests__/fixtures/arrow-functions.ts'], + exclude: fixtureExcludes, + }); + + const exportedFn = result.documents.find( + (d) => d.type === 'variable' && d.metadata.name === 'exportedArrow' + ); + expect(exportedFn).toBeDefined(); + expect(exportedFn?.metadata.exported).toBe(true); + }); + + it('should extract non-exported arrow functions', async () => { + const result = await scanRepository({ + repoRoot, + include: ['packages/core/src/scanner/__tests__/fixtures/arrow-functions.ts'], + exclude: fixtureExcludes, + }); + + const privateFn = result.documents.find( + (d) => d.type === 'variable' && d.metadata.name === 'privateHelper' + ); + expect(privateFn).toBeDefined(); + expect(privateFn?.metadata.exported).toBe(false); + }); + + it('should extract function expressions assigned to variables', async () => { + const result = await scanRepository({ + repoRoot, + include: ['packages/core/src/scanner/__tests__/fixtures/arrow-functions.ts'], + exclude: fixtureExcludes, + }); + + const funcExpr = result.documents.find( + (d) => d.type === 'variable' && d.metadata.name === 'legacyFunction' + ); + expect(funcExpr).toBeDefined(); + expect(funcExpr?.metadata.isArrowFunction).toBe(false); + }); + + it('should include signature for arrow functions', async () => { + const result = await scanRepository({ + repoRoot, + include: ['packages/core/src/scanner/__tests__/fixtures/arrow-functions.ts'], + exclude: fixtureExcludes, + }); + + const fn = result.documents.find( + (d) => d.type === 'variable' && d.metadata.name === 'typedArrow' + ); + expect(fn).toBeDefined(); + expect(fn?.metadata.signature).toContain('typedArrow'); + expect(fn?.metadata.signature).toContain('=>'); + }); + + it('should extract callees from arrow functions', async () => { + const result = await scanRepository({ + repoRoot, + include: ['packages/core/src/scanner/__tests__/fixtures/arrow-functions.ts'], + exclude: fixtureExcludes, + }); + + const fn = result.documents.find( + (d) => d.type === 'variable' && d.metadata.name === 'composedFunction' + ); + expect(fn).toBeDefined(); + expect(fn?.metadata.callees).toBeDefined(); + expect(fn?.metadata.callees?.length).toBeGreaterThan(0); + }); + + it('should extract JSDoc from arrow functions', async () => { + const result = await scanRepository({ + repoRoot, + include: ['packages/core/src/scanner/__tests__/fixtures/arrow-functions.ts'], + exclude: fixtureExcludes, + }); + + const fn = result.documents.find( + (d) => d.type === 'variable' && d.metadata.name === 'documentedArrow' + ); + expect(fn).toBeDefined(); + expect(fn?.metadata.docstring).toBeDefined(); + expect(fn?.metadata.docstring).toContain('documented'); + }); + + it('should not extract non-exported variables without function initializers', async () => { + const result = await scanRepository({ + repoRoot, + include: ['packages/core/src/scanner/__tests__/fixtures/arrow-functions.ts'], + exclude: fixtureExcludes, + }); + + // Should NOT find non-exported plain constants + const constant = result.documents.find( + (d) => d.type === 'variable' && d.metadata.name === 'plainConstant' + ); + expect(constant).toBeUndefined(); + + // Should NOT find non-exported object constants + const objectConst = result.documents.find( + (d) => d.type === 'variable' && d.metadata.name === 'configObject' + ); + expect(objectConst).toBeUndefined(); + + // Should NOT find exported primitive constants (low semantic value) + const primitiveExport = result.documents.find( + (d) => d.type === 'variable' && d.metadata.name === 'API_ENDPOINT' + ); + expect(primitiveExport).toBeUndefined(); + }); + }); + + describe('Exported Constant Extraction', () => { + // Note: We override exclude to allow fixtures directory (excluded by default) + const fixtureExcludes = ['**/node_modules/**', '**/dist/**']; + + it('should extract exported object constants', async () => { + const result = await scanRepository({ + repoRoot, + include: ['packages/core/src/scanner/__tests__/fixtures/arrow-functions.ts'], + exclude: fixtureExcludes, + }); + + const config = result.documents.find( + (d) => d.type === 'variable' && d.metadata.name === 'API_CONFIG' + ); + expect(config).toBeDefined(); + expect(config?.metadata.exported).toBe(true); + expect(config?.metadata.isConstant).toBe(true); + expect(config?.metadata.constantKind).toBe('object'); + }); + + it('should extract exported array constants', async () => { + const result = await scanRepository({ + repoRoot, + include: ['packages/core/src/scanner/__tests__/fixtures/arrow-functions.ts'], + exclude: fixtureExcludes, + }); + + const languages = result.documents.find( + (d) => d.type === 'variable' && d.metadata.name === 'SUPPORTED_LANGUAGES' + ); + expect(languages).toBeDefined(); + expect(languages?.metadata.exported).toBe(true); + expect(languages?.metadata.isConstant).toBe(true); + expect(languages?.metadata.constantKind).toBe('array'); + }); + + it('should extract exported call expression constants (factories)', async () => { + const result = await scanRepository({ + repoRoot, + include: ['packages/core/src/scanner/__tests__/fixtures/arrow-functions.ts'], + exclude: fixtureExcludes, + }); + + const context = result.documents.find( + (d) => d.type === 'variable' && d.metadata.name === 'AppContext' + ); + expect(context).toBeDefined(); + expect(context?.metadata.exported).toBe(true); + expect(context?.metadata.isConstant).toBe(true); + expect(context?.metadata.constantKind).toBe('value'); + }); + + it('should extract typed exported constants with signature', async () => { + const result = await scanRepository({ + repoRoot, + include: ['packages/core/src/scanner/__tests__/fixtures/arrow-functions.ts'], + exclude: fixtureExcludes, + }); + + const theme = result.documents.find( + (d) => d.type === 'variable' && d.metadata.name === 'THEME_CONFIG' + ); + expect(theme).toBeDefined(); + expect(theme?.metadata.signature).toContain('THEME_CONFIG'); + expect(theme?.metadata.signature).toContain('dark'); + }); + + it('should extract JSDoc from exported constants', async () => { + const result = await scanRepository({ + repoRoot, + include: ['packages/core/src/scanner/__tests__/fixtures/arrow-functions.ts'], + exclude: fixtureExcludes, + }); + + const config = result.documents.find( + (d) => d.type === 'variable' && d.metadata.name === 'API_CONFIG' + ); + expect(config).toBeDefined(); + expect(config?.metadata.docstring).toBeDefined(); + expect(config?.metadata.docstring).toContain('API configuration'); + }); + + it('should not extract non-exported object constants', async () => { + const result = await scanRepository({ + repoRoot, + include: ['packages/core/src/scanner/__tests__/fixtures/arrow-functions.ts'], + exclude: fixtureExcludes, + }); + + // configObject is not exported, should not be extracted + const config = result.documents.find( + (d) => d.type === 'variable' && d.metadata.name === 'configObject' + ); + expect(config).toBeUndefined(); + }); + }); }); diff --git a/packages/core/src/scanner/types.ts b/packages/core/src/scanner/types.ts index b83b784..90043d3 100644 --- a/packages/core/src/scanner/types.ts +++ b/packages/core/src/scanner/types.ts @@ -7,7 +7,8 @@ export type DocumentType = | 'type' | 'struct' | 'method' - | 'documentation'; + | 'documentation' + | 'variable'; /** * Information about a function/method that calls this component @@ -57,6 +58,13 @@ export interface DocumentMetadata { callees?: CalleeInfo[]; // Functions/methods this component calls // Note: callers are computed at query time via reverse lookup + // Variable/function metadata + isArrowFunction?: boolean; // True if variable initialized with arrow function + isHook?: boolean; // True if name starts with 'use' (React convention) + isAsync?: boolean; // True if async function/arrow function + isConstant?: boolean; // True if exported constant (object/array/call expression) + constantKind?: 'object' | 'array' | 'value'; // Kind of constant initializer + // Extensible for future use custom?: Record; } diff --git a/packages/core/src/scanner/typescript.ts b/packages/core/src/scanner/typescript.ts index ab518a8..37fb4b3 100644 --- a/packages/core/src/scanner/typescript.ts +++ b/packages/core/src/scanner/typescript.ts @@ -1,8 +1,10 @@ import * as path from 'node:path'; import { + type ArrowFunction, type CallExpression, type ClassDeclaration, type FunctionDeclaration, + type FunctionExpression, type InterfaceDeclaration, type MethodDeclaration, type Node, @@ -10,6 +12,8 @@ import { type SourceFile, SyntaxKind, type TypeAliasDeclaration, + type VariableDeclaration, + type VariableStatement, } from 'ts-morph'; import type { CalleeInfo, Document, Scanner, ScannerCapabilities } from './types'; @@ -115,6 +119,38 @@ export class TypeScriptScanner implements Scanner { if (doc) documents.push(doc); } + // Extract variables with arrow functions, function expressions, or exported constants + for (const varStmt of sourceFile.getVariableStatements()) { + for (const decl of varStmt.getDeclarations()) { + const initializer = decl.getInitializer(); + if (!initializer) continue; + + const kind = initializer.getKind(); + + // Arrow functions and function expressions (any export status) + if (kind === SyntaxKind.ArrowFunction || kind === SyntaxKind.FunctionExpression) { + const doc = this.extractVariableWithFunction( + decl, + varStmt, + relativeFile, + imports, + sourceFile + ); + if (doc) documents.push(doc); + } + // Exported constants with object/array/call expression initializers + else if ( + varStmt.isExported() && + (kind === SyntaxKind.ObjectLiteralExpression || + kind === SyntaxKind.ArrayLiteralExpression || + kind === SyntaxKind.CallExpression) + ) { + const doc = this.extractExportedConstant(decl, varStmt, relativeFile, imports); + if (doc) documents.push(doc); + } + } + } + return documents; } @@ -371,6 +407,150 @@ export class TypeScriptScanner implements Scanner { }; } + /** + * Extract a variable declaration that is initialized with an arrow function or function expression. + * Captures React hooks, utility functions, and other function-valued constants. + */ + private extractVariableWithFunction( + decl: VariableDeclaration, + varStmt: VariableStatement, + file: string, + imports: string[], + sourceFile: SourceFile + ): Document | null { + const name = decl.getName(); + if (!name) return null; + + const initializer = decl.getInitializer(); + if (!initializer) return null; + + const isArrowFunction = initializer.getKind() === SyntaxKind.ArrowFunction; + const funcNode = initializer as ArrowFunction | FunctionExpression; + + const startLine = decl.getStartLineNumber(); + const endLine = decl.getEndLineNumber(); + const fullText = decl.getText(); + const docComment = this.getDocComment(varStmt); + const isExported = varStmt.isExported(); + const snippet = this.truncateSnippet(fullText); + const callees = this.extractCallees(funcNode, sourceFile); + + // Check if async + const isAsync = funcNode.isAsync?.() ?? false; + + // Check if React hook (name starts with 'use' followed by uppercase) + const isHook = /^use[A-Z]/.test(name); + + // Build signature from the variable declaration + // e.g., "const useAuth = (options: AuthOptions) => AuthResult" + const params = funcNode + .getParameters() + .map((p) => p.getText()) + .join(', '); + const returnType = funcNode.getReturnType()?.getText() ?? ''; + const asyncPrefix = isAsync ? 'async ' : ''; + const arrowOrFunction = isArrowFunction ? '=>' : 'function'; + const signature = `const ${name} = ${asyncPrefix}(${params}) ${arrowOrFunction}${returnType ? `: ${returnType}` : ''}`; + + const text = this.buildEmbeddingText({ + type: 'variable', + name, + signature, + docComment, + language: 'typescript', + }); + + return { + id: `${file}:${name}:${startLine}`, + text, + type: 'variable', + language: 'typescript', + metadata: { + file, + startLine, + endLine, + name, + signature, + exported: isExported, + docstring: docComment, + snippet, + imports, + callees: callees.length > 0 ? callees : undefined, + isArrowFunction, + isHook, + isAsync, + }, + }; + } + + /** + * Extract an exported constant with object literal, array literal, or call expression initializer. + * Captures configuration objects, contexts, and factory-created values. + */ + private extractExportedConstant( + decl: VariableDeclaration, + varStmt: VariableStatement, + file: string, + imports: string[] + ): Document | null { + const name = decl.getName(); + if (!name) return null; + + const initializer = decl.getInitializer(); + if (!initializer) return null; + + const startLine = decl.getStartLineNumber(); + const endLine = decl.getEndLineNumber(); + const fullText = decl.getText(); + const docComment = this.getDocComment(varStmt); + const snippet = this.truncateSnippet(fullText); + + // Determine the kind of constant for better embedding text + const kind = initializer.getKind(); + let constantKind: 'object' | 'array' | 'value'; + if (kind === SyntaxKind.ObjectLiteralExpression) { + constantKind = 'object'; + } else if (kind === SyntaxKind.ArrayLiteralExpression) { + constantKind = 'array'; + } else { + constantKind = 'value'; // Call expression or other + } + + // Build signature + const typeAnnotation = decl.getTypeNode()?.getText(); + const signature = typeAnnotation + ? `export const ${name}: ${typeAnnotation}` + : `export const ${name}`; + + const text = this.buildEmbeddingText({ + type: 'constant', + name, + signature, + docComment, + language: 'typescript', + }); + + return { + id: `${file}:${name}:${startLine}`, + text, + type: 'variable', + language: 'typescript', + metadata: { + file, + startLine, + endLine, + name, + signature, + exported: true, // Always true for this method + docstring: docComment, + snippet, + imports, + isConstant: true, + constantKind, + }, + }; + } + private getDocComment(node: Node): string | undefined { // ts-morph doesn't export getJsDocs on base Node type, but it exists on declarations const nodeWithJsDocs = node as unknown as { diff --git a/website/content/_meta.js b/website/content/_meta.js index e445689..2b28fa0 100644 --- a/website/content/_meta.js +++ b/website/content/_meta.js @@ -1,4 +1,5 @@ export default { index: 'Home', docs: 'Documentation', + updates: 'Updates', }; diff --git a/website/content/docs/quickstart.mdx b/website/content/docs/quickstart.mdx index 8b26901..1cd5a52 100644 --- a/website/content/docs/quickstart.mdx +++ b/website/content/docs/quickstart.mdx @@ -71,7 +71,7 @@ Try these prompts: When you run `dev index .`: 1. **Scanner** parses your code using the TypeScript Compiler API -2. **Extractor** identifies functions, classes, interfaces, types +2. **Extractor** identifies functions, classes, interfaces, types, arrow functions, hooks, and exported constants 3. **Embedder** generates vector embeddings locally (MiniLM-L6-v2) 4. **Storage** saves everything to LanceDB (`~/.dev-agent/indexes/`) diff --git a/website/content/docs/tools/dev-search.mdx b/website/content/docs/tools/dev-search.mdx index eac0f64..df5b5ca 100644 --- a/website/content/docs/tools/dev-search.mdx +++ b/website/content/docs/tools/dev-search.mdx @@ -86,11 +86,13 @@ This surfaces test files without affecting semantic search rankings — tests ar dev_search indexes these code components: -- **Functions** — Named functions and arrow functions +- **Functions** — Named function declarations +- **Arrow functions** — Arrow functions assigned to variables (e.g., `const handler = () => {}`) ✨ v0.5 +- **React hooks** — Automatically detected by `use*` naming pattern ✨ v0.5 - **Classes** — Class declarations and their methods - **Interfaces** — TypeScript interfaces - **Types** — Type aliases -- **Variables** — Exported constants and variables +- **Exported constants** — Objects, arrays, and factory calls (e.g., `export const CONFIG = {}`) ✨ v0.5 - **Markdown sections** — Headers and content blocks ## Tips diff --git a/website/content/docs/tools/index.mdx b/website/content/docs/tools/index.mdx index 504c43b..8eea744 100644 --- a/website/content/docs/tools/index.mdx +++ b/website/content/docs/tools/index.mdx @@ -16,7 +16,12 @@ dev-agent provides nine tools through the Model Context Protocol (MCP). These to | [`dev_status`](/docs/tools/dev-status) | Check repository indexing status | | [`dev_health`](/docs/tools/dev-health) | Monitor MCP server health | -## New in v0.4.0 +## New in v0.5.0 + +- **Enhanced indexing** — Arrow functions, React hooks, and exported constants now extracted +- **`dev_search`** — Better coverage of modern JavaScript patterns (hooks, utilities, configs) + +## v0.4.0 - **`dev_history`** — Semantic search over git commit history - **`dev_map`** — Change frequency indicators (🔥 hot, ✏️ active, 📝 recent) diff --git a/website/content/index.mdx b/website/content/index.mdx index 97835d7..ad2b474 100644 --- a/website/content/index.mdx +++ b/website/content/index.mdx @@ -11,7 +11,7 @@ Local semantic code search for Cursor and Claude Code via MCP. [Get Started](/docs) · [View on GitHub](https://github.com/lytics/dev-agent) - **v0.4.4** — Search results now show related test files automatically. [See what's new →](https://github.com/lytics/dev-agent/releases) + **v0.5.0 coming soon** — Arrow functions, React hooks, and exported constants now indexed. [See what's new →](/updates) diff --git a/website/content/updates/_meta.js b/website/content/updates/_meta.js new file mode 100644 index 0000000..845eabc --- /dev/null +++ b/website/content/updates/_meta.js @@ -0,0 +1,3 @@ +export default { + index: 'Release Notes', +}; diff --git a/website/content/updates/index.mdx b/website/content/updates/index.mdx new file mode 100644 index 0000000..f4c1f2a --- /dev/null +++ b/website/content/updates/index.mdx @@ -0,0 +1,199 @@ +# Updates + +What's new in dev-agent. We ship improvements regularly to help AI assistants understand your code better. + +--- + +## v0.5.0 — Modern JavaScript Support + +*Coming soon* + +**Better coverage for how developers actually write code today.** + +Modern JavaScript and TypeScript codebases look very different from traditional OOP. React hooks, arrow functions, and configuration objects are the primary APIs — and dev-agent now understands them. + +### What's New + +**🎯 Arrow Functions** + +```typescript +// Now indexed! AI can find these via semantic search +export const useAuth = () => { ... } +const validateEmail = (email: string) => email.includes('@') +``` + +**⚛️ React Hooks Detection** + +Hooks are automatically detected by the `use*` naming pattern. When you search for "authentication hook", dev-agent knows `useAuth` is a hook — not just a function. + +**📦 Exported Constants** + +Configuration objects, context providers, and factory-created values are now searchable: + +```typescript +export const API_CONFIG = { baseUrl: '/api', timeout: 5000 } +export const ThemeContext = createContext({ dark: false }) +export const SUPPORTED_LANGUAGES = ['typescript', 'javascript'] +``` + +### Why This Matters + +Before v0.5.0, searching for "API configuration" wouldn't find `export const API_CONFIG = {...}`. Now it does. This means: + +- **Better search results** for modern React/TypeScript codebases +- **More context** for AI assistants to understand your code +- **Fewer "I couldn't find..." moments** when asking about hooks or configs + +--- + +## v0.4.4 — Test File Discovery + +*Released November 2024* + +**Search results now show related test files automatically.** + +When `dev_search` returns source files, it checks for matching test files (`*.test.ts`, `*.spec.ts`) and includes them in the response: + +``` +1. [89%] function: authenticate (src/auth.ts:15) +2. [84%] class: AuthMiddleware (src/middleware.ts:5) + +--- +Related test files: + • src/auth.test.ts + • src/middleware.test.ts +``` + +This helps AI assistants find both implementation *and* tests without extra searches — useful when fixing bugs or understanding expected behavior. + +--- + +## v0.4.0 — Intelligent Git History + +*Released November 2024* + +**Semantic search over your commit history.** + +Ever tried to find "when did we add rate limiting?" by scrolling through `git log`? Now you can just ask: + +### New Tool: `dev_history` + +Search commits by meaning, not just keywords: + +``` +> "Find commits related to authentication changes" + +1. [92%] fix(auth): resolve token refresh race condition + Author: jane@example.com | 3 days ago + Files: src/auth/refresh.ts, src/auth/token.ts + +2. [87%] feat(auth): add OAuth2 provider support + Author: john@example.com | 2 weeks ago + Files: src/auth/oauth.ts, src/auth/providers/ +``` + +### Change Frequency in `dev_map` + +See which parts of your codebase are actively changing: + +``` +packages/ +├── 🔥 core/ (45 components) — 12 commits in 30d +├── ✏️ mcp-server/ (28 components) — 3 commits in 30d +├── 📝 cli/ (15 components) — 2 commits in 90d +└── subagents/ (35 components) +``` + +- 🔥 **Hot** — 10+ commits in last 30 days +- ✏️ **Active** — 3-9 commits in last 30 days +- 📝 **Recent** — Any commits in last 90 days + +### Context Assembly Improvements + +`dev_plan` now includes related commits when assembling context for GitHub issues. AI assistants can see both *what* the issue asks for and *how* similar changes were made before. + +--- + +## v0.3.0 — Code Relationships + +*Released November 2024* + +**Understand how your code connects.** + +### New Tool: `dev_refs` + +Find what calls what — and what's called by what: + +``` +> "What functions call validateUser?" + +Callers of validateUser: +1. AuthMiddleware.authenticate (src/middleware.ts:45) +2. LoginController.handleLogin (src/controllers/login.ts:23) +3. SessionManager.validateSession (src/session.ts:67) +``` + +Essential for impact analysis before refactoring. + +### Hot Paths in `dev_map` + +See the most-referenced files in your codebase: + +``` +Hot Paths (most referenced): + 🔥 src/utils/logger.ts (47 references) + 🔥 src/types/index.ts (34 references) + 🔥 src/config.ts (28 references) +``` + +### Smarter Context Assembly + +`dev_plan` was refactored from generating task breakdowns to assembling rich context. Instead of making assumptions about *how* to implement something, it now gives AI assistants the context they need to figure it out themselves. + +--- + +## v0.1.0 — Initial Release + +*Released November 2024* + +**The foundation: local-first code understanding for AI tools.** + +### Core Features + +- **Semantic Search** — Find code by meaning, not keywords +- **MCP Integration** — Works with Cursor, Claude Code, and any MCP client +- **Multi-language** — TypeScript, JavaScript, Go, Python, Rust, Markdown +- **100% Local** — No cloud, no telemetry, all processing on your machine + +### Five Tools at Launch + +| Tool | Purpose | +|------|---------| +| `dev_search` | Semantic code search | +| `dev_plan` | Context assembly for issues | +| `dev_explore` | Pattern discovery | +| `dev_gh` | GitHub issue/PR search | +| `dev_status` | Repository health | + +### Installation + +One command to index, one command to install: + +```bash +npm install -g dev-agent +dev index . +dev mcp install --cursor +``` + +--- + +## What's Next? + +We're working on: + +- **Incremental indexing** — Only re-index changed files +- **More languages** — Better Go and Python support via tree-sitter +- **Parallel search** — Query multiple repos simultaneously + +Have ideas? [Open an issue](https://github.com/lytics/dev-agent/issues) or join the discussion. +