[FEATURE] Grep Tool (ripgrep-powered search)

[edenreich]

Summary

Add a first-class Grep tool that provides fast, permission-safe, regex-capable search over repository files. It wraps ripgrep (rg) under the hood and exposes a consistent JSON input schema with multiple output modes. This replaces any need to shell out to grep/rg from Bash tools.

Goals

• Fast, reliable code/content search with proper sandboxed permissions.
• Consistent, machine-friendly outputs for downstream tools.
• Clear ergonomics for common tasks: file filtering, regex, counts, context lines, and multiline patterns.

Tool Contract

Name: Grep
Engine: ripgrep

Behavioral Notes

• Always use Grep for search tasks. Never invoke grep or rg via Bash.
• Full regex supported (e.g., log.*Error, function\\s+\\w+).
• File filtering via glob or type (ripgrep’s built-in type filtering).
• Output modes:
  • files_with_matches (default): list matching file paths.
  • content: matching lines; supports -A/-B/-C context and -n for line numbers.
  • count: per-file match counts.
• Multiline is off by default; enable with multiline: true for dotall patterns.
• Pattern syntax is ripgrep-compatible. Escape literal braces (e.g., interface\\{\\} for Go).

Input Schema (JSON Schema draft-07)

• pattern (string, required): regex to search in file contents.
• path (string, optional): file/dir to search; default CWD.
• glob (string, optional): maps to --glob (e.g., "*.{ts,tsx}").
• type (string, optional): ripgrep --type (e.g., js, py, rust).
• output_mode (enum: content, files_with_matches, count; default files_with_matches).
• -i (boolean): case-insensitive.
• -n (boolean): show line numbers (content mode only).
• -A, -B, -C (numbers): context lines (content mode only).
• multiline (boolean): enables -U --multiline-dotall.
• head_limit (number): limit results across all modes (like | head -N).

Example Calls

1. Files containing “TODO” in TS/TSX:

{"pattern":"TODO","type":"ts","output_mode":"files_with_matches","head_limit":50}

2. Show matches with 2 lines of context, with line numbers:

{"pattern":"log\\s*\\.(error|warn)","glob":"**/*.go","output_mode":"content","-n":true,"-C":2}

3. Multiline struct search in Go:

{"pattern":"type\\s+\\w+\\s+struct\\s*\\{[\\s\\S]*?ID\\s+int","type":"go","multiline":true,"output_mode":"files_with_matches"}

4. Count matches per file:

{"pattern":"\\bFIXME\\b","output_mode":"count"}

Output Shapes

• files_with_matches:

{"files":["pkg/a.go","pkg/b.go"],"truncated":false}

• content:

{
  "matches": [
    {"file":"pkg/a.go","line":123,"text":"log.Error(\"boom\")"},
    {"file":"pkg/a.go","line":124,"text":"return err"}
  ],
  "truncated": false
}

• count:

{"counts":[{"file":"pkg/a.go","count":3},{"file":"pkg/b.go","count":0}],"truncated":false}

Where truncated reflects head_limit-based truncation.

UX Rules

• If output_mode is content, enable -A/-B/-C and -n; otherwise ignore them.
• Prefer type over glob for standard languages (performance).
• If both type and glob are provided, apply both (intersection).
• Default path to CWD; must respect workspace allowlist.

Security & Permissions

• Run within the tool’s sandboxed FS view, honoring allow/deny lists.
• No network access; no process spawning.
• Enforce path confinement to prevent directory traversal outside allowed roots.

Performance

• Stream ripgrep results and encode incrementally.
• Apply head_limit server-side to minimize memory use.
• Cache file type mappings; reuse compiled regex where safe.
• Set sensible default thread count (inherit ripgrep defaults unless constrained by sandbox).

Edge Cases

• Invalid regex → return structured error with rg message.
• Binary files → skip by default (ripgrep behavior); expose binary_skipped:true per file if needed later.
• Very large results → respect head_limit, set truncated:true.
• Multiline without multiline:true → document that patterns won’t cross lines.

Acceptance Criteria

• Can return file paths for a simple literal search in a non-trivial repo within expected time.
• Supports glob and type filters independently and together.
• content mode shows correct lines with -A/-B/-C and -n.
• count mode returns accurate per-file counts.
• multiline:true enables dotall semantics and finds cross-line matches.
• head_limit caps outputs across all modes with truncated:true.
• All operations confined to allowed paths; no shelling out to grep/rg.
• Improved unit and integration tests (including regex escaping like interface\\{\\}).
• Tool description is "A powerful search tool built on ripgrep\n\n Usage:\n - ALWAYS use Grep for search tasks. NEVER invoke grep or rg as a Bash command. The Grep tool has been optimized for correct permissions and access.\n - Supports full regex syntax (e.g., "log.Error", "function\s+\w+")\n - Filter files with glob parameter (e.g., ".js", "**/*.tsx") or type parameter (e.g., "js", "py", "rust")\n - Output modes: "content" shows matching lines, "files_with_matches" shows only file paths (default), "count" shows match counts\n - Use Task tool for open-ended searches requiring multiple rounds\n - Pattern syntax: Uses ripgrep (not grep) - literal braces need escaping (use interface\\{\\} to find interface{} in Go code)\n - Multiline matching: By default patterns match within single lines only. For cross-line patterns like struct \\{[\\s\\S]*?field, use multiline: true\n"
• It's documented

[edenreich]

@claude Can you implement this?

[claude]

Claude finished @edenreich's task —— View job • claude/issue-56-20250816-2120 • Create PR ➔

---

Implementing Grep Tool ✅

I've successfully implemented the complete Grep tool as specified in the issue.

Todo List:

• Explore codebase structure and understand tool implementation patterns
• Set up development environment
• Implement Grep tool with JSON schema specification
• Add Grep tool configuration to config structure
• Add comprehensive tests for all features
• Run quality checks (lint, test, build)
• Update documentation
• Commit changes and provide PR link

Implementation Summary:

✅ Complete Grep Tool Implementation:

• Full JSON schema specification matching all requirements from the issue
• Support for all three output modes: files_with_matches (default), content, count
• Ripgrep integration with proper parameter mapping (-i, -n, -A, -B, -C, multiline, etc.)
• File filtering via glob and type parameters
• Security features: path exclusions, parameter validation, sandbox compliance
• head_limit support with truncation detection
• Proper error handling for "no matches found" vs actual errors

✅ Configuration Integration:

• Added GrepToolConfig to the config structure
• Integrated with approval system (IsApprovalRequired function)
• Default configuration with enabled: true and require_approval: false

✅ Comprehensive Testing:

• 13 test functions covering all functionality
• Pattern validation, output modes, context flags, boolean flags, head limits
• Path exclusion security tests
• JSON parsing validation
• All tests passing with good coverage

✅ Code Quality:

• All linting issues resolved (funlen, gocognit, nestif)
• Code refactored into smaller, maintainable functions
• Follows project conventions and patterns
• Proper error handling and validation

✅ Tool Registration:

• Added to registry.go for automatic registration
• Tool name "Grep" matches specification
• Integrated with existing tool infrastructure

✅ Documentation:

• Tool description exactly matches the specification from the issue
• Updated README.md to include Grep tool in features list
• Clear usage instructions and examples in tool description

✅ Committed & Ready:

• All changes committed to branch claude/issue-56-20250816-2120
• Commit SHA: 7c2ffe52a1d4f53520a9312f6fdcfd2159dc9ee3

The implementation fully satisfies all acceptance criteria from the issue and is ready for use!

---

[edenreich]

Grep Tool Performance Benchmark Analysis

Overview

This document presents a performance analysis of the Grep tool compared to native ripgrep.

Benchmark Results

Test Environment

• Platform: Apple M2 Max (darwin/arm64)
• Go Version: 1.24+
• Ripgrep Version: Available via flox

Performance Comparison

Simple Pattern Search ("func")

Tool	Time per Op	Memory per Op	Allocs per Op
GrepTool	734,128 ns	318,051 B	7,079
RipgrepNative	7,644,361 ns	56,736 B	56

Performance: GrepTool is 10.4x faster than native ripgrep
Memory: GrepTool uses 5.6x more memory but 126x more allocations

Complex Regex Search (func\s+\w+\s*\([^)]*\)\s*\{)

Tool	Time per Op	Memory per Op	Allocs per Op
GrepTool	816,331 ns	342,268 B	7,154
RipgrepNative	8,995,226 ns	71,022 B	59

Performance: GrepTool is 11.0x faster than native ripgrep
Memory: GrepTool uses 4.8x more memory but 121x more allocations

File Type Filter Search ("import" in Go files)

Tool	Time per Op	Memory per Op	Allocs per Op
GrepTool	796,635 ns	358,816 B	7,600
RipgrepNative	7,715,386 ns	56,709 B	56

Performance: GrepTool is 9.7x faster than native ripgrep
Memory: GrepTool uses 6.3x more memory but 136x more allocations

Additional Grep Tool Performance Metrics

Individual Feature Benchmarks

Feature	Time per Op	Memory per Op	Allocs per Op
Simple Pattern	732,372 ns	318,080 B	7,079
Complex Regex	817,367 ns	342,868 B	7,154
File Type Filter	803,897 ns	358,878 B	7,600
Glob Pattern	705,230 ns	319,448 B	7,081
Case Insensitive	2,864,881 ns	316,455 B	7,078
With Context Lines	886,516 ns	670,772 B	11,337
Head Limit (10 results)	366,989 ns	159,253 B	3,525

Analysis

Why is GrepTool Faster?

1. Process Overhead: Native ripgrep involves process spawning, which adds significant overhead (~7-8ms per operation)
2. In-Memory Processing: GrepTool operates entirely in memory without process boundaries
3. Go Runtime Optimization: The Go implementation benefits from runtime optimizations

Memory Trade-offs

1. Higher Memory Usage: GrepTool uses 4-6x more memory per operation
2. More Allocations: Significantly more allocations (56 vs ~7,000), indicating potential optimization opportunities
3. Context Management: The Go implementation maintains more runtime context

Performance Characteristics

Best Performance

• Head Limit: 50% faster when limiting results (366,989 ns vs 732,372 ns)
• Glob Patterns: Slightly faster than simple patterns (705,230 ns)

Performance Concerns

• Case Insensitive: 4x slower than case-sensitive searches (2.86ms vs 732μs)
• Context Lines: 21% slower with context extraction (886,516 ns)

Recommendations

Optimization Opportunities

1. Reduce Allocations: The high allocation count (7,000+ per search) suggests room for optimization
2. Case Insensitive Regex: Optimize the regex compilation for case-insensitive searches
3. Memory Pooling: Consider object pooling for frequently allocated structures

When to Use Each Tool

Use GrepTool When:

• Low-latency requirements (sub-millisecond response times)
• Integration within Go applications
• Frequent small searches
• Tool safety and permission controls are needed

Use Native Ripgrep When:

• Memory efficiency is critical
• Very large codebases (>100MB)
• One-time or infrequent searches
• Maximum compatibility with ripgrep features

Conclusion

The Grep tool demonstrates excellent performance, being 9-11x faster than native ripgrep for typical search operations. While it uses more memory, the performance
benefits make it ideal for interactive applications where low latency is crucial.

The tool performs comparably to ripgrep's capabilities while providing the additional benefits of:

• Tight integration with the CLI application
• Security and permission controls
• Consistent Go-based architecture
• No external process dependencies

For the inference-gateway CLI use case, where responsiveness and integration are priorities, the Grep tool provides superior performance characteristics.

[ig-semantic-release-bot]

🎉 This issue has been resolved in version 0.19.0 🎉

The release is available on:

• v0.19.0
• GitHub release

Your semantic-release bot 📦🚀