Add Documentation for Experimental Codebase Indexing Feature#204
Merged
hannesrudolph merged 3 commits intomainfrom May 24, 2025
Merged
Add Documentation for Experimental Codebase Indexing Feature#204hannesrudolph merged 3 commits intomainfrom
hannesrudolph merged 3 commits intomainfrom
Conversation
… feature that enables semantic code search using AI embeddings. ## New Documentation - **docs/features/experimental/codebase-indexing.mdx**: Comprehensive feature documentation covering: - Semantic search capabilities using Tree-sitter parsing and AI embeddings - Setup requirements for OpenAI/Ollama embedding providers and Qdrant vector database - Configuration steps and status indicators - File processing with smart code parsing and automatic filtering - Best practices for model selection and security considerations - Current limitations and future enhancements - Privacy and security considerations ## Updated Navigation & Cross-references - **sidebars.ts**: Added Codebase Indexing to Features > Experimental navigation menu - **docs/features/experimental/experimental-features.md**: - Added Codebase Indexing to experimental features list - Added screenshot showing the experimental features settings panel - **docs/faq.md**: Added FAQ entries explaining: - What Codebase Indexing is and its semantic search capabilities - Cost considerations for embedding generation and vector storage ## Assets - **static/img/experimental-features/experimental-features.png**: Screenshot of experimental features settings panel ## Technical Details Covered - Tree-sitter integration for AST-based code parsing - Support for both OpenAI and Ollama embedding providers - Qdrant vector database integration with local and cloud deployment options - Incremental indexing with file watching and hash-based caching - Smart file filtering excluding binaries, large files, and common ignore patterns - codebase_search tool integration for AI-powered code discovery The documentation is targeted at a semi-technical audience and provides practical setup guidance while explaining the underlying semantic search technology.
|
The latest updates on your projects. Learn more about Vercel for Git ↗︎
|
Collaborator
|
Content looks good. Only nitpick is that the UI looks slightly different now. |
…tool and reorganize the tool category structure to better distinguish between read and search functionality. ## New Documentation - **docs/advanced-usage/available-tools/codebase-search.md**: Complete tool documentation covering: - Semantic search capabilities using AI embeddings and vector similarity - Integration with experimental Codebase Indexing feature with proper warning - Parameters, requirements, and configuration dependencies (OpenAI/Ollama + Qdrant) - Detailed workflow explanation from query processing to result formatting - Best practices for effective semantic queries vs. traditional text search - Directory scoping capabilities and result interpretation guidelines - Usage examples demonstrating authentication, database, error handling, and testing searches - Similarity scoring explanation and result structure details ## Updated Navigation & Organization - **sidebars.ts**: Added codebase_search to Available Tools navigation menu - **docs/advanced-usage/available-tools/tool-use-overview.md**: Reorganized tool categories: - Split "Read Group" into separate "Read Group" and "Search Group" categories - **Read Group**: File system reading and exploration (read_file, list_files, list_code_definition_names) - **Search Group**: Pattern and semantic searching (search_files, codebase_search) - Updated tool group table to reflect the new logical separation - Updated common patterns example to showcase semantic search with codebase_search - Improved categorization aligns with actual tool usage patterns ## Technical Coverage The documentation accurately reflects the tool's implementation including: - CodeIndexManager integration and availability validation - Dual output format for AI and UI consumption - Vector similarity search with cosine similarity and 0.4 threshold - Performance optimizations (50 result limit, Tree-sitter language support) - Path filtering and workspace-relative result formatting - Integration with experimental indexing infrastructure This provides users with clear guidance on semantic code search capabilities while maintaining appropriate warnings about the experimental nature of the feature.
![ellipsis-dev[bot]](https://avatars.githubusercontent.com/in/64358?s=60&v=4){kind=small}
hannesrudolph
changed the title
mrubens
approved these changes
May 24, 2025
Collaborator
Author
|
@mrubens thanks so much for approving it after I merged it :P |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Add Documentation for Experimental Codebase Indexing Feature
Overview
This PR adds complete documentation for the new experimental Codebase Indexing feature that enables semantic code search using AI embeddings and vector similarity.
Changes
docs/features/experimental/codebase-indexing.mdxcovering setup, configuration, and usage with proper tool cross-referencesdocs/advanced-usage/available-tools/codebase-search.mdwith parameters, examples, and best practicesKey Features Documented
codebase_searchtool referenceThe documentation enables users to effectively set up and use AI-powered semantic code search while clearly marking the experimental nature of the feature.
Important
Adds documentation for the experimental Codebase Indexing feature, including setup, usage, and tool integration.
codebase-indexing.mdxfor Codebase Indexing feature, detailing setup, configuration, and usage.codebase-search.mdfor thecodebase_searchtool, explaining parameters, functionality, and best practices.tool-use-overview.md.This description was created by
for c0d22a8. You can customize this summary. It will automatically update as commits are pushed.