diff --git a/fern/customization/bring-your-own-vectors/trieve.mdx b/fern/customization/bring-your-own-vectors/trieve.mdx index 34bec1faf..bf2bfa6e1 100644 --- a/fern/customization/bring-your-own-vectors/trieve.mdx +++ b/fern/customization/bring-your-own-vectors/trieve.mdx @@ -1,62 +1,360 @@ --- title: Bring your own chunks/vectors from Trieve -subtitle: Use existing chunks/vectors from [Trieve](https://trieve.ai) +subtitle: Using Trieve for improved RAG with Vapi slug: customization/bring-your-own-vectors/trieve --- -Vapi supports Trieve as a knowledgebase provider, allowing you to leverage your existing document embeddings and chunks. While Vapi maintains its own storage of documents and vectors, you can seamlessly integrate with your Trieve datasets. +# Using Trieve with Vapi -## Use Cases +Vapi offers two ways to integrate with [Trieve](https://trieve.ai): -### Existing Knowledge Base Migration +1. **Direct Integration**: Create and manage Trieve datasets directly through Vapi +2. **BYOK (Bring Your Own Key)**: Use your Trieve API key to import your existing Trieve datasets into Vapi -If you've already invested time in building and organizing your knowledge base in Trieve, you can continue using those vectors without having to reprocess your documents. This is particularly useful for: +## Direct Integration with Trieve -- Large document collections that took significant time to process -- Carefully curated and cleaned datasets -- Custom-chunked documents with specific segmentation rules +When using Trieve directly through Vapi, you can create and manage datasets, but they'll be tied to Vapi's account. This approach offers: -### Parallel Systems +- Quick setup with minimal configuration +- Basic dataset management through Vapi's API +- Limited customization options -You might want to use both Trieve's native interface and Vapi simultaneously: +### Setting up Direct Integration -- Use Trieve's UI for content management and organization -- Leverage Vapi's chat interface and API capabilities -- Maintain consistency across both platforms +1. Navigate to the [Vapi dashboard credentials page](https://dashboard.vapi.ai/keys) +2. Add your Trieve API key from [Trieve's dashboard](https://dashboard.trieve.ai/org/keys) +3. Create a new knowledge base with Trieve as the provider: -## Integration Steps +```json +{ + "name": "my-trieve-kb", + "provider": "trieve", + "searchPlan": { + "scoreThreshold": 0.2, + "searchType": "semantic" + }, + "createPlan": { + "type": "create", + "chunkPlans": [ + { + "fileIds": ["file-123", "file-456"], + "websites": ["https://example.com"], + "targetSplitsPerChunk": 50, + "rebalanceChunks": true + } + ] + } +} +``` -1. **Configure Trieve Credentials** +## BYOK with Trieve (Recommended) - - Navigate to the credentials page in your [Vapi dashboard](https://dashboard.vapi.ai/keys) - - Add your Trieve API key for authentication from [Trieve](https://dashboard.trieve.ai/org/keys) +The BYOK approach offers more flexibility and control over your datasets. You can: -2. **Create a New Knowledge Base** +- Fully manage your datasets in Trieve's native interface +- Use Trieve's advanced features like: + - Custom chunking rules + - Search playground testing + - Manual chunk editing + - Website crawling + - Dataset visualization - - When setting up a new knowledge base, provide: - - Your Trieve datasetId as the providerId. - - Appropriate search configuration parameters. - - Vapi will then automatically use your Trieve dataset as the knowledge base. +### Step 1: Set Up Trieve Dataset - Example configuration: +1. Create an account at [Trieve](https://trieve.ai) +2. Create a new dataset using Trieve's dashboard - ```json - { - "name": "trieve-byok", - "provider": "trieve", - "searchPlan": { - "scoreThreshold": 0.2, - "searchType": "semantic" - }, - "createPlan": { - "type": "import", - "providerId": "" - } - } - ``` +![Create dataset in Trieve](../../static/images/knowledge-base/create-dataset.png) + +When creating your dataset in Trieve, selecting the right embedding model is crucial for optimizing performance and accuracy. Here are some of the available options: + +### jina-base-en + +- **Provider**: Jina AI (Hosted by Trieve) +- **Performance**: Fast +- **Description**: This model is designed for speed and efficiency, making it suitable for applications where quick response times are critical. It provides a good balance of performance and accuracy for general use cases. + +### text-embedding-3-small + +- **Provider**: OpenAI +- **Performance**: Moderate +- **Description**: A smaller model from OpenAI that offers a compromise between speed and accuracy. It is suitable for applications that require a balance between computational efficiency and the quality of embeddings. + +### text-embedding-3-large + +- **Provider**: OpenAI +- **Performance**: Slow +- **Description**: This larger model provides the highest accuracy among the options but at the cost of slower processing times. It is ideal for applications where the quality of embeddings is prioritized over speed. + +3. Add content through various methods: + +#### Upload Documents + +Upload documents directly through Trieve's interface: + +![Upload files in Trieve](../../static/images/knowledge-base/upload-files.png) + +When uploading files, you can configure advanced chunking options: + +![Upload files advanced options in Trieve](../../static/images/knowledge-base/upload-files-advanced.png) + +#### Edit Individual Chunks + +After uploading documents, you can edit individual chunks to refine their content: + +![Edit chunk interface in Trieve](../../static/images/knowledge-base/edit-chunk.png) + +##### Editing Options + +- **Chunk Content**: Modify the text directly in the rich text editor + + - Fix formatting issues + - Correct errors or typos + - Split or combine chunks manually + - Add or remove content + +- **Metadata Fields**: + - Date: Update document timestamps + - Number Value: Adjust numeric metadata for filtering + - Location: Set or modify geographical coordinates + - Weight: Fine-tune search relevance with custom weights + - Fulltext Boost: Add terms to enhance search visibility + - Semantic Boost: Adjust vector embedding influence + +##### Best Practices for Chunk Editing + +1. **Content Length** + + - Keep chunks between 200-1000 tokens + - Maintain logical content boundaries + - Ensure complete thoughts within each chunk + +2. **Metadata Optimization** + + - Use consistent date formats + - Add relevant numeric values for filtering + - Apply weights strategically for important content + +3. **Search Enhancement** + - Use boost terms for critical keywords + - Balance semantic and fulltext boosts + - Test search results after significant edits + +### Advanced Chunking Options + +#### Metadata + +- Add custom metadata as JSON to associate with your chunks + - Useful for filtering and organizing content (e.g., `{"author": "John Doe", "category": "technical"}`) + - Keep metadata concise and relevant to avoid storage overhead + - Use consistent keys across related documents for better searchability + +#### Date + +- Specify the creation or relevant date for the document + - Important for version control and content freshness + - Helps with filtering outdated information + - Use actual document creation dates when possible + +#### Split Delimiters + +- Define custom delimiters (e.g., ".,?\n") to control where chunks are split + - Recommended defaults: ".,?\n" for general content + - Add semicolons (;) for technical documentation + - Use "\n\n" for markdown or structured content + - Avoid over-aggressive splitting that might break context + +#### Target Splits Per Chunk + +- Set the desired number of splits per chunk + - Default: 20 splits + - Recommended ranges: + - 15-25 for general content + - 10-15 for technical documentation + - 25-30 for narrative content + - Lower values create more granular chunks, better for precise retrieval + - Higher values maintain more context but may retrieve irrelevant information + +#### Rebalance Chunks + +- Enable to redistribute content evenly across chunks + - Recommended for documents with varying section lengths + - Helps maintain consistent chunk sizes + - May slightly impact natural content boundaries + - Best used with technical documentation or structured content + +#### Use gpt4o chunking + +- Enable GPT-4 optimized chunking for improved semantic coherence + - Recommended for: + - Complex technical documentation + - Content with intricate relationships + - Documents where context preservation is crucial + - Note: Increases processing time and cost + - Best for high-value content where accuracy is paramount + +#### Heading Based Chunking + +- Split content based on document headings + - Ideal for well-structured documents (e.g., documentation, reports) + - Works best with consistent heading hierarchy + - Consider enabling for: + - Technical documentation + - User manuals + - Research papers + - May create uneven chunk sizes based on section lengths + +#### System Prompt + +- Provide custom instructions for the chunking process + - Optional but powerful for specific use cases + - Example prompts: + - "Preserve code blocks as single chunks" + - "Keep API endpoint descriptions together" + - "Maintain question-answer pairs in the same chunk" + - Keep prompts clear and specific + - Test different prompts with sample content to optimize results + +#### Website Crawling + +Trieve offers powerful website crawling capabilities with extensive configuration options: + +![Website crawling in Trieve](../../static/images/knowledge-base/crawl.png) + +##### Crawl Configuration Options + +- **Crawl Interval**: Set how often to refresh content + + - Options: Daily, Weekly, Monthly + - Recommended: Daily for frequently updated content + +- **Page Limit**: Control the maximum number of pages to crawl + + - Default: 1000 pages + - Adjust based on your site size and content relevance + +- **URL Patterns** + + - Include/Exclude specific URL patterns using regex + - Example includes: `https://docs.example.com/*` + - Example excludes: `https://example.com/internal/*` + +- **Query Selectors** + + - Include specific HTML elements for targeted content extraction + - Exclude navigation, footers, and other non-content elements + - Common excludes: `navbar`, `footer`, `aside`, `nav`, `form` + +- **Special Content Types** + + - OpenAPI Spec: Toggle for API documentation crawling + - Shopify: Enable for e-commerce content + - YouTube Channel: Include video transcripts and descriptions + +- **Advanced Options** + - Boost Titles: Increase weight of page titles in search results + - Allow External Links: Include content from linked domains + - Ignore Sitemap: Skip sitemap-based crawling + - Remove Strings: Clean up headers and body content + +##### Best Practices for Crawling + +1. **Start Small** + + - Begin with a low page limit + - Test with specific sections of your site + - Gradually expand coverage + +2. **Optimize Selectors** + + - Remove navigation and UI elements + - Focus on main content areas + - Use browser inspector to identify key selectors + +3. **Monitor Performance** + - Check crawl logs regularly + - Adjust patterns based on results + - Balance frequency with server load + +### Step 2: Test and Refine + +Use Trieve's search playground to: + +- Test semantic search queries +- Adjust chunk sizes +- Edit chunks manually + +- Visualize vector embeddings +- Fine-tune relevance scores + +![Search playground in Trieve](../../static/images/knowledge-base/search-playground.png) + +### Step 3: Import to Vapi + +1. Create your Trieve API key from [Trieve's dashboard](https://dashboard.trieve.ai/org/keys) +2. Add your Trieve API key to Vapi [Provider Credentials](https://dashboard.vapi.ai/keys) + ![Add Trieve API key in Vapi](../../static/images/knowledge-base/trieve-credential.png) +3. Once your dataset is optimized in Trieve, import it to Vapi: + +```json +{ + "name": "trieve-byok", + "provider": "trieve", + "searchPlan": { + "scoreThreshold": 0.2, + "searchType": "semantic" + }, + "createPlan": { + "type": "import", + "providerId": "" + } +} +``` ## Best Practices -- Ensure your Trieve API key has appropriate permissions -- Keep track of which datasetIds correspond to which knowledge bases -- Monitor vector synchronization to ensure consistency +1. **Dataset Organization** + + - Segment datasets by domain knowledge boundaries + - Use semantic-based dataset naming (e.g., "api-docs-v2", "user-guides-2024") + - Version control chunking configurations in your codebase + +2. **Content Quality** + + - Implement text normalization (Unicode normalization, whitespace standardization) + - Use regex patterns to clean formatting artifacts + - Validate chunk semantic coherence through embedding similarity scores + +3. **Performance Optimization** + + - Target chunk sizes: 200-1000 tokens (optimal for current embedding models) + - Configure hybrid search with BM25 boost = 0.3 for technical content + - Set score thresholds dynamically based on embedding model (0.2 for text-embedding-3-small, 0.25 for text-embedding-3-large) + +4. **Maintenance** + - Implement automated content refresh cycles via Trieve's API + - Track search result relevance metrics (MRR, NDCG) + - Rotate API keys on 90-day cycles + +## Troubleshooting + +Common issues and solutions: + +1. **Search Relevance Issues** + + - Implement cross-encoder reranking for critical queries + - Fine-tune BM25 vs semantic weights (recommended ratio: 0.3:0.7) + - Analyze chunk boundary overlap percentage (aim for 15-20%) + +2. **Integration Errors** + + - Validate dataset permissions (READ_DATASET scope required) + - Check for dataset ID format compliance (UUID v4) + - Monitor rate limits (default: 100 requests/min) + +3. **Performance Optimization** + - Implement chunk size normalization (max variance: 20%) + - Enable query caching for frequent searches + - Use batch operations for bulk updates (max 100 chunks/request) + +Need help? Contact [support@vapi.ai](mailto:support@vapi.ai) for assistance. diff --git a/fern/static/images/knowledge-base/crawl.png b/fern/static/images/knowledge-base/crawl.png new file mode 100644 index 000000000..05cf5ccfc Binary files /dev/null and b/fern/static/images/knowledge-base/crawl.png differ diff --git a/fern/static/images/knowledge-base/create-dataset.png b/fern/static/images/knowledge-base/create-dataset.png new file mode 100644 index 000000000..0d6557cf8 Binary files /dev/null and b/fern/static/images/knowledge-base/create-dataset.png differ diff --git a/fern/static/images/knowledge-base/edit-chunk.png b/fern/static/images/knowledge-base/edit-chunk.png new file mode 100644 index 000000000..4d736d65f Binary files /dev/null and b/fern/static/images/knowledge-base/edit-chunk.png differ diff --git a/fern/static/images/knowledge-base/search-playground.png b/fern/static/images/knowledge-base/search-playground.png new file mode 100644 index 000000000..e4950834a Binary files /dev/null and b/fern/static/images/knowledge-base/search-playground.png differ diff --git a/fern/static/images/knowledge-base/trieve-credential.png b/fern/static/images/knowledge-base/trieve-credential.png new file mode 100644 index 000000000..af07a7060 Binary files /dev/null and b/fern/static/images/knowledge-base/trieve-credential.png differ diff --git a/fern/static/images/knowledge-base/upload-files-advanced.png b/fern/static/images/knowledge-base/upload-files-advanced.png new file mode 100644 index 000000000..afc1c80d5 Binary files /dev/null and b/fern/static/images/knowledge-base/upload-files-advanced.png differ diff --git a/fern/static/images/knowledge-base/upload-files.png b/fern/static/images/knowledge-base/upload-files.png new file mode 100644 index 000000000..63e2ce71a Binary files /dev/null and b/fern/static/images/knowledge-base/upload-files.png differ