From 28d0939afde67446cd5d66fc52d52b6531b644a4 Mon Sep 17 00:00:00 2001 From: Mahdiyar Date: Tue, 30 Sep 2025 04:23:48 -0400 Subject: [PATCH] Add Roundtable MCP Server Integration for LlamaIndex MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit This integration brings unified AI assistant access to LlamaIndex applications through the Roundtable MCP Server, enabling: • Multi-provider AI access (OpenAI, Anthropic, Google) through single interface • Cost-optimized query routing based on task complexity • Robust fallback support for production reliability • Enhanced RAG workflows with specialized model selection • Advanced multi-modal capabilities across different AI providers The integration includes: - Comprehensive example notebook with practical use cases - Cost optimization strategies for LlamaIndex applications - Production-ready configuration examples - Multi-modal workflow demonstrations - Integration with existing LlamaIndex query engines This addition enhances LlamaIndex's ecosystem by providing seamless access to multiple AI providers while maintaining the framework's core principles of modularity and extensibility. 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude --- .../tools/roundtable_mcp_server.ipynb | 574 ++++++++++++++++++ .../docs/framework/community/integrations.md | 1 + 2 files changed, 575 insertions(+) create mode 100644 docs/examples/examples/tools/roundtable_mcp_server.ipynb diff --git a/docs/examples/examples/tools/roundtable_mcp_server.ipynb b/docs/examples/examples/tools/roundtable_mcp_server.ipynb new file mode 100644 index 0000000000..29b8acee0e --- /dev/null +++ b/docs/examples/examples/tools/roundtable_mcp_server.ipynb @@ -0,0 +1,574 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Using Roundtable MCP Server with LlamaIndex\n", + "\n", + "Integrate unified AI assistant access into your LlamaIndex workflows using the [Roundtable MCP Server](https://github.com/punkpeye/roundtable)." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Overview\n", + "\n", + "**Roundtable** is a unified AI assistant access system that allows you to interact with multiple AI providers (OpenAI, Anthropic, Google, etc.) through a single, consistent interface. The Roundtable MCP Server brings this powerful capability directly into your LlamaIndex applications.\n", + "\n", + "### Key Benefits for LlamaIndex Developers:\n", + "\n", + "- **Unified AI Access**: Switch between AI providers without changing your LlamaIndex code\n", + "- **Enhanced RAG Workflows**: Combine multiple AI assistants for complex reasoning tasks\n", + "- **Multi-Model Strategies**: Use different models for different parts of your pipeline\n", + "- **Cost Optimization**: Route queries to the most cost-effective model for each task\n", + "- **Fallback Support**: Automatic failover between AI providers for reliability\n", + "\n", + "This integration is particularly powerful for LlamaIndex applications that need:\n", + "- Complex multi-step reasoning with different AI capabilities\n", + "- Robust production systems with multiple AI provider fallbacks\n", + "- Cost-optimized AI usage across different model types\n", + "- Flexible AI provider switching without code changes" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Installation\n", + "\n", + "First, install the required packages:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# Install LlamaIndex MCP tools\n", + "!pip install llama-index-tools-mcp\n", + "\n", + "# Install Roundtable MCP Server\n", + "!npm install -g @punkpeye/roundtable" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Setup Roundtable MCP Server\n", + "\n", + "Configure your AI provider credentials and start the Roundtable MCP Server:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# Create Roundtable configuration\n", + "import json\n", + "import os\n", + "\n", + "# Configure your AI providers\n", + "roundtable_config = {\n", + " \"providers\": {\n", + " \"openai\": {\n", + " \"apiKey\": os.getenv(\"OPENAI_API_KEY\"),\n", + " \"models\": [\"gpt-4\", \"gpt-3.5-turbo\"]\n", + " },\n", + " \"anthropic\": {\n", + " \"apiKey\": os.getenv(\"ANTHROPIC_API_KEY\"),\n", + " \"models\": [\"claude-3-sonnet\", \"claude-3-haiku\"]\n", + " },\n", + " \"google\": {\n", + " \"apiKey\": os.getenv(\"GOOGLE_API_KEY\"),\n", + " \"models\": [\"gemini-pro\", \"gemini-pro-vision\"]\n", + " }\n", + " },\n", + " \"defaultProvider\": \"openai\",\n", + " \"fallbackProviders\": [\"anthropic\", \"google\"]\n", + "}\n", + "\n", + "# Save configuration\n", + "with open(\"roundtable-config.json\", \"w\") as f:\n", + " json.dump(roundtable_config, f, indent=2)\n", + "\n", + "print(\"Roundtable configuration created!\")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Start the Roundtable MCP Server (run this in a separate terminal):\n", + "\n", + "```bash\n", + "roundtable mcp --config roundtable-config.json --port 8080\n", + "```" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Basic Usage: Connect to Roundtable MCP Server\n", + "\n", + "Load tools from the Roundtable MCP Server:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "from llama_index.tools.mcp import aget_tools_from_mcp_url\n", + "import asyncio\n", + "\n", + "# Connect to Roundtable MCP Server\n", + "roundtable_tools = await aget_tools_from_mcp_url(\"http://127.0.0.1:8080/mcp\")\n", + "\n", + "print(f\"Loaded {len(roundtable_tools)} tools from Roundtable MCP Server:\")\n", + "for tool in roundtable_tools:\n", + " print(f\"- {tool.metadata.name}: {tool.metadata.description}\")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Example 1: Enhanced RAG with Multi-Model Strategy\n", + "\n", + "Use different AI models for different aspects of your RAG pipeline:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "from llama_index.core import SimpleDirectoryReader, VectorStoreIndex\n", + "from llama_index.core.agent.workflow import FunctionAgent\n", + "from llama_index.llms.openai import OpenAI\n", + "\n", + "# Load your documents\n", + "documents = SimpleDirectoryReader(\"your_documents\").load_data()\n", + "index = VectorStoreIndex.from_documents(documents)\n", + "\n", + "# Create an agent with Roundtable tools\n", + "agent = FunctionAgent(\n", + " tools=roundtable_tools,\n", + " llm=OpenAI(model=\"gpt-4\"),\n", + " system_prompt=\"\"\"\n", + " You are an advanced RAG assistant with access to multiple AI providers through Roundtable.\n", + " \n", + " For complex reasoning tasks, use Claude models.\n", + " For quick factual queries, use GPT-3.5-turbo.\n", + " For creative tasks, use GPT-4.\n", + " For visual analysis, use Gemini Pro Vision.\n", + " \n", + " Always choose the most appropriate model for each subtask.\n", + " \"\"\"\n", + ")\n", + "\n", + "# Enhanced query with multi-model reasoning\n", + "response = await agent.run(\"\"\"\n", + "Analyze the uploaded documents and:\n", + "1. Use Claude for deep reasoning about the main themes\n", + "2. Use GPT-3.5 for quick fact extraction\n", + "3. Use GPT-4 for creative synthesis\n", + "4. Provide a comprehensive analysis combining all perspectives\n", + "\"\"\")\n", + "\n", + "print(response)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Example 2: Cost-Optimized Query Routing\n", + "\n", + "Automatically route queries to the most cost-effective model:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# Create a cost-optimized agent\n", + "cost_optimized_agent = FunctionAgent(\n", + " tools=roundtable_tools,\n", + " llm=OpenAI(model=\"gpt-3.5-turbo\"),\n", + " system_prompt=\"\"\"\n", + " You are a cost-optimization specialist. For each query:\n", + " \n", + " - Simple factual questions → Use GPT-3.5-turbo (lowest cost)\n", + " - Complex reasoning → Use Claude Haiku (balanced cost/performance)\n", + " - Creative tasks → Use GPT-4 only when necessary\n", + " - Code generation → Use Claude Sonnet\n", + " \n", + " Always justify your model choice based on cost-effectiveness.\n", + " \"\"\"\n", + ")\n", + "\n", + "# Example queries with automatic cost optimization\n", + "queries = [\n", + " \"What is the capital of France?\", # Simple → GPT-3.5\n", + " \"Analyze the philosophical implications of AI consciousness\", # Complex → Claude\n", + " \"Write a creative story about time travel\", # Creative → GPT-4\n", + " \"Generate Python code for data analysis\" # Code → Claude Sonnet\n", + "]\n", + "\n", + "for query in queries:\n", + " print(f\"\\nQuery: {query}\")\n", + " response = await cost_optimized_agent.run(query)\n", + " print(f\"Response: {response}\")\n", + " print(\"-\" * 50)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Example 3: Robust Production System with Fallbacks\n", + "\n", + "Build resilient systems with automatic provider fallbacks:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# Create a production-ready agent with fallback support\n", + "production_agent = FunctionAgent(\n", + " tools=roundtable_tools,\n", + " llm=OpenAI(model=\"gpt-4\"),\n", + " system_prompt=\"\"\"\n", + " You are a production AI assistant with robust fallback capabilities.\n", + " \n", + " Primary strategy: Use OpenAI GPT-4\n", + " Fallback 1: If OpenAI fails, use Anthropic Claude\n", + " Fallback 2: If Anthropic fails, use Google Gemini\n", + " \n", + " Always ensure responses are delivered even if the primary provider fails.\n", + " Log which provider was used for monitoring purposes.\n", + " \"\"\"\n", + ")\n", + "\n", + "# Simulate production queries with error handling\n", + "async def robust_query(query: str):\n", + " try:\n", + " response = await production_agent.run(f\"\"\"\n", + " Process this query with fallback support: {query}\n", + " \n", + " If the primary provider fails, automatically try fallback providers.\n", + " Include in your response which AI provider successfully handled the query.\n", + " \"\"\")\n", + " return response\n", + " except Exception as e:\n", + " print(f\"Error with all providers: {e}\")\n", + " return \"All AI providers are currently unavailable. Please try again later.\"\n", + "\n", + "# Test production resilience\n", + "production_query = \"Analyze market trends for Q4 2024 and provide strategic recommendations\"\n", + "result = await robust_query(production_query)\n", + "print(f\"Production Result: {result}\")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Example 4: Advanced Multi-Modal Workflow\n", + "\n", + "Combine text, image, and code analysis using different specialized models:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# Multi-modal analysis agent\n", + "multimodal_agent = FunctionAgent(\n", + " tools=roundtable_tools,\n", + " llm=OpenAI(model=\"gpt-4\"),\n", + " system_prompt=\"\"\"\n", + " You are a multi-modal AI assistant specialist. Route tasks to the best model:\n", + " \n", + " - Image analysis → Gemini Pro Vision\n", + " - Code review → Claude Sonnet\n", + " - Document analysis → GPT-4\n", + " - Creative writing → GPT-4\n", + " - Technical reasoning → Claude Sonnet\n", + " - Quick facts → GPT-3.5-turbo\n", + " \n", + " Coordinate between models to provide comprehensive analysis.\n", + " \"\"\"\n", + ")\n", + "\n", + "# Complex multi-modal workflow\n", + "workflow_query = \"\"\"\n", + "I have a project with:\n", + "1. A screenshot of a UI mockup (image)\n", + "2. Python backend code\n", + "3. A requirements document (text)\n", + "\n", + "Please:\n", + "1. Analyze the UI mockup using vision capabilities\n", + "2. Review the Python code for best practices\n", + "3. Check if the code meets the requirements\n", + "4. Provide a comprehensive project assessment\n", + "\"\"\"\n", + "\n", + "comprehensive_analysis = await multimodal_agent.run(workflow_query)\n", + "print(f\"Multi-modal Analysis: {comprehensive_analysis}\")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Integration with LlamaIndex Query Engines\n", + "\n", + "Enhance your existing LlamaIndex query engines with Roundtable's multi-provider capabilities:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "from llama_index.core.query_engine import SubQuestionQueryEngine\n", + "from llama_index.core.tools import QueryEngineTool\n", + "\n", + "# Create enhanced query engines with Roundtable\n", + "base_query_engine = index.as_query_engine()\n", + "\n", + "# Wrap with Roundtable-enhanced capabilities\n", + "enhanced_query_engine = SubQuestionQueryEngine.from_defaults(\n", + " query_engine_tools=[\n", + " QueryEngineTool.from_defaults(\n", + " query_engine=base_query_engine,\n", + " description=\"Base document search and analysis\"\n", + " )\n", + " ] + roundtable_tools, # Add Roundtable tools\n", + " llm=OpenAI(model=\"gpt-4\")\n", + ")\n", + "\n", + "# Enhanced querying with multiple AI providers\n", + "enhanced_response = enhanced_query_engine.query(\"\"\"\n", + "Analyze the documents using multiple AI perspectives:\n", + "1. Use Claude for philosophical analysis\n", + "2. Use GPT-4 for technical insights\n", + "3. Use Gemini for alternative viewpoints\n", + "4. Synthesize all perspectives into a unified answer\n", + "\"\"\")\n", + "\n", + "print(f\"Enhanced Response: {enhanced_response}\")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Monitoring and Analytics\n", + "\n", + "Track usage patterns and optimize your AI provider selection:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# Analytics and monitoring agent\n", + "analytics_agent = FunctionAgent(\n", + " tools=roundtable_tools,\n", + " llm=OpenAI(model=\"gpt-3.5-turbo\"),\n", + " system_prompt=\"\"\"\n", + " You are an AI usage analytics specialist. Track:\n", + " \n", + " - Which providers are used most frequently\n", + " - Response time patterns\n", + " - Cost optimization opportunities\n", + " - Error rates and fallback usage\n", + " \n", + " Provide actionable insights for optimization.\n", + " \"\"\"\n", + ")\n", + "\n", + "# Get usage analytics\n", + "analytics_query = \"\"\"\n", + "Analyze our AI provider usage and provide:\n", + "1. Most cost-effective provider combinations\n", + "2. Recommendations for query routing optimization\n", + "3. Suggestions for reducing overall AI costs\n", + "4. Performance optimization recommendations\n", + "\"\"\"\n", + "\n", + "analytics_result = await analytics_agent.run(analytics_query)\n", + "print(f\"Usage Analytics: {analytics_result}\")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Best Practices\n", + "\n", + "### 1. Provider Selection Strategy\n", + "- **GPT-3.5-turbo**: Quick factual queries, simple tasks\n", + "- **GPT-4**: Complex reasoning, creative tasks\n", + "- **Claude Sonnet**: Code analysis, technical documentation\n", + "- **Claude Haiku**: Fast reasoning, cost-effective analysis\n", + "- **Gemini Pro**: Alternative perspectives, multimodal tasks\n", + "\n", + "### 2. Cost Optimization\n", + "- Route simple queries to cheaper models\n", + "- Use expensive models only for complex tasks\n", + "- Implement intelligent caching for repeated queries\n", + "- Monitor usage patterns and adjust routing\n", + "\n", + "### 3. Error Handling\n", + "- Always configure fallback providers\n", + "- Implement retry logic with exponential backoff\n", + "- Log provider failures for monitoring\n", + "- Test failover scenarios regularly\n", + "\n", + "### 4. Performance Monitoring\n", + "- Track response times by provider\n", + "- Monitor error rates and success rates\n", + "- Analyze cost per query by provider\n", + "- Set up alerts for service degradation" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Configuration Options\n", + "\n", + "Advanced Roundtable configuration for production use:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# Advanced production configuration\n", + "production_config = {\n", + " \"providers\": {\n", + " \"openai\": {\n", + " \"apiKey\": os.getenv(\"OPENAI_API_KEY\"),\n", + " \"models\": [\"gpt-4\", \"gpt-3.5-turbo\"],\n", + " \"rateLimits\": {\n", + " \"requestsPerMinute\": 60,\n", + " \"tokensPerMinute\": 90000\n", + " },\n", + " \"timeout\": 30000,\n", + " \"retryAttempts\": 3\n", + " },\n", + " \"anthropic\": {\n", + " \"apiKey\": os.getenv(\"ANTHROPIC_API_KEY\"),\n", + " \"models\": [\"claude-3-sonnet\", \"claude-3-haiku\"],\n", + " \"rateLimits\": {\n", + " \"requestsPerMinute\": 50,\n", + " \"tokensPerMinute\": 100000\n", + " },\n", + " \"timeout\": 45000,\n", + " \"retryAttempts\": 2\n", + " }\n", + " },\n", + " \"routing\": {\n", + " \"strategy\": \"cost-optimized\",\n", + " \"fallbackEnabled\": True,\n", + " \"loadBalancing\": True\n", + " },\n", + " \"monitoring\": {\n", + " \"enabled\": True,\n", + " \"logLevel\": \"info\",\n", + " \"metricsEndpoint\": \"/metrics\"\n", + " }\n", + "}\n", + "\n", + "# Save production configuration\n", + "with open(\"roundtable-production.json\", \"w\") as f:\n", + " json.dump(production_config, f, indent=2)\n", + "\n", + "print(\"Production configuration created!\")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Conclusion\n", + "\n", + "The Roundtable MCP Server integration with LlamaIndex provides:\n", + "\n", + "1. **Unified AI Access**: Single interface to multiple AI providers\n", + "2. **Enhanced Reliability**: Automatic failover and error handling\n", + "3. **Cost Optimization**: Intelligent routing to the most cost-effective models\n", + "4. **Improved Performance**: Task-specific model selection for optimal results\n", + "5. **Production Ready**: Robust configuration and monitoring capabilities\n", + "\n", + "This integration is ideal for LlamaIndex applications that require:\n", + "- High availability and reliability\n", + "- Cost-effective AI usage\n", + "- Complex multi-model workflows\n", + "- Production-grade AI systems\n", + "\n", + "### Next Steps\n", + "\n", + "1. **Explore the [Roundtable documentation](https://github.com/punkpeye/roundtable)** for advanced features\n", + "2. **Configure your production setup** with appropriate provider credentials\n", + "3. **Implement monitoring and analytics** to optimize your AI usage\n", + "4. **Experiment with different routing strategies** for your specific use case\n", + "\n", + "### Resources\n", + "\n", + "- [Roundtable GitHub Repository](https://github.com/punkpeye/roundtable)\n", + "- [LlamaIndex MCP Documentation](/python/framework/module_guides/mcp/llamaindex_mcp)\n", + "- [LlamaIndex Tools Documentation](/python/framework/module_guides/deploying/agents/tools)\n", + "- [MCP Protocol Specification](https://spec.modelcontextprotocol.io/)" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.9.6" + } + }, + "nbformat": 4, + "nbformat_minor": 4 +} \ No newline at end of file diff --git a/docs/src/content/docs/framework/community/integrations.md b/docs/src/content/docs/framework/community/integrations.md index 40e3810fa1..1ad792033a 100644 --- a/docs/src/content/docs/framework/community/integrations.md +++ b/docs/src/content/docs/framework/community/integrations.md @@ -19,6 +19,7 @@ The full set of data loaders are found on [LlamaHub](https://llamahub.ai/) The full set of agent tools are found on [LlamaHub](https://llamahub.ai/) - [MCP Toolbox](/python/examples/tools/mcp_toolbox) +- [Roundtable MCP Server](/python/examples/tools/roundtable_mcp_server) ## LLMs