feat: Make OpenAI optional and use local model by default - Update main.py to use LocalRAGAgent when no OpenAI key is present Update README to reflect that OpenAI is optional - Make Mistral-7B the default RAG model

Author: jasperan

agentic_rag/README.md

@@ -9,7 +9,7 @@ The system has the following features:
 - Persistent vector storage with ChromaDB
 - Smart context retrieval and response generation
 - FastAPI-based REST API
-- Support for both OpenAI-based agents or local, transformer-based agents
+- Support for both OpenAI-based agents or local, transformer-based agents (Mistral-7B by default)
 
 ## Setup
 
@@ -21,13 +21,15 @@ The system has the following features:
     pip install -r requirements.txt
     ```
 
-2. In case you're going to use the OpenAI-based agent, create a `.env` file with your OpenAI API key:
+2. (Optional) If you want to use the OpenAI-based agent instead of the default local model, create a `.env` file with your OpenAI API key:
 
    ```bash
    OPENAI_API_KEY=your-api-key-here
    ```
 
-3. If you're planning to use the local open-source model alternative for RAG, there's no additional configuration needed. The system will automatically use `Mistral-7B-Instruct-v0.2` for text generation.
+   If no API key is provided, the system will automatically use the local Mistral-7B model for text generation.
+
+3. The system will automatically download and use `Mistral-7B-Instruct-v0.2` for text generation when using the local model. No additional configuration is needed.
 
 ## 1. Getting Started
 

agentic_rag/main.py

@@ -8,6 +8,7 @@
 
 from pdf_processor import PDFProcessor
 from store import VectorStore
+from local_rag_agent import LocalRAGAgent
 from rag_agent import RAGAgent
 
 # Load environment variables
@@ -32,10 +33,15 @@
 # Initialize components
 pdf_processor = PDFProcessor()
 vector_store = VectorStore()
-rag_agent = RAGAgent(
-    vector_store=vector_store,
-    openai_api_key=os.getenv("OPENAI_API_KEY")
-)
+
+# Initialize RAG agent - use OpenAI if API key is available, otherwise use local model
+openai_api_key = os.getenv("OPENAI_API_KEY")
+if openai_api_key:
+    print("\nUsing OpenAI GPT-4 for RAG...")
+    rag_agent = RAGAgent(vector_store=vector_store, openai_api_key=openai_api_key)
+else:
+    print("\nOpenAI API key not found. Using local Mistral model for RAG...")
+    rag_agent = LocalRAGAgent(vector_store=vector_store)
 
 class QueryRequest(BaseModel):
     query: str