🦙 Offline RAG System (Ollama + ChromaDB)

A fully local Retrieval Augmented Generation (RAG) system that allows you to chat with your documents using offline LLMs. This project ensures privacy and zero data leakage by running entirely on your machine using Ollama.

🎥 Demo

Watch the Demo Video Here: Google Drive link

🚀 Features

• 100% Offline: No API keys or internet required for inference.
• Multi-Model Support: Switch between Gemma 2, Phi-3, and Mistral.
• Custom Knowledge Base: Ingests your own PDFs and text files.
• Diagnostics Tools: Built-in scripts to verify offline readiness.
• Low-Spec Optimized: Runs efficiently on standard hardware (Check setup.md for details).

🛠️ Prerequisites

1. Install Ollama: Download from ollama.com.
2. Python 3.10+ installed.

📥 Installation

1. Clone the repository

   git clone https://github.com/Vishwanath-06/Offline_RAG
   cd Offline_RAG

2. Install Python Dependencies

   pip install -r requirements.txt

---

⚙️ Automated Setup (Run Once Online)

Before going offline, run the automated setup script. This downloads the embedding models and pulls the required LLM (Phi-3 by default) to your machine.

python cache_models.py

Tip: If you prefer manual setup, you can run ollama pull phi3 (or mistral, gemma2:2b) and ensure sentence-transformers is installed.

---

📚 Building Your Knowledge Base

1. Add Your Documents Create a folder named study_materials and add your files.

   mkdir study_materials

Copy your .pdf, .txt, or .md files into this folder

2. Ingest Documents Process the documents into the local vector database.

   python ingest.py
   

🧪 Verification (Optional)

Want to make sure everything will work when you disconnect the internet? We have included diagnostic scripts:

1. Full System Test: Checks Ollama, Database, Embeddings, and Internet status.

   python offline_test.py

2. Cache Verification: Strictly checks if the embedding model is accessible without internet.

   python verify_cache.py

💬 Usage (Offline)

Once setup and ingestion are complete, you can disconnect your internet.

Start the Application:

python main.py

(Open interface.html in your browser if it doesn't open automatically).

---

⚡ Performance & Low-End PC Guide

Running on a laptop with 4GB-8GB RAM? Check setup.md for a detailed guide on:

• Selecting smaller models (TinyLlama/Gemma 2B).
• Optimizing chunk sizes and retrieval speed.
• Reducing memory usage.

---

📂 Project Structure

├── chroma_db/            # (Generated) Local vector database
├── study_materials/      # (User Required) Place your documents here
├── v2/                   # (Ignored) Future experimental versions
├── cache_models.py       # [SETUP] Downloads models for offline use
├── ingest.py             # [SETUP] Processes documents -> Database
├── offline_test.py       # [TEST] Verifies system health & offline readiness
├── verify_cache.py       # [TEST] Strict embedding cache check
├── main.py               # [RUN] Main API application
├── interface.html        # [RUN] User Interface
├── setup.md              # Performance optimization guide
└── requirements.txt      # Dependencies

🤝 Contributing

Made by Vishwanath.T

Feel free to open issues or submit pull requests if you have suggestions for improving the retrieval accuracy or UI.