Self-Correcting RAG Pipeline

A multi-stage Retrieval-Augmented Generation (RAG) system that reduces AI hallucinations through automated fact-checking and verification.

🎯 Problem It Solves

Traditional RAG systems suffer from hallucination - generating answers not supported by source documents. This project implements a 4-stage pipeline that:

1. Retrieves relevant documents
2. Filters for relevance
3. Generates answers
4. Fact-checks against sources before delivery

🚀 Features

• ✅ 100% Free - Uses Ollama (runs locally, no API costs)
• ✅ No API Keys Required - Completely offline after setup
• ✅ Multi-Stage Verification - 4 distinct agents working together
• ✅ Hallucination Detection - Automatically rejects unsupported answers
• ✅ Privacy-First - All data stays on your machine

🛠️ Tech Stack

• LLM: Ollama (Llama 3.2)
• Framework: LangChain
• Vector Store: FAISS
• Embeddings: HuggingFace (sentence-transformers)
• Language: Python 3.9+

📋 Prerequisites

• macOS (or Linux)
• Python 3.9+
• Ollama

🔧 Installation

1. Install Ollama

Download from: https://ollama.com/download

Or install via Homebrew:

brew install ollama

2. Pull the LLM Model

ollama pull llama3.2

3. Clone This Repository

git clone https://github.com/mchavva413/self-correcting-rag.git
cd self-correcting-rag

4. Install Python Dependencies

pip3 install langchain langchain-community faiss-cpu sentence-transformers

🎮 Usage

Run the Pipeline

python3 main.py

Ask Questions

The system will prompt you to enter questions. Try:

• "Who created Python?"
• "When was JavaScript created?"
• "What is React?"

Type quit to exit.

📁 Project Structure

self-correcting-rag/
├── agents/
│   ├── __init__.py
│   ├── relevance_agent.py      # Stage 1: Filters relevant documents
│   ├── generator_agent.py      # Stage 2: Generates answers
│   └── factcheck_agent.py      # Stage 3: Verifies factual consistency
├── utils/
│   ├── __init__.py
│   └── vector_store.py         # Document embedding & retrieval
├── data/
│   └── documents.txt           # Knowledge base
├── main.py                     # Main pipeline orchestrator
└── README.md

🔄 How It Works

Stage 1: Document Retrieval

Uses FAISS vector similarity search to find potentially relevant documents.

Stage 2: Relevance Filtering

An LLM agent evaluates each retrieved document for relevance to the query.

Stage 3: Answer Generation

Generates an answer using ONLY the relevant documents (no external knowledge).

Stage 4: Fact-Checking

Verifies the answer against source documents and assigns a consistency score (0-100).

• Score ≥ 70: Answer APPROVED ✅
• Score < 70: Answer REJECTED ❌

📊 Example Output

🔎 Enter your question: Who created Python?

STAGE 1️⃣: DOCUMENT RETRIEVAL
✓ Retrieved 4 documents

STAGE 2️⃣: RELEVANCE FILTERING
✓ Found 1/4 relevant documents

STAGE 3️⃣: ANSWER GENERATION
💡 Generated Answer:
   Python was created by Guido van Rossum and first released in 1991.

STAGE 4️⃣: FACT-CHECKING
✓ Fact-check complete: PASSED ✅

🏁 FINAL STATUS: APPROVED ✅

🎓 Learning Outcomes

This project demonstrates:

• Multi-agent AI systems
• RAG architecture
• Vector embeddings and similarity search
• Prompt engineering
• Hallucination detection and mitigation

🤝 Contributing

Pull requests are welcome! For major changes, please open an issue first.

📝 License

MIT License

🙏 Acknowledgments

• Anthropic for Claude (used for development guidance)
• Meta for Llama models
• LangChain community