insight-engine-version.1

🚀 Insight Engine

Insight Engine transforms a collection of PDF documents into an intelligent, conversational knowledge base.
This release provides all the necessary backend infrastructure to upload documents, process them with AI, and ask complex questions about their content.

---

✨ Key Features in This Release

📄 Document Upload API

A robust endpoint for uploading multiple PDF documents at once.

🚀 Asynchronous Processing Pipeline

Utilizes a BullMQ queue with dedicated workers to handle document processing (text extraction, chunking, and AI embedding) without blocking the main server.
This ensures the application remains responsive even when handling large files.

🧠 AI-Powered Ingestion

Leverages the Google Gemini API to generate high-quality vector embeddings for document chunks, capturing the semantic meaning of the text.

📚 Vector Storage

Uses ChromaDB as a dedicated vector store for persisting and efficiently searching through the document knowledge base.

💬 Retrieval-Augmented Generation (RAG) API

A core /chat endpoint that:

• Retrieves the most relevant context from your documents based on the user's question.
• Augments a prompt with this context.
• Generates a final, accurate, source-grounded answer using the Gemini generative model.

🗑️ Session-Based Memory

Each new file upload session automatically clears the previous knowledge base, ensuring that answers are always relevant only to the documents currently being analyzed.

🐳 Fully Containerized

The entire application (API Server, Worker, Redis, ChromaDB) is orchestrated with Docker Compose, allowing for a simple, one-command startup for both development and deployment.