AI assistant built with Streamlit, LangGraph, and multiple LLM backends (Gemini, Llama). The system provides document processing, OCR capabilities, and tool integration for enhanced question-answering and data analysis.
- Multi-Model Support: Switch between Gemini and Llama LLMs with easy configuration
- Document Processing: Upload and query PDF/HTML documents using vector store retrieval
- OCR Integration: Extract text from images using EasyOCR with multi-language support
- Tool Integration: Built-in tools for calculations, weather, Wikipedia search, and document retrieval
- Data Analysis: Query CSV files using pandas DataFrame agents
- Conversation Memory: Persistent conversation history with SQLite checkpointing
- Web Interface: Clean Streamlit UI with intuitive tab-based navigation
- Vector Store Management: Add, search, and reset document collections
- Python 3.12
- Google API Key (for Gemini model - option 1)
- Ollama (for Llama model - option 2)
-
Clone the repository:
git clone <repository-url> cd ai-assistant-with-tools
-
Install dependencies using Make:
make setup
This will install Python dependencies and set up Ollama with the Llama model.
-
Run the application:
make run-app
If Make is not available:
# Install Python dependencies
pip install -r requirements.txt
# Install Ollama (for Llama support)
curl -fsSL https://ollama.com/install.sh | sh
ollama serve &
ollama pull llama3.2:3b
# Run the application
python -m streamlit run ui/app.py- Choose LLM: Select either Llama (local) or Gemini (cloud-based) from the sidebar
- API Configuration: For Gemini, enter your Google API key in the form
- Upload Documents: Use the sidebar to upload PDF or HTML files
- Vector Store: Documents are automatically processed and added to the searchable vector store
- Reset Option: Clear all documents using the reset button when needed
- Ask Questions: Use the chat tab to ask questions about your documents
- Image Context: Optionally upload images for OCR text extraction
- Tool Integration: The agent automatically uses appropriate tools for calculations, weather, etc.
- Upload CSV: Use the CSV tab to upload data files
- Data Preview: View your data before querying
- Natural Language Queries: Ask questions about your data in plain English
ai-assistant-with-tools/
├── models/
│ ├── llm.py # LLM factory for model creation
│ └── agents.py # Agent management and creation
├── services/
│ ├── auth.py # Authentication manager (placeholder)
│ ├── ocr.py # OCR text extraction from images
│ └── vectorstore.py # Vector store management and document processing
├── tools/
│ └── tools.py # Tool definitions and manager
├── ui/
│ └── app.py # Main Streamlit application
├── config.py # Configuration settings
├── requirements.txt # Python dependencies
└── Makefile # Build and run automation
- Frontend: Streamlit for web interface
- AI Framework: LangGraph for agent orchestration
- LLM Backends: Google Gemini, Ollama/Llama
- Vector Store: FAISS for document retrieval
- OCR: EasyOCR for text extraction from images
- Data Analysis: pandas for CSV processing
- Embeddings: SentenceTransformers for text embeddings
- Memory: SQLite for conversation checkpointing
The system can be configured through config.py and environment variables:
# API Keys (set via environment variables)
GOOGLE_API_KEY = "your-google-api-key"
# Model configurations
GEMINI_MODEL = "gemini-2.5-flash-lite-preview-06-17"
LLAMA_MODEL = "llama3.2:3b"
EMBEDDING_MODEL = "sentence-transformers/all-MiniLM-L6-v2"
# Data paths
VECTORSTORE_PATH = "./data/vectorstore_index"
UPLOAD_DIR = "./data/uploaded_docs"
CHECKPOINT_DB = "./data/checkpoints/checkpoint.db"| Command | Description |
|---|---|
make all |
Setup and run complete application |
make setup |
Install all dependencies |
make install_deps |
Install Python dependencies only |
make install_ollama |
Install Ollama runtime |
make start_ollama |
Start Ollama service |
make pull_llama |
Download Llama model |
make run_llama |
Run Llama model in background |
make run-app |
Run Streamlit application |
make pull_qwen |
Download Qwen model (alternative) |
- Function: Mathematical calculations using numexpr
- Usage: Automatic for math questions
- Example: "Calculate 37593 times 67"
- Function: Current weather information
- Usage: Location-based weather queries
- Example: "What's the weather in Cairo?"
- Function: Search uploaded documents
- Usage: Automatic for document-related queries
- Example: "Find information about machine learning"
- Function: Wikipedia knowledge retrieval
- Usage: General knowledge questions
- Example: "Tell me about the Eiffel Tower"
- Purpose: Store conversation history and agent states
- Location:
data/checkpoints/checkpoint.db - Tables: Automatically managed by LangGraph SQLiteSaver
- English (
en) - Arabic (
ar) - Extensible to other languages
- Formats: PNG, JPG, JPEG
- Text Extraction: Multi-line text recognition
- Integration: Automatic context addition to queries
- Gemini API Errors: Verify your Google API key is valid and has Gemini access
- Ollama Connection Issues: Ensure Ollama service is running (
ollama serve) - Vector Store Errors: Use reset function to clear corrupted vector stores
- Memory Issues: For large documents, consider increasing system RAM
- Memory: 4GB+ RAM recommended (8GB for Llama)
- Storage: 10GB+ for models and data
- Network: Required for Gemini, optional for Llama (local)
- API keys are stored in environment variables
- Documents processed locally; no external data transmission for Llama
- Consider secure storage for sensitive documents
- Authentication system is placeholder; implement proper auth for production
This project is licensed under the MIT License - see the LICENSE file for details.
- Fork the repository
- Create a feature branch
- Make your changes
- Test thoroughly with both model backends
- Submit a pull request
For bug reports and feature requests, please use the issue tracker.
Note: This AI assistant is designed for educational and development purposes. For production deployment, consider implementing proper authentication, rate limiting, and security measures.