Local AI Analyst

A minimal, fully local agentic AI system that routes your questions to the right tool — document retrieval, SQL queries, finance calculations, or open-ended LLM answers — all running on your machine with no cloud dependencies.

Portfolio write-up: From CLI Model to Local Agentic AI (Medium article coming soon)

---

Features

• RAG over local documents — index .txt, .md, and .pdf files; retrieve with TF-IDF cosine similarity
• SQL tool — query a local SQLite database with natural language (Run SQL: ...)
• Finance tools — quarterly tax estimation and percentage calculations
• Ollama + Qwen — local LLM inference, no API keys required
• FastAPI — REST endpoint at /ask for integration
• Streamlit UI — browser-based chat interface
• Evaluation harness — benchmark your queries with eval/test_cases.json
• Clean package layout — pip install -e . ready, easy to extend

---

Repo structure

local-ai-analyst/
├── config/
│   └── settings.yaml          # Default config reference
├── data/
│   ├── raw/                   # Drop your .txt/.md/.pdf files here
│   └── processed/             # Generated index + SQLite DB (git-ignored)
├── docs/
│   └── architecture.md        # System design overview
├── notebooks/
│   └── exploration.ipynb      # Experiments
├── scripts/
│   ├── ingest_docs.py         # Build the document index
│   └── run_agent.py           # CLI query interface
├── src/
│   └── local_ai_analyst/
│       ├── __init__.py
│       ├── agent.py           # LocalAIAgent + router + tool helpers
│       ├── config.py          # Settings (Pydantic + dotenv)
│       ├── ingest.py          # Document loading, chunking, indexing
│       ├── llm.py             # QwenClient (Ollama HTTP wrapper)
│       ├── prompts.py         # Prompt templates
│       ├── retriever.py       # TfidfVectorStore + LocalRetriever
│       └── utils.py           # GenerationResult, finance tools, SQL helpers
├── eval/
│   ├── benchmark.py
│   ├── metrics.py
│   └── test_cases.json
├── tests/
│   └── test_imports.py        # Smoke tests
├── ui/
│   └── streamlit_app.py       # Web UI
├── .env.example
├── requirements.txt
└── setup.py

---

Prerequisites

1. Python 3.10+
2. Ollama installed and running
3. A local model pulled, e.g.:

ollama pull qwen2.5:7b

---

Setup

# 1. Clone and enter the repo
git clone https://github.com/your-username/local-ai-analyst.git
cd local-ai-analyst

# 2. Create a virtual environment
python3 -m venv .venv
source .venv/bin/activate      # Windows: .venv\Scripts\activate

# 3. Install dependencies
pip install -r requirements.txt
pip install -e .

# 4. Configure environment (optional — defaults work out of the box)
cp .env.example .env
# Edit .env if you use a different model or port

---

Usage

Ingest your documents

Drop .txt, .md, or .pdf files into data/raw/, then:

python scripts/ingest_docs.py

Ask questions via CLI

python scripts/run_agent.py --query "What can you do?"

Run the API

uvicorn app.main:app --reload
# API docs at http://127.0.0.1:8000/docs

Run the Streamlit UI

streamlit run ui/streamlit_app.py

---

Example queries

General

What are the main capabilities of this system?

RAG (document retrieval)

Summarize the project notes
What does the architecture document say about retrieval?

Finance tool

Estimate quarterly taxes for income 46000 and taxes_paid 5200
What is 30 percent of 7900?

SQL tool

Run SQL: SELECT * FROM demo_sales
Run SQL: SELECT COUNT(*) AS n FROM demo_sales

---

Running tests

pip install pytest
pytest tests/

---

Roadmap

• Replace TF-IDF with FAISS + sentence-transformers for semantic search
• Add structured tool calling with JSON schemas
• Add conversation memory / multi-turn context
• Add structured logging and request tracing
• Benchmark comparisons across local models (Llama 3, Mistral, Phi)
• Docker Compose one-command setup

---

License

MIT