Simple RAG: upload document and chat with it

[zjy4fun]

Goal

Allow users to upload a document (PDF/TXT/MD) and ask questions about it with retrieval-augmented generation.

Tasks

• POST /documents/upload — accept file, extract text (pdf-parse for PDF, raw for txt/md)
• Chunk text into ~500 token segments with overlap
• Generate embeddings via OpenAI embeddings API (or local model)
• Store chunks + vectors in SQLite with sqlite-vec extension (or simple cosine similarity)
• On chat: embed the query → find top-K similar chunks → inject into system prompt as context
• Show "sources" in the response: which chunks were used, with page/line references
• Frontend: file upload area, source citation display

Why

RAG is THE killer feature of modern LLM applications. Understanding the full pipeline (chunk → embed → retrieve → augment → generate) is essential knowledge. This also bridges nicely with your prompt-hub project.

Learning Points

• Text chunking strategies (fixed size, semantic, recursive)
• Vector similarity search (cosine similarity, approximate nearest neighbors)
• Embedding models: dimensions, cost, quality trade-offs
• Prompt engineering for RAG: how to inject context without confusing the model
• Hallucination mitigation: "answer only based on provided context"