citations, contextual retrieval updates, ui, adaptive querying, history

[manasmhasakar]

This PR adds six major enhancements to TokenSmith: detailed citations, contextual chunk retrieval, adaptive query planning, conversation memory, performance optimizations, and a Streamlit web interface. These improvements significantly enhance answer quality, user experience, and system performance.

Core Enhancements

• Citation System: Page numbers, chapters, and sections displayed with each answer
• Contextual Retrieval: New ContextualRetriever with neighbor expansion and cross-reference boosting
• Adaptive Query Planning: HeuristicQueryPlanner adjusts retrieval based on question type
• Conversation Memory: Maintains 5-turn history for contextual follow-ups
• Performance: Parallel embedding + incremental indexing (reduces rebuild time from ~3.5 hours)
• Web UI: Streamlit interface with multiple chat sessions, history, and citation display