BiteBuddy.AI — Ahmedabad Restaurant Intelligence System

A production-grade Retrieval-Augmented Generation (RAG) system for hyper-local restaurant discovery in Ahmedabad, India. Built with a multi-model LLM architecture, semantic vector search, and strict grounding guarantees.

---

Overview

BiteBuddy.AI is a domain-specific conversational AI designed to help users discover the best dining options in Ahmedabad. Rather than using a generic LLM that can hallucinate restaurant details, BiteBuddy.AI is grounded entirely in a curated, structured restaurant dataset — every recommendation is fetched, filtered, and narrated from real, verified data.

The system is engineered around a multi-stage RAG pipeline with production-level controls: per-session rate limiting, hardcoded LLM token budgets, off-topic guardrails, regional language routing, and progressive preference elicitation.

---

RAG Pipeline: How It Works

The system follows a 5-stage RAG pipeline for every chat turn:

Stage 1 — Preference Elicitation

BiteBuddy.AI does not immediately query its knowledge base. Instead, it engages in a structured multi-turn conversation to collect four key parameters before searching:

Parameter	Examples
group_type	Solo, Couple, Friends, Family, Colleagues
budget_per_person	Under ₹150, ₹150–300, ₹300–500, ₹500+
locality	Satellite, Vastrapur, Bodakdev, Navrangpura, Bopal...
cuisines	Gujarati, Punjabi, South Indian, Cafe, Fast Food...

Once all four parameters are collected (or the user explicitly requests recommendations), the retrieval stage is triggered. The system also supports force-recommend intent detection — if the user says "just recommend something", it searches immediately with whatever prefs are available.

Stage 2 — Dual-Layer Preference Extraction

Each user message is parsed by two parallel extraction systems:

1. Heuristic Extractor — A deterministic regex and keyword matcher covering all known preference values. Runs instantly with zero LLM cost. Handles chip clicks and simple responses.
2. LLM Extractor — A structured extraction call to the active language model, returning a JSON object of detected preference slots from free-form conversational input.

Results are merged with the LLM taking precedence over heuristics. This hybrid design ensures fast chip-click responses while still understanding complex natural language like "I'm taking my in-laws out for a proper Gujarati thali dinner somewhere near CG Road".

Stage 3 — Semantic Vector Retrieval

The retrieval engine uses cosine similarity search against a precomputed offline embedding matrix:

• Embedding Model: NVIDIA nv-embedqa-e5-v5 via the LiteLLM interface
• Index: NumPy matrix of 1024-dimensional embeddings built offline over the enriched restaurant corpus and cached in restaurant_embeddings.npy
• Query Construction: A natural language query string is built from user preferences (e.g., "Punjabi in Vastrapur good for couple")
• Retrieval: Top-10 semantic matches are fetched, then re-ranked by user_avg_rating and user_rating_count to surface the most reputable options

Stage 4 — Filter & Relaxation Engine

Semantic results are passed through a hard-constraint filter:

Budget Filter: cost_for_two is checked against the selected budget tier
Open Now Filter: timings_json is parsed against current day + IST local time

If fewer than 3 restaurants survive strict filtering, the system applies automatic relaxation in order:

1. Level 0 — All constraints applied
2. Level 1 — Budget constraint relaxed, other filters maintained
3. Level 2 — Full semantic fallback (pure embedding similarity, no hard filters)

The relaxation level is surfaced to the user in the UI as a "Strict Match" or "Relaxed Constraints" badge.

Stage 5 — Grounded Narration Generation

The final LLM call generates the assistant's conversational response. This stage enforces the strictest grounding rules:

• The model is given only the JSON data of the matched restaurants
• The system prompt contains 5 explicit non-fabrication rules, e.g.: "If the must_try_dish field is absent from a restaurant's data object, do NOT mention any dish for that restaurant."
• Temperature is set to 0.1 with top_p=0.9 to minimize hallucination
• Output is hard-capped at 600 tokens

---

Multi-Model Architecture

BiteBuddy.AI routes requests across multiple specialized models via the LiteLLM abstraction layer, allowing any OpenAI-compatible model to be swapped in without code changes:

Role	Model	Purpose
Primary Generation	meta/llama-3.3-70b-instruct	Preference extraction & narration for English input
Regional Language	sarvamai/sarvam-m	Handles Hindi and Gujarati input natively
Embeddings	nvidia/nv-embedqa-e5-v5	Offline restaurant corpus indexing + online query embedding
Safety Guard	nvidia/llama-3.1-nemoguard-8b-topic-control	Topic-relevance guardrail for off-topic detection

Language Detection & Routing

Every incoming message is scanned for Unicode character ranges at zero cost before any LLM call:

# Gujarati Unicode block: U+0A80–U+0AFF
# Devanagari (Hindi) Unicode block: U+0900–U+097F

If Gujarati or Hindi script is detected, the request is routed to Sarvam-M, a model purpose-built for Indic languages. English messages are routed to Llama 3.3 70B.

---

Production Hardening

The system is designed to be safe to expose publicly with the following controls:

Rate Limiting

In-memory sliding window limiter per session using Python collections.deque:

• 10 requests/minute — prevents flooding
• 60 requests/hour — prevents sustained abuse

Input Guard

• Messages exceeding 250 characters are rejected with HTTP 400 before any LLM call is made
• Both the backend and client enforce this limit independently (zero-latency client-side feedback)

Safety & Off-Topic Guardrail

A fast-path regex whitelist covers all known safe preference values (chip labels, locality names, cuisine types), bypassing the LLM safety call entirely for those inputs. For free-form text, a dedicated safety LLM call classifies the message as safe or unsafe. Off-topic requests receive a hard refusal:

"I can only help you with Ahmedabad food and restaurant recommendations."

Token Budgets

Every LLM call has a hardcoded max_tokens ceiling:

Call	Limit	Reason
Safety check	10	Only needs one word: safe / unsafe
Preference extraction	200	JSON preference object is small
Guide narration	600	3 restaurant summaries fit comfortably

Session History Pruning

Each session retains only the last 20 messages in memory. Older turns are dropped automatically to prevent unbounded memory growth in long-running server instances.

---

Data Pipeline

The restaurant corpus is built offline via clean_and_enrich.py:

1. Raw data from ahmedabad_cafes.csv is cleaned and deduplicated
2. Enrichment adds vibe classification, must-try dish tagging, timing parsing into timings_json, and good-for labeling
3. Output is written to ahmedabad_restaurants_enriched.csv
4. Embeddings are computed in batches of 32 via the NVIDIA embedding model and cached as restaurant_embeddings.npy (21MB NumPy binary)

At startup, app.py loads the CSV and embedding cache directly into memory for sub-millisecond retrieval latency.

---

Technology Stack

Layer	Technology
API Server	FastAPI + Uvicorn
LLM Interface	LiteLLM (OpenAI-compatible)
Vector Search	NumPy (cosine similarity)
Data Validation	Pydantic
Timezone Handling	pytz (Asia/Kolkata)
Frontend	HTML5, Vanilla CSS3, Native JavaScript
Video Streaming	hls.js (Mux HLS stream)
Markdown Rendering	marked.js

---

Setup & Installation

1. Install Dependencies

pip install fastapi uvicorn litellm numpy pytz pydantic python-dotenv

2. Configure Environment Variables

Copy the provided template to create your local .env file:

# Windows
copy .env.example .env

# Linux / macOS
cp .env.example .env

Then open .env and fill in your real values:

# Your LLM provider API key
ANTHROPIC_API_KEY=your-api-key-here

# The base URL of your OpenAI-compatible LLM provider
ANTHROPIC_BASE_URL=https://your-provider-base-url-here

# Optional: override the primary generation model (default: meta/llama-3.3-70b-instruct)
ANTHROPIC_DEFAULT_SONNET_MODEL=meta/llama-3.3-70b-instruct

# Optional: override the regional language model (default: sarvamai/sarvam-m)
BITEBUDDY_MODEL_SARVAM=sarvamai/sarvam-m

# Optional: override the embedding model (default: nvidia/nv-embedqa-e5-v5)
BITEBUDDY_MODEL_EMBED=nvidia/nv-embedqa-e5-v5

Note: The .env file is listed in .gitignore and will never be committed. Only .env.example (with placeholder values) is tracked in version control.

3. Build the Embedding Index

python clean_and_enrich.py

This enriches the restaurant CSV and builds the restaurant_embeddings.npy cache. Only needs to be run once, or when the dataset changes.

4. Start the Server

uvicorn app:app --host 127.0.0.1 --port 8000

Visit http://localhost:8000 in your browser.

---

Project Structure

restaurant_suggestions/
├── app.py                              # FastAPI server, RAG pipeline, multi-model routing
├── clean_and_enrich.py                 # Offline dataset enrichment & embedding builder
├── ahmedabad_restaurants_enriched.csv  # Enriched restaurant corpus (source of truth)
├── restaurant_embeddings.npy           # Precomputed 1024-dim embedding matrix (gitignored)
├── .env.example                        # Environment variable template (committed, safe)
├── .env                                # Your real secrets — gitignored, never committed
├── .gitignore
├── static/
│   ├── index.html                      # Frontend entrypoint
│   ├── styles.css                      # Liquid glassmorphism UI design system
│   └── app.js                          # Chat state, HLS video, chip logic
└── README.md

---

API Reference

POST /api/chat

Send a user message and receive a response with narration and structured recommendation data.

Request:

{
  "message": "I want Gujarati food in Vastrapur for family",
  "session_id": "session_abc123"
}

Response:

{
  "response": "Chalo, Bhai! Here are the top picks...",
  "recommendations": [ { "name": "...", "locality": "...", "avg_rating": 4.3, "cuisines": "..." } ],
  "preferences": { "group_type": "family", "locality": "Vastrapur", "cuisines": ["Gujarati"] },
  "questions_complete": true,
  "relaxed_status": null
}

POST /api/session/reset

Clears all session state and preference memory for a given session.

{ "session_id": "session_abc123" }