NyRAG

NyRAG (pronounced as knee-RAG) is a simple tool for building RAG applications by crawling websites or processing documents, then deploying to Vespa for hybrid search with an integrated chat UI.

How It Works

When a user asks a question, NyRAG performs a multi-stage retrieval process:

1. Query Enhancement: An LLM generates additional search queries based on the user's question and initial context to improve retrieval coverage
2. Embedding Generation: Each query is converted to embeddings using the configured SentenceTransformer model
3. Vespa Search: Queries are executed against Vespa using nearestNeighbor search with the best_chunk_score ranking profile to find the most relevant document chunks
4. Chunk Fusion: Results from all queries are aggregated, deduplicated, and ranked by score to select the top-k most relevant chunks
5. Answer Generation: The retrieved context is sent to an LLM which generates a grounded answer based only on the provided chunks

This multi-query RAG approach with chunk-level retrieval ensures answers are comprehensive and grounded in your actual content, whether from crawled websites or processed documents.

LLM Support

NyRAG works with any OpenAI-compatible API, including:

• OpenRouter (100+ models from various providers)
• Ollama (local models: Llama, Mistral, Qwen, etc.)
• LM Studio (local GUI for running models)
• vLLM (high-performance local or remote inference)
• LocalAI (local OpenAI drop-in replacement)
• OpenAI (GPT-4, GPT-3.5, etc.)
• Any other service implementing the OpenAI API format

Installation

pip install nyrag

We recommend uv:

uv init --python 3.10
uv venv
uv sync
source .venv/bin/activate
uv pip install -U nyrag

For development:

git clone https://github.com/abhishekkrthakur/nyrag.git
cd nyrag
pip install -e .

Usage

NyRAG is designed to be used primarily through its web UI, which manages the entire lifecycle from data processing to chat.

1. Start the UI

Local Mode (requires Docker):

nyrag ui

Cloud Mode (requires Vespa Cloud account):

nyrag ui --cloud

Open http://localhost:8000 in your browser.

2. Configure & Process

In the UI, you can create a new configuration for your data source.

Example Web Crawl Config:

name: mywebsite
mode: web
start_loc: https://example.com/
crawl_params:
  respect_robots_txt: true
rag_params:
  embedding_model: sentence-transformers/all-MiniLM-L6-v2

Example Docs Processing Config:

name: mydocs
mode: docs
start_loc: /path/to/documents/
doc_params:
  recursive: true
rag_params:
  embedding_model: sentence-transformers/all-mpnet-base-v2

3. Chat

Once processing is complete, you can start chatting with your data immediately in the UI. Make sure your configuration includes your LLM API key and model selection.

Configuration Reference

Cloud Deploy Parameters

Parameter	Type	Default	Description
cloud_tenant	str	None	Vespa Cloud tenant (required for cloud mode if no env/CLI target)

Connection Parameters

Parameter	Type	Default	Description
vespa_url	str	None	Vespa endpoint URL (auto-filled into conf.yml after deploy)
vespa_port	int	None	Vespa endpoint port (auto-filled into conf.yml after deploy)

Web Mode Parameters (crawl_params)

Parameter	Type	Default	Description
respect_robots_txt	bool	true	Respect robots.txt rules
aggressive_crawl	bool	false	Faster crawling with more concurrent requests
follow_subdomains	bool	true	Follow links to subdomains
strict_mode	bool	false	Only crawl URLs matching start pattern
user_agent_type	str	chrome	chrome, firefox, safari, mobile, bot
custom_user_agent	str	None	Custom user agent string
allowed_domains	list	None	Explicitly allowed domains

Docs Mode Parameters (doc_params)

Parameter	Type	Default	Description
recursive	bool	true	Process subdirectories
include_hidden	bool	false	Include hidden files
follow_symlinks	bool	false	Follow symbolic links
max_file_size_mb	float	None	Max file size in MB
file_extensions	list	None	Only process these extensions

RAG Parameters (rag_params)

Parameter	Type	Default	Description
embedding_model	str	sentence-transformers/all-MiniLM-L6-v2	Embedding model
embedding_dim	int	384	Embedding dimension
chunk_size	int	1024	Chunk size for text splitting
chunk_overlap	int	50	Overlap between chunks
distance_metric	str	angular	Distance metric
max_tokens	int	8192	Max tokens per document
llm_base_url	str	None	LLM API base URL (OpenAI-compatible)
llm_model	str	None	LLM model name
llm_api_key	str	None	LLM API key

---

LLM Provider Support

NyRAG works with any OpenAI-compatible API. Just configure the rag_params in your UI settings.

Provider	Base URL	Model Example	API Key
Ollama	http://localhost:11434/v1	llama3.2	dummy
LM Studio	http://localhost:1234/v1	local-model	dummy
vLLM	http://localhost:8000/v1	meta-llama/Llama-3.2-3B-Instruct	dummy
OpenRouter	https://openrouter.ai/api/v1	openai/gpt-5.2	your-key
OpenAI	None (default)	openai/gpt-4o	your-key

Example Config:

llm_config:
  llm_base_url: https://openrouter.ai/api/v1
  llm_model: llama3.2
  llm_api_key: dummy

---