Skip to content

Commit 826cd27

Browse files
Merge pull request #7 from davidvonthenen/add-hybrid-rag-projects
Add Graph RAG, BM25 RAG, and Hybrid RAG Projects
2 parents 3d7ba57 + 21f82df commit 826cd27

File tree

4 files changed

+264
-0
lines changed

4 files changed

+264
-0
lines changed

README.md

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -11,6 +11,9 @@ This is your gateway to explore and experiment with our Early Access Software. B
1111

1212
- [NetApp Neo: Connector for M365 Copilot](https://netapp.github.io/Innovation-Labs/projects/neo/core/overview.html)
1313
- [NetApp Console Plugins for Red Hat OpenShift](./netapp-openshift-consoles/README.md)
14+
- [NetApp Hybrid RAG (BM25-based) Deployment Guide](./docs/projects/hybrid-rag-bm25/README.md)
15+
- [NetApp Graph RAG Deployment Guide](./docs/projects/graph-rag/README.md)
16+
- [NetApp BM25-based/Document RAG Deployment Guide](./docs/projects/document-rag/README.md)
1417

1518
## Getting Started
1619

Lines changed: 87 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,87 @@
1+
# Document-Based RAG with NetApp
2+
3+
## 1. Introduction
4+
5+
This project captures a **Document-centric Retrieval-Augmented Generation (RAG)** architecture developed and validated by **NetApp**.
6+
7+
The focus is on building RAG systems that are **explainable, deterministic, and governance-ready** from day one. Instead of defaulting to vector-only retrieval, this architecture uses **BM25 lexical search**, enriched with **explicit entity extraction**, to make every retrieval decision observable and reproducible.
8+
9+
The complete reference implementation, including open source code and step-by-step guides, lives here:
10+
👉 **[https://github.com/NetApp/document-rag-guide](https://github.com/NetApp/document-rag-guide)**
11+
12+
This page serves as the **NetApp-specific overview and entry point**.
13+
14+
## 2. Why Document RAG
15+
16+
Most RAG stacks begin with embeddings and end with uncomfortable questions:
17+
18+
* Why did this document match?
19+
* Which terms mattered?
20+
* Can we reproduce this result tomorrow?
21+
* Can we prove compliance?
22+
23+
Document-based RAG flips that model.
24+
25+
![Document RAG with Reinforcement Learning](https://raw.githubusercontent.com/NetApp/document-rag-guide/refs/heads/main/images/enterprise_deployment.png)
26+
27+
Instead of treating retrieval as an opaque side effect of embeddings, it treats retrieval as a **first-class, auditable system**.
28+
29+
Key reasons this approach works:
30+
31+
* **Explainability by default**: BM25 matches explicit fields and terms. You can point to the exact reason a document was retrieved.
32+
33+
* **Deterministic behavior**: The same query over the same data produces the same result. No hidden ranking drift.
34+
35+
* **Reduced hallucinations**: LLM responses are grounded in retrieved documents, not semantic "near matches."
36+
37+
* **Clear governance boundaries**: Explicit document metadata, entity fields, and retention policies make audits practical instead of theoretical.
38+
39+
Vectors still exist, but only as **augmentation**, never as the sole authority.
40+
41+
## 3. How NetApp Enhances This Architecture
42+
43+
NetApp extends Document RAG with **enterprise-grade data management and storage capabilities** that turn a clean design into a deployable system.
44+
45+
Key NetApp-specific enhancements include:
46+
47+
* **Dual-tier memory model**
48+
49+
* **Long-Term (LT)**: authoritative, durable document store
50+
* **HOT (unstable)**: short-lived, user- or session-specific working set
51+
52+
* **Governance-driven isolation**
53+
54+
* HOT exists to enforce retention, policy asymmetry, and blast-radius control
55+
* LT remains stable, conservative, and audit-ready
56+
57+
* **High-performance locality**
58+
59+
* NetApp FlexCache keeps frequently accessed documents close to compute
60+
* Cache eviction is explicit and policy-driven, not accidental
61+
62+
* **Enterprise resilience**
63+
64+
* SnapMirror and MetroCluster support replication and disaster recovery
65+
* Snapshots enable point-in-time audits of "what the AI knew"
66+
67+
* **Safe experimentation**
68+
69+
* FlexClone enables instant copies of indices for testing new analyzers or embeddings without impacting production
70+
71+
The result is a Document RAG architecture that aligns with how enterprises already manage data: **explicit, observable, and controlled**.
72+
73+
## 4. Visit the GitHub Project for More Details
74+
75+
This page is intentionally high-level.
76+
77+
For full technical details, code, and deployment guidance, visit the main project:
78+
79+
👉 **[https://github.com/NetApp/document-rag-guide](https://github.com/NetApp/document-rag-guide)**
80+
81+
There you'll find:
82+
83+
* A fully open source, community-runnable implementation
84+
* An enterprise architecture with HOT/LT separation and promotion workflows
85+
* Clear patterns for explainable, compliant retrieval
86+
87+
If your goal is AI you can **explain, reproduce, and defend**, start there.

docs/projects/graph-rag/README.md

Lines changed: 86 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,86 @@
1+
# Graph RAG with NetApp
2+
3+
## 1. Introduction
4+
5+
This project documents a **Graph-based Retrieval-Augmented Generation (Graph RAG)** architecture developed and tested by **NetApp**.
6+
7+
The goal is simple: show how enterprises can build AI systems that are **explainable, governable, and production-ready**, not just clever demos. Instead of relying only on vector embeddings, this architecture uses **knowledge graphs with explicit relationships**, combined with a dual-memory model that separates authoritative knowledge from fast, conversational context.
8+
9+
The full reference implementation, including open source code and detailed walkthroughs, lives here:
10+
👉 **[https://github.com/NetApp/graph-rag-guide](https://github.com/NetApp/graph-rag-guide)**
11+
12+
This repository serves as the **NetApp-focused entry point** and architectural overview.
13+
14+
## 2. Why Graph RAG
15+
16+
![Graph RAG](https://raw.githubusercontent.com/NetApp/graph-rag-guide/refs/heads/main/images/rag-graph.png)
17+
18+
Traditional RAG pipelines usually start and end with vector search. That works for similarity matching, but it breaks down when teams need:
19+
20+
* Clear explanations for why an answer was returned
21+
* Auditable data lineage and provenance
22+
* Multi-hop reasoning across related facts
23+
* Strong governance and compliance controls
24+
25+
Graph RAG addresses these gaps by storing knowledge as **nodes and relationships** instead of opaque embeddings.
26+
27+
Key advantages include:
28+
29+
* **Reduced hallucinations**: Responses are grounded in explicit graph paths, not nearest-neighbor guesses.
30+
31+
* **Explainability by design**: Every answer can be traced through readable graph queries.
32+
33+
* **Better governance**: Provenance, confidence, and promotion logic live directly in the data model.
34+
35+
* **Multi-step reasoning**: Graphs naturally support traversals across documents, entities, and concepts.
36+
37+
This architecture treats retrieval as a **first-class system**, not a side effect of embeddings.
38+
39+
## 3. How NetApp Enhances This Architecture
40+
41+
NetApp extends the core Graph RAG design with **enterprise-grade data and storage capabilities** that make it practical at scale.
42+
43+
Key enhancements include:
44+
45+
* **Dual-memory architecture**
46+
47+
* Long-term memory for authoritative, durable knowledge
48+
* Short-term memory for fast, conversational context
49+
50+
* **High-performance caching**
51+
52+
* NetApp FlexCache enables microsecond-level access to hot graph data
53+
* Cached data expires automatically to prevent stale knowledge
54+
55+
* **Data mobility and resilience**
56+
57+
* SnapMirror provides replication and recovery across sites
58+
* Storage follows workloads, not the other way around
59+
60+
* **Promotion and reinforcement workflows**
61+
62+
* Frequently used or validated facts are promoted from cache to long-term memory
63+
* Confidence, provenance, and audit metadata are preserved end-to-end
64+
65+
* **Operational readiness**
66+
67+
* Designed to integrate with streaming pipelines and production infrastructure
68+
* Supports regulated environments where traceability is non-negotiable
69+
70+
The result is a Graph RAG architecture that aligns with real enterprise constraints: performance, governance, and scale.
71+
72+
## 4. Visit the GitHub Project for More Details
73+
74+
This page is only a summary.
75+
76+
For full architecture diagrams, implementation details, and runnable examples, visit the main project:
77+
78+
👉 **[https://github.com/NetApp/graph-rag-guide](https://github.com/NetApp/graph-rag-guide)**
79+
80+
There you'll find:
81+
82+
* A community, open source reference implementation
83+
* An enterprise-grade architecture with promotion and governance patterns
84+
* Clear upgrade paths from laptop demos to production deployments
85+
86+
If you're building AI systems that need to be trusted, explained, and operated long-term, start there.
Lines changed: 88 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,88 @@
1+
# Hybrid RAG with NetApp
2+
3+
**BM25 + Vector Retrieval with Governance Built In**
4+
5+
## 1. Introduction
6+
7+
This project highlights a **Hybrid Retrieval-Augmented Generation (Hybrid RAG)** architecture developed and validated by **NetApp**.
8+
9+
The design combines **BM25 lexical search** for deterministic, explainable grounding with **vector embeddings** for semantic coverage. The result is a retrieval system that balances **precision and recall** while remaining **observable, auditable, and enterprise-ready**.
10+
11+
This page provides a NetApp-focused overview of the architecture and its enterprise implications.
12+
The full open source reference implementation lives here:
13+
👉 **[https://github.com/davidvonthenen-com/hybrid-rag-bm25-with-ai-governance](https://github.com/davidvonthenen-com/hybrid-rag-bm25-with-ai-governance)**
14+
15+
## 2. Why Hybrid RAG
16+
17+
Pure vector RAG is good at "semantic vibes" but weak at answering hard questions like:
18+
19+
* Why did this document match?
20+
* Which terms actually mattered?
21+
* Can we reproduce this result next week?
22+
* Can we defend it to auditors?
23+
24+
Hybrid RAG addresses those gaps by **anchoring retrieval in BM25 first**, then using vectors as **supporting context**, not the source of truth.
25+
26+
![Hybrid RAG Using BM25](https://raw.githubusercontent.com/NetApp/hybrid-rag-bm25-with-ai-governance/refs/heads/main/images/enterprise_deployment.png)
27+
28+
Key reasons Hybrid RAG works:
29+
30+
* **Deterministic grounding**: BM25 provides explicit, traceable matches against known terms and entities.
31+
32+
* **Semantic coverage without drift**: Vector embeddings expand recall for paraphrases and long-tail phrasing without replacing lexical evidence.
33+
34+
* **Explainability by design**: Every result can be tied back to fields, terms, and highlights rather than opaque similarity scores.
35+
36+
* **Lower hallucination risk**: LLM responses are grounded in retrieved documents with clear provenance before any stylistic refinement.
37+
38+
* **Practical governance**: Retrieval behavior is inspectable and reproducible, which matters in regulated environments.
39+
40+
This approach delivers many of the governance benefits people look to Graph RAG for, **without the operational overhead of graph databases or ontology management**.
41+
42+
## 3. How NetApp Enhances This Architecture
43+
44+
NetApp extends Hybrid RAG with **enterprise-grade data management and storage primitives** that make the architecture operational at scale.
45+
46+
Key NetApp contributions include:
47+
48+
* **Dual-tier memory model**
49+
50+
* **Long-Term (LT)**: authoritative, durable knowledge store
51+
* **HOT (unstable)**: short-lived, user- or session-specific working set
52+
53+
* **Governance-first tiering**
54+
55+
* HOT exists for retention control, policy asymmetry, and isolation
56+
* LT remains conservative, stable, and audit-ready
57+
58+
* **High-performance locality**
59+
60+
* NetApp FlexCache keeps frequently accessed shards close to compute
61+
* Eviction is explicit and policy-driven, not accidental
62+
63+
* **Enterprise resilience**
64+
65+
* SnapMirror and MetroCluster support replication and disaster recovery
66+
* Snapshots enable point-in-time audits of "what the AI knew"
67+
68+
* **Safe experimentation**
69+
70+
* FlexClone allows instant, space-efficient copies of indices for testing new analyzers or embedding models without touching production
71+
72+
NetApp's role is not to change how Hybrid RAG works logically, but to **make it reliable, governable, and operable in real enterprise environments**.
73+
74+
## 4. Visit the GitHub Project for More Details
75+
76+
This page is intentionally concise.
77+
78+
For full technical details, code, and deployment guidance, visit the open source project:
79+
80+
👉 **[https://github.com/NetApp/hybrid-rag-bm25-with-ai-governance](https://github.com/NetApp/hybrid-rag-bm25-with-ai-governance)**
81+
82+
There you'll find:
83+
84+
* A complete Hybrid RAG reference implementation
85+
* Community and enterprise deployment paths
86+
* Detailed explanations of BM25 grounding, vector augmentation, and HOT/LT promotion workflows
87+
88+
If you're building RAG systems that need to be **accurate, explainable, and defensible**, that repository is the place to start.

0 commit comments

Comments
 (0)