Merge pull request #7 from davidvonthenen/add-hybrid-rag-projects

Add Graph RAG, BM25 RAG, and Hybrid RAG Projects

Author: davidvonthenen

README.md

@@ -11,6 +11,9 @@ This is your gateway to explore and experiment with our Early Access Software. B
 
 - [NetApp Neo: Connector for M365 Copilot](https://netapp.github.io/Innovation-Labs/projects/neo/core/overview.html)
 - [NetApp Console Plugins for Red Hat OpenShift](./netapp-openshift-consoles/README.md)
+- [NetApp Hybrid RAG (BM25-based) Deployment Guide](./docs/projects/hybrid-rag-bm25/README.md)
+- [NetApp Graph RAG Deployment Guide](./docs/projects/graph-rag/README.md)
+- [NetApp BM25-based/Document RAG Deployment Guide](./docs/projects/document-rag/README.md)
 
 ## Getting Started
 

docs/projects/document-rag/README.md

@@ -0,0 +1,87 @@
+# Document-Based RAG with NetApp
+
+## 1. Introduction
+
+This project captures a **Document-centric Retrieval-Augmented Generation (RAG)** architecture developed and validated by **NetApp**.
+
+The focus is on building RAG systems that are **explainable, deterministic, and governance-ready** from day one. Instead of defaulting to vector-only retrieval, this architecture uses **BM25 lexical search**, enriched with **explicit entity extraction**, to make every retrieval decision observable and reproducible.
+
+The complete reference implementation, including open source code and step-by-step guides, lives here:
+👉 **[https://github.com/NetApp/document-rag-guide](https://github.com/NetApp/document-rag-guide)**
+
+This page serves as the **NetApp-specific overview and entry point**.
+
+## 2. Why Document RAG
+
+Most RAG stacks begin with embeddings and end with uncomfortable questions:
+
+* Why did this document match?
+* Which terms mattered?
+* Can we reproduce this result tomorrow?
+* Can we prove compliance?
+
+Document-based RAG flips that model.
+
+![Document RAG with Reinforcement Learning](https://raw.githubusercontent.com/NetApp/document-rag-guide/refs/heads/main/images/enterprise_deployment.png)
+
+Instead of treating retrieval as an opaque side effect of embeddings, it treats retrieval as a **first-class, auditable system**.
+
+Key reasons this approach works:
+
+* **Explainability by default**: BM25 matches explicit fields and terms. You can point to the exact reason a document was retrieved.
+
+* **Deterministic behavior**: The same query over the same data produces the same result. No hidden ranking drift.
+
+* **Reduced hallucinations**: LLM responses are grounded in retrieved documents, not semantic "near matches."
+
+* **Clear governance boundaries**: Explicit document metadata, entity fields, and retention policies make audits practical instead of theoretical.
+
+Vectors still exist, but only as **augmentation**, never as the sole authority.
+
+## 3. How NetApp Enhances This Architecture
+
+NetApp extends Document RAG with **enterprise-grade data management and storage capabilities** that turn a clean design into a deployable system.
+
+Key NetApp-specific enhancements include:
+
+* **Dual-tier memory model**
+
+  * **Long-Term (LT)**: authoritative, durable document store
+  * **HOT (unstable)**: short-lived, user- or session-specific working set
+
+* **Governance-driven isolation**
+
+  * HOT exists to enforce retention, policy asymmetry, and blast-radius control
+  * LT remains stable, conservative, and audit-ready
+
+* **High-performance locality**
+
+  * NetApp FlexCache keeps frequently accessed documents close to compute
+  * Cache eviction is explicit and policy-driven, not accidental
+
+* **Enterprise resilience**
+
+  * SnapMirror and MetroCluster support replication and disaster recovery
+  * Snapshots enable point-in-time audits of "what the AI knew"
+
+* **Safe experimentation**
+
+  * FlexClone enables instant copies of indices for testing new analyzers or embeddings without impacting production
+
+The result is a Document RAG architecture that aligns with how enterprises already manage data: **explicit, observable, and controlled**.
+
+## 4. Visit the GitHub Project for More Details
+
+This page is intentionally high-level.
+
+For full technical details, code, and deployment guidance, visit the main project:
+
+👉 **[https://github.com/NetApp/document-rag-guide](https://github.com/NetApp/document-rag-guide)**
+
+There you'll find:
+
+* A fully open source, community-runnable implementation
+* An enterprise architecture with HOT/LT separation and promotion workflows
+* Clear patterns for explainable, compliant retrieval
+
+If your goal is AI you can **explain, reproduce, and defend**, start there.

docs/projects/graph-rag/README.md

@@ -0,0 +1,86 @@
+# Graph RAG with NetApp
+
+## 1. Introduction
+
+This project documents a **Graph-based Retrieval-Augmented Generation (Graph RAG)** architecture developed and tested by **NetApp**.
+
+The goal is simple: show how enterprises can build AI systems that are **explainable, governable, and production-ready**, not just clever demos. Instead of relying only on vector embeddings, this architecture uses **knowledge graphs with explicit relationships**, combined with a dual-memory model that separates authoritative knowledge from fast, conversational context.
+
+The full reference implementation, including open source code and detailed walkthroughs, lives here:
+👉 **[https://github.com/NetApp/graph-rag-guide](https://github.com/NetApp/graph-rag-guide)**
+
+This repository serves as the **NetApp-focused entry point** and architectural overview.
+
+## 2. Why Graph RAG
+
+![Graph RAG](https://raw.githubusercontent.com/NetApp/graph-rag-guide/refs/heads/main/images/rag-graph.png)
+
+Traditional RAG pipelines usually start and end with vector search. That works for similarity matching, but it breaks down when teams need:
+
+* Clear explanations for why an answer was returned
+* Auditable data lineage and provenance
+* Multi-hop reasoning across related facts
+* Strong governance and compliance controls
+
+Graph RAG addresses these gaps by storing knowledge as **nodes and relationships** instead of opaque embeddings.
+
+Key advantages include:
+
+* **Reduced hallucinations**: Responses are grounded in explicit graph paths, not nearest-neighbor guesses.
+
+* **Explainability by design**: Every answer can be traced through readable graph queries.
+
+* **Better governance**: Provenance, confidence, and promotion logic live directly in the data model.
+
+* **Multi-step reasoning**: Graphs naturally support traversals across documents, entities, and concepts.
+
+This architecture treats retrieval as a **first-class system**, not a side effect of embeddings.
+
+## 3. How NetApp Enhances This Architecture
+
+NetApp extends the core Graph RAG design with **enterprise-grade data and storage capabilities** that make it practical at scale.
+
+Key enhancements include:
+
+* **Dual-memory architecture**
+
+  * Long-term memory for authoritative, durable knowledge
+  * Short-term memory for fast, conversational context
+
+* **High-performance caching**
+
+  * NetApp FlexCache enables microsecond-level access to hot graph data
+  * Cached data expires automatically to prevent stale knowledge
+
+* **Data mobility and resilience**
+
+  * SnapMirror provides replication and recovery across sites
+  * Storage follows workloads, not the other way around
+
+* **Promotion and reinforcement workflows**
+
+  * Frequently used or validated facts are promoted from cache to long-term memory
+  * Confidence, provenance, and audit metadata are preserved end-to-end
+
+* **Operational readiness**
+
+  * Designed to integrate with streaming pipelines and production infrastructure
+  * Supports regulated environments where traceability is non-negotiable
+
+The result is a Graph RAG architecture that aligns with real enterprise constraints: performance, governance, and scale.
+
+## 4. Visit the GitHub Project for More Details
+
+This page is only a summary.
+
+For full architecture diagrams, implementation details, and runnable examples, visit the main project:
+
+👉 **[https://github.com/NetApp/graph-rag-guide](https://github.com/NetApp/graph-rag-guide)**
+
+There you'll find:
+
+* A community, open source reference implementation
+* An enterprise-grade architecture with promotion and governance patterns
+* Clear upgrade paths from laptop demos to production deployments
+
+If you're building AI systems that need to be trusted, explained, and operated long-term, start there.

docs/projects/hybrid-rag-bm25/README.md

@@ -0,0 +1,88 @@
+# Hybrid RAG with NetApp
+
+**BM25 + Vector Retrieval with Governance Built In**
+
+## 1. Introduction
+
+This project highlights a **Hybrid Retrieval-Augmented Generation (Hybrid RAG)** architecture developed and validated by **NetApp**.
+
+The design combines **BM25 lexical search** for deterministic, explainable grounding with **vector embeddings** for semantic coverage. The result is a retrieval system that balances **precision and recall** while remaining **observable, auditable, and enterprise-ready**.
+
+This page provides a NetApp-focused overview of the architecture and its enterprise implications.
+The full open source reference implementation lives here:
+👉 **[https://github.com/davidvonthenen-com/hybrid-rag-bm25-with-ai-governance](https://github.com/davidvonthenen-com/hybrid-rag-bm25-with-ai-governance)**
+
+## 2. Why Hybrid RAG
+
+Pure vector RAG is good at "semantic vibes" but weak at answering hard questions like:
+
+* Why did this document match?
+* Which terms actually mattered?
+* Can we reproduce this result next week?
+* Can we defend it to auditors?
+
+Hybrid RAG addresses those gaps by **anchoring retrieval in BM25 first**, then using vectors as **supporting context**, not the source of truth.
+
+![Hybrid RAG Using BM25](https://raw.githubusercontent.com/NetApp/hybrid-rag-bm25-with-ai-governance/refs/heads/main/images/enterprise_deployment.png)
+
+Key reasons Hybrid RAG works:
+
+* **Deterministic grounding**: BM25 provides explicit, traceable matches against known terms and entities.
+
+* **Semantic coverage without drift**: Vector embeddings expand recall for paraphrases and long-tail phrasing without replacing lexical evidence.
+
+* **Explainability by design**: Every result can be tied back to fields, terms, and highlights rather than opaque similarity scores.
+
+* **Lower hallucination risk**: LLM responses are grounded in retrieved documents with clear provenance before any stylistic refinement.
+
+* **Practical governance**: Retrieval behavior is inspectable and reproducible, which matters in regulated environments.
+
+This approach delivers many of the governance benefits people look to Graph RAG for, **without the operational overhead of graph databases or ontology management**.
+
+## 3. How NetApp Enhances This Architecture
+
+NetApp extends Hybrid RAG with **enterprise-grade data management and storage primitives** that make the architecture operational at scale.
+
+Key NetApp contributions include:
+
+* **Dual-tier memory model**
+
+  * **Long-Term (LT)**: authoritative, durable knowledge store
+  * **HOT (unstable)**: short-lived, user- or session-specific working set
+
+* **Governance-first tiering**
+
+  * HOT exists for retention control, policy asymmetry, and isolation
+  * LT remains conservative, stable, and audit-ready
+
+* **High-performance locality**
+
+  * NetApp FlexCache keeps frequently accessed shards close to compute
+  * Eviction is explicit and policy-driven, not accidental
+
+* **Enterprise resilience**
+
+  * SnapMirror and MetroCluster support replication and disaster recovery
+  * Snapshots enable point-in-time audits of "what the AI knew"
+
+* **Safe experimentation**
+
+  * FlexClone allows instant, space-efficient copies of indices for testing new analyzers or embedding models without touching production
+
+NetApp's role is not to change how Hybrid RAG works logically, but to **make it reliable, governable, and operable in real enterprise environments**.
+
+## 4. Visit the GitHub Project for More Details
+
+This page is intentionally concise.
+
+For full technical details, code, and deployment guidance, visit the open source project:
+
+👉 **[https://github.com/NetApp/hybrid-rag-bm25-with-ai-governance](https://github.com/NetApp/hybrid-rag-bm25-with-ai-governance)**
+
+There you'll find:
+
+* A complete Hybrid RAG reference implementation
+* Community and enterprise deployment paths
+* Detailed explanations of BM25 grounding, vector augmentation, and HOT/LT promotion workflows
+
+If you're building RAG systems that need to be **accurate, explainable, and defensible**, that repository is the place to start.