Skip to content

Commit 46d34fb

Browse files
anniesurlaanniesurla
authored
Add Readme (#367)
1 parent d967961 commit 46d34fb

File tree

1 file changed

+217
-0
lines changed

1 file changed

+217
-0
lines changed

‎README.md‎

Lines changed: 217 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,217 @@
1+
<!--
2+
SPDX-FileCopyrightText: Copyright (c) 2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
3+
SPDX-License-Identifier: Apache-2.0
4+
-->
5+
6+
![](docs/images/[email protected])
7+
8+
# NVIDIA Generative AI Examples
9+
10+
This repository is a starting point for developers looking to integrate with the NVIDIA software ecosystem to speed up their generative AI systems. Whether you are building RAG pipelines, agentic workflows, or fine-tuning models, this repository will help you integrate NVIDIA, seamlessly and natively, with your development stack.
11+
12+
## Table of Contents
13+
<!-- TOC -->
14+
15+
* [What's New?](#whats-new)
16+
* [Data Flywheel](#data-flywheel)
17+
* [Safer Agentic AI](#safer-agentic-ai)
18+
* [Knowledge Graph RAG](#knowledge-graph-rag)
19+
* [Agentic Workflows with Llama 3.1](#agentic-workflows-with-llama-31)
20+
* [RAG with Local NIM Deployment and LangChain](#rag-with-local-nim-deployment-and-langchain)
21+
* [Vision NIM Workflows](#vision-nim-workflows)
22+
* [Try it Now!](#try-it-now)
23+
* [Data Flywheel](#data-flywheel)
24+
* [Tool-Calling Notebooks](#tool-calling-notebooks)
25+
* [RAG](#rag)
26+
* [RAG Notebooks](#rag-notebooks)
27+
* [RAG Examples](#rag-examples)
28+
* [RAG Tools](#rag-tools)
29+
* [RAG Projects](#rag-projects)
30+
* [Documentation](#documentation)
31+
* [Getting Started](#getting-started)
32+
* [How To's](#how-tos)
33+
* [Reference](#reference)
34+
* [Community](#community)
35+
36+
<!-- /TOC -->
37+
38+
## What's New?
39+
40+
### Data Flywheel
41+
42+
These tutorials demonstrate Data Flywheel workflows that use NVIDIA NeMo Microservices. They include components such as NVIDIA NeMo Datastore, NeMo Entity Store, NeMo Customizer, NeMo Evaluator, NeMo Guardrails microservices, and NVIDIA NIMs.
43+
44+
- [Tool Calling Fine-tuning, Inference, Evaluation, and Guardrailing with NVIDIA NeMo Microservices and NIMs](./nemo/data-flywheel/tool-calling)
45+
- [Embedding Fine-tuning, Inference, and Evaluation with NVIDIA NeMo Microservices and NIMs](./nemo/data-flywheel/embedding-finetuning/)
46+
47+
### Safer Agentic AI
48+
49+
The following tutorials illustrate how to audit your large language models with NeMo Auditor to identify vulnerabilities to unsafe prompts, and how to run inference with multiple rails in parallel to reduce latency and improve throughput.
50+
51+
- [Audit your LLMs](./nemo/NeMo-Auditor/Getting_Started_With_NeMo_Auditor.ipynb)
52+
- [Inference with Parallel Rails](./nemo/NeMo-Guardrails/Parallel_Rails_Tutorial.ipynb)
53+
54+
### Knowledge Graph RAG
55+
56+
This example implements a GPU-accelerated pipeline for creating and querying knowledge graphs using RAG by leveraging NIM microservices and the RAPIDS ecosystem to process large-scale datasets efficiently.
57+
58+
- [Knowledge Graphs for RAG with NVIDIA AI Foundation Models and Endpoints](community/knowledge_graph_rag)
59+
60+
### Agentic Workflows with Llama 3.1
61+
62+
- Build an Agentic RAG Pipeline with Llama 3.1 and NVIDIA NeMo Retriever NIM microservices [[Blog](https://developer.nvidia.com/blog/build-an-agentic-rag-pipeline-with-llama-3-1-and-nvidia-nemo-retriever-nims/), [Notebook](RAG/notebooks/langchain/agentic_rag_with_nemo_retriever_nim.ipynb)]
63+
- [NVIDIA Morpheus, NIM microservices, and RAG pipelines integrated to create LLM-based agent pipelines](https://github.com/NVIDIA/GenerativeAIExamples/blob/v0.7.0/experimental/event-driven-rag-cve-analysis)
64+
65+
66+
### RAG with Local NIM Deployment and LangChain
67+
68+
- Tips for Building a RAG Pipeline with NVIDIA AI LangChain AI Endpoints by Amit Bleiweiss. [[Blog](https://developer.nvidia.com/blog/tips-for-building-a-rag-pipeline-with-nvidia-ai-langchain-ai-endpoints/), [Notebook](https://github.com/NVIDIA/GenerativeAIExamples/blob/v0.7.0/notebooks/08_RAG_Langchain_with_Local_NIM.ipynb)]
69+
70+
For more information, refer to the [Generative AI Example releases](https://github.com/NVIDIA/GenerativeAIExamples/releases/).
71+
72+
### Vision NIM Workflows
73+
A collection of Jupyter notebooks, sample code and reference applications built with Vision NIMs.
74+
75+
To pull the vision NIM workflows, clone this repository recursively:
76+
```
77+
git clone https://github.com/nvidia/GenerativeAIExamples --recurse-submodules
78+
```
79+
80+
The workflows will then be located at [GenerativeAIExamples/vision_workflows](vision_workflows/README.md)
81+
82+
Follow the links below to learn more:
83+
- [Learn how to use VLMs to automatically monitor a video stream for custom events.](nim_workflows/vlm_alerts/README.md)
84+
- [Learn how to search images with natural language using NV-CLIP.](nim_workflows/nvclip_multimodal_search/README.md)
85+
- [Learn how to combine VLMs, LLMs and CV models to build a robust text extraction pipeline.](nim_workflows/vision_text_extraction/README.md)
86+
- [Learn how to use embeddings with NVDINOv2 and a Milvus VectorDB to build a few shot classification model.](nim_workflows/nvdinov2_few_shot/README.md)
87+
88+
89+
## Try it Now!
90+
91+
Experience NVIDIA RAG Pipelines with just a few steps!
92+
93+
1. Get your NVIDIA API key.
94+
1. Go to the [NVIDIA API Catalog](https://build.ngc.nvidia.com/explore/).
95+
1. Select any model.
96+
1. Click **Get API Key**.
97+
1. Run:
98+
```console
99+
export NVIDIA_API_KEY=nvapi-...
100+
```
101+
102+
1. Clone the repository.
103+
104+
```console
105+
git clone https://github.com/nvidia/GenerativeAIExamples.git
106+
```
107+
108+
1. Build and run the basic RAG pipeline.
109+
110+
```console
111+
cd GenerativeAIExamples/RAG/examples/basic_rag/langchain/
112+
docker compose up -d --build
113+
```
114+
115+
1. Go to <https://localhost:8090/> and submit queries to the sample RAG Playground.
116+
117+
1. Stop containers when done.
118+
119+
```console
120+
docker compose down
121+
```
122+
123+
124+
125+
## Data Flywheel
126+
127+
A [Data Flywheel](https://www.nvidia.com/en-us/glossary/data-flywheel/) is a self-reinforcing cycle where user interactions generate data that improves AI models or products, leading to better outcomes that attract more users and further enhance data quality. This feedback loop relies on continuous data processing, model refinement, and guardrails to ensure accuracy and compliance while compounding value over time. Real-world applications range from personalized customer experiences to operational systems like inventory management, where improved predictions drive efficiency and growth.
128+
129+
### Tool-Calling Notebooks
130+
131+
Tool calling empowers Large Language Models (LLMs) to integrate with external APIs, execute dynamic workflows, and retrieve real-time data beyond their training scope. The NVIDIA NeMo microservices platform offers a modular infrastructure for deploying AI pipelines that includes fine-tuning, evaluation, inference, and guardrail enforcement—across Kubernetes clusters in cloud or on-premises environments.
132+
133+
This end-to-end [tutorial](./nemo/data-flywheel/tool-calling) demonstrates how to leverage NeMo Microservices to customize [Llama-3.2-1B-Instruct](https://huggingface.co/meta-llama/Llama-3.2-1B-Instruct) by using the [xLAM](https://huggingface.co/datasets/Salesforce/xlam-function-calling-60k) function-calling dataset, assess its accuracy, and implement safety constraints to govern its behavior.
134+
135+
## RAG
136+
137+
### RAG Notebooks
138+
139+
NVIDIA has first-class support for popular generative AI developer frameworks like [LangChain](https://python.langchain.com/v0.2/docs/integrations/chat/nvidia_ai_endpoints/), [LlamaIndex](https://docs.llamaindex.ai/en/stable/examples/llm/nvidia/), and [Haystack](https://haystack.deepset.ai/integrations/nvidia). These end-to-end notebooks show how to integrate NIM microservices using your preferred generative AI development framework.
140+
141+
Use these [notebooks](./RAG/notebooks/README.md) to learn about the LangChain and LlamaIndex connectors.
142+
143+
#### LangChain Notebooks
144+
145+
- RAG
146+
- [Basic RAG with CHATNVIDIA LangChain Integration](./RAG/notebooks/langchain/langchain_basic_RAG.ipynb)
147+
- [RAG using local NIM microservices for LLMs and Retrieval](./RAG/notebooks/langchain/RAG_Langchain_with_Local_NIM.ipynb)
148+
- [RAG for HTML Documents](./RAG/notebooks/langchain/RAG_for_HTML_docs_with_Langchain_NVIDIA_AI_Endpoints.ipynb)
149+
- [Chat with NVIDIA Financial Reports](./RAG/notebooks/langchain/Chat_with_nvidia_financial_reports.ipynb)
150+
- Agents
151+
- [NIM Tool Calling 101](https://github.com/langchain-ai/langchain-nvidia/blob/main/cookbook/nvidia_nim_agents_llama3.1.ipynb)
152+
- [Agentic RAG with NeMo Retriever](./RAG/notebooks/langchain/agentic_rag_with_nemo_retriever_nim.ipynb)
153+
- [Agents with Human in the Loop](./RAG/notebooks/langchain/LangGraph_HandlingAgent_IntermediateSteps.ipynb)
154+
155+
156+
#### LlamaIndex Notebooks
157+
158+
- [Basic RAG with LlamaIndex Integration](./RAG/notebooks/llamaindex/llamaindex_basic_RAG.ipynb)
159+
160+
### RAG Examples
161+
162+
By default, these end-to-end [examples](RAG/examples/README.md) use preview NIM endpoints on [NVIDIA API Catalog](https://catalog.ngc.nvidia.com). Alternatively, you can run any of the examples [on premises](./RAG/examples/local_deploy/).
163+
164+
#### Basic RAG Examples
165+
166+
- [LangChain example](./RAG/examples/basic_rag/langchain/README.md)
167+
- [LlamaIndex example](./RAG/examples/basic_rag/llamaindex/README.md)
168+
169+
#### Advanced RAG Examples
170+
171+
- [Multi-Turn](./RAG/examples/advanced_rag/multi_turn_rag/README.md)
172+
- [Multimodal Data](./RAG/examples/advanced_rag/multimodal_rag/README.md)
173+
- [Structured Data](./RAG/examples/advanced_rag/structured_data_rag/README.md) (CSV)
174+
- [Query Decomposition](./RAG/examples/advanced_rag/query_decomposition_rag/README.md)
175+
176+
### RAG Tools
177+
178+
Example tools and tutorials to enhance LLM development and productivity when using NVIDIA RAG pipelines.
179+
180+
- [Evaluation](./RAG/tools/evaluation/README.md)
181+
- [Observability](./RAG/tools/observability/README.md)
182+
183+
### RAG Projects
184+
185+
- [NVIDIA Tokkio LLM-RAG](https://docs.nvidia.com/ace/latest/workflows/tokkio/text/Tokkio_LLM_RAG_Bot.html): Use Tokkio to add avatar animation for RAG responses.
186+
- [Hybrid RAG Project on AI Workbench](https://github.com/NVIDIA/workbench-example-hybrid-rag): Run an NVIDIA AI Workbench example project for RAG.
187+
188+
## Documentation
189+
190+
### Getting Started
191+
192+
- [Prerequisites](./docs/common-prerequisites.md)
193+
194+
### How To's
195+
196+
- [Changing the Inference or Embedded Model](./docs/change-model.md)
197+
- [Customizing the Vector Database](./docs/vector-database.md)
198+
- [Customizing the Chain Server](./docs/chain-server.md):
199+
- [Chunking Strategy](./docs/text-splitter.md)
200+
- [Prompting Template Engineering](./docs/prompt-customization.md)
201+
- [Configuring LLM Parameters at Runtime](./docs/llm-params.md)
202+
- [Supporting Multi-Turn Conversations](./docs/multiturn.md)
203+
- [Speaking Queries and Listening to Responses with NVIDIA Riva](./docs/riva-asr-tts.md)
204+
205+
### Reference
206+
207+
- [Support Matrix](./docs/support-matrix.md)
208+
- [Architecture](./docs/architecture.md)
209+
- [Using the Sample Chat Web Application](./docs/using-sample-web-application.md)
210+
- [RAG Playground Web Application](./docs/frontend.md)
211+
- [Software Component Configuration](./docs/configuration.md)
212+
213+
214+
## Community
215+
We're posting these examples on GitHub to support the NVIDIA LLM community and facilitate feedback.
216+
We invite contributions! Open a GitHub issue or pull request! See [contributing](docs/contributing.md) Check out the [community](./community/README.md) examples and notebooks.
217+

0 commit comments

Comments
 (0)