Skip to content

Commit 517be52

Browse files
sdiazlorsdiazlor
authored
docs: add llamaindex tutorial (#5402)
# Description <!-- Please include a summary of the changes and the related issue. Please also include relevant motivation and context. List any dependencies that are required for this change. --> Closes #<issue_number> **Type of change** <!-- Please delete options that are not relevant. Remember to title the PR according to the type of change --> - Bug fix (non-breaking change which fixes an issue) - New feature (non-breaking change which adds functionality) - Breaking change (fix or feature that would cause existing functionality to not work as expected) - Refactor (change restructuring the codebase without changing functionality) - Improvement (change adding some improvement to an existing functionality) - Documentation update **How Has This Been Tested** <!-- Please add some reference about how your feature has been tested. --> **Checklist** <!-- Please go over the list and make sure you've taken everything into account --> - I added relevant documentation - I followed the style guidelines of this project - I did a self-review of my code - I made corresponding changes to the documentation - I confirm My changes generate no new warnings - I have added tests that prove my fix is effective or that my feature works - I have added relevant notes to the CHANGELOG.md file (See https://keepachangelog.com/)
1 parent 78575e6 commit 517be52

File tree

3 files changed

+341
-0
lines changed

3 files changed

+341
-0
lines changed

‎argilla/docs/assets/images/community/integrations/llamaindex_rag_1.png‎

153 KB
Loading

‎argilla/docs/community/integrations/llamaindex_rag_github.ipynb‎

Lines changed: 339 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,339 @@
1+
{
2+
"cells": [
3+
{
4+
"cell_type": "markdown",
5+
"metadata": {},
6+
"source": [
7+
"# 🕵🏻‍♀️ Create a RAG system expert in a GitHub repository and log your predictions in Argilla\n",
8+
"\n",
9+
"In this tutorial, we'll show you how to create a RAG system that can answer questions about a specific GitHub repository. As example, we will target the [Argilla repository](https://github.com/argilla-io/argilla). This RAG system will target the docs of the repository, as that's where most of the natural language information about the repository can be found.\n",
10+
"\n",
11+
"This tutorial includes the following steps:\n",
12+
"\n",
13+
"- Setting up the Argilla callback handler for LlamaIndex.\n",
14+
"- Initializing a GitHub client\n",
15+
"- Creating an index with a specific set of files from the GitHub repository of our choice.\n",
16+
"- Create a RAG system out of the Argilla repository, ask questions, and automatically log the answers to Argilla.\n",
17+
"\n",
18+
"This tutorial is based on the [Github Repository Reader](https://docs.llamaindex.ai/en/stable/examples/data_connectors/GithubRepositoryReaderDemo/) made by LlamaIndex."
19+
]
20+
},
21+
{
22+
"cell_type": "markdown",
23+
"metadata": {},
24+
"source": [
25+
"## Getting started\n",
26+
"\n",
27+
"### Deploy the Argilla server¶\n",
28+
"\n",
29+
"If you already have deployed Argilla, you can skip this step. Otherwise, you can quickly deploy Argilla following [this guide](https://docs.argilla.io/latest/getting_started/quickstart/)."
30+
]
31+
},
32+
{
33+
"cell_type": "markdown",
34+
"metadata": {},
35+
"source": [
36+
"### Set up the environment¶\n",
37+
"\n",
38+
"To complete this tutorial, you need to install this integration and a third-party library via pip.\n",
39+
"\n",
40+
"!!! note\n",
41+
" Check the integration GitHub repository [here](https://github.com/argilla-io/argilla-llama-index)."
42+
]
43+
},
44+
{
45+
"cell_type": "code",
46+
"execution_count": null,
47+
"metadata": {},
48+
"outputs": [],
49+
"source": [
50+
"!pip install \"argilla-llama-index\"\n",
51+
"!pip install \"llama-index-readers-github==0.1.9\""
52+
]
53+
},
54+
{
55+
"cell_type": "markdown",
56+
"metadata": {},
57+
"source": [
58+
"Let's make the required imports:"
59+
]
60+
},
61+
{
62+
"cell_type": "code",
63+
"execution_count": 1,
64+
"metadata": {},
65+
"outputs": [],
66+
"source": [
67+
"from llama_index.core import (\n",
68+
" Settings,\n",
69+
" VectorStoreIndex,\n",
70+
" set_global_handler,\n",
71+
")\n",
72+
"from llama_index.llms.openai import OpenAI\n",
73+
"from llama_index.readers.github import (\n",
74+
" GithubClient,\n",
75+
" GithubRepositoryReader,\n",
76+
")"
77+
]
78+
},
79+
{
80+
"cell_type": "markdown",
81+
"metadata": {},
82+
"source": [
83+
"We need to set the OpenAI API key and the GitHub token. The OpenAI API key is required to run queries using GPT models, while the GitHub token ensures you have access to the repository you're using. Although the GitHub token might not be necessary for public repositories, it is still recommended."
84+
]
85+
},
86+
{
87+
"cell_type": "code",
88+
"execution_count": 2,
89+
"metadata": {},
90+
"outputs": [],
91+
"source": [
92+
"import os\n",
93+
"\n",
94+
"os.environ[\"OPENAI_API_KEY\"] = \"sk-...\"\n",
95+
"openai_api_key = os.getenv(\"OPENAI_API_KEY\")\n",
96+
"\n",
97+
"os.environ[\"GITHUB_TOKEN\"] = \"ghp_...\"\n",
98+
"github_token = os.getenv(\"GITHUB_TOKEN\")"
99+
]
100+
},
101+
{
102+
"cell_type": "markdown",
103+
"metadata": {},
104+
"source": [
105+
"## Set the Argilla's LlamaIndex handler\n",
106+
"\n",
107+
"To easily log your data into Argilla within your LlamaIndex workflow, you only need a simple step. Just call the Argilla global handler for Llama Index before starting production with your LLM. This ensured that the predictions obtained using Llama Index are automatically logged to the Argilla instance.\n",
108+
"\n",
109+
"- `dataset_name`: The name of the dataset. If the dataset does not exist, it will be created with the specified name. Otherwise, it will be updated.\n",
110+
"- `api_url`: The URL to connect to the Argilla instance.\n",
111+
"- `api_key`: The API key to authenticate with the Argilla instance.\n",
112+
"- `number_of_retrievals`: The number of retrieved documents to be logged. Defaults to 0.\n",
113+
"- `workspace_name`: The name of the workspace to log the data. By default, the first available workspace.\n",
114+
"\n",
115+
"> For more information about the credentials, check the documentation for [users](https://docs.argilla.io/latest/how_to_guides/user/) and [workspaces](https://docs.argilla.io/latest/how_to_guides/workspace/)."
116+
]
117+
},
118+
{
119+
"cell_type": "code",
120+
"execution_count": null,
121+
"metadata": {},
122+
"outputs": [],
123+
"source": [
124+
"set_global_handler(\n",
125+
" \"argilla\",\n",
126+
" dataset_name=\"github_query_model\",\n",
127+
" api_url=\"http://localhost:6900\",\n",
128+
" api_key=\"argilla.apikey\",\n",
129+
" number_of_retrievals=2,\n",
130+
")"
131+
]
132+
},
133+
{
134+
"cell_type": "markdown",
135+
"metadata": {},
136+
"source": [
137+
"## Retrieve the data from GitHub\n",
138+
"\n",
139+
"First, we need to initialize the GitHub client, which will include the GitHub token for repository access."
140+
]
141+
},
142+
{
143+
"cell_type": "code",
144+
"execution_count": 4,
145+
"metadata": {},
146+
"outputs": [],
147+
"source": [
148+
"github_client = GithubClient(github_token=github_token, verbose=True)"
149+
]
150+
},
151+
{
152+
"cell_type": "markdown",
153+
"metadata": {},
154+
"source": [
155+
"Before creating our `GithubRepositoryReader` instance, we need to adjust the nesting. Since the Jupyter kernel operates on an event loop, we must prevent this loop from finishing before the repository is fully read."
156+
]
157+
},
158+
{
159+
"cell_type": "code",
160+
"execution_count": 5,
161+
"metadata": {},
162+
"outputs": [],
163+
"source": [
164+
"import nest_asyncio\n",
165+
"\n",
166+
"nest_asyncio.apply()"
167+
]
168+
},
169+
{
170+
"cell_type": "markdown",
171+
"metadata": {},
172+
"source": [
173+
"Now, let’s create a GithubRepositoryReader instance with the necessary repository details. In this case, we'll target the `main` branch of the `argilla` repository. As we will focus on the documentation, we will focus on the `argilla/docs/` folder, excluding images, json files, and ipynb files."
174+
]
175+
},
176+
{
177+
"cell_type": "code",
178+
"execution_count": 6,
179+
"metadata": {},
180+
"outputs": [],
181+
"source": [
182+
"documents = GithubRepositoryReader(\n",
183+
" github_client=github_client,\n",
184+
" owner=\"argilla-io\",\n",
185+
" repo=\"argilla\",\n",
186+
" use_parser=False,\n",
187+
" verbose=False,\n",
188+
" filter_directories=(\n",
189+
" [\"argilla/docs/\"],\n",
190+
" GithubRepositoryReader.FilterType.INCLUDE,\n",
191+
" ),\n",
192+
" filter_file_extensions=(\n",
193+
" [\n",
194+
" \".png\",\n",
195+
" \".jpg\",\n",
196+
" \".jpeg\",\n",
197+
" \".gif\",\n",
198+
" \".svg\",\n",
199+
" \".ico\",\n",
200+
" \".json\",\n",
201+
" \".ipynb\", # Erase this line if you want to include notebooks\n",
202+
"\n",
203+
" ],\n",
204+
" GithubRepositoryReader.FilterType.EXCLUDE,\n",
205+
" ),\n",
206+
").load_data(branch=\"main\")"
207+
]
208+
},
209+
{
210+
"cell_type": "markdown",
211+
"metadata": {},
212+
"source": [
213+
"## Create the index and make some queries"
214+
]
215+
},
216+
{
217+
"cell_type": "markdown",
218+
"metadata": {},
219+
"source": [
220+
"Now, let's create a LlamaIndex index out of this document, and we can start querying the RAG system."
221+
]
222+
},
223+
{
224+
"cell_type": "code",
225+
"execution_count": 7,
226+
"metadata": {},
227+
"outputs": [],
228+
"source": [
229+
"# LLM settings\n",
230+
"Settings.llm = OpenAI(\n",
231+
" model=\"gpt-3.5-turbo\", temperature=0.8, openai_api_key=openai_api_key\n",
232+
")\n",
233+
"\n",
234+
"# Load the data and create the index\n",
235+
"index = VectorStoreIndex.from_documents(documents)\n",
236+
"\n",
237+
"# Create the query engine\n",
238+
"query_engine = index.as_query_engine()"
239+
]
240+
},
241+
{
242+
"cell_type": "code",
243+
"execution_count": null,
244+
"metadata": {},
245+
"outputs": [],
246+
"source": [
247+
"response = query_engine.query(\"How do I create a Dataset in Argilla?\")\n",
248+
"response"
249+
]
250+
},
251+
{
252+
"cell_type": "markdown",
253+
"metadata": {},
254+
"source": [
255+
"The generated response will be automatically logged in our Argilla instance. Check it out! From Argilla you can quickly have a look at your predictions and annotate them, so you can combine both synthetic data and human feedback.\n",
256+
"\n",
257+
"![Argilla UI](../../assets/images/community/integrations/llamaindex_rag_1.png)\n"
258+
]
259+
},
260+
{
261+
"cell_type": "markdown",
262+
"metadata": {},
263+
"source": [
264+
"Let's ask a couple of more questions to see the overall behavior of the RAG chatbot. Remember that the answers are automatically logged into your Argilla instance."
265+
]
266+
},
267+
{
268+
"cell_type": "code",
269+
"execution_count": 10,
270+
"metadata": {},
271+
"outputs": [
272+
{
273+
"name": "stdout",
274+
"output_type": "stream",
275+
"text": [
276+
"Question: How can I list the available datasets?\n",
277+
"Answer: You can list all the datasets available in a workspace by utilizing the `datasets` attribute of the `Workspace` class. Additionally, you can determine the number of datasets in a workspace by using `len(workspace.datasets)`. To list the datasets, you can iterate over them and print out each dataset. Remember that dataset settings are not preloaded when listing datasets, and if you need to work with settings, you must load them explicitly for each dataset.\n",
278+
"----------------------------\n",
279+
"Question: Which are the user credentials?\n",
280+
"Answer: The user credentials in Argilla consist of a username, password, and API key.\n",
281+
"----------------------------\n",
282+
"Question: Can I use markdown in Argilla?\n",
283+
"Answer: Yes, you can use Markdown in Argilla.\n",
284+
"----------------------------\n",
285+
"Question: Could you explain how to annotate datasets in Argilla?\n",
286+
"Answer: To annotate datasets in Argilla, users can manage their data annotation projects by setting up `Users`, `Workspaces`, `Datasets`, and `Records`. By deploying Argilla on the Hugging Face Hub or with `Docker`, installing the Python SDK with `pip`, and creating the first project, users can get started in just 5 minutes. The tool allows for interacting with data in a more engaging way through features like quick labeling with filters, AI feedback suggestions, and semantic search, enabling users to focus on training models and monitoring their performance effectively.\n",
287+
"----------------------------\n"
288+
]
289+
},
290+
{
291+
"name": "stderr",
292+
"output_type": "stream",
293+
"text": [
294+
"\n"
295+
]
296+
}
297+
],
298+
"source": [
299+
"questions = [\n",
300+
" \"How can I list the available datasets?\",\n",
301+
" \"Which are the user credentials?\",\n",
302+
" \"Can I use markdown in Argilla?\",\n",
303+
" \"Could you explain how to annotate datasets in Argilla?\",\n",
304+
"]\n",
305+
"\n",
306+
"answers = []\n",
307+
"\n",
308+
"for question in questions:\n",
309+
" answers.append(query_engine.query(question))\n",
310+
"\n",
311+
"for question, answer in zip(questions, answers):\n",
312+
" print(f\"Question: {question}\")\n",
313+
" print(f\"Answer: {answer}\")\n",
314+
" print(\"----------------------------\")"
315+
]
316+
}
317+
],
318+
"metadata": {
319+
"kernelspec": {
320+
"display_name": ".venv",
321+
"language": "python",
322+
"name": "python3"
323+
},
324+
"language_info": {
325+
"codemirror_mode": {
326+
"name": "ipython",
327+
"version": 3
328+
},
329+
"file_extension": ".py",
330+
"mimetype": "text/x-python",
331+
"name": "python",
332+
"nbconvert_exporter": "python",
333+
"pygments_lexer": "ipython3",
334+
"version": "3.10.12"
335+
}
336+
},
337+
"nbformat": 4,
338+
"nbformat_minor": 2
339+
}

‎argilla/mkdocs.yml‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -187,5 +187,7 @@ nav:
187187
- How to contribute?: community/contributor.md
188188
- Issue dashboard: community/popular_issues.md
189189
- Changelog: community/changelog.md
190+
- Integrations:
191+
- LlamaIndex: community/integrations/llamaindex_rag_github.ipynb
190192
- UI Demo ↗:
191193
- https://demo.argilla.io/sign-in?auth=ZGVtbzoxMjM0NTY3OA==

0 commit comments

Comments
 (0)