diff --git a/docs/docs/tutorials/graph_vectorstore.ipynb b/docs/docs/tutorials/graph_vectorstore.ipynb new file mode 100644 index 0000000000000..d618edfc02847 --- /dev/null +++ b/docs/docs/tutorials/graph_vectorstore.ipynb @@ -0,0 +1,489 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Build a RAG application with graph links between documents\n", + "\n", + "This tutorial shows how to create links between documents in a GraphVectorStore and use it to get more relevant responses when querying.\n", + "\n", + "It contains 2 main sections:\n", + "* [Preparation: load the DataStaxAstra Documentation into GraphVectorStore](#preparation-load-the-datastax-astra-documentation-into-graphvectorstore)\n", + "* [Create and execute the RAG Chains](#create-and-execute-the-rag-chains)\n", + "\n", + "## Preliminaries" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "%pip install -q langchain-community beautifulsoup4 markdownify python-dotenv" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Preparation: load the DataStax Astra Documentation into GraphVectorStore\n", + "\n", + "First, we'll crawl the DataStax documentation. At the moment, `SiteMapLoader` loads all of the pages into memory simultaneously, which makes it impossible to index larger sites from small environments (such as CoLab). So, we'll scrape the sitemap ourselves and iterate over the URLs, allowing us to process documents in batches and flush them to Astra DB. " + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Scrape the URLs from the Site Maps\n", + "First, we use Beautiful Soup to parse the XML content of each sitemap and get the list of URLs.\n", + "We also add a few extra URLs for external sites that are useful to include in the index." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "import requests\n", + "from bs4 import BeautifulSoup\n", + "\n", + "# Use sitemaps to crawl the content\n", + "SITEMAPS = [\n", + " \"https://docs.datastax.com/en/sitemap-astra-db-vector.xml\",\n", + " \"https://docs.datastax.com/en/sitemap-cql.xml\",\n", + " \"https://docs.datastax.com/en/sitemap-dev-app-drivers.xml\",\n", + " \"https://docs.datastax.com/en/sitemap-glossary.xml\",\n", + " \"https://docs.datastax.com/en/sitemap-astra-db-serverless.xml\",\n", + "]\n", + "\n", + "# Additional URLs to crawl for content.\n", + "EXTRA_URLS = [\"https://github.com/jbellis/jvector\"]\n", + "\n", + "\n", + "def load_pages(sitemap_url):\n", + " r = requests.get(\n", + " sitemap_url,\n", + " headers={\n", + " # Astra docs only return a sitemap with a user agent set.\n", + " \"User-Agent\": \"Mozilla/5.0 (X11; Linux x86_64; rv:58.0) Gecko/20100101 \"\n", + " \"Firefox/58.0\",\n", + " },\n", + " timeout=30,\n", + " )\n", + " xml = r.text\n", + "\n", + " soup = BeautifulSoup(xml, features=\"xml\")\n", + " url_tags = soup.find_all(\"url\")\n", + " for url in url_tags:\n", + " yield (url.find(\"loc\").text)\n", + "\n", + "\n", + "# For maintenance purposes, we could check only the new articles since a given time.\n", + "URLS = [url for sitemap_url in SITEMAPS for url in load_pages(sitemap_url)] + EXTRA_URLS\n", + "len(URLS)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Load the content from each URL\n", + "Next, we create the code to load each page. This performs the following steps:\n", + "\n", + "1. Parses the HTML with BeautifulSoup\n", + "2. Locates the \"content\" of the HTML using an appropriate selector based on the URL\n", + "3. Use an HtmlLinkExtractor to find the link (``) tags in the content and collect the absolute URLs (for creating edges).\n", + "\n", + "Adding the URLs of these references to the metadata allows the graph store to create edges between the documents." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "from typing import AsyncIterator, Iterable\n", + "\n", + "from langchain_community.document_loaders import AsyncHtmlLoader\n", + "from langchain_community.graph_vectorstores.extractors import HtmlInput, HtmlLinkExtractor\n", + "from langchain_core.documents import Document\n", + "from langchain_core.graph_vectorstores.links import add_links\n", + "from markdownify import MarkdownConverter\n", + "\n", + "markdown_converter = MarkdownConverter(heading_style=\"ATX\")\n", + "html_link_extractor = HtmlLinkExtractor()\n", + "\n", + "\n", + "def select_content(soup: BeautifulSoup, url: str) -> BeautifulSoup:\n", + " if url.startswith(\"https://docs.datastax.com/en/\"):\n", + " return soup.select_one(\"article.doc\")\n", + " if url.startswith(\"https://github.com\"):\n", + " return soup.select_one(\"article.entry-content\")\n", + " return soup\n", + "\n", + "\n", + "async def load_pages(urls: Iterable[str]) -> AsyncIterator[Document]:\n", + " loader = AsyncHtmlLoader(\n", + " urls,\n", + " requests_per_second=4,\n", + " # Astra docs require a user agent\n", + " header_template={\n", + " \"User-Agent\": \"Mozilla/5.0 (X11; Linux x86_64; rv:58.0) Gecko/20100101 \"\n", + " \"Firefox/58.0\"\n", + " },\n", + " )\n", + " async for html in loader.alazy_load():\n", + " url = html.metadata[\"source\"]\n", + "\n", + " # Use the URL as the doc ID.\n", + " html.id = url\n", + "\n", + " # Apply the selectors while loading. This reduces the size of\n", + " # the document as early as possible for reduced memory usage.\n", + " soup = BeautifulSoup(html.page_content, \"html.parser\")\n", + " content = select_content(soup, url)\n", + "\n", + " # Extract HTML links from the content.\n", + " add_links(html, html_link_extractor.extract_one(HtmlInput(content, url)))\n", + "\n", + " # Convert the content to markdown\n", + " html.page_content = markdown_converter.convert_soup(content)\n", + "\n", + " yield html" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Initialize Environment\n", + "Before we initialize the Graph Store and write the documents we need to set some environment variables." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "import getpass\n", + "import os\n", + "from dotenv import load_dotenv\n", + "\n", + "load_dotenv()\n", + "\n", + "if \"OPENAI_API_KEY\" not in os.environ:\n", + " os.environ[\"OPENAI_API_KEY\"] = getpass.getpass(\"Enter OpenAI API Key: \")\n", + " \n", + "if \"ASTRA_DB_DATABASE_ID\" not in os.environ:\n", + " os.environ[\"ASTRA_DB_DATABASE_ID\"] = input(\"Enter Astra DB Database ID: \")\n", + " \n", + "if \"ASTRA_DB_APPLICATION_TOKEN\" not in os.environ:\n", + " os.environ[\"ASTRA_DB_APPLICATION_TOKEN\"] = getpass.getpass(\n", + " \"Enter Astra DB Application Token: \"\n", + ")\n", + " \n", + "if \"ASTRA_DB_KEYSPACE\" not in os.environ:\n", + " keyspace = input(\"Enter Astra DB Keyspace (Empty for default): \")\n", + " if keyspace:\n", + " os.environ[\"ASTRA_DB_KEYSPACE\"] = keyspace" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Initialize Cassio and GraphVectorStore\n", + "With the environment variables set, we initialize the Cassio library for talking to Cassandra / Astra DB.\n", + "We also create the `GraphVectorStore`." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "import cassio\n", + "from langchain_openai import OpenAIEmbeddings\n", + "from langchain_community.graph_vectorstores import CassandraGraphVectorStore\n", + "\n", + "cassio.init(auto=True)\n", + "embeddings = OpenAIEmbeddings()\n", + "graph_vectorstore = CassandraGraphVectorStore(\n", + " embeddings,\n", + " node_table=f\"astra_docs_nodes\",\n", + ")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Load the Documents\n", + "Finally, we fetch pages and write them to the graph store in batches of 50." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "not_found = 0\n", + "found = 0\n", + "BATCH_SIZE = 50\n", + "\n", + "docs = []\n", + "async for doc in load_pages(URLS):\n", + " if doc.page_content.startswith(\"\\n# Page Not Found\"):\n", + " not_found += 1\n", + " continue\n", + "\n", + " docs.append(doc)\n", + " found += 1\n", + "\n", + " if len(docs) >= BATCH_SIZE:\n", + " graph_vectorstore.add_documents(docs)\n", + " docs.clear()\n", + "\n", + "if docs:\n", + " graph_vectorstore.add_documents(docs)\n", + "print(f\"{not_found} (of {not_found + found}) URLs were not found\")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Create and execute the RAG Chains" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "from langchain_core.output_parsers import StrOutputParser\n", + "from langchain_core.prompts import ChatPromptTemplate\n", + "from langchain_core.runnables import RunnablePassthrough\n", + "from langchain_openai import ChatOpenAI\n", + "\n", + "llm = ChatOpenAI(model=\"gpt-4o\")\n", + "\n", + "template = \"\"\"You are a helpful technical support bot. You should provide complete answers explaining the options the user has available to address their problem. Answer the question based only on the following context:\n", + "{context}\n", + "\n", + "Question: {question}\n", + "\"\"\" # noqa: E501\n", + "prompt = ChatPromptTemplate.from_template(template)\n", + "\n", + "\n", + "def format_docs(docs):\n", + " return \"\\n\\n\".join(\n", + " f\"From {doc.metadata['content_id']}: {doc.page_content}\" for doc in docs\n", + " )" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "We'll use the following question. This is an interesting question because the ideal answer should be concise and in-depth, based on how the vector indexing is actually implemented." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "QUESTION = \"What vector indexing algorithms does Astra use?\"" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "from IPython.display import Markdown, display\n", + "\n", + "\n", + "# Helper method to render markdown in responses to a chain.\n", + "def run_and_render(chain, question):\n", + " result = chain.invoke(question)\n", + " display(Markdown(result))" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Vector-Only Retrieval" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# Depth 0 doesn't traverses edges and is equivalent to vector similarity only.\n", + "vector_retriever = graph_vectorstore.as_retriever(search_kwargs={\"depth\": 0})\n", + "\n", + "vector_rag_chain = (\n", + " {\"context\": vector_retriever | format_docs, \"question\": RunnablePassthrough()}\n", + " | prompt\n", + " | llm\n", + " | StrOutputParser()\n", + ")" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "run_and_render(vector_rag_chain, QUESTION)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Graph Traversal Retrieval" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# Depth 1 does vector similarity and then traverses 1 level of edges.\n", + "graph_retriever = graph_vectorstore.as_retriever(search_kwargs={\"depth\": 1})\n", + "\n", + "graph_rag_chain = (\n", + " {\"context\": graph_retriever | format_docs, \"question\": RunnablePassthrough()}\n", + " | prompt\n", + " | llm\n", + " | StrOutputParser()\n", + ")" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "run_and_render(graph_rag_chain, QUESTION)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### MMR Graph Traversal" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "mmr_graph_retriever = graph_vectorstore.as_retriever(\n", + " search_type=\"mmr_traversal\",\n", + " search_kwargs={\n", + " \"k\": 4,\n", + " \"fetch_k\": 10,\n", + " \"depth\": 2,\n", + " # \"score_threshold\": 0.2,\n", + " },\n", + ")\n", + "\n", + "mmr_graph_rag_chain = (\n", + " {\"context\": mmr_graph_retriever | format_docs, \"question\": RunnablePassthrough()}\n", + " | prompt\n", + " | llm\n", + " | StrOutputParser()\n", + ")" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "run_and_render(mmr_graph_rag_chain, QUESTION)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Check Retrieval Results" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# Set the question and see what documents each technique retrieves.\n", + "for i, doc in enumerate(vector_retriever.invoke(QUESTION)):\n", + " print(f\"Vector [{i}]: {doc.id}\")\n", + "\n", + "for i, doc in enumerate(graph_retriever.invoke(QUESTION)):\n", + " print(f\"Graph [{i}]: {doc.id}\")\n", + "\n", + "for i, doc in enumerate(mmr_graph_retriever.invoke(QUESTION)):\n", + " print(f\"MMR Graph [{i}]: {doc.id}\")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Conclusion\n", + "With vector only we retrieved chunks from the Astra documentation explaining that it used JVector.\n", + "Since it didn't follow the link to [JVector on GitHub](https://github.com/jbellis/jvector) it didn't actually answer the question.\n", + "\n", + "The graph retrieval started with the same set of chunks, but it followed the edge to the documents we loaded from GitHub.\n", + "This allowed the LLM to read in more depth how JVector is implemented, which allowed it to answer the question more clearly and with more detail." + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "agent-framework-aiP65pJh-py3.11", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.11.4" + } + }, + "nbformat": 4, + "nbformat_minor": 2 +} diff --git a/docs/docs/tutorials/index.mdx b/docs/docs/tutorials/index.mdx index a4e1840bf9744..81d31e7a47c53 100644 --- a/docs/docs/tutorials/index.mdx +++ b/docs/docs/tutorials/index.mdx @@ -15,6 +15,7 @@ New to LangChain or to LLM app development in general? Read this material to qui ## Working with external knowledge - [Build a Retrieval Augmented Generation (RAG) Application](/docs/tutorials/rag) - [Build a Conversational RAG Application](/docs/tutorials/qa_chat_history) +- [Build a RAG application with graph links between documents](/docs/tutorials/graph_vectorstore) - [Build a Question/Answering system over SQL data](/docs/tutorials/sql_qa) - [Build a Query Analysis System](/docs/tutorials/query_analysis) - [Build a local RAG application](/docs/tutorials/local_rag)