Merge pull request #353 from oracle-samples/opensearch_notebook

adding opensearch notebook

Author: qiuosier

llm_application/Use_of_Cohere_embed_models_for_Semantic_Search_in_OCI_OpenSearch.ipynb

@@ -0,0 +1,344 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# Retrieval Augmented Generative Question Answer Using OCI OpenSearch as Retriever\n",
+    "\n",
+    "In this tutorial, we will walk through the steps to set up a retrieval-augmented generative QA using OCI OpenSearch as retriever.\n",
+    "\n",
+    "### Prerequesites\n",
+    "- You have a Running Instance of OCI Search.\n",
+    "- OpenSearch version has to be at least 2.8.0.\n",
+    "- You need to install langchain, opensearch-py, and oracle-ads.\n",
+    "\n",
+    "To check how to spin up an instance of OCI search, see [Search and visualize data using OCI Search Service with OpenSearch](https://docs.oracle.com/en/learn/oci-opensearch/index.html#introduction)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "!pip install --upgrade oracle-ads langchain opensearch-py"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "### Step 1: Load and Split your Documents Into Chunks\n",
+    "Let's say you're looking to create a search engine that enables users to search through documentation stored as markdown files. Actually, it does not matter what file format your documentation are in as Langchain offers support for various types of document loaders. In this tutorial, we will just use markdown file as an example."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 1,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "import fsspec\n",
+    "from langchain.text_splitter import MarkdownHeaderTextSplitter\n",
+    "\n",
+    "with fsspec.open(\n",
+    "    \"https://raw.githubusercontent.com/oracle-samples/oci-data-science-ai-samples/main/distributed_training/Tensorboard.md\",\n",
+    "    \"r\"\n",
+    ") as f:\n",
+    "    report = f.read()\n",
+    "    \n",
+    "    \n",
+    "headers_to_split_on = [\n",
+    "    (\"#\", \"Header 1\"),\n",
+    "    (\"##\", \"Header 2\"),\n",
+    "    (\"###\", \"Header 3\"),\n",
+    "]\n",
+    "\n",
+    "markdown_splitter = MarkdownHeaderTextSplitter(headers_to_split_on=headers_to_split_on)\n",
+    "md_header_splits = markdown_splitter.split_text(report)\n",
+    "texts = [text.page_content for text in md_header_splits]"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 2,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Number of documents: 4\n",
+      "First document:\n",
+      "TensorBoard helps visualizing your experiments. You bring up a ``TensorBoard`` session on your workstation and point to\n",
+      "the directory that contains the TensorBoard logs.  \n",
+      "`OCI` = Oracle Cloud Infrastructure\n",
+      "`DT` = Distributed Training\n",
+      "`ADS` = Oracle Accelerated Data Science Library\n",
+      "`OCIR` = Oracle Cloud Infrastructure Registry\n"
+     ]
+    }
+   ],
+   "source": [
+    "print(f\"Number of documents: {len(texts)}\")\n",
+    "print(f\"First document:\\n{texts[0]}\")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "### Step 2: Embed your Documents\n",
+    "\n",
+    "You can use oracle-ads to access the GenerativeAI embedding models. The embedding models returns embedding vectors of length 1024. oracle-ads is an open source library. It speeds up common data science activities by providing tools that automate and simplify common data science tasks. Additionally, provides data scientists a friendly pythonic interface to OCI services. Check [oracle-ads github](https://github.com/oracle/accelerated-data-science) for more information."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from ads.llm import GenerativeAIEmbeddings\n",
+    " \n",
+    "oci_embedings = GenerativeAIEmbeddings(\n",
+    "    compartment_id=\"ocid1.compartment.oc1.######\",\n",
+    "    client_kwargs=dict(service_endpoint=\"https://generativeai.aiservice.us-chicago-1.oci.oraclecloud.com\") # this can be omitted after Generative AI service is GA.\n",
+    ")\n",
+    "embeddings = oci_embedings.embed_documents(texts=texts)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 4,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Number of embeddings: 4\n",
+      "Embedding dimensions: 1024\n"
+     ]
+    }
+   ],
+   "source": [
+    "print(f\"Number of embeddings: {len(embeddings)}\")\n",
+    "print(f\"Embedding dimensions: {len(embeddings[0])}\")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "### Step 3: Create an Index for your Documents\n",
+    "\n",
+    "First connect to your OCI search cluster. We can use the opensearchpy library to connect to the OpenSearch cluster."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 6,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# Connect to the opensearch cluster.\n",
+    "from opensearchpy import OpenSearch\n",
+    " \n",
+    "# Create a connection to your OpenSearch cluster\n",
+    "es = OpenSearch(\n",
+    "    ['https://####'],  # Replace with your OpenSearch endpoint URL\n",
+    "    http_auth=('username', 'password'),  # Replace with your credentials\n",
+    "    verify_certs=False,  # Set to True if you want to verify SSL certificates\n",
+    "    timeout=30\n",
+    ")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "First, you must create a k-NN index and set the ``index.knn`` parameter to true. This settings tells the plugin to generate native library indexes specifically tailored for k-NN searches. \n",
+    "\n",
+    "Next, you must add one or more fields of the knn_vector data type. This example creates an index with one ``knn_vector``: ``embedding_vector`` and one ``text``: ``text``. \n",
+    "\n",
+    "The knn_vector uses Lucene fields that specify the configuration of the k-NN search algorithms. It employs the Hierarchical Navigable Small Worlds [HNSW](https://www.pinecone.io/learn/series/faiss/hnsw/) algorithm  for super fast search and fantastic recall and consine similarity to measure distance. \n",
+    "\n",
+    "- ``efSearch`` controls how many entry points will be explored between layers during the search. A higher value of ef_search typically results in a more thorough and potentially higher-quality search, but increased computational cost. \n",
+    "\n",
+    "- ``efConstruction`` controls how many entry points will be explored when building the index. A higher value of \"ef_constructions\" typically results in a higher-quality graph structure but may also increase the computational cost of building the index.\n",
+    "\n",
+    "The ``dimension`` field defines the size of the embedding vector. In our case, we are using embedding vectors returned from the genAI embedding model, which is of length 1024. \n",
+    "\n",
+    "See [documentation](https://opensearch.org/docs/2.8/search-plugins/knn/knn-index#method-definitions) for more details on parameters' definitions. You\n",
+    "\n",
+    "**Note**: The Lucene engine can support dimension up to 1,024."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 7,
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "{'acknowledged': True, 'shards_acknowledged': True, 'index': 'tensorboard'}"
+      ]
+     },
+     "execution_count": 7,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "INDEX_NAME = \"tensorboard\"\n",
+    "VECTOR_1_NAME = \"embedding_vector\"\n",
+    "VECTOR_2_NAME = \"text\"\n",
+    " \n",
+    "body = {\n",
+    "    # Index setting: https://opensearch.org/docs/2.11/search-plugins/knn/knn-index\n",
+    "    \"settings\": {\"index\": {\"knn\": \"true\", \"knn.algo_param.ef_search\": 100}},\n",
+    "    # Explicit mapping: https://opensearch.org/docs/2.11/field-types/index/#explicit-mapping\n",
+    "    \"mappings\": { \n",
+    "        \"properties\": {\n",
+    "            VECTOR_1_NAME: {\n",
+    "                # Supported field types: https://opensearch.org/docs/2.11/field-types/supported-field-types/index/\n",
+    "                \"type\": \"knn_vector\", \n",
+    "                \"dimension\": 1024,\n",
+    "                # Method definition: https://opensearch.org/docs/2.11/search-plugins/knn/knn-index#method-definitions\n",
+    "                \"method\": { \n",
+    "                    \"name\": \"hnsw\",\n",
+    "                    \"space_type\": \"cosinesimil\",\n",
+    "                    \"engine\": \"lucene\",\n",
+    "                    \"parameters\": {\"ef_construction\": 128, \"m\": 24},\n",
+    "                },\n",
+    "            },\n",
+    "            VECTOR_2_NAME: {\n",
+    "                 \"type\": \"text\"\n",
+    "               },\n",
+    "        }\n",
+    "    },\n",
+    "}\n",
+    "response = es.indices.create(INDEX_NAME, body=body)\n",
+    "response"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "### Step 4: Insert the Embedding Vectors for your Documents\n",
+    "Now let's populate the index using the embedding vectors calculated from your documents using Cohere Embedding Models. "
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 8,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "i = 0\n",
+    "# insert each row one-at-a-time to the document index\n",
+    "for text, embed in zip(texts, embeddings):\n",
+    " \n",
+    "    try:\n",
+    "         \n",
+    "        body = {\n",
+    "            VECTOR_1_NAME: embed,\n",
+    "            VECTOR_2_NAME: text,\n",
+    "        }\n",
+    "        response = es.index(index=INDEX_NAME, body=body)\n",
+    "    except Exception as e:\n",
+    "        print(f\"[ERROR]: {e}\")\n",
+    "        continue\n",
+    "    i += 1"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "\n",
+    "A new query coming in, first calcualte the embedding vector and then conduct a semantic search.\n",
+    "\n",
+    "- `k`: the number of neighbors the search will return\n",
+    "- `size`: (required) how many results the query actually returns. The plugin returns k amount of results for each shard (and each segment) and size amount of results for the entire query. The plugin supports a maximum k value of 10,000."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 15,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "It is required that ``tensorboard`` is installed in a dedicated conda environment or virtual environment. Prepare an\n",
+      "environment yaml file for creating conda environment with following command -  \n",
+      "**tensorboard-dep.yaml**:  \n",
+      "```yaml\n",
+      "dependencies:\n",
+      "- python=3.8\n",
+      "- pip\n",
+      "- pip:\n",
+      "- ocifs\n",
+      "- tensorboard\n",
+      "name: tensorboard\n",
+      "```  \n",
+      "Create the conda environment from the yaml file generated in the preceeding step  \n",
+      "```bash\n",
+      "conda env create -f tensorboard-dep.yaml\n",
+      "```  \n",
+      "This will create a conda environment called tensorboard. Activate the conda environment by running -  \n",
+      "```bash\n",
+      "conda activate tensorboard\n",
+      "```  \n",
+      "**Using TensorBoard Logs:**  \n",
+      "To launch a TensorBoard session on your local workstation, run -  \n",
+      "```bash\n",
+      "export OCIFS_IAM_KEY=api_key\n",
+      "tensorboard --logdir oci://my-bucket@my-namespace/path/to/logs\n",
+      "```  \n",
+      "`OCIFS_IAM_KEY=api_key` - If you are using resource principal, set `resource_principal`  \n",
+      "This will bring up TensorBoard app on your workstation. Access TensorBoard at ``http://localhost:6006/``  \n",
+      "**Note**: The logs take some initial time (few minutes) to reflect on the tensorboard dashboard.\n"
+     ]
+    }
+   ],
+   "source": [
+    "query_vector = oci_embedings.embed_query(text=\"how to set up tensorboard in oci?\")\n",
+    "query = {\n",
+    "    \"size\": 2,\n",
+    "    \"query\": {\"knn\": {VECTOR_1_NAME: {\"vector\": query_vector, \"k\": 2}}},\n",
+    "}\n",
+    " \n",
+    "response = es.search(body=query, index=INDEX_NAME)  # the same as before\n",
+    "print(response[\"hits\"][\"hits\"][0]['_source']['text'])"
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3 (ipykernel)",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.8.18"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 4
+}