\n"
+ ],
+ "application/vnd.google.colaboratory.intrinsic+json": {
+ "type": "dataframe",
+ "summary": "{\n \"name\": \"evaluate_retrieval\",\n \"rows\": 1,\n \"fields\": [\n {\n \"column\": \"precision\",\n \"properties\": {\n \"dtype\": \"number\",\n \"std\": null,\n \"min\": 0.67,\n \"max\": 0.67,\n \"num_unique_values\": 1,\n \"samples\": [\n 0.67\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"recall\",\n \"properties\": {\n \"dtype\": \"number\",\n \"std\": null,\n \"min\": 0.5,\n \"max\": 0.5,\n \"num_unique_values\": 1,\n \"samples\": [\n 0.5\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"map\",\n \"properties\": {\n \"dtype\": \"number\",\n \"std\": null,\n \"min\": 0.83,\n \"max\": 0.83,\n \"num_unique_values\": 1,\n \"samples\": [\n 0.83\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n }\n ]\n}"
+ }
+ },
+ "metadata": {},
+ "execution_count": 9
+ }
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "What are the figures above telling us?\n",
+ "\n",
+ "* Precision (0.67) tells us that 2 out of 3 of the retrieved docs are correct\n",
+ "* Recall (0.5) means that 2 out of 4 relevant docs have been retrieved\n",
+ "* MAP (0.83) is computed as the average of 1/1 (the highest ranked doc is correct) and 2/3 (the 2nd ranked doc is wrong, the 3rd is correct).\n",
+ "\n",
+ "While the example here focuses on a single datapoint, you can easily apply the same metrics to all your dataset and get the overall performance of your Retrieve phase."
+ ],
+ "metadata": {
+ "id": "sL-sIiSl2vw9"
+ }
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "\n",
+ "## Generation Evaluation\n",
+ "\n",
+ "Evaluating grounded generation (the second step of RAG) is notoriously difficult, because generations are usually complex and rich of information, and simply labelling an answer as \"good\" or \"bad\" is not enough.\n",
+ "To overcome this issue, we first decompose complex answers into a set of basic *claims*, where a claim is any sentence or part of a sentence in the answer that expresses a verifiable fact. Subsequently, we check the validity of each claim independently, defining the overall quality of the answer based on the correctness of the claims it includes.\n",
+ "\n",
+ "We use claims to compute three metrics:\n",
+ "\n",
+ "* **Faithfulness**, which measures how many of the claims in the generated response are supported by the retrieved documents. This is a fundamental metric, as it tells us how *grounded* in the documents the response is, and, contextually, it allows us to spot hallucinations\n",
+ "\n",
+ "* **Correctness**, which checks which claims in the response also occur in the gold answer\n",
+ "\n",
+ "* And **Coverage**, by which we assess how many of the claims in the gold answer are included in the generated response.\n",
+ "\n",
+ "Note that Faithfulness and Correctness share the exact same approach, the difference being that the former checks the claims against the supporting docs, while the latter against the golden answer.\n",
+ "Also, while Correctness is measuring the precision of the claims in the response, Coverage can be seen as complementary, as it measures recall."
+ ],
+ "metadata": {
+ "id": "mwEyE9IaklPL"
+ }
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "### Claim Extraction\n",
+ "\n",
+ "Let's now see how we implement the evaluation described above using LLMs. Let's start with **claim extraction**."
+ ],
+ "metadata": {
+ "id": "ySgduJrTTjvK"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# first, let's define a function which extracts the claims from a response\n",
+ "def extract_claims(query, response, model, client):\n",
+ "\n",
+ " # define the instructions on how to extract the claims\n",
+ " preamble = \"You are shown a prompt and a completion. You have to identify the main claims stated in the completion. A claim is any sentence or part of a sentence that expresses a verifiable fact. Please return a bullet list, in which every line includes one of the claims you identified. Do not add any further explanation to the bullet points.\"\n",
+ "\n",
+ " # build the prompt\n",
+ " prompt = f\"{preamble}\\n\\nPROMPT: {query}\\n\\nCOMPLETION: {response}\"\n",
+ "\n",
+ " # get the claims\n",
+ " claims = get_response(model, client, prompt)\n",
+ "\n",
+ " return claims\n"
+ ],
+ "metadata": {
+ "id": "X2fQa0Vbuy3W"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# now, let's consider this answer, which we previously generated with command-r\n",
+ "response = \"Apple's total net sales experienced a decline over the last year. The three-month period ended July 1, 2023, saw a total net sale of $81,797 million, which was a 1% decrease from the same period in 2022. The nine-month period ended July 1, 2023, fared slightly better, with a 3% decrease in net sales compared to the first nine months of 2022.\\nThis downward trend continued into the three and six-month periods ending April 1, 2023. Apple's total net sales decreased by 3% and 4% respectively, compared to the same periods in 2022.\"\n",
+ "\n",
+ "# let's extract the claims\n",
+ "claims = extract_claims(query, response, model, client)\n",
+ "\n",
+ "# and see what the model returns\n",
+ "print(f\"List of claims extracted from the model's response:\\n\\n{claims}\")"
+ ],
+ "metadata": {
+ "colab": {
+ "base_uri": "https://localhost:8080/"
+ },
+ "id": "y_0NXZEfu0AJ",
+ "outputId": "3fa32ab2-6898-449f-f062-187df7aea41e"
+ },
+ "execution_count": null,
+ "outputs": [
+ {
+ "output_type": "stream",
+ "name": "stdout",
+ "text": [
+ "List of claims extracted from the model's response:\n",
+ "\n",
+ "- Apple's total net sales experienced a decline over the last year.\n",
+ "- The three-month period ended July 1, 2023, saw a total net sale of $81,797 million.\n",
+ "- This was a 1% decrease from the same period in 2022.\n",
+ "- The nine-month period ended July 1, 2023, had a 3% decrease in net sales compared to the first nine months of 2022.\n",
+ "- The downward trend continued into the three and six-month periods ending April 1, 2023.\n",
+ "- Apple's total net sales decreased by 3% and 4% respectively, compared to the same periods in 2022.\n"
+ ]
+ }
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "### Claim Assessment\n",
+ "\n",
+ "Nice! now that we have the list of claims, we can go ahead and **assess the validity** of each claim."
+ ],
+ "metadata": {
+ "id": "50QRlCVe7dZf"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Let's create a function that checks each claim against a reference text,\n",
+ "# which here we will call \"context\". As you will see, we will use different contexts,\n",
+ "# depending on the metric we want to compute.\n",
+ "\n",
+ "def assess_claims(query, claims, context, model, client):\n",
+ "\n",
+ " # define the instructions on how to perform the assessment.\n",
+ " # the model has to append to each row a binary SUPPORTED tag\n",
+ " preamble = \"You are shown a prompt, a context and a list of claims. You have to check which of the claims in the list are supported by the context. Please return the list of claims exactly as is it, just append to each row “SUPPORTED=1” if the claim is supported by the context, or “SUPPORTED=0” if the claim is not supported by the context. Do not add any further explanation to the bullet points.\"\n",
+ "\n",
+ " # turn list into string\n",
+ " context = '\\n'.join(context)\n",
+ "\n",
+ " # build the prompt\n",
+ " prompt = f\"{preamble}\\n\\nPROMPT: {query}\\n\\nCONTEXT:\\n{context}\\n\\nCLAIMS:\\n{claims}\"\n",
+ "\n",
+ " # get the response\n",
+ " assessment = get_response(model, client, prompt)\n",
+ "\n",
+ " return assessment"
+ ],
+ "metadata": {
+ "id": "jrg9-dlMTRpB"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "### Faithfulness"
+ ],
+ "metadata": {
+ "id": "1OmD9pKxArk7"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Let's start with Faithfulness: in this case, we want to assess the claims\n",
+ "# in the response against the retrieved documents (i.e., context = retrieved documents)\n",
+ "\n",
+ "# for the sake of clarity, we report the actual text of the retrieved documents\n",
+ "retrieved_documents = ['Products and Services Performance\\nThe following table shows net sales by category for the three- and six-month periods ended April 1, 2023 and March 26, 2022 (dollars in millions):\\nThree Months Ended Six Months Ended\\nApril 1,\\n2023March 26,\\n2022 ChangeApril 1,\\n2023March 26,\\n2022 Change\\nNet sales by category:\\niPhone $ 51,334 $ 50,570 2 %$ 117,109 $ 122,198 (4)%\\nMac 7,168 10,435 (31)% 14,903 21,287 (30)%\\niPad 6,670 7,646 (13)% 16,066 14,894 8 %\\nWearables, Home and Accessories 8,757 8,806 (1)% 22,239 23,507 (5)%\\nServices 20,907 19,821 5 % 41,673 39,337 6 %\\nTotal net sales $ 94,836 $ 97,278 (3)%$ 211,990 $ 221,223 (4)%\\niPhone\\niPhone net sales were relatively flat during the second quarter of 2023 compared to the secon d quarter of 2022. Year-over-year iPhone net sales decreased\\nduring the first six months of 2023 due primarily to lower net sales from the Company’ s new iPhone models launched in the fourth quarter of 2022.\\nMac\\nMac net sales decreased during the second quarter and first six months of 2023 compared to the same periods in 2022 due primarily to lower net sales of\\nMacBook Pro.\\niPad\\niPad net sales decreased during the second quarter of 2023 compared to the second quarter of 2022 due primarily to lower net sales of iPad Pro and iPad Air.\\nYear-over-year iPad net sales increased during the first six months of 2023 due primarily to higher net sales of iPad, partially offset by lower net sales of iPad\\nmini .\\nWearables, Home and Accessories\\nWearables, Home and Accessories net sales were relatively flat during the second quarter of 2023 compared to the second quarter of 2022. Year-over-year\\nWearables, Home and Accessories net sales decreased during the first six months of 2023 due primarily to lower net sales of AirPods .\\nServices\\nServices net sales increased during the second quarter and first six months of 2023 compared to the same periods in 2022 due primarily to higher net sales from\\ncloud services, music and advertising.® ®\\n®\\n®\\nApple Inc. | Q2 2023 Form 10-Q | 16', 'Products and Services Performance\\nThe following table shows net sales by category for the three- and nine-month periods ended July 1, 2023 and June 25, 2022 (dollars in millions):\\nThree Months Ended Nine Months Ended\\nJuly 1,\\n2023June 25,\\n2022 ChangeJuly 1,\\n2023June 25,\\n2022 Change\\nNet sales by category:\\niPhone $ 39,669 $ 40,665 (2)%$ 156,778 $ 162,863 (4)%\\nMac 6,840 7,382 (7)% 21,743 28,669 (24)%\\niPad 5,791 7,224 (20)% 21,857 22,118 (1)%\\nWearables, Home and Accessories 8,284 8,084 2 % 30,523 31,591 (3)%\\nServices 21,213 19,604 8 % 62,886 58,941 7 %\\nTotal net sales $ 81,797 $ 82,959 (1)%$ 293,787 $ 304,182 (3)%\\niPhone\\niPhone net sales decreased during the third quarter and first nine months of 2023 compared to the same periods in 2022 due primarily to lower net sales from\\ncertain iPhone models, partially of fset by higher net sales of iPhone 14 Pro models.\\nMac\\nMac net sales decreased during the third quarter and first nine months of 2023 compared to the same periods in 2022 due primarily to lower net sales of laptops.\\niPad\\niPad net sales decreased during the third quarter of 2023 compared to the third quarter of 2022 due primarily to lower net sales across most iPad models. Year-\\nover-year iPad net sales were relatively flat during the first nine months of 2023.\\nWearables, Home and Accessories\\nWearables, Home and Accessories net sales increased during the third quarter of 2023 compare d to the third quarter of 2022 due primarily to higher net sales of\\nWearables, which includes AirPods , Apple Watch and Beats products, partially offset by lower net sales of accessories. Year-over-year Wearables, Home\\nand Accessories net sales decreased during the first nine months of 2023 due primarily to lower net sales of W earables and accessories.\\nServices\\nServices net sales increased during the third quarter of 2023 compared to the third quarter of 2022 due primarily to higher net sales from advertising, cloud\\nservices and the App Store . Year-over-year Services net sales increased during the first nine months of 2023 due primarily to higher net sales from cloud\\nservices, advertising and music.® ® ®\\n®\\nApple Inc. | Q3 2023 Form 10-Q | 16']\n",
+ "\n",
+ "# get the Faithfulness assessment for each claim\n",
+ "assessed_claims_faithfulness = assess_claims(query=query,\n",
+ " claims=claims,\n",
+ " context=retrieved_documents,\n",
+ " model=model,\n",
+ " client=client)\n",
+ "\n",
+ "print(f\"Assessment of the claims extracted from the model's response:\\n\\n{assessed_claims_faithfulness}\")"
+ ],
+ "metadata": {
+ "colab": {
+ "base_uri": "https://localhost:8080/"
+ },
+ "id": "5VmLE4yCwEMe",
+ "outputId": "10874986-1f91-45a3-c081-dc2aae3fd618"
+ },
+ "execution_count": null,
+ "outputs": [
+ {
+ "output_type": "stream",
+ "name": "stdout",
+ "text": [
+ "Assessment of the claims extracted from the model's response:\n",
+ "\n",
+ "- Apple's total net sales experienced a decline over the last year. SUPPORTED=1\n",
+ "- The three-month period ended July 1, 2023, saw a total net sale of $81,797 million. SUPPORTED=1\n",
+ "- This was a 1% decrease from the same period in 2022. SUPPORTED=1\n",
+ "- The nine-month period ended July 1, 2023, had a 3% decrease in net sales compared to the first nine months of 2022. SUPPORTED=1\n",
+ "- The downward trend continued into the three and six-month periods ending April 1, 2023. SUPPORTED=1\n",
+ "- Apple's total net sales decreased by 3% and 4% respectively, compared to the same periods in 2022. SUPPORTED=1\n"
+ ]
+ }
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "Great, we now have an assessment for each of the claims: in the last step, we just need to use these assessments to define the final score."
+ ],
+ "metadata": {
+ "id": "vBbLuSkO-iN7"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# given the list of claims and their label, compute the final score\n",
+ "# as the proportion of correct claims over the full list of claims\n",
+ "def get_final_score(claims_list):\n",
+ " supported = len(re.findall(\"SUPPORTED=1\", claims_list))\n",
+ " non_supported = len(re.findall(\"SUPPORTED=0\", claims_list))\n",
+ " score = supported / (supported+non_supported)\n",
+ " return round(score, 2)"
+ ],
+ "metadata": {
+ "id": "tFk-IxK_wDVT"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "score_faithfulness = get_final_score(assessed_claims_faithfulness)\n",
+ "print(f'Faithfulness: {score_faithfulness}')"
+ ],
+ "metadata": {
+ "colab": {
+ "base_uri": "https://localhost:8080/"
+ },
+ "id": "Z680DvFOW2Ea",
+ "outputId": "1666057d-7981-43d3-f4bb-c10fdc861cae"
+ },
+ "execution_count": null,
+ "outputs": [
+ {
+ "output_type": "stream",
+ "name": "stdout",
+ "text": [
+ "Faithfulness: 1.0\n"
+ ]
+ }
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "The final Faithfulness score is 1, which means that the model's response is fully grounded in the retrieved documents: that's a very good news :)\n",
+ "\n",
+ "Before moving on, let's modify the model's response by adding a piece of information which is **not** grounded in any document, and re-compute Faithfulness."
+ ],
+ "metadata": {
+ "id": "3_Ks8ELQrdQO"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# let's mess up the century, changing 2022 to 1922\n",
+ "modified_response = response.replace('2022', '1922')\n",
+ "\n",
+ "# extract the claims from the modified response\n",
+ "modified_claims = extract_claims(query, modified_response, model, client)\n",
+ "\n",
+ "# and get assess the modified claims\n",
+ "assessed_modified_claims = assess_claims(query=query,\n",
+ " claims=modified_claims,\n",
+ " context=retrieved_documents,\n",
+ " model=model,\n",
+ " client=client)\n",
+ "\n",
+ "print(f\"Assessment of the modified claims:\\n\\n{assessed_modified_claims}\\n\")\n",
+ "\n",
+ "score_faithfulness_modified_claims = get_final_score(assessed_modified_claims)\n",
+ "print(f'Faithfulness: {score_faithfulness_modified_claims}')"
+ ],
+ "metadata": {
+ "colab": {
+ "base_uri": "https://localhost:8080/"
+ },
+ "id": "YEZUA8XMAxLE",
+ "outputId": "9b6e59b1-a4f9-46bf-cdf8-a4f520462d12"
+ },
+ "execution_count": null,
+ "outputs": [
+ {
+ "output_type": "stream",
+ "name": "stdout",
+ "text": [
+ "Assessment of the modified claims:\n",
+ "\n",
+ "- Apple's total net sales experienced a decline over the last year. SUPPORTED=1\n",
+ "- The three-month period ended July 1, 2023, saw a total net sale of $81,797 million. SUPPORTED=1\n",
+ "- This was a 1% decrease from the same period in 1922. SUPPORTED=0\n",
+ "- The nine-month period ended July 1, 2023, had a 3% decrease in net sales compared to the first nine months of 1922. SUPPORTED=0\n",
+ "- The downward trend continued into the three and six-month periods ending April 1, 2023. SUPPORTED=1\n",
+ "- Apple's total net sales decreased by 3% and 4% respectively, compared to the same periods in 1922. SUPPORTED=0\n",
+ "\n",
+ "Faithfulness: 0.5\n"
+ ]
+ }
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "As you can see, by assessing claims one by one, we are able to spot **hallucinations**, that is, the (corrupted) cases in which the information provided by the model is not grounded in any of the retrieved documents."
+ ],
+ "metadata": {
+ "id": "TccmzE9TCQsN"
+ }
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "### Correctness\n",
+ "\n",
+ "As said, Faithfulness and Correctness share the same logic, the only difference being that we will check the claims against the gold answer. We can therefore repeat the process above, and just substitute the `context`."
+ ],
+ "metadata": {
+ "id": "bPK-MND1riQD"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# let's get the gold answer from the dataset\n",
+ "golden_answer = rag_dataset[idx].reference_answer\n",
+ "\n",
+ "# and check the claims in the response against the gold.\n",
+ "# note that assess_claims takes exactly the same args as with Faithfulness\n",
+ "# except for the context, that now is the golden_answer\n",
+ "assessed_claims_correctness = assess_claims(query=query,\n",
+ " claims=claims,\n",
+ " context=golden_answer, # note the different context\n",
+ " model=model,\n",
+ " client=client)\n",
+ "\n",
+ "\n",
+ "print(f\"Assess the claims extracted from the model's response against the golden answer:\\n\\n{assessed_claims_correctness}\")"
+ ],
+ "metadata": {
+ "colab": {
+ "base_uri": "https://localhost:8080/"
+ },
+ "id": "qVeQDVDYEr45",
+ "outputId": "a14a5b09-b8f7-4563-ade5-fe2cd1384cfb"
+ },
+ "execution_count": null,
+ "outputs": [
+ {
+ "output_type": "stream",
+ "name": "stdout",
+ "text": [
+ "Assess the claims extracted from the model's response against the golden answer:\n",
+ "\n",
+ "- Apple's total net sales experienced a decline over the last year. SUPPORTED=1\n",
+ "- The three-month period ended July 1, 2023, saw a total net sale of $81,797 million. SUPPORTED=1\n",
+ "- This was a 1% decrease from the same period in 2022. SUPPORTED=0\n",
+ "- The nine-month period ended July 1, 2023, had a 3% decrease in net sales compared to the first nine months of 2022. SUPPORTED=0\n",
+ "- The downward trend continued into the three and six-month periods ending April 1, 2023. SUPPORTED=1\n",
+ "- Apple's total net sales decreased by 3% and 4% respectively, compared to the same periods in 2022. SUPPORTED=0\n"
+ ]
+ }
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "As mentioned above, automatic evaluation is a hard task, and even when using powerful models, claim assessment can present problems: for example, the third claim is labelled as 0, even if it might be inferred from the information in the gold answer."
+ ],
+ "metadata": {
+ "id": "V4pCRJ0qPQWT"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# we can now compute the final Correctness score\n",
+ "score_correctness = get_final_score(assessed_claims_correctness)\n",
+ "print(f'Correctness: {score_correctness}')"
+ ],
+ "metadata": {
+ "colab": {
+ "base_uri": "https://localhost:8080/"
+ },
+ "id": "nObA30PzxQha",
+ "outputId": "2e354428-ea91-4c7e-94b6-31433018058c"
+ },
+ "execution_count": null,
+ "outputs": [
+ {
+ "output_type": "stream",
+ "name": "stdout",
+ "text": [
+ "Correctness: 0.5\n"
+ ]
+ }
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "For Correctness, we found that only half of the claims in the generated response are found in the gold answer. Note that this is not necessarily an issue: reference answers are often non-exhaustive, especially in dataset including open-ended questions, like the one we are considering in this post, and *both* the generated and golden answer can include relevant information.\n"
+ ],
+ "metadata": {
+ "id": "4Nd0bQ3rR1c3"
+ }
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "### Coverage\n",
+ "\n",
+ "We finally move to Coverage. Remember that, in this case, we want to check how many of the claims *in the gold answer* are included in the generated response. To do it, we first need to extract the claims from the gold answer."
+ ],
+ "metadata": {
+ "id": "YhsEJDNXR8OY"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# let's extract the golden claims\n",
+ "gold_claims = extract_claims(query, golden_answer, model, client)\n",
+ "\n",
+ "print(f\"List of claims extracted from the gold answer:\\n\\n{gold_claims}\")"
+ ],
+ "metadata": {
+ "colab": {
+ "base_uri": "https://localhost:8080/"
+ },
+ "id": "wJeHJhiAxQjy",
+ "outputId": "8da8edf0-5605-4548-aab4-002d0202ee88"
+ },
+ "execution_count": null,
+ "outputs": [
+ {
+ "output_type": "stream",
+ "name": "stdout",
+ "text": [
+ "List of claims extracted from the gold answer:\n",
+ "\n",
+ "- For the quarterly period ended June 25, 2022, the total net sales were $82,959 million.\n",
+ "- For the quarterly period ended December 31, 2022, the total net sales were $117,154 million.\n",
+ "- For the quarterly period ended April 1, 2023, the total net sales were $94,836 million.\n",
+ "- For the quarterly period ended July 1, 2023, the total net sales were $81,797 million.\n",
+ "- There was an increase in total net sales from the quarter ended June 25, 2022, to the quarter ended December 31, 2022.\n",
+ "- There was a decrease in total net sales in the quarters ended April 1, 2023, and July 1, 2023.\n"
+ ]
+ }
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "Then, we check which of these claims is present in the response generated by the model."
+ ],
+ "metadata": {
+ "id": "6VXxxrL1SuFv"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# note that in, this case, the context is the model's response\n",
+ "assessed_claims_coverage = assess_claims(query=query,\n",
+ " claims=gold_claims,\n",
+ " context=response,\n",
+ " model=model,\n",
+ " client=client)\n",
+ "\n",
+ "\n",
+ "print(f\"Assess which of the gold claims is in the model's response:\\n\\n{assessed_claims_coverage}\")"
+ ],
+ "metadata": {
+ "colab": {
+ "base_uri": "https://localhost:8080/"
+ },
+ "id": "Ro-sYsp5SnOo",
+ "outputId": "d334a72e-a87a-4900-b9e6-9a6c2140dffb"
+ },
+ "execution_count": null,
+ "outputs": [
+ {
+ "output_type": "stream",
+ "name": "stdout",
+ "text": [
+ "Assess which of the gold claims is in the model's response:\n",
+ "\n",
+ "- For the quarterly period ended June 25, 2022, the total net sales were $82,959 million. SUPPORTED=0\n",
+ "- For the quarterly period ended December 31, 2022, the total net sales were $117,154 million. SUPPORTED=0\n",
+ "- For the quarterly period ended April 1, 2023, the total net sales were $94,836 million. SUPPORTED=0\n",
+ "- For the quarterly period ended July 1, 2023, the total net sales were $81,797 million. SUPPORTED=1\n",
+ "- There was an increase in total net sales from the quarter ended June 25, 2022, to the quarter ended December 31, 2022. SUPPORTED=0\n",
+ "- There was a decrease in total net sales in the quarters ended April 1, 2023, and July 1, 2023. SUPPORTED=1\n"
+ ]
+ }
+ ]
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# we compute the final Coverage score\n",
+ "score_coverage = get_final_score(assessed_claims_coverage)\n",
+ "print(f'Coverage: {score_coverage}')"
+ ],
+ "metadata": {
+ "colab": {
+ "base_uri": "https://localhost:8080/"
+ },
+ "id": "FUvVUN8qSnG2",
+ "outputId": "e854432a-f13b-4596-fb7c-f9b51ee870ec"
+ },
+ "execution_count": null,
+ "outputs": [
+ {
+ "output_type": "stream",
+ "name": "stdout",
+ "text": [
+ "Coverage: 0.33\n"
+ ]
+ }
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "The Coverage score is telling us that 1/3 of the information in the gold answer is present in the generated answer. This is a useful information, that, similarly to what said above regarding Correctness, can raise further questions, such as: is it acceptable to have diverging information in the generated answer? Is any crucial piece of information missing in the generated answer?\n",
+ "\n",
+ "The answer to these questions is use case-specific, and has to be made by the end user: The claim-based approach implemented here supports the user by providing a clear and detailed view on what the model is assessing and how."
+ ],
+ "metadata": {
+ "id": "pduSBr2-U_R7"
+ }
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "\n",
+ "## Final Comments\n",
+ "\n",
+ "RAG evaluation is a hard task, especially the evaluation of the generated response. In this notebook we offer a clear, robust and replicable approach to evaluation, on which you can build on to build your evaluation pipeline."
+ ],
+ "metadata": {
+ "id": "0-1FsN2dHzMS"
+ }
+ }
+ ]
+}
diff --git a/notebooks/.ipynb_checkpoints/Document_Parsing_For_Enterprises-checkpoint.ipynb b/notebooks/.ipynb_checkpoints/Document_Parsing_For_Enterprises-checkpoint.ipynb
new file mode 100644
index 00000000..9a80a714
--- /dev/null
+++ b/notebooks/.ipynb_checkpoints/Document_Parsing_For_Enterprises-checkpoint.ipynb
@@ -0,0 +1,1830 @@
+{
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "ssw34RVKHmsJ"
+ },
+ "source": [
+ "# Advanced Document Parsing For Enterprises"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "b2oYelmWW35Q"
+ },
+ "source": [
+ "## Introduction\n",
+ "\n",
+ "The bread and butter of natural language processing technology is text. Once we can reduce a set of data into text, we can do all kinds of things with it: question answering, summarization, classification, sentiment analysis, searching and indexing, and more.\n",
+ " \n",
+ " \n",
+ "In the context of enterprise Retrieval Augmented Generation (RAG), the information is often locked in complex file types such as PDFs. These formats are made for sharing information between humans, but not so much with language models.\n",
+ " \n",
+ " \n",
+ "In this notebook, we will use a real-world pharmaceutical drug label to test out various performant approaches to parsing PDFs. This will allow us to use [Cohere's Command-R model](https://txt.cohere.com/command-r/) in a RAG setting to answer questions and asks about this label, such as \"I need a succinct summary of the compound name, indication, route of administration, and mechanism of action of\" a given pharmaceutical.\n",
+ " \n",
+ " \n",
+ ""
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "f-mq1FCojI2p"
+ },
+ "source": [
+ "\n",
+ "## PDF Parsing\n",
+ "\n",
+ "We will go over five proprietary as well as open source options for processing PDFs. The parsing mechanisms demonstrated in the following sections are\n",
+ "- [Google Document AI](#gcp)\n",
+ "- [AWS Textract](#aws)\n",
+ "- [Unstructured.io](#unstructured)\n",
+ "- [LlamaParse](#llama)\n",
+ "- [pdf2image + pytesseract](#pdf2image)\n",
+ "\n",
+ "By way of example, we will be parsing a [21-page PDF](https://www.accessdata.fda.gov/drugsatfda_docs/label/2023/215500s000lbl.pdf) containing the label for a recent FDA drug approval, the beginning of which is shown below. Then, we will perform a series of basic RAG tasks with our different parsings and evaluate their performance.\n",
+ "\n",
+ ""
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "lB0o4L4jh5Sv"
+ },
+ "source": [
+ "## Getting Set Up\n",
+ "\n",
+ "Before we dive into the technical weeds, we need to set up the notebook's runtime and filesystem environments. The code cells below do the following:\n",
+ "- Install required libraries (note - we assume a Ubuntu/Debian runtime below, but other environments will work as well so long as `tesseract` and `poppler` can be installed. Note that the exact names of the packages may vary. These are only needed for Solution 5.)\n",
+ "- Confirm that data dependencies from the GitHub repo have been downloaded. These will be under `data/document-parsing` and contain the following:\n",
+ " - the PDF document that we will be working with, `fda-approved-drug.pdf` (this can also be found here: https://www.accessdata.fda.gov/drugsatfda_docs/label/2023/215500s000lbl.pdf)\n",
+ " - precomputed parsed documents for each parsing solution. While the point of this notebook is to illustrate how this is done, we provide the parsed final results to allow readers to skip ahead to the RAG section without having to set up the required infrastructure for each solution.)\n",
+ "- Add utility functions needed for later sections"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "ZP1v0aF6Ij3U"
+ },
+ "outputs": [],
+ "source": [
+ "%%capture\n",
+ "! sudo apt install tesseract-ocr poppler-utils\n",
+ "! pip install cohere fsspec hnswlib google-cloud-documentai google-cloud-storage boto3 langchain-text-splitters llama_parse pytesseract pdf2image pandas\n"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "A5yWsKDSL_kY"
+ },
+ "outputs": [],
+ "source": [
+ "data_dir = \"data/document-parsing\"\n",
+ "source_filename = \"example-drug-label\"\n",
+ "extension = \"pdf\""
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "from pathlib import Path\n",
+ "\n",
+ "sources = [\"gcp\", \"aws\", \"unstructured-io\", \"llamaparse-text\", \"llamaparse-markdown\", \"pytesseract\"]\n",
+ "\n",
+ "filenames = [\"{}-parsed-fda-approved-drug.txt\".format(source) for source in sources]\n",
+ "filenames.append(\"fda-approved-drug.pdf\")\n",
+ "\n",
+ "for filename in filenames: \n",
+ " file_path = Path(f\"{data_dir}/{filename}\")\n",
+ " if file_path.is_file() == False:\n",
+ " print(f\"File {filename} not found at {data_dir}!\")"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "BoM6-Tq-Sm-1"
+ },
+ "source": [
+ "### Utility Functions\n",
+ "Make sure to include the notebook's utility functions in the runtime."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "jl7BJsvdSrr5"
+ },
+ "outputs": [],
+ "source": [
+ "def store_document(path: str, doc_content: str):\n",
+ " with open(path, 'w') as f:\n",
+ " f.write(doc_content)"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "R2u-drbt7SOQ"
+ },
+ "outputs": [],
+ "source": [
+ "import json\n",
+ "\n",
+ "def insert_citations_in_order(text, citations, documents):\n",
+ " \"\"\"\n",
+ " A helper function to pretty print citations.\n",
+ " \"\"\"\n",
+ "\n",
+ " citations_reference = {}\n",
+ " for index, doc in enumerate(documents):\n",
+ " citations_reference[index] = doc\n",
+ "\n",
+ " offset = 0\n",
+ " # Process citations in the order they were provided\n",
+ " for citation in citations:\n",
+ " # Adjust start/end with offset\n",
+ " start, end = citation['start'] + offset, citation['end'] + offset\n",
+ " citation_numbers = []\n",
+ " for doc_id in citation[\"document_ids\"]:\n",
+ " for citation_index, doc in citations_reference.items():\n",
+ " if doc[\"id\"] == doc_id:\n",
+ " citation_numbers.append(citation_index)\n",
+ " references = \"(\" + \", \".join(\"[{}]\".format(num) for num in citation_numbers) + \")\"\n",
+ " modification = f'{text[start:end]} {references}'\n",
+ " # Replace the cited text with its bolded version + placeholder\n",
+ " text = text[:start] + modification + text[end:]\n",
+ " # Update the offset for subsequent replacements\n",
+ " offset += len(modification) - (end - start)\n",
+ "\n",
+ " # Add the citations at the bottom of the text\n",
+ " text_with_citations = f'{text}'\n",
+ " citations_reference = [\"[{}]: {}\".format(x[\"id\"], x[\"text\"]) for x in citations_reference.values()]\n",
+ "\n",
+ " return text_with_citations, \"\\n\".join(citations_reference)"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "SULVG_HwKdm5"
+ },
+ "outputs": [],
+ "source": [
+ "def format_docs_for_chat(documents):\n",
+ " return [{\"id\": str(index), \"text\": x} for index, x in enumerate(documents)]"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "y-VohvR3X6S6"
+ },
+ "source": [
+ "## Document Parsing Solutions\n",
+ "\n",
+ "For demonstration purposes, we have collected and saved the parsed documents from each solution in this notebook. Skip to the [next section](#document-questions) to run RAG with Command-R on the pre-fetched versions. You can find all parsed resources in detail at the link [here](https://github.com/gchatz22/temp-cohere-resources/tree/main/data)."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "jLtrGjXiJE9M",
+ "jp-MarkdownHeadingCollapsed": true
+ },
+ "source": [
+ "\n",
+ "### Solution 1: Google Cloud Document AI [[Back to Solutions]](#top)\n",
+ "\n",
+ "Document AI helps developers create high-accuracy processors to extract, classify, and split documents.\n",
+ " \n",
+ "External documentation: https://cloud.google.com/document-ai"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "cxwJ_jZpgNDo"
+ },
+ "source": [
+ "#### Parsing the document\n",
+ "\n",
+ "The following block can be executed in one of two ways\n",
+ "1. Inside a Google Vertex AI environment\n",
+ " - No authentication is needed\n",
+ "2. Inside the notebook\n",
+ " - Authentication needed\n",
+ " - There are pointers inside the code on which lines to uncomment in order to achieve that\n",
+ " \n",
+ "**Note: You can skip to the next block if you want to use the pre-existing parsed version.**"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "uZdjNlfxJEXv"
+ },
+ "outputs": [],
+ "source": [
+ "\"\"\"\n",
+ "Extracted from https://cloud.google.com/document-ai/docs/samples/documentai-batch-process-document\n",
+ "\"\"\"\n",
+ "\n",
+ "import re\n",
+ "from typing import Optional\n",
+ "\n",
+ "from google.api_core.client_options import ClientOptions\n",
+ "from google.api_core.exceptions import InternalServerError\n",
+ "from google.api_core.exceptions import RetryError\n",
+ "from google.cloud import documentai # type: ignore\n",
+ "from google.cloud import storage\n",
+ "\n",
+ "project_id = \"\"\n",
+ "location = \"\"\n",
+ "processor_id = \"\"\n",
+ "gcs_output_uri = \"\"\n",
+ "# credentials_file = \"populate if you are running in a non Vertex AI environment.\"\n",
+ "gcs_input_prefix = \"\"\n",
+ "\n",
+ "\n",
+ "def batch_process_documents(\n",
+ " project_id: str,\n",
+ " location: str,\n",
+ " processor_id: str,\n",
+ " gcs_output_uri: str,\n",
+ " gcs_input_prefix: str,\n",
+ " timeout: int = 400\n",
+ ") -> None:\n",
+ " parsed_documents = []\n",
+ "\n",
+ " # Client configs\n",
+ " opts = ClientOptions(api_endpoint=f\"{location}-documentai.googleapis.com\")\n",
+ " # With credentials\n",
+ " # opts = ClientOptions(api_endpoint=f\"{location}-documentai.googleapis.com\", credentials_file=credentials_file)\n",
+ "\n",
+ " client = documentai.DocumentProcessorServiceClient(client_options=opts)\n",
+ " processor_name = client.processor_path(project_id, location, processor_id)\n",
+ "\n",
+ " # Input storage configs\n",
+ " gcs_prefix = documentai.GcsPrefix(gcs_uri_prefix=gcs_input_prefix)\n",
+ " input_config = documentai.BatchDocumentsInputConfig(gcs_prefix=gcs_prefix)\n",
+ "\n",
+ " # Output storage configs\n",
+ " gcs_output_config = documentai.DocumentOutputConfig.GcsOutputConfig(gcs_uri=gcs_output_uri, field_mask=None)\n",
+ " output_config = documentai.DocumentOutputConfig(gcs_output_config=gcs_output_config)\n",
+ " storage_client = storage.Client()\n",
+ " # With credentials\n",
+ " # storage_client = storage.Client.from_service_account_json(json_credentials_path=credentials_file)\n",
+ "\n",
+ " # Batch process docs request\n",
+ " request = documentai.BatchProcessRequest(\n",
+ " name=processor_name,\n",
+ " input_documents=input_config,\n",
+ " document_output_config=output_config,\n",
+ " )\n",
+ "\n",
+ " # batch_process_documents returns a long running operation\n",
+ " operation = client.batch_process_documents(request)\n",
+ "\n",
+ " # Continually polls the operation until it is complete.\n",
+ " # This could take some time for larger files\n",
+ " try:\n",
+ " print(f\"Waiting for operation {operation.operation.name} to complete...\")\n",
+ " operation.result(timeout=timeout)\n",
+ " except (RetryError, InternalServerError) as e:\n",
+ " print(e.message)\n",
+ "\n",
+ " # Get output document information from completed operation metadata\n",
+ " metadata = documentai.BatchProcessMetadata(operation.metadata)\n",
+ " if metadata.state != documentai.BatchProcessMetadata.State.SUCCEEDED:\n",
+ " raise ValueError(f\"Batch Process Failed: {metadata.state_message}\")\n",
+ "\n",
+ " print(\"Output files:\")\n",
+ " # One process per Input Document\n",
+ " for process in list(metadata.individual_process_statuses):\n",
+ " matches = re.match(r\"gs://(.*?)/(.*)\", process.output_gcs_destination)\n",
+ " if not matches:\n",
+ " print(\"Could not parse output GCS destination:\", process.output_gcs_destination)\n",
+ " continue\n",
+ "\n",
+ " output_bucket, output_prefix = matches.groups()\n",
+ " output_blobs = storage_client.list_blobs(output_bucket, prefix=output_prefix)\n",
+ "\n",
+ " # Document AI may output multiple JSON files per source file\n",
+ " # (Large documents get split in multiple file \"versions\" doc --> parsed_doc_0 + parsed_doc_1 ...)\n",
+ " for blob in output_blobs:\n",
+ " # Document AI should only output JSON files to GCS\n",
+ " if blob.content_type != \"application/json\":\n",
+ " print(f\"Skipping non-supported file: {blob.name} - Mimetype: {blob.content_type}\")\n",
+ " continue\n",
+ "\n",
+ " # Download JSON file as bytes object and convert to Document Object\n",
+ " print(f\"Fetching {blob.name}\")\n",
+ " document = documentai.Document.from_json(blob.download_as_bytes(), ignore_unknown_fields=True)\n",
+ " # Store the filename and the parsed versioned document content as a tuple\n",
+ " parsed_documents.append((blob.name.split(\"/\")[-1].split(\".\")[0], document.text))\n",
+ "\n",
+ " print(\"Finished document parsing process.\")\n",
+ " return parsed_documents\n",
+ "\n",
+ "# Call service\n",
+ "# versioned_parsed_documents = batch_process_documents(\n",
+ "# project_id=project_id,\n",
+ "# location=location,\n",
+ "# processor_id=processor_id,\n",
+ "# gcs_output_uri=gcs_output_uri,\n",
+ "# gcs_input_prefix=gcs_input_prefix\n",
+ "# )"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "colab": {
+ "base_uri": "https://localhost:8080/",
+ "height": 35
+ },
+ "id": "CT4QwMTwLPad",
+ "outputId": "1c85c3aa-a3b5-48c4-bb6e-5033c2c604a0"
+ },
+ "outputs": [
+ {
+ "data": {
+ "application/vnd.google.colaboratory.intrinsic+json": {
+ "type": "string"
+ },
+ "text/plain": [
+ "'\\nPost process parsed document and store it locally.\\nMake sure to run in Google Vertex AI environment or include a credentials file.\\n'"
+ ]
+ },
+ "execution_count": 10,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "\"\"\"\n",
+ "Post process parsed document and store it locally.\n",
+ "Make sure to run in Google Vertex AI environment or include a credentials file.\n",
+ "\"\"\"\n",
+ "\n",
+ "# from pathlib import Path\n",
+ "# from collections import defaultdict\n",
+ "\n",
+ "# parsed_documents = []\n",
+ "# combined_versioned_parsed_documents = defaultdict(list)\n",
+ "\n",
+ "# # Assemble versioned documents together ({\"doc_name\": [(0, doc_content_0), (1, doc_content_1), ...]}).\n",
+ "# for filename, doc_content in versioned_parsed_documents:\n",
+ "# filename, version = \"-\".join(filename.split(\"-\")[:-1]), filename.split(\"-\")[-1]\n",
+ "# combined_versioned_parsed_documents[filename].append((version, doc_content))\n",
+ "\n",
+ "# # Sort documents by version and join the content together.\n",
+ "# for filename, docs in combined_versioned_parsed_documents.items():\n",
+ "# doc_content = \" \".join([x[1] for x in sorted(docs, key=lambda x: x[0])])\n",
+ "# parsed_documents.append((filename, doc_content))\n",
+ "\n",
+ "# # Store parsed documents in local storage.\n",
+ "# for filename, doc_content in parsed_documents:\n",
+ "# file_path = \"{}/{}-parsed-{}.txt\".format(data_dir, \"gcp\", source_filename)\n",
+ "# store_document(file_path, doc_content)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "dmkya9saFN8R"
+ },
+ "source": [
+ "#### Visualize the parsed document"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "S6SMCDb1FhqZ"
+ },
+ "outputs": [],
+ "source": [
+ "filename = \"gcp-parsed-{}.txt\".format(source_filename)\n",
+ "with open(\"{}/{}\".format(data_dir, filename), \"r\") as doc:\n",
+ " parsed_document = doc.read()\n",
+ "\n",
+ "print(parsed_document[:1000])"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "dlRxuddi5w0E",
+ "jp-MarkdownHeadingCollapsed": true
+ },
+ "source": [
+ "\n",
+ "### Solution 2: AWS Textract [[Back to Solutions]](#top)\n",
+ "\n",
+ "[Amazon Textract](https://aws.amazon.com/textract/) is an OCR service offered by AWS. It can detect text, forms, tables, and more in PDFs and images. In this section, we go over how to use Textract's asynchronous API.\n",
+ " \n",
+ " "
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "1YCszLJnge4j"
+ },
+ "source": [
+ "#### Parsing the document"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "vFBasw782Gho"
+ },
+ "source": [
+ "We assume you are working within the AWS ecosystem (from a SageMaker notebook, EC2 instance, a Lambda function, ...) with valid credentials. Much of the code here is from supplemental materials created by AWS and offered here:\n",
+ "\n",
+ "- https://github.com/awsdocs/aws-doc-sdk-examples/tree/main/python/example_code/textract\n",
+ "- https://github.com/aws-samples/textract-paragraph-identification/tree/main\n",
+ "\n",
+ "At minimum, you will need access to the following AWS resources to get started:\n",
+ "\n",
+ "- Textract\n",
+ "- an S3 bucket containing the document(s) to process - in this case, our `example-drug-label.pdf` file\n",
+ "- an SNS topic that Textract can publish to. This is used to send a notification that parsing is complete.\n",
+ "- an IAM role that Textract will assume, granting access to the S3 bucket and SNS topic\n",
+ "\n",
+ "First, we bring in the `TextractWrapper` class provided in the [AWS Code Examples repository](https://github.com/awsdocs/aws-doc-sdk-examples/blob/main/python/example_code/textract/textract_wrapper.py). This class makes it simpler to interface with the Textract service."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "casiElly3G2C"
+ },
+ "outputs": [],
+ "source": [
+ "# source: https://github.com/awsdocs/aws-doc-sdk-examples/tree/main/python/example_code/textract\n",
+ "\n",
+ "# Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.\n",
+ "# SPDX-License-Identifier: Apache-2.0\n",
+ "\n",
+ "\"\"\"\n",
+ "Purpose\n",
+ "\n",
+ "Shows how to use the AWS SDK for Python (Boto3) with Amazon Textract to\n",
+ "detect text, form, and table elements in document images.\n",
+ "\"\"\"\n",
+ "\n",
+ "import json\n",
+ "import logging\n",
+ "from botocore.exceptions import ClientError\n",
+ "\n",
+ "logger = logging.getLogger(__name__)\n",
+ "\n",
+ "\n",
+ "# snippet-start:[python.example_code.textract.TextractWrapper]\n",
+ "class TextractWrapper:\n",
+ " \"\"\"Encapsulates Textract functions.\"\"\"\n",
+ "\n",
+ " def __init__(self, textract_client, s3_resource, sqs_resource):\n",
+ " \"\"\"\n",
+ " :param textract_client: A Boto3 Textract client.\n",
+ " :param s3_resource: A Boto3 Amazon S3 resource.\n",
+ " :param sqs_resource: A Boto3 Amazon SQS resource.\n",
+ " \"\"\"\n",
+ " self.textract_client = textract_client\n",
+ " self.s3_resource = s3_resource\n",
+ " self.sqs_resource = sqs_resource\n",
+ "\n",
+ " # snippet-end:[python.example_code.textract.TextractWrapper]\n",
+ "\n",
+ " # snippet-start:[python.example_code.textract.DetectDocumentText]\n",
+ " def detect_file_text(self, *, document_file_name=None, document_bytes=None):\n",
+ " \"\"\"\n",
+ " Detects text elements in a local image file or from in-memory byte data.\n",
+ " The image must be in PNG or JPG format.\n",
+ "\n",
+ " :param document_file_name: The name of a document image file.\n",
+ " :param document_bytes: In-memory byte data of a document image.\n",
+ " :return: The response from Amazon Textract, including a list of blocks\n",
+ " that describe elements detected in the image.\n",
+ " \"\"\"\n",
+ " if document_file_name is not None:\n",
+ " with open(document_file_name, \"rb\") as document_file:\n",
+ " document_bytes = document_file.read()\n",
+ " try:\n",
+ " response = self.textract_client.detect_document_text(\n",
+ " Document={\"Bytes\": document_bytes}\n",
+ " )\n",
+ " logger.info(\"Detected %s blocks.\", len(response[\"Blocks\"]))\n",
+ " except ClientError:\n",
+ " logger.exception(\"Couldn't detect text.\")\n",
+ " raise\n",
+ " else:\n",
+ " return response\n",
+ "\n",
+ " # snippet-end:[python.example_code.textract.DetectDocumentText]\n",
+ "\n",
+ " # snippet-start:[python.example_code.textract.AnalyzeDocument]\n",
+ " def analyze_file(\n",
+ " self, feature_types, *, document_file_name=None, document_bytes=None\n",
+ " ):\n",
+ " \"\"\"\n",
+ " Detects text and additional elements, such as forms or tables, in a local image\n",
+ " file or from in-memory byte data.\n",
+ " The image must be in PNG or JPG format.\n",
+ "\n",
+ " :param feature_types: The types of additional document features to detect.\n",
+ " :param document_file_name: The name of a document image file.\n",
+ " :param document_bytes: In-memory byte data of a document image.\n",
+ " :return: The response from Amazon Textract, including a list of blocks\n",
+ " that describe elements detected in the image.\n",
+ " \"\"\"\n",
+ " if document_file_name is not None:\n",
+ " with open(document_file_name, \"rb\") as document_file:\n",
+ " document_bytes = document_file.read()\n",
+ " try:\n",
+ " response = self.textract_client.analyze_document(\n",
+ " Document={\"Bytes\": document_bytes}, FeatureTypes=feature_types\n",
+ " )\n",
+ " logger.info(\"Detected %s blocks.\", len(response[\"Blocks\"]))\n",
+ " except ClientError:\n",
+ " logger.exception(\"Couldn't detect text.\")\n",
+ " raise\n",
+ " else:\n",
+ " return response\n",
+ "\n",
+ " # snippet-end:[python.example_code.textract.AnalyzeDocument]\n",
+ "\n",
+ " # snippet-start:[python.example_code.textract.helper.prepare_job]\n",
+ " def prepare_job(self, bucket_name, document_name, document_bytes):\n",
+ " \"\"\"\n",
+ " Prepares a document image for an asynchronous detection job by uploading\n",
+ " the image bytes to an Amazon S3 bucket. Amazon Textract must have permission\n",
+ " to read from the bucket to process the image.\n",
+ "\n",
+ " :param bucket_name: The name of the Amazon S3 bucket.\n",
+ " :param document_name: The name of the image stored in Amazon S3.\n",
+ " :param document_bytes: The image as byte data.\n",
+ " \"\"\"\n",
+ " try:\n",
+ " bucket = self.s3_resource.Bucket(bucket_name)\n",
+ " bucket.upload_fileobj(document_bytes, document_name)\n",
+ " logger.info(\"Uploaded %s to %s.\", document_name, bucket_name)\n",
+ " except ClientError:\n",
+ " logger.exception(\"Couldn't upload %s to %s.\", document_name, bucket_name)\n",
+ " raise\n",
+ "\n",
+ " # snippet-end:[python.example_code.textract.helper.prepare_job]\n",
+ "\n",
+ " # snippet-start:[python.example_code.textract.helper.check_job_queue]\n",
+ " def check_job_queue(self, queue_url, job_id):\n",
+ " \"\"\"\n",
+ " Polls an Amazon SQS queue for messages that indicate a specified Textract\n",
+ " job has completed.\n",
+ "\n",
+ " :param queue_url: The URL of the Amazon SQS queue to poll.\n",
+ " :param job_id: The ID of the Textract job.\n",
+ " :return: The status of the job.\n",
+ " \"\"\"\n",
+ " status = None\n",
+ " try:\n",
+ " queue = self.sqs_resource.Queue(queue_url)\n",
+ " messages = queue.receive_messages()\n",
+ " if messages:\n",
+ " msg_body = json.loads(messages[0].body)\n",
+ " msg = json.loads(msg_body[\"Message\"])\n",
+ " if msg.get(\"JobId\") == job_id:\n",
+ " messages[0].delete()\n",
+ " status = msg.get(\"Status\")\n",
+ " logger.info(\n",
+ " \"Got message %s with status %s.\", messages[0].message_id, status\n",
+ " )\n",
+ " else:\n",
+ " logger.info(\"No messages in queue %s.\", queue_url)\n",
+ " except ClientError:\n",
+ " logger.exception(\"Couldn't get messages from queue %s.\", queue_url)\n",
+ " else:\n",
+ " return status\n",
+ "\n",
+ " # snippet-end:[python.example_code.textract.helper.check_job_queue]\n",
+ "\n",
+ " # snippet-start:[python.example_code.textract.StartDocumentTextDetection]\n",
+ " def start_detection_job(\n",
+ " self, bucket_name, document_file_name, sns_topic_arn, sns_role_arn\n",
+ " ):\n",
+ " \"\"\"\n",
+ " Starts an asynchronous job to detect text elements in an image stored in an\n",
+ " Amazon S3 bucket. Textract publishes a notification to the specified Amazon SNS\n",
+ " topic when the job completes.\n",
+ " The image must be in PNG, JPG, or PDF format.\n",
+ "\n",
+ " :param bucket_name: The name of the Amazon S3 bucket that contains the image.\n",
+ " :param document_file_name: The name of the document image stored in Amazon S3.\n",
+ " :param sns_topic_arn: The Amazon Resource Name (ARN) of an Amazon SNS topic\n",
+ " where the job completion notification is published.\n",
+ " :param sns_role_arn: The ARN of an AWS Identity and Access Management (IAM)\n",
+ " role that can be assumed by Textract and grants permission\n",
+ " to publish to the Amazon SNS topic.\n",
+ " :return: The ID of the job.\n",
+ " \"\"\"\n",
+ " try:\n",
+ " response = self.textract_client.start_document_text_detection(\n",
+ " DocumentLocation={\n",
+ " \"S3Object\": {\"Bucket\": bucket_name, \"Name\": document_file_name}\n",
+ " },\n",
+ " NotificationChannel={\n",
+ " \"SNSTopicArn\": sns_topic_arn,\n",
+ " \"RoleArn\": sns_role_arn,\n",
+ " },\n",
+ " )\n",
+ " job_id = response[\"JobId\"]\n",
+ " logger.info(\n",
+ " \"Started text detection job %s on %s.\", job_id, document_file_name\n",
+ " )\n",
+ " except ClientError:\n",
+ " logger.exception(\"Couldn't detect text in %s.\", document_file_name)\n",
+ " raise\n",
+ " else:\n",
+ " return job_id\n",
+ "\n",
+ " # snippet-end:[python.example_code.textract.StartDocumentTextDetection]\n",
+ "\n",
+ " # snippet-start:[python.example_code.textract.GetDocumentTextDetection]\n",
+ " def get_detection_job(self, job_id):\n",
+ " \"\"\"\n",
+ " Gets data for a previously started text detection job.\n",
+ "\n",
+ " :param job_id: The ID of the job to retrieve.\n",
+ " :return: The job data, including a list of blocks that describe elements\n",
+ " detected in the image.\n",
+ " \"\"\"\n",
+ " try:\n",
+ " response = self.textract_client.get_document_text_detection(JobId=job_id)\n",
+ " job_status = response[\"JobStatus\"]\n",
+ " logger.info(\"Job %s status is %s.\", job_id, job_status)\n",
+ " except ClientError:\n",
+ " logger.exception(\"Couldn't get data for job %s.\", job_id)\n",
+ " raise\n",
+ " else:\n",
+ " return response\n",
+ "\n",
+ " # snippet-end:[python.example_code.textract.GetDocumentTextDetection]\n",
+ "\n",
+ " # snippet-start:[python.example_code.textract.StartDocumentAnalysis]\n",
+ " def start_analysis_job(\n",
+ " self,\n",
+ " bucket_name,\n",
+ " document_file_name,\n",
+ " feature_types,\n",
+ " sns_topic_arn,\n",
+ " sns_role_arn,\n",
+ " ):\n",
+ " \"\"\"\n",
+ " Starts an asynchronous job to detect text and additional elements, such as\n",
+ " forms or tables, in an image stored in an Amazon S3 bucket. Textract publishes\n",
+ " a notification to the specified Amazon SNS topic when the job completes.\n",
+ " The image must be in PNG, JPG, or PDF format.\n",
+ "\n",
+ " :param bucket_name: The name of the Amazon S3 bucket that contains the image.\n",
+ " :param document_file_name: The name of the document image stored in Amazon S3.\n",
+ " :param feature_types: The types of additional document features to detect.\n",
+ " :param sns_topic_arn: The Amazon Resource Name (ARN) of an Amazon SNS topic\n",
+ " where job completion notification is published.\n",
+ " :param sns_role_arn: The ARN of an AWS Identity and Access Management (IAM)\n",
+ " role that can be assumed by Textract and grants permission\n",
+ " to publish to the Amazon SNS topic.\n",
+ " :return: The ID of the job.\n",
+ " \"\"\"\n",
+ " try:\n",
+ " response = self.textract_client.start_document_analysis(\n",
+ " DocumentLocation={\n",
+ " \"S3Object\": {\"Bucket\": bucket_name, \"Name\": document_file_name}\n",
+ " },\n",
+ " NotificationChannel={\n",
+ " \"SNSTopicArn\": sns_topic_arn,\n",
+ " \"RoleArn\": sns_role_arn,\n",
+ " },\n",
+ " FeatureTypes=feature_types,\n",
+ " )\n",
+ " job_id = response[\"JobId\"]\n",
+ " logger.info(\n",
+ " \"Started text analysis job %s on %s.\", job_id, document_file_name\n",
+ " )\n",
+ " except ClientError:\n",
+ " logger.exception(\"Couldn't analyze text in %s.\", document_file_name)\n",
+ " raise\n",
+ " else:\n",
+ " return job_id\n",
+ "\n",
+ " # snippet-end:[python.example_code.textract.StartDocumentAnalysis]\n",
+ "\n",
+ " # snippet-start:[python.example_code.textract.GetDocumentAnalysis]\n",
+ " def get_analysis_job(self, job_id):\n",
+ " \"\"\"\n",
+ " Gets data for a previously started detection job that includes additional\n",
+ " elements.\n",
+ "\n",
+ " :param job_id: The ID of the job to retrieve.\n",
+ " :return: The job data, including a list of blocks that describe elements\n",
+ " detected in the image.\n",
+ " \"\"\"\n",
+ " try:\n",
+ " response = self.textract_client.get_document_analysis(JobId=job_id)\n",
+ " job_status = response[\"JobStatus\"]\n",
+ " logger.info(\"Job %s status is %s.\", job_id, job_status)\n",
+ " except ClientError:\n",
+ " logger.exception(\"Couldn't get data for job %s.\", job_id)\n",
+ " raise\n",
+ " else:\n",
+ " return response\n",
+ "\n",
+ "\n",
+ "# snippet-end:[python.example_code.textract.GetDocumentAnalysis]"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "kpF9UagY01-l"
+ },
+ "source": [
+ "Next, we set up Textract and S3, and provide this to an instance of `TextractWrapper`.\n",
+ "\n"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "aetKe-lg0-58"
+ },
+ "outputs": [],
+ "source": [
+ "import boto3\n",
+ "\n",
+ "textract_client = boto3.client('textract')\n",
+ "s3_client = boto3.client('s3')\n",
+ "\n",
+ "textractWrapper = TextractWrapper(textract_client, s3_client, None)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "U3Qr-MTL4dkT"
+ },
+ "source": [
+ "We are now ready to make calls to Textract. At a high level, Textract has two modes: synchronous and asynchronous. Synchronous calls return the parsed output once it is completed. As of the time of writing (March 2024), however, multipage PDF processing is only supported [asynchronously](https://docs.aws.amazon.com/textract/latest/dg/sync.html). So for our purposes here, we will only explore the asynchronous route.\n",
+ "\n",
+ "Asynchronous calls follow the below process:\n",
+ "\n",
+ "1. Send a request to Textract with an SNS topic, S3 bucket, and the name (key) of the document inside that bucket to process. Textract returns a Job ID that can be used to track the status of the request\n",
+ "2. Textract fetches the document from S3 and processes it\n",
+ "3. Once the request is complete, Textract sends out a message to the SNS topic. This can be used in conjunction with other services such as Lambda or SQS for downstream processes.\n",
+ "4. The parsed results can be fetched from Textract in chunks via the job ID."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "pb9AIE4v4Yww"
+ },
+ "outputs": [],
+ "source": [
+ "bucket_name = \"your-bucket-name\"\n",
+ "sns_topic_arn = \"your-sns-arn\" # this can be found under the topic you created in the Amazon SNS dashboard\n",
+ "sns_role_arn = \"sns-role-arn\" # this is an IAM role that allows Textract to interact with SNS\n",
+ "\n",
+ "file_name = \"example-drug-label.pdf\""
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "tPG-P7c79xln"
+ },
+ "outputs": [],
+ "source": [
+ "# kick off a text detection job. This returns a job ID.\n",
+ "job_id = textractWrapper.start_detection_job(bucket_name=bucket_name, document_file_name=file_name,\n",
+ " sns_topic_arn=sns_topic_arn, sns_role_arn=sns_role_arn)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "M0e0hZZJ-yCf"
+ },
+ "source": [
+ "Once the job completes, this will return a dictionary with the following keys:\n",
+ "\n",
+ "```dict_keys(['DocumentMetadata', 'JobStatus', 'NextToken', 'Blocks', 'AnalyzeDocumentModelVersion', 'ResponseMetadata'])```\n",
+ "\n",
+ "This response corresponds to one chunk of information parsed by Textract. The number of chunks a document is parsed into depends on the length of the document. The two keys we are most interested in are `Blocks` and `NextToken`. `Blocks` contains all of the information that was extracted from this chunk, while `NextToken` tells us what chunk comes next, if any.\n",
+ "\n",
+ "Textract returns an information-rich representation of the extracted text, such as their position on the page and hierarchical relationships with other entities, all the way down to the individual word level. Since we are only interested in the raw text, we need a way to parse through all of the chunks and their `Blocks`. Lucky for us, Amazon provides some [helper functions](https://github.com/aws-samples/textract-paragraph-identification/tree/main) for this purpose, which we utilize below."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "9h-6yBBJCYC-"
+ },
+ "outputs": [],
+ "source": [
+ "def get_text_results_from_textract(job_id):\n",
+ " response = textract_client.get_document_text_detection(JobId=job_id)\n",
+ " collection_of_textract_responses = []\n",
+ " pages = [response]\n",
+ "\n",
+ " collection_of_textract_responses.append(response)\n",
+ "\n",
+ " while 'NextToken' in response:\n",
+ " next_token = response['NextToken']\n",
+ " response = textract_client.get_document_text_detection(JobId=job_id, NextToken=next_token)\n",
+ " pages.append(response)\n",
+ " collection_of_textract_responses.append(response)\n",
+ " return collection_of_textract_responses\n",
+ "\n",
+ "def get_the_text_with_required_info(collection_of_textract_responses):\n",
+ " total_text = []\n",
+ " total_text_with_info = []\n",
+ " running_sequence_number = 0\n",
+ "\n",
+ " font_sizes_and_line_numbers = {}\n",
+ " for page in collection_of_textract_responses:\n",
+ " per_page_text = []\n",
+ " blocks = page['Blocks']\n",
+ " for block in blocks:\n",
+ " if block['BlockType'] == 'LINE':\n",
+ " block_text_dict = {}\n",
+ " running_sequence_number += 1\n",
+ " block_text_dict.update(text=block['Text'])\n",
+ " block_text_dict.update(page=block['Page'])\n",
+ " block_text_dict.update(left_indent=round(block['Geometry']['BoundingBox']['Left'], 2))\n",
+ " font_height = round(block['Geometry']['BoundingBox']['Height'], 3)\n",
+ " line_number = running_sequence_number\n",
+ " block_text_dict.update(font_height=round(block['Geometry']['BoundingBox']['Height'], 3))\n",
+ " block_text_dict.update(indent_from_top=round(block['Geometry']['BoundingBox']['Top'], 2))\n",
+ " block_text_dict.update(text_width=round(block['Geometry']['BoundingBox']['Width'], 2))\n",
+ " block_text_dict.update(line_number=running_sequence_number)\n",
+ "\n",
+ " if font_height in font_sizes_and_line_numbers:\n",
+ " line_numbers = font_sizes_and_line_numbers[font_height]\n",
+ " line_numbers.append(line_number)\n",
+ " font_sizes_and_line_numbers[font_height] = line_numbers\n",
+ " else:\n",
+ " line_numbers = []\n",
+ " line_numbers.append(line_number)\n",
+ " font_sizes_and_line_numbers[font_height] = line_numbers\n",
+ "\n",
+ " total_text.append(block['Text'])\n",
+ " per_page_text.append(block['Text'])\n",
+ " total_text_with_info.append(block_text_dict)\n",
+ "\n",
+ " return total_text, total_text_with_info, font_sizes_and_line_numbers\n",
+ "\n",
+ "def get_text_with_line_spacing_info(total_text_with_info):\n",
+ " i = 1\n",
+ " text_info_with_line_spacing_info = []\n",
+ " while (i < len(total_text_with_info) - 1):\n",
+ " previous_line_info = total_text_with_info[i - 1]\n",
+ " current_line_info = total_text_with_info[i]\n",
+ " next_line_info = total_text_with_info[i + 1]\n",
+ " if current_line_info['page'] == next_line_info['page'] and previous_line_info['page'] == current_line_info[\n",
+ " 'page']:\n",
+ " line_spacing_after = round((next_line_info['indent_from_top'] - current_line_info['indent_from_top']), 2)\n",
+ " spacing_with_prev = round((current_line_info['indent_from_top'] - previous_line_info['indent_from_top']), 2)\n",
+ " current_line_info.update(line_space_before=spacing_with_prev)\n",
+ " current_line_info.update(line_space_after=line_spacing_after)\n",
+ " text_info_with_line_spacing_info.append(current_line_info)\n",
+ " else:\n",
+ " text_info_with_line_spacing_info.append(None)\n",
+ " i += 1\n",
+ " return text_info_with_line_spacing_info"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "McBARKH_ClEU"
+ },
+ "source": [
+ "We feed in the Job ID from before into the function `get_text_results_from_textract` to fetch all of the chunks associated with this job. Then, we pass the resulting list into `get_the_text_with_required_info` and `get_text_with_line_spacing_info` to organize the text into lines.\n",
+ "\n",
+ "Finally, we can concatenate the lines into one string to pass into our downstream RAG pipeline."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "0CieHFURIhBm"
+ },
+ "outputs": [],
+ "source": [
+ "all_text = \"\\n\".join([line[\"text\"] if line else \"\" for line in text_info_with_line_spacing])\n",
+ "\n",
+ "with open(f\"aws-parsed-{source_filename}.txt\", \"w\") as f:\n",
+ " f.write(all_text)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "fy8iAFpbggcD"
+ },
+ "source": [
+ "#### Visualize the parsed document"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "3AwZQnFRghv8"
+ },
+ "outputs": [],
+ "source": [
+ "filename = \"aws-parsed-{}.txt\".format(source_filename)\n",
+ "with open(\"{}/{}\".format(data_dir, filename), \"r\") as doc:\n",
+ " parsed_document = doc.read()\n",
+ "\n",
+ "print(parsed_document[:1000])"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "LEO2tP6WJmtn",
+ "jp-MarkdownHeadingCollapsed": true
+ },
+ "source": [
+ "\n",
+ "### Solution 3: Unstructured.io [[Back to Solutions]](#top)\n",
+ "\n",
+ "Unstructured.io provides libraries with open-source components for pre-processing text documents such as PDFs, HTML and Word Documents.\n",
+ " \n",
+ "External documentation: https://github.com/Unstructured-IO/unstructured-api"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "julG3fBrgmqQ"
+ },
+ "source": [
+ "#### Parsing the document\n",
+ "\n",
+ "The guide assumes an endpoint exists that hosts this service. The API is offered in two forms\n",
+ "1. [a hosted version](https://unstructured.io/)\n",
+ "2. [an OSS docker image](https://github.com/Unstructured-IO/unstructured-api?tab=readme-ov-file#dizzy-instructions-for-using-the-docker-image)\n",
+ " \n",
+ "**Note: You can skip to the next block if you want to use the pre-existing parsed version.**"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "5-jaa4-aJiJG"
+ },
+ "outputs": [],
+ "source": [
+ "import os\n",
+ "import requests\n",
+ "\n",
+ "UNSTRUCTURED_URL = \"\" # enter service endpoint\n",
+ "\n",
+ "parsed_documents = []\n",
+ "\n",
+ "input_path = \"{}/{}.{}\".format(data_dir, source_filename, extension)\n",
+ "with open(input_path, 'rb') as file_data:\n",
+ " response = requests.post(\n",
+ " url=UNSTRUCTURED_URL,\n",
+ " files={\"files\": (\"{}.{}\".format(source_filename, extension), file_data)},\n",
+ " data={\n",
+ " \"output_format\": (None, \"application/json\"),\n",
+ " \"stratergy\": \"hi_res\",\n",
+ " \"pdf_infer_table_structure\": \"true\",\n",
+ " \"include_page_breaks\": \"true\"\n",
+ " },\n",
+ " headers={\"Accept\": \"application/json\"}\n",
+ " )\n",
+ "\n",
+ "parsed_response = response.json()\n",
+ "\n",
+ "parsed_document = \" \".join([parsed_entry[\"text\"] for parsed_entry in parsed_response])\n",
+ "print(\"Parsed {}\".format(source_filename))"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "5qI5Q85vTfxs"
+ },
+ "outputs": [],
+ "source": [
+ "\"\"\"\n",
+ "Post process parsed document and store it locally.\n",
+ "\"\"\"\n",
+ "\n",
+ "file_path = \"{}/{}-parsed-fda-approved-drug.txt\".format(data_dir, \"unstructured-io\")\n",
+ "store_document(file_path, parsed_document)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "kOAYMvt1HlAj"
+ },
+ "source": [
+ "#### Visualize the parsed document"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "cBr_roPTHonz"
+ },
+ "outputs": [],
+ "source": [
+ "filename = \"unstructured-io-parsed-{}.txt\".format(source_filename)\n",
+ "with open(\"{}/{}\".format(data_dir, filename), \"r\") as doc:\n",
+ " parsed_document = doc.read()\n",
+ "\n",
+ "print(parsed_document[:1000])"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "KF528hYyEQZx",
+ "jp-MarkdownHeadingCollapsed": true
+ },
+ "source": [
+ "\n",
+ "\n",
+ "### Solution 4: LlamaParse [[Back to Solutions]](#top)\n",
+ "\n",
+ "LlamaParse is an API created by LlamaIndex to efficiently parse and represent files for efficient retrieval and context augmentation using LlamaIndex frameworks.\n",
+ " \n",
+ "External documentation: https://github.com/run-llama/llama_parse"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "oBLNlEVXgshj"
+ },
+ "source": [
+ "#### Parsing the document\n",
+ "\n",
+ "The following block uses the LlamaParse cloud offering. You can learn more and fetch a respective API key for the service [here](https://cloud.llamaindex.ai/parse).\n",
+ " \n",
+ "Parsing documents with LlamaParse offers an option for two output modes both of which we will explore and compare below\n",
+ "- Text\n",
+ "- Markdown\n",
+ " \n",
+ "\n",
+ "**Note: You can skip to the next block if you want to use the pre-existing parsed version.**"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "hR_85riD9ApM"
+ },
+ "outputs": [],
+ "source": [
+ "import os\n",
+ "from llama_parse import LlamaParse\n",
+ "\n",
+ "import nest_asyncio # needed to notebook env\n",
+ "nest_asyncio.apply() # needed to notebook env\n",
+ "\n",
+ "llama_index_api_key = \"{API_KEY}\"\n",
+ "input_path = \"{}/{}.{}\".format(data_dir, source_filename, extension)"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "e4TskJQ4EdpA"
+ },
+ "outputs": [],
+ "source": [
+ "# Text mode\n",
+ "text_parser = LlamaParse(\n",
+ " api_key=llama_index_api_key,\n",
+ " result_type=\"text\"\n",
+ ")\n",
+ "\n",
+ "text_response = text_parser.load_data(input_path)\n",
+ "text_parsed_document = \" \".join([parsed_entry.text for parsed_entry in text_response])\n",
+ "\n",
+ "print(\"Parsed {} to text\".format(source_filename))"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "GKca2BfI9X5O"
+ },
+ "outputs": [],
+ "source": [
+ "\"\"\"\n",
+ "Post process parsed document and store it locally.\n",
+ "\"\"\"\n",
+ "\n",
+ "file_path = \"{}/{}-text-parsed-fda-approved-drug.txt\".format(data_dir, \"llamaparse\")\n",
+ "store_document(file_path, text_parsed_document)"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "Mjig7g-f89PQ"
+ },
+ "outputs": [],
+ "source": [
+ "# Markdown mode\n",
+ "markdown_parser = LlamaParse(\n",
+ " api_key=llama_index_api_key,\n",
+ " result_type=\"markdown\"\n",
+ ")\n",
+ "\n",
+ "markdown_response = markdown_parser.load_data(input_path)\n",
+ "markdown_parsed_document = \" \".join([parsed_entry.text for parsed_entry in markdown_response])\n",
+ "\n",
+ "print(\"Parsed {} to markdown\".format(source_filename))"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "upmHMI8SLcXZ"
+ },
+ "outputs": [],
+ "source": [
+ "\"\"\"\n",
+ "Post process parsed document and store it locally.\n",
+ "\"\"\"\n",
+ "\n",
+ "file_path = \"{}/{}-markdown-parsed-fda-approved-drug.txt\".format(data_dir, \"llamaparse\")\n",
+ "store_document(file_path, markdown_parsed_document)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "P88GmKarHsTQ"
+ },
+ "source": [
+ "#### Visualize the parsed document"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "Y0Cs-a03Huob"
+ },
+ "outputs": [],
+ "source": [
+ "# Text parsing\n",
+ "\n",
+ "filename = \"llamaparse-text-parsed-{}.txt\".format(source_filename)\n",
+ "\n",
+ "with open(\"{}/{}\".format(data_dir, filename), \"r\") as doc:\n",
+ " parsed_document = doc.read()\n",
+ " \n",
+ "print(parsed_document[:1000])"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "XW6SIVEg_fRN"
+ },
+ "outputs": [],
+ "source": [
+ "# Markdown parsing\n",
+ "\n",
+ "filename = \"llamaparse-markdown-parsed-fda-approved-drug.txt\"\n",
+ "with open(\"{}/{}\".format(data_dir, filename), \"r\") as doc:\n",
+ " parsed_document = doc.read()\n",
+ " \n",
+ "print(parsed_document[:1000])"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "8QQ_RopYZ_mf",
+ "jp-MarkdownHeadingCollapsed": true
+ },
+ "source": [
+ "\n",
+ "\n",
+ "### Solution 5: pdf2image + pytesseract [[Back to Solutions]](#top)\n",
+ "\n",
+ "The final parsing method we examine does not rely on cloud services, but rather relies on two libraries: `pdf2image`, and `pytesseract`. `pytesseract` lets you perform OCR locally on images, but not PDF files. So, we first convert our PDF into a set of images via `pdf2image`."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "unM8RfqYgzWC"
+ },
+ "source": [
+ "#### Parsing the document"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "4kt-bmbGeWsf"
+ },
+ "outputs": [],
+ "source": [
+ "from matplotlib import pyplot as plt\n",
+ "from pdf2image import convert_from_path\n",
+ "import pytesseract"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "5bE2Sebtf_eU"
+ },
+ "outputs": [],
+ "source": [
+ "# pdf2image extracts as a list of PIL.Image objects\n",
+ "pages = convert_from_path(filename)"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "ViLTaLPYeT-o"
+ },
+ "outputs": [],
+ "source": [
+ "# we look at the first page as a sanity check:\n",
+ "\n",
+ "plt.imshow(pages[0])\n",
+ "plt.axis('off')\n",
+ "plt.show()"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "mZhXeNTbBKU_"
+ },
+ "source": [
+ "Now, we can process the image of each page with `pytesseract` and concatenate the results to get our parsed document."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "W1SNCEd91k7N"
+ },
+ "outputs": [],
+ "source": [
+ "label_ocr_pytesseract = \"\".join([pytesseract.image_to_string(page) for page in pages])"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "colab": {
+ "base_uri": "https://localhost:8080/"
+ },
+ "id": "tv9hLiJgBANJ",
+ "outputId": "1c3bd5dc-cfe9-43e6-9cc7-bc695a210d31"
+ },
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "HIGHLIGHTS OF PRESCRIBING INFORMATION\n",
+ "\n",
+ "These highlights do not include all the information needed to use\n",
+ "IWILFIN™ safely and effectively. See full prescribing information for\n",
+ "IWILFIN.\n",
+ "\n",
+ "IWILFIN™ (eflor\n"
+ ]
+ }
+ ],
+ "source": [
+ "print(label_ocr_pytesseract[:200])"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "TIwcDmjXejKV"
+ },
+ "outputs": [],
+ "source": [
+ "label_ocr_pytesseract = \"\".join([pytesseract.image_to_string(page) for page in pages])\n",
+ "\n",
+ "with open(f\"pytesseract-parsed-{source_filename}.txt\", \"w\") as f:\n",
+ " f.write(label_ocr_pytesseract)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "j0u1UfNdaFt4"
+ },
+ "source": [
+ "#### Visualize the parsed document"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "HaJutaBkf9Yj"
+ },
+ "outputs": [],
+ "source": [
+ "filename = \"pytesseract-parsed-{}.txt\".format(source_filename)\n",
+ "with open(\"{}/{}\".format(data_dir, filename), \"r\") as doc:\n",
+ " parsed_document = doc.read()\n",
+ "\n",
+ "print(parsed_document[:1000])"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "SCbkT4oZSfs9",
+ "jp-MarkdownHeadingCollapsed": true
+ },
+ "source": [
+ "\n",
+ "## Document Questions\n",
+ "\n",
+ "We can now ask a set of simple + complex questions and see how each parsing solution performs with Command-R. The questions are\n",
+ "- **What are the most common adverse reactions of Iwilfin?**\n",
+ " - Task: Simple information extraction\n",
+ "- **What is the recommended dosage of IWILFIN on body surface area between 0.5 and 0.75?**\n",
+ " - Task: Tabular data extraction\n",
+ "- **I need a succinct summary of the compound name, indication, route of administration, and mechanism of action of Iwilfin.**\n",
+ " - Task: Overall document summary"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "WSo0xA4vZAHV"
+ },
+ "outputs": [],
+ "source": [
+ "import cohere\n",
+ "co = cohere.Client(api_key=\"{API_KEY}\")"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "gyPYqKErY7ni"
+ },
+ "outputs": [],
+ "source": [
+ "\"\"\"\n",
+ "Document Questions\n",
+ "\"\"\"\n",
+ "prompt = \"What are the most common adverse reactions of Iwilfin?\"\n",
+ "# prompt = \"What is the recommended dosage of Iwilfin on body surface area between 0.5 m2 and 0.75 m2?\"\n",
+ "# prompt = \"I need a succinct summary of the compound name, indication, route of administration, and mechanism of action of Iwilfin.\"\n",
+ "\n",
+ "\"\"\"\n",
+ "Choose one of the above solutions\n",
+ "\"\"\"\n",
+ "source = \"gcp\"\n",
+ "# source = \"aws\"\n",
+ "# source = \"unstructured-io\"\n",
+ "# source = \"llamaparse-text\"\n",
+ "# source = \"llamaparse-markdown\"\n",
+ "# source = \"pytesseract\""
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "BwdK7trMKykt",
+ "jp-MarkdownHeadingCollapsed": true
+ },
+ "source": [
+ "## Data Ingestion\n",
+ "\n",
+ "\n",
+ "In order to set up our RAG implementation, we need to separate the parsed text into chunks and load the chunks to an index. The index will allow us to retrieve relevant passages from the document for different queries. Here, we use a simple implementation of indexing using the `hnswlib` library. Note that there are many different indexing solutions that are appropriate for specific production use cases."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "46rPn1uoLQDa"
+ },
+ "outputs": [],
+ "source": [
+ "\"\"\"\n",
+ "Read parsed document content and chunk data\n",
+ "\"\"\"\n",
+ "\n",
+ "import os\n",
+ "from langchain_text_splitters import RecursiveCharacterTextSplitter\n",
+ "\n",
+ "documents = []\n",
+ "\n",
+ "with open(\"{}/{}-parsed-fda-approved-drug.txt\".format(data_dir, source), \"r\") as doc:\n",
+ " doc_content = doc.read()\n",
+ "\n",
+ "\"\"\"\n",
+ "Personal notes on chunking\n",
+ "https://medium.com/@ayhamboucher/llm-based-context-splitter-for-large-documents-445d3f02b01b\n",
+ "\"\"\"\n",
+ "\n",
+ "\n",
+ "# Chunk doc content\n",
+ "text_splitter = RecursiveCharacterTextSplitter(\n",
+ " chunk_size=512,\n",
+ " chunk_overlap=200,\n",
+ " length_function=len,\n",
+ " is_separator_regex=False\n",
+ ")\n",
+ "\n",
+ "# Split the text into chunks with some overlap\n",
+ "chunks_ = text_splitter.create_documents([doc_content])\n",
+ "documents = [c.page_content for c in chunks_]\n",
+ "\n",
+ "print(\"Source document has been broken down to {} chunks\".format(len(documents)))"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "lk4YgREV7LgC"
+ },
+ "outputs": [],
+ "source": [
+ "\"\"\"\n",
+ "Embed document chunks\n",
+ "\"\"\"\n",
+ "document_embeddings = co.embed(texts=documents, model=\"embed-english-v3.0\", input_type=\"search_document\").embeddings"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "colab": {
+ "base_uri": "https://localhost:8080/"
+ },
+ "id": "xtG3eblo7Mkd",
+ "outputId": "6dfb3a1f-d4a5-480b-b8e3-233950fce701"
+ },
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "Count: 115\n"
+ ]
+ }
+ ],
+ "source": [
+ "\"\"\"\n",
+ "Create document index and add embedded chunks\n",
+ "\"\"\"\n",
+ "\n",
+ "import hnswlib\n",
+ "\n",
+ "index = hnswlib.Index(space='ip', dim=1024) # space: inner product\n",
+ "index.init_index(max_elements=len(document_embeddings), ef_construction=512, M=64)\n",
+ "index.add_items(document_embeddings, list(range(len(document_embeddings))))\n",
+ "print(\"Count:\", index.element_count)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "YIncJz3qhWkg",
+ "jp-MarkdownHeadingCollapsed": true
+ },
+ "source": [
+ "## Retrieval\n",
+ "\n",
+ "In this step, we use k-nearest neighbors to fetch the most relevant documents for our query. Once the nearest neighbors are retrieved, we use Cohere's reranker to reorder the documents in the most relevant order with regards to our input search query."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "5rTZcKQ48tAJ"
+ },
+ "outputs": [],
+ "source": [
+ "\"\"\"\n",
+ "Embed search query\n",
+ "Fetch k nearest neighbors\n",
+ "\"\"\"\n",
+ "\n",
+ "query_emb = co.embed(texts=[prompt], model='embed-english-v3.0', input_type=\"search_query\").embeddings\n",
+ "default_knn = 10\n",
+ "knn = default_knn if default_knn <= index.element_count else index.element_count\n",
+ "result = index.knn_query(query_emb, k=knn)\n",
+ "neighbors = [(result[0][0][i], result[1][0][i]) for i in range(len(result[0][0]))]\n",
+ "relevant_docs = [documents[x[0]] for x in sorted(neighbors, key=lambda x: x[1])]"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "vz8jbX8A9RO_"
+ },
+ "outputs": [],
+ "source": [
+ "\"\"\"\n",
+ "Rerank retrieved documents\n",
+ "\"\"\"\n",
+ "\n",
+ "rerank_results = co.rerank(query=prompt, documents=relevant_docs, top_n=3, model='rerank-english-v2.0').results\n",
+ "reranked_relevant_docs = format_docs_for_chat([x.document[\"text\"] for x in rerank_results])"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "KX0RYtx1HW_h",
+ "jp-MarkdownHeadingCollapsed": true
+ },
+ "source": [
+ "## Final Step: Call Command-R + RAG!"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "beUFAEKnHYCQ"
+ },
+ "outputs": [],
+ "source": [
+ "\"\"\"\n",
+ "Call the /chat endpoint with command-r\n",
+ "\"\"\"\n",
+ "\n",
+ "response = co.chat(\n",
+ " message=prompt,\n",
+ " model=\"command-r\",\n",
+ " documents=reranked_relevant_docs\n",
+ ")\n",
+ "\n",
+ "cited_response, citations_reference = insert_citations_in_order(response.text, response.citations, reranked_relevant_docs)\n",
+ "print(cited_response)\n",
+ "print(\"\\n\")\n",
+ "print(\"References:\")\n",
+ "print(citations_reference)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "lzJclPa6hnpi",
+ "jp-MarkdownHeadingCollapsed": true
+ },
+ "source": [
+ "## Head-to-head Comparisons\n",
+ "\n",
+ "Run the code cells below to make head to head comparisons of the different parsing techniques across different questions."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "52wdFoILLy85"
+ },
+ "outputs": [],
+ "source": [
+ "import pandas as pd\n",
+ "results = pd.read_csv(\"{}/results-table.csv\".format(data_dir))"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "colab": {
+ "base_uri": "https://localhost:8080/"
+ },
+ "id": "l5WswlWbO4Da",
+ "outputId": "ddf65c9d-ef12-4b07-a000-b8c1e6c1319e"
+ },
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "\n",
+ "Question 1: What are the most common adverse reactions of Iwilfin?\n",
+ "Question 2: What is the recommended dosage of Iwilfin on body surface area between 0.5 m2 and 0.75 m2?\n",
+ "Question 3: I need a succinct summary of the compound name, indication, route of administration, and mechanism of action of Iwilfin.\n",
+ "\n",
+ "Pick which question you want to see (1,2,3): 3\n",
+ "Do you want to see the references as well? References are long and noisy (y/n): n\n",
+ "\n",
+ "\n",
+ "\n",
+ "| gcp |\n",
+ "\n",
+ "\n",
+ "Compound Name: eflornithine hydrochloride ([0], [1], [2]) (IWILFIN ([1])™)\n",
+ "\n",
+ "Indication: used to reduce the risk of relapse in adult and paediatric patients with high-risk neuroblastoma (HRNB) ([1], [3]), who have responded at least partially to prior multiagent, multimodality therapy. ([1], [3], [4])\n",
+ "\n",
+ "Route of Administration: IWILFIN™ tablets ([1], [3], [4]) are taken orally twice daily ([3], [4]), with doses ranging from 192 to 768 mg based on body surface area. ([3], [4])\n",
+ "\n",
+ "Mechanism of Action: IWILFIN™ is an ornithine decarboxylase inhibitor. ([0], [2])\n",
+ "\n",
+ "\n",
+ "\n",
+ "| aws |\n",
+ "\n",
+ "\n",
+ "Compound Name: eflornithine ([0], [1], [2], [3]) (IWILFIN ([0])™)\n",
+ "\n",
+ "Indication: used to reduce the risk of relapse ([0], [3]) in adults ([0], [3]) and paediatric patients ([0], [3]) with high-risk neuroblastoma (HRNB) ([0], [3]) who have responded to prior therapies. ([0], [3], [4])\n",
+ "\n",
+ "Route of Administration: Oral ([2], [4])\n",
+ "\n",
+ "Mechanism of Action: IWILFIN is an ornithine decarboxylase inhibitor. ([1])\n",
+ "\n",
+ "\n",
+ "| unstructured-io |\n",
+ "\n",
+ "\n",
+ "Compound Name: Iwilfin ([1], [2], [3], [4]) (eflornithine) ([0], [2], [3], [4])\n",
+ "\n",
+ "Indication: Iwilfin is indicated to reduce the risk of relapse ([1], [3]) in adult and paediatric patients ([1], [3]) with high-risk neuroblastoma (HRNB) ([1], [3]), who have responded to prior anti-GD2 ([1]) immunotherapy ([1], [4]) and multi-modality therapy. ([1])\n",
+ "\n",
+ "Route of Administration: Oral ([0], [3])\n",
+ "\n",
+ "Mechanism of Action: Iwilfin is an ornithine decarboxylase inhibitor. ([1], [2], [3], [4])\n",
+ "\n",
+ "\n",
+ "| llamaparse-text |\n",
+ "\n",
+ "\n",
+ "Compound Name: IWILFIN ([2], [3]) (eflornithine) ([3])\n",
+ "\n",
+ "Indication: IWILFIN is used to reduce the risk of relapse ([1], [2], [3]) in adult and paediatric patients ([1], [2], [3]) with high-risk neuroblastoma (HRNB) ([1], [2], [3]), who have responded at least partially to certain prior therapies. ([2], [3])\n",
+ "\n",
+ "Route of Administration: IWILFIN is administered as a tablet. ([2])\n",
+ "\n",
+ "Mechanism of Action: IWILFIN is an ornithine decarboxylase inhibitor. ([0], [1], [4])\n",
+ "\n",
+ "\n",
+ "| llamaparse-markdown |\n",
+ "\n",
+ "\n",
+ "Compound Name: IWILFIN ([1], [2]) (eflornithine) ([1])\n",
+ "\n",
+ "Indication: IWILFIN is indicated to reduce the risk of relapse ([1], [2]) in adult and paediatric patients ([1], [2]) with high-risk neuroblastoma (HRNB) ([1], [2]), who have responded at least partially ([1], [2], [3]) to prior anti-GD2 immunotherapy ([1], [2]) and multiagent, multimodality therapy. ([1], [2], [3])\n",
+ "\n",
+ "Route of Administration: Oral ([0], [1], [3], [4])\n",
+ "\n",
+ "Mechanism of Action: IWILFIN acts as an ornithine decarboxylase inhibitor. ([1])\n",
+ "\n",
+ "\n",
+ "| pytesseract |\n",
+ "\n",
+ "\n",
+ "Compound Name: IWILFIN™ ([0], [2]) (eflornithine) ([0], [2])\n",
+ "\n",
+ "Indication: IWILFIN is indicated to reduce the risk of relapse ([0], [2]) in adult and paediatric patients ([0], [2]) with high-risk neuroblastoma (HRNB) ([0], [2]), who have responded positively to prior anti-GD2 immunotherapy and multiagent, multimodality therapy. ([0], [2], [4])\n",
+ "\n",
+ "Route of Administration: IWILFIN is administered orally ([0], [1], [3], [4]), in the form of a tablet. ([1])\n",
+ " \n",
+ "Mechanism of Action: IWILFIN acts as an ornithine decarboxylase inhibitor. ([0])\n",
+ "\n",
+ "\n"
+ ]
+ }
+ ],
+ "source": [
+ "question = input(\"\"\"\n",
+ "Question 1: What are the most common adverse reactions of Iwilfin?\n",
+ "Question 2: What is the recommended dosage of Iwilfin on body surface area between 0.5 m2 and 0.75 m2?\n",
+ "Question 3: I need a succinct summary of the compound name, indication, route of administration, and mechanism of action of Iwilfin.\n",
+ "\n",
+ "Pick which question you want to see (1,2,3): \"\"\")\n",
+ "references = input(\"Do you want to see the references as well? References are long and noisy (y/n): \")\n",
+ "print(\"\\n\\n\")\n",
+ "\n",
+ "index = {\"1\": 0, \"2\": 3, \"3\": 6}[question]\n",
+ "\n",
+ "for src in [\"gcp\", \"aws\", \"unstructured-io\", \"llamaparse-text\", \"llamaparse-markdown\", \"pytesseract\"]:\n",
+ " print(\"| {} |\".format(src))\n",
+ " print(\"\\n\")\n",
+ " print(results[src][index])\n",
+ " if references == \"y\":\n",
+ " print(\"\\n\")\n",
+ " print(\"References:\")\n",
+ " print(results[src][index+1])\n",
+ " print(\"\\n\")"
+ ]
+ }
+ ],
+ "metadata": {
+ "colab": {
+ "collapsed_sections": [
+ "BoM6-Tq-Sm-1",
+ "jLtrGjXiJE9M",
+ "dlRxuddi5w0E",
+ "1YCszLJnge4j",
+ "LEO2tP6WJmtn",
+ "julG3fBrgmqQ",
+ "KF528hYyEQZx",
+ "oBLNlEVXgshj",
+ "8QQ_RopYZ_mf",
+ "unM8RfqYgzWC",
+ "BwdK7trMKykt"
+ ],
+ "provenance": []
+ },
+ "kernelspec": {
+ "display_name": "Python 3 (ipykernel)",
+ "language": "python",
+ "name": "python3"
+ },
+ "language_info": {
+ "codemirror_mode": {
+ "name": "ipython",
+ "version": 3
+ },
+ "file_extension": ".py",
+ "mimetype": "text/x-python",
+ "name": "python",
+ "nbconvert_exporter": "python",
+ "pygments_lexer": "ipython3",
+ "version": "3.12.2"
+ }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 4
+}
diff --git a/notebooks/.ipynb_checkpoints/RAG_with_Chat_Embed_and_Rerank-checkpoint.ipynb b/notebooks/.ipynb_checkpoints/RAG_with_Chat_Embed_and_Rerank-checkpoint.ipynb
new file mode 100644
index 00000000..e138717e
--- /dev/null
+++ b/notebooks/.ipynb_checkpoints/RAG_with_Chat_Embed_and_Rerank-checkpoint.ipynb
@@ -0,0 +1,873 @@
+{
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "id": "ctaLvRUsfpj8",
+ "metadata": {
+ "id": "ctaLvRUsfpj8"
+ },
+ "source": [
+ "\n",
+ " \n",
+ ""
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "61bac5b5",
+ "metadata": {
+ "id": "61bac5b5"
+ },
+ "source": [
+ "# RAG with Chat, Embed, and Rerank\n",
+ "\n",
+ "This notebook shows how to build a RAG-powered chatbot with Cohere's Chat endpoint. The chatbot can extract relevant information from external documents and produce verifiable, inline citations in its responses.\n",
+ "\n",
+ "This application will use several Cohere API endpoints:\n",
+ "\n",
+ "- Chat: For handling the main logic of the chatbot, including turning a user message into queries, generating responses, and producing citations\n",
+ "- Embed: For turning textual documents into their embeddings representation, later to be used in retrieval (we’ll use the latest, state-of-the-art Embed v3 model)\n",
+ "- Rerank: For reranking the retrieved documents according to their relevance to a query\n",
+ "\n",
+ "The diagram below provides an overview of what we’ll build."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "CLNMDV4IuA3C",
+ "metadata": {
+ "id": "CLNMDV4IuA3C"
+ },
+ "source": [
+ ""
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "f6ab2d5d",
+ "metadata": {
+ "id": "f6ab2d5d"
+ },
+ "source": [
+ "Here is a summary of the steps involved.\n",
+ "\n",
+ "Initial phase:\n",
+ "- **Step 0**: Ingest the documents – get documents, chunk, embed, and index.\n",
+ "\n",
+ "For each user-chatbot interaction:\n",
+ "- **Step 1**: Get the user message\n",
+ "- **Step 2**: Call the Chat endpoint in query-generation mode\n",
+ "- If at least one query is generated\n",
+ " - **Step 3**: Retrieve and rerank relevant documents\n",
+ " - **Step 4**: Call the Chat endpoint in document mode to generate a grounded response with citations\n",
+ "- If no query is generated\n",
+ " - **Step 4**: Call the Chat endpoint in normal mode to generate a response"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "TWyo_5WoNUM-",
+ "metadata": {
+ "id": "TWyo_5WoNUM-"
+ },
+ "source": [
+ "# Setup"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "5pLAhQmTOKiV",
+ "metadata": {
+ "id": "5pLAhQmTOKiV"
+ },
+ "outputs": [],
+ "source": [
+ "! pip install cohere hnswlib unstructured -q"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "f3a03a57",
+ "metadata": {
+ "id": "f3a03a57"
+ },
+ "outputs": [],
+ "source": [
+ "import cohere\n",
+ "import uuid\n",
+ "import hnswlib\n",
+ "from typing import List, Dict\n",
+ "from unstructured.partition.html import partition_html\n",
+ "from unstructured.chunking.title import chunk_by_title\n",
+ "\n",
+ "co = cohere.Client(\"COHERE_API_KEY\") # Get your API key here: https://dashboard.cohere.com/api-keys"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "Dx1cncziCWBB",
+ "metadata": {
+ "cellView": "form",
+ "id": "Dx1cncziCWBB"
+ },
+ "outputs": [],
+ "source": [
+ "#@title Enable text wrapping in Google Colab\n",
+ "\n",
+ "from IPython.display import HTML, display\n",
+ "\n",
+ "def set_css():\n",
+ " display(HTML('''\n",
+ " \n",
+ " '''))\n",
+ "get_ipython().events.register('pre_run_cell', set_css)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "2f7e7d1c",
+ "metadata": {
+ "id": "2f7e7d1c"
+ },
+ "source": [
+ "# Create a vector store for ingestion and retrieval\n",
+ "\n",
+ "First, we define the list of documents we want to ingest and make available for retrieval. As an example, we'll use the contents from the first module of Cohere's *LLM University: What are Large Language Models?*."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "3dca4a88",
+ "metadata": {
+ "colab": {
+ "base_uri": "https://localhost:8080/",
+ "height": 17
+ },
+ "id": "3dca4a88",
+ "outputId": "b05da1ee-0456-4387-c232-a43e0ffed54c"
+ },
+ "outputs": [
+ {
+ "data": {
+ "text/html": [
+ "\n",
+ " \n",
+ " "
+ ],
+ "text/plain": [
+ ""
+ ]
+ },
+ "metadata": {},
+ "output_type": "display_data"
+ }
+ ],
+ "source": [
+ "raw_documents = [\n",
+ " {\n",
+ " \"title\": \"Text Embeddings\",\n",
+ " \"url\": \"https://docs.cohere.com/docs/text-embeddings\"},\n",
+ " {\n",
+ " \"title\": \"Similarity Between Words and Sentences\",\n",
+ " \"url\": \"https://docs.cohere.com/docs/similarity-between-words-and-sentences\"},\n",
+ " {\n",
+ " \"title\": \"The Attention Mechanism\",\n",
+ " \"url\": \"https://docs.cohere.com/docs/the-attention-mechanism\"},\n",
+ " {\n",
+ " \"title\": \"Transformer Models\",\n",
+ " \"url\": \"https://docs.cohere.com/docs/transformer-models\"}\n",
+ "]"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "5e2a8968",
+ "metadata": {
+ "id": "5e2a8968"
+ },
+ "source": [
+ "Usually the number of documents for practical applications is vast, and so we'll need to be able to search documents efficiently. This involves breaking the documents into chunks, generating embeddings, and indexing the embeddings, as shown in the image below. \n",
+ "\n",
+ "We implement this in the `Vectorstore` class below, which takes the `raw_documents` list as input. Three methods are immediately called when creating an object of the `Vectorstore` class:\n",
+ "\n",
+ "\n",
+ "`load_and_chunk()` \n",
+ "This method uses the `partition_html()` method from the `unstructured` library to load the documents from URL and break them into smaller chunks. Each chunk is turned into a dictionary object with three fields:\n",
+ "- `title` - the web page’s title,\n",
+ "- `text` - the textual content of the chunk, and\n",
+ "- `url` - the web page’s URL. \n",
+ " \n",
+ " \n",
+ "`embed()` \n",
+ "This method uses Cohere's `embed-english-v3.0` model to generate embeddings of the chunked documents. Since our documents will be used for retrieval, we set `input_type=\"search_document\"`. We send the documents to the Embed endpoint in batches, because the endpoint has a limit of 96 documents per call.\n",
+ "\n",
+ "`index()` \n",
+ "This method uses the `hsnwlib` package to index the document chunk embeddings. This will ensure efficient similarity search during retrieval. Note that `hnswlib` uses a vector library, and we have chosen it for its simplicity."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "7c33412c",
+ "metadata": {
+ "colab": {
+ "base_uri": "https://localhost:8080/",
+ "height": 17
+ },
+ "id": "7c33412c",
+ "outputId": "cf04f8ed-8000-4433-f976-2d37747f21e7"
+ },
+ "outputs": [
+ {
+ "data": {
+ "text/html": [
+ "\n",
+ " \n",
+ " "
+ ],
+ "text/plain": [
+ ""
+ ]
+ },
+ "metadata": {},
+ "output_type": "display_data"
+ }
+ ],
+ "source": [
+ "class Vectorstore:\n",
+ " \"\"\"\n",
+ " A class representing a collection of documents indexed into a vectorstore.\n",
+ "\n",
+ " Parameters:\n",
+ " raw_documents (list): A list of dictionaries representing the sources of the raw documents. Each dictionary should have 'title' and 'url' keys.\n",
+ "\n",
+ " Attributes:\n",
+ " raw_documents (list): A list of dictionaries representing the raw documents.\n",
+ " docs (list): A list of dictionaries representing the chunked documents, with 'title', 'text', and 'url' keys.\n",
+ " docs_embs (list): A list of the associated embeddings for the document chunks.\n",
+ " docs_len (int): The number of document chunks in the collection.\n",
+ " idx (hnswlib.Index): The index used for document retrieval.\n",
+ "\n",
+ " Methods:\n",
+ " load_and_chunk(): Loads the data from the sources and partitions the HTML content into chunks.\n",
+ " embed(): Embeds the document chunks using the Cohere API.\n",
+ " index(): Indexes the document chunks for efficient retrieval.\n",
+ " retrieve(): Retrieves document chunks based on the given query.\n",
+ " \"\"\"\n",
+ "\n",
+ " def __init__(self, raw_documents: List[Dict[str, str]]):\n",
+ " self.raw_documents = raw_documents\n",
+ " self.docs = []\n",
+ " self.docs_embs = []\n",
+ " self.retrieve_top_k = 10\n",
+ " self.rerank_top_k = 3\n",
+ " self.load_and_chunk()\n",
+ " self.embed()\n",
+ " self.index()\n",
+ "\n",
+ "\n",
+ " def load_and_chunk(self) -> None:\n",
+ " \"\"\"\n",
+ " Loads the text from the sources and chunks the HTML content.\n",
+ " \"\"\"\n",
+ " print(\"Loading documents...\")\n",
+ "\n",
+ " for raw_document in self.raw_documents:\n",
+ " elements = partition_html(url=raw_document[\"url\"])\n",
+ " chunks = chunk_by_title(elements)\n",
+ " for chunk in chunks:\n",
+ " self.docs.append(\n",
+ " {\n",
+ " \"title\": raw_document[\"title\"],\n",
+ " \"text\": str(chunk),\n",
+ " \"url\": raw_document[\"url\"],\n",
+ " }\n",
+ " )\n",
+ "\n",
+ " def embed(self) -> None:\n",
+ " \"\"\"\n",
+ " Embeds the document chunks using the Cohere API.\n",
+ " \"\"\"\n",
+ " print(\"Embedding document chunks...\")\n",
+ "\n",
+ " batch_size = 90\n",
+ " self.docs_len = len(self.docs)\n",
+ " for i in range(0, self.docs_len, batch_size):\n",
+ " batch = self.docs[i : min(i + batch_size, self.docs_len)]\n",
+ " texts = [item[\"text\"] for item in batch]\n",
+ " docs_embs_batch = co.embed(\n",
+ " texts=texts, model=\"embed-english-v3.0\", input_type=\"search_document\"\n",
+ " ).embeddings\n",
+ " self.docs_embs.extend(docs_embs_batch)\n",
+ "\n",
+ " def index(self) -> None:\n",
+ " \"\"\"\n",
+ " Indexes the document chunks for efficient retrieval.\n",
+ " \"\"\"\n",
+ " print(\"Indexing document chunks...\")\n",
+ "\n",
+ " self.idx = hnswlib.Index(space=\"ip\", dim=1024)\n",
+ " self.idx.init_index(max_elements=self.docs_len, ef_construction=512, M=64)\n",
+ " self.idx.add_items(self.docs_embs, list(range(len(self.docs_embs))))\n",
+ "\n",
+ " print(f\"Indexing complete with {self.idx.get_current_count()} document chunks.\")\n",
+ "\n",
+ " def retrieve(self, query: str) -> List[Dict[str, str]]:\n",
+ " \"\"\"\n",
+ " Retrieves document chunks based on the given query.\n",
+ "\n",
+ " Parameters:\n",
+ " query (str): The query to retrieve document chunks for.\n",
+ "\n",
+ " Returns:\n",
+ " List[Dict[str, str]]: A list of dictionaries representing the retrieved document chunks, with 'title', 'text', and 'url' keys.\n",
+ " \"\"\"\n",
+ "\n",
+ " # Dense retrieval\n",
+ " query_emb = co.embed(\n",
+ " texts=[query], model=\"embed-english-v3.0\", input_type=\"search_query\"\n",
+ " ).embeddings\n",
+ "\n",
+ " doc_ids = self.idx.knn_query(query_emb, k=self.retrieve_top_k)[0][0]\n",
+ "\n",
+ " # Reranking\n",
+ " docs_to_rerank = [self.docs[doc_id][\"text\"] for doc_id in doc_ids]\n",
+ "\n",
+ " rerank_results = co.rerank(\n",
+ " query=query,\n",
+ " documents=docs_to_rerank,\n",
+ " top_n=self.rerank_top_k,\n",
+ " model=\"rerank-english-v2.0\",\n",
+ " )\n",
+ "\n",
+ " doc_ids_reranked = [doc_ids[result.index] for result in rerank_results.results]\n",
+ "\n",
+ " docs_retrieved = []\n",
+ " for doc_id in doc_ids_reranked:\n",
+ " docs_retrieved.append(\n",
+ " {\n",
+ " \"title\": self.docs[doc_id][\"title\"],\n",
+ " \"text\": self.docs[doc_id][\"text\"],\n",
+ " \"url\": self.docs[doc_id][\"url\"],\n",
+ " }\n",
+ " )\n",
+ "\n",
+ " return docs_retrieved"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "e1bf5d85",
+ "metadata": {
+ "id": "e1bf5d85"
+ },
+ "source": [
+ "In the code cell below, we initialize an instance of the `Vectorstore` class and pass in the `raw_documents` list as input."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "4643e630",
+ "metadata": {
+ "colab": {
+ "base_uri": "https://localhost:8080/",
+ "height": 178
+ },
+ "id": "4643e630",
+ "outputId": "fe01fcb6-3574-4322-d8d0-57d37aad397d"
+ },
+ "outputs": [
+ {
+ "data": {
+ "text/html": [
+ "\n",
+ " \n",
+ " "
+ ],
+ "text/plain": [
+ ""
+ ]
+ },
+ "metadata": {},
+ "output_type": "display_data"
+ },
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "Loading documents...\n"
+ ]
+ },
+ {
+ "name": "stderr",
+ "output_type": "stream",
+ "text": [
+ "[nltk_data] Downloading package punkt to /root/nltk_data...\n",
+ "[nltk_data] Unzipping tokenizers/punkt.zip.\n",
+ "[nltk_data] Downloading package averaged_perceptron_tagger to\n",
+ "[nltk_data] /root/nltk_data...\n",
+ "[nltk_data] Unzipping taggers/averaged_perceptron_tagger.zip.\n"
+ ]
+ },
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "Embedding document chunks...\n",
+ "Indexing document chunks...\n",
+ "Indexing complete with 136 document chunks.\n"
+ ]
+ }
+ ],
+ "source": [
+ "# Create an instance of the Vectorstore class with the given sources\n",
+ "vectorstore = Vectorstore(raw_documents)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "61928287",
+ "metadata": {
+ "id": "61928287"
+ },
+ "source": [
+ "The `Vectorstore` class also has a `retrieve()` method, which we'll use to retrieve relevant document chunks given a query (as in Step 3 in the diagram shared at the beginning of this notebook). This method has two components: (1) dense retrieval, and (2) reranking.\n",
+ "\n",
+ "### Dense retrieval\n",
+ "\n",
+ "First, we embed the query using the same `embed-english-v3.0` model we used to embed the document chunks, but this time we set `input_type=\"search_query\"`.\n",
+ "\n",
+ "Search is performed by the `knn_query()` method from the `hnswlib` library. Given a query, it returns the document chunks most similar to the query. We can define the number of document chunks to return using the attribute `self.retrieve_top_k=10`.\n",
+ "\n",
+ "### Reranking\n",
+ "\n",
+ "After semantic search, we implement a reranking step. While our semantic search component is already highly capable of retrieving relevant sources, the [Rerank endpoint](https://cohere.com/rerank) provides an additional boost to the quality of the search results, especially for complex and domain-specific queries. It takes the search results and sorts them according to their relevance to the query.\n",
+ "\n",
+ "We call the Rerank endpoint with the `co.rerank()` method and define the number of top reranked document chunks to retrieve using the attribute `self.rerank_top_k=3`. The model we use is `rerank-english-v2.0`. \n",
+ "\n",
+ "This method returns the top retrieved document chunks `chunks_retrieved` so that they can be passed to the chatbot.\n",
+ "\n",
+ "In the code cell below, we check the document chunks that are retrieved for the query `\"multi-head attention definition\"`."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "OwozNf_uPEyX",
+ "metadata": {
+ "id": "OwozNf_uPEyX"
+ },
+ "source": [
+ "## Test Retrieval"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "82617b91",
+ "metadata": {
+ "colab": {
+ "base_uri": "https://localhost:8080/",
+ "height": 267
+ },
+ "id": "82617b91",
+ "outputId": "7f1f2bc8-8ed9-4190-bd6b-7af2d9dc1980"
+ },
+ "outputs": [
+ {
+ "data": {
+ "text/html": [
+ "\n",
+ " \n",
+ " "
+ ],
+ "text/plain": [
+ ""
+ ]
+ },
+ "metadata": {},
+ "output_type": "display_data"
+ },
+ {
+ "data": {
+ "text/plain": [
+ "[{'title': 'Transformer Models',\n",
+ " 'text': 'The attention step used in transformer models is actually much more powerful, and it’s called multi-head attention. In multi-head attention, several different embeddings are used to modify the vectors and add context to them. Multi-head attention has helped language models reach much higher levels of efficacy when processing and generating text.',\n",
+ " 'url': 'https://docs.cohere.com/docs/transformer-models'},\n",
+ " {'title': 'The Attention Mechanism',\n",
+ " 'text': \"What you learned in this chapter is simple self-attention. However, we can do much better than that. There is a method called multi-head attention, in which one doesn't only consider one embedding, but several different ones. These are all obtained from the original by transforming it in different ways. Multi-head attention has been very successful at the task of adding context to text. If you'd like to learn more about the self and multi-head attention, you can check out the following two\",\n",
+ " 'url': 'https://docs.cohere.com/docs/the-attention-mechanism'},\n",
+ " {'title': 'Transformer Models',\n",
+ " 'text': 'Attention helps give context to each word, based on the other words in the sentence (or text).',\n",
+ " 'url': 'https://docs.cohere.com/docs/transformer-models'}]"
+ ]
+ },
+ "execution_count": 8,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "vectorstore.retrieve(\"multi-head attention definition\")"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "e69fbca9",
+ "metadata": {
+ "id": "e69fbca9"
+ },
+ "source": [
+ "# Create a chatbot\n",
+ "\n",
+ "Next, we implement a class to handle the interaction between the user and the chatbot. It takes an instance of the `Vectorstore` class as input.\n",
+ "\n",
+ "The `run()` method will be used to run the chatbot application. It begins with the logic for getting the user message, along with a way for the user to end the conversation. \n",
+ "\n",
+ "Based on the user message, the chatbot needs to decide if it needs to consult external information before responding. If so, the chatbot determines an optimal set of search queries to use for retrieval. When we call `co.chat()` with `search_queries_only=True`, the Chat endpoint handles this for us automatically.\n",
+ "\n",
+ "The generated queries can be accessed from the `search_queries` field of the object that is returned. Then, what happens next depends on how many queries are returned.\n",
+ "- If queries are returned, we call the `retrieve()` method of the Vectorstore object for the retrieval step. The retrieved document chunks are then passed to the Chat endpoint by adding a `documents` parameter when we call `co.chat()` again.\n",
+ "- Otherwise, if no queries are returned, we call the Chat endpoint another time, passing the user message and without needing to add any documents to the call.\n",
+ "\n",
+ "In either case, we also pass the `conversation_id` parameter, which retains the interactions between the user and the chatbot in the same conversation thread. We also enable the `stream` parameter so we can stream the chatbot response.\n",
+ "\n",
+ "We then print the chatbot's response. In the case that the external information was used to generate a response, we also display citations."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "d2c15a1f",
+ "metadata": {
+ "colab": {
+ "base_uri": "https://localhost:8080/",
+ "height": 17
+ },
+ "id": "d2c15a1f",
+ "outputId": "8daa9159-338c-45ec-e9ed-830aedcdf0d8"
+ },
+ "outputs": [
+ {
+ "data": {
+ "text/html": [
+ "\n",
+ " \n",
+ " "
+ ],
+ "text/plain": [
+ ""
+ ]
+ },
+ "metadata": {},
+ "output_type": "display_data"
+ }
+ ],
+ "source": [
+ "class Chatbot:\n",
+ " def __init__(self, vectorstore: Vectorstore):\n",
+ " \"\"\"\n",
+ " Initializes an instance of the Chatbot class.\n",
+ "\n",
+ " Parameters:\n",
+ " vectorstore (Vectorstore): An instance of the Vectorstore class.\n",
+ "\n",
+ " \"\"\"\n",
+ " self.vectorstore = vectorstore\n",
+ " self.conversation_id = str(uuid.uuid4())\n",
+ "\n",
+ " def run(self):\n",
+ " \"\"\"\n",
+ " Runs the chatbot application.\n",
+ "\n",
+ " \"\"\"\n",
+ " while True:\n",
+ " # Get the user message\n",
+ " message = input(\"User: \")\n",
+ "\n",
+ " # Typing \"quit\" ends the conversation\n",
+ " if message.lower() == \"quit\":\n",
+ " print(\"Ending chat.\")\n",
+ " break\n",
+ " # else: # Uncomment for Google Colab to avoid printing the same thing twice\n",
+ " # print(f\"User: {message}\") # Uncomment for Google Colab to avoid printing the same thing twice\n",
+ "\n",
+ " # Generate search queries (if any)\n",
+ " response = co.chat(message=message,\n",
+ " model=\"command-r\",\n",
+ " search_queries_only=True)\n",
+ "\n",
+ " # If there are search queries, retrieve document chunks and respond\n",
+ " if response.search_queries:\n",
+ " print(\"Retrieving information...\", end=\"\")\n",
+ "\n",
+ " # Retrieve document chunks for each query\n",
+ " documents = []\n",
+ " for query in response.search_queries:\n",
+ " documents.extend(self.vectorstore.retrieve(query.text))\n",
+ "\n",
+ " # Use document chunks to respond\n",
+ " response = co.chat_stream(\n",
+ " message=message,\n",
+ " model=\"command-r\",\n",
+ " documents=documents,\n",
+ " conversation_id=self.conversation_id,\n",
+ " )\n",
+ "\n",
+ " # If there is no search query, directly respond\n",
+ " else:\n",
+ " response = co.chat_stream(\n",
+ " message=message,\n",
+ " model=\"command-r\",\n",
+ " conversation_id=self.conversation_id,\n",
+ " )\n",
+ "\n",
+ " # Print the chatbot response, citations, and documents\n",
+ " print(\"\\nChatbot:\")\n",
+ " citations = []\n",
+ " cited_documents = []\n",
+ "\n",
+ " # Display response\n",
+ " for event in response:\n",
+ " if event.event_type == \"text-generation\":\n",
+ " print(event.text, end=\"\")\n",
+ " elif event.event_type == \"citation-generation\":\n",
+ " citations.extend(event.citations)\n",
+ " elif event.event_type == \"search-results\":\n",
+ " cited_documents = event.documents\n",
+ "\n",
+ " # Display citations and source documents\n",
+ " if citations:\n",
+ " print(\"\\n\\nCITATIONS:\")\n",
+ " for citation in citations:\n",
+ " print(citation)\n",
+ "\n",
+ " print(\"\\nDOCUMENTS:\")\n",
+ " for document in cited_documents:\n",
+ " print(document)\n",
+ "\n",
+ " print(f\"\\n{'-'*100}\\n\")"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "F0X9FZqwOwQ6",
+ "metadata": {
+ "id": "F0X9FZqwOwQ6"
+ },
+ "source": [
+ "# Run the chatbot"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "3cb442f7",
+ "metadata": {
+ "id": "3cb442f7"
+ },
+ "source": [
+ "We can now run the chatbot. For this, we create the instance of `Chatbot` and run the chatbot by invoking the `run()` method.\n",
+ "\n",
+ "The format of each citation is:\n",
+ "- `start`: The starting point of a span where one or more documents are referenced\n",
+ "- `end`: The ending point of a span where one or more documents are referenced\n",
+ "- `text`: The text representing this span\n",
+ "- `document_ids`: The IDs of the documents being referenced (`doc_0` being the ID of the first document passed to the `documents` creating parameter in the endpoint call, and so on)"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "42d3f345",
+ "metadata": {
+ "colab": {
+ "base_uri": "https://localhost:8080/",
+ "height": 1000
+ },
+ "id": "42d3f345",
+ "outputId": "8b935c8b-b1d4-4913-bdf8-73ba503402b8"
+ },
+ "outputs": [
+ {
+ "data": {
+ "text/html": [
+ "\n",
+ " \n",
+ " "
+ ],
+ "text/plain": [
+ ""
+ ]
+ },
+ "metadata": {},
+ "output_type": "display_data"
+ },
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "User: Hello, I have a question\n",
+ "\n",
+ "Chatbot:\n",
+ "Hello! What's your question? I'm here to help you in any way I can.\n",
+ "----------------------------------------------------------------------------------------------------\n",
+ "\n",
+ "User: What’s the difference between word and sentence embeddings\n",
+ "Retrieving information...\n",
+ "Chatbot:\n",
+ "Word embeddings associate words with lists of numbers. Similar words are assigned numbers that are mathematically close while dissimilar words are assigned numbers that are far apart. \n",
+ "\n",
+ "Sentence embeddings do the same thing as word embeddings, but for sentences. Each sentence is associated with a vector of numbers in a coherent way. This means that similar sentences are assigned similar vectors and dissimilar sentences are assigned different vectors.\n",
+ "\n",
+ "CITATIONS:\n",
+ "start=0 end=15 text='Word embeddings' document_ids=['doc_0']\n",
+ "start=16 end=54 text='associate words with lists of numbers.' document_ids=['doc_0']\n",
+ "start=55 end=68 text='Similar words' document_ids=['doc_0']\n",
+ "start=82 end=119 text='numbers that are mathematically close' document_ids=['doc_0']\n",
+ "start=126 end=142 text='dissimilar words' document_ids=['doc_0']\n",
+ "start=156 end=183 text='numbers that are far apart.' document_ids=['doc_0']\n",
+ "start=186 end=205 text='Sentence embeddings' document_ids=['doc_0', 'doc_2']\n",
+ "start=213 end=242 text='same thing as word embeddings' document_ids=['doc_0', 'doc_2']\n",
+ "start=263 end=276 text='Each sentence' document_ids=['doc_0', 'doc_2']\n",
+ "start=298 end=315 text='vector of numbers' document_ids=['doc_0', 'doc_2']\n",
+ "start=321 end=329 text='coherent' document_ids=['doc_2']\n",
+ "start=351 end=368 text='similar sentences' document_ids=['doc_0', 'doc_2']\n",
+ "start=382 end=397 text='similar vectors' document_ids=['doc_0', 'doc_2']\n",
+ "start=402 end=422 text='dissimilar sentences' document_ids=['doc_0', 'doc_2']\n",
+ "start=436 end=454 text='different vectors.' document_ids=['doc_0', 'doc_2']\n",
+ "\n",
+ "DOCUMENTS:\n",
+ "{'id': 'doc_0', 'text': 'In the previous chapters, you learned about word and sentence embeddings and similarity between words and sentences. In short, a word embedding is a way to associate words with lists of numbers (vectors) in such a way that similar words are associated with numbers that are close by, and dissimilar words with numbers that are far away from each other. A sentence embedding does the same thing, but associating a vector to every sentence. Similarity is a way to measure how similar two words (or', 'title': 'The Attention Mechanism', 'url': 'https://docs.cohere.com/docs/the-attention-mechanism'}\n",
+ "{'id': 'doc_1', 'text': 'Sentence embeddings\\n\\nSo word embeddings seem to be pretty useful, but in reality, human language is much more complicated than simply a bunch of words put together. Human language has structure, sentences, etc. How would one be able to represent, for instance, a sentence? Well, here’s an idea. How about the sums of scores of all the words? For example, say we have a word embedding that assigns the following scores to these words:\\n\\nNo: [1,0,0,0]\\n\\nI: [0,2,0,0]\\n\\nAm: [-1,0,1,0]\\n\\nGood: [0,0,1,3]', 'title': 'Text Embeddings', 'url': 'https://docs.cohere.com/docs/text-embeddings'}\n",
+ "{'id': 'doc_2', 'text': 'This is where sentence embeddings come into play. A sentence embedding is just like a word embedding, except it associates every sentence with a vector full of numbers, in a coherent way. By coherent, I mean that it satisfies similar properties as a word embedding. For instance, similar sentences are assigned to similar vectors, different sentences are assigned to different vectors, and most importantly, each of the coordinates of the vector identifies some (whether clear or obscure) property of', 'title': 'Text Embeddings', 'url': 'https://docs.cohere.com/docs/text-embeddings'}\n",
+ "\n",
+ "----------------------------------------------------------------------------------------------------\n",
+ "\n",
+ "User: And what are their similarities\n",
+ "Retrieving information...\n",
+ "Chatbot:\n",
+ "The similarity between word and sentence embeddings lies in the fact that they both measure similarity between items. For example, if two sentences are very similar, their corresponding vectors will also be similar. This is best illustrated with an example: \n",
+ "\n",
+ "The similarities between the following sentences can be computed using sentence embeddings:\n",
+ "1. Who was the 16th president of the US and fought in the American Civil War?\n",
+ "2. The American Civil War saw the 16th President, Abraham Lincoln, attempt to preserve the Union.\n",
+ "3. Lincoln was the 16th president of the United States.\n",
+ "\n",
+ "The similarity between sentences 1 and 2 is 6738.2859, which is very high. On the other hand, the similarities between sentences 1 and 3, and 2 and 3, are much lower at -122.2267 and -3.4946 respectively.\n",
+ "\n",
+ "CITATIONS:\n",
+ "start=84 end=102 text='measure similarity' document_ids=['doc_0', 'doc_2']\n",
+ "start=131 end=215 text='if two sentences are very similar, their corresponding vectors will also be similar.' document_ids=['doc_0']\n",
+ "start=589 end=638 text='similarity between sentences 1 and 2 is 6738.2859' document_ids=['doc_1']\n",
+ "start=683 end=734 text='similarities between sentences 1 and 3, and 2 and 3' document_ids=['doc_1']\n",
+ "start=754 end=775 text='-122.2267 and -3.4946' document_ids=['doc_1']\n",
+ "\n",
+ "DOCUMENTS:\n",
+ "{'id': 'doc_0', 'text': 'Notice that these sentences are all very similar. In particular, the three highlighted sentences pretty much have the same meaning. If you look at their corresponding vectors, these are also really similar. That is exactly what an embedding should do.', 'title': 'Text Embeddings', 'url': 'https://docs.cohere.com/docs/text-embeddings'}\n",
+ "{'id': 'doc_1', 'text': 'And the results are:\\n\\nThe similarity between sentences 1 and 2: 6738.2858668486715\\n\\nThe similarity between sentences 1 and 3: -122.22666955510499\\n\\nThe similarity between sentences 2 and 3: -3.494608113647928\\n\\nThese results certainly confirm our predictions. The similarity between sentences 1 and 2 is 6738, which is high. The similarities between sentences 1 and 3, and 2 and 3, are -122 and -3.5 (dot products are allowed to be negative too!), which are much lower.', 'title': 'Similarity Between Words and Sentences', 'url': 'https://docs.cohere.com/docs/similarity-between-words-and-sentences'}\n",
+ "{'id': 'doc_2', 'text': 'The similarity between each sentence and itself is 1 (the diagonal in the plot), which is consistent with our expectations. Furthermore, a sentence and itself represent the same point in space, which gives an angle of 0 with the origin, so it makes sense that the similarity is the cosine of 0, which is 1!\\n\\nConclusion', 'title': 'Similarity Between Words and Sentences', 'url': 'https://docs.cohere.com/docs/similarity-between-words-and-sentences'}\n",
+ "\n",
+ "----------------------------------------------------------------------------------------------------\n",
+ "\n",
+ "User: What do you know about 5G networks\n",
+ "Retrieving information...\n",
+ "Chatbot:\n",
+ "Unfortunately, I could not find any information about 5G networks in the available documentation. However, I can tell you about the 4G networks which have preceded 5G. 4G networks enable a high-speed connection and were designed to support a wide range of functions on mobile devices, including video streaming and high-quality music streaming. They also support a wider coverage area and better spectral efficiency, allowing more devices to connect simultaneously.\n",
+ "----------------------------------------------------------------------------------------------------\n",
+ "\n",
+ "User: quit\n",
+ "Ending chat.\n"
+ ]
+ }
+ ],
+ "source": [
+ "# Create an instance of the Chatbot class\n",
+ "chatbot = Chatbot(vectorstore)\n",
+ "\n",
+ "# Run the chatbot\n",
+ "chatbot.run()"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "9e69aa7c",
+ "metadata": {
+ "id": "9e69aa7c"
+ },
+ "source": [
+ "In the conversation above, notice a few observations that reflect the different components of what we built:\n",
+ "\n",
+ "- **Direct response**: For user messages that don’t require retrieval, such as \"Hello, I have a question\", the chatbot responds directly without requiring retrieval.\n",
+ "- **Citation generation**: For responses that do require retrieval (\"What’s the difference between word and sentence embeddings\"), the endpoint returns the response together with the citations.\n",
+ "- **State management**: The endpoint maintains the state of the conversation via the `conversation_id` parameter, for example, by being able to correctly respond to a vague user message of \"And what are their similarities\".\n",
+ "- **Response synthesis**: The model can decide if none of the retrieved documents provide the necessary information required to answer a user message. For example, when asked the question \"What do you know about 5G networks\", the chatbot goes on and retrieves external information from the index. However, it doesn’t use any of the information in its response as none of them is relevant to the question."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "-JBWZVz9ObcV",
+ "metadata": {
+ "id": "-JBWZVz9ObcV"
+ },
+ "outputs": [],
+ "source": []
+ }
+ ],
+ "metadata": {
+ "colab": {
+ "provenance": []
+ },
+ "kernelspec": {
+ "display_name": "Python 3 (ipykernel)",
+ "language": "python",
+ "name": "python3"
+ },
+ "language_info": {
+ "codemirror_mode": {
+ "name": "ipython",
+ "version": 3
+ },
+ "file_extension": ".py",
+ "mimetype": "text/x-python",
+ "name": "python",
+ "nbconvert_exporter": "python",
+ "pygments_lexer": "ipython3",
+ "version": "3.11.5"
+ }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}
diff --git a/notebooks/.ipynb_checkpoints/Vanilla_Multi_Step_Tool_Use-checkpoint.ipynb b/notebooks/.ipynb_checkpoints/Vanilla_Multi_Step_Tool_Use-checkpoint.ipynb
new file mode 100644
index 00000000..550e7d87
--- /dev/null
+++ b/notebooks/.ipynb_checkpoints/Vanilla_Multi_Step_Tool_Use-checkpoint.ipynb
@@ -0,0 +1,979 @@
+{
+ "nbformat": 4,
+ "nbformat_minor": 0,
+ "metadata": {
+ "colab": {
+ "provenance": []
+ },
+ "kernelspec": {
+ "name": "python3",
+ "display_name": "Python 3"
+ },
+ "language_info": {
+ "name": "python"
+ }
+ },
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "source": [
+ "# Multi-Step Tool Use\n",
+ "\n",
+ "Tool use allows developers to connect Cohere's models to external tools like search engines, APIs, functions, databases, etc.\n",
+ "\n",
+ "Multi-step tool use is an extension of this basic idea, and allows the model to call more than one tool in a sequence of steps, using the results from one tool call in a subsequent step. This process allows the language model to reason, perform dynamic actions, and quickly adapt on the basis of information coming from external sources.\n",
+ "\n",
+ "The recommended way to achieve [multi-step tool use with Cohere](https://docs.cohere.com/docs/multi-step-tool-use) is by leveraging the [Langchain framework](https://python.langchain.com/docs/integrations/providers/cohere#react-agent) in Python."
+ ],
+ "metadata": {
+ "id": "aH1iAdZiURXh"
+ }
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "## Install Dependencies"
+ ],
+ "metadata": {
+ "id": "PVT6Sl3msjNe"
+ }
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 1,
+ "metadata": {
+ "colab": {
+ "base_uri": "https://localhost:8080/"
+ },
+ "id": "0m31LaFDjCIY",
+ "outputId": "7177e241-a864-47ed-cf0a-5fc9fa70a7ec"
+ },
+ "outputs": [
+ {
+ "output_type": "stream",
+ "name": "stdout",
+ "text": [
+ "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m812.8/812.8 kB\u001b[0m \u001b[31m6.5 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
+ "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m194.1/194.1 kB\u001b[0m \u001b[31m4.7 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
+ "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m1.9/1.9 MB\u001b[0m \u001b[31m9.6 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
+ "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m276.8/276.8 kB\u001b[0m \u001b[31m7.7 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
+ "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m87.5/87.5 kB\u001b[0m \u001b[31m3.7 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
+ "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m145.9/145.9 kB\u001b[0m \u001b[31m2.8 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
+ "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m3.1/3.1 MB\u001b[0m \u001b[31m21.7 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
+ "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m75.6/75.6 kB\u001b[0m \u001b[31m7.1 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
+ "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m49.4/49.4 kB\u001b[0m \u001b[31m4.1 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
+ "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m53.0/53.0 kB\u001b[0m \u001b[31m4.8 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
+ "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m144.8/144.8 kB\u001b[0m \u001b[31m4.9 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
+ "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m77.9/77.9 kB\u001b[0m \u001b[31m1.9 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
+ "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m58.3/58.3 kB\u001b[0m \u001b[31m5.5 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
+ "\u001b[?25h"
+ ]
+ }
+ ],
+ "source": [
+ "! pip install --quiet langchain langchain_cohere langchain_experimental"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# LLM\n",
+ "import os\n",
+ "os.environ['COHERE_API_KEY'] = "
+ ],
+ "metadata": {
+ "id": "K0GELKJVnadW"
+ },
+ "execution_count": 2,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "## Define tools\n",
+ "\n",
+ "Your agent will be equipped with the following tools. The model can pick between\n",
+ "\n",
+ "- doing a search on the web\n",
+ "- and/or retrieving data from a vector store\n",
+ "- and/or using a python interpreter\n",
+ "- and/or directly answering [ this tool comes out of the box! ]\n",
+ "\n",
+ "Plus the model can self-reflect."
+ ],
+ "metadata": {
+ "id": "6l1pDbAmsptW"
+ }
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "#### Web search\n",
+ "You can easily equip your agent with web search!"
+ ],
+ "metadata": {
+ "id": "_9riBTvIVnaN"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "from langchain_community.tools.tavily_search import TavilySearchResults\n",
+ "\n",
+ "os.environ[\"TAVILY_API_KEY\"] = # you can create an API key for free on Tavily's website\n",
+ "\n",
+ "internet_search = TavilySearchResults()\n",
+ "internet_search.name = \"internet_search\"\n",
+ "internet_search.description = \"Returns a list of relevant document snippets for a textual query retrieved from the internet.\"\n",
+ "\n",
+ "\n",
+ "from langchain_core.pydantic_v1 import BaseModel, Field\n",
+ "class TavilySearchInput(BaseModel):\n",
+ " query: str = Field(description=\"Query to search the internet with\")\n",
+ "internet_search.args_schema = TavilySearchInput"
+ ],
+ "metadata": {
+ "id": "6wMNTOGNVvwA"
+ },
+ "execution_count": 3,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "#### Vector store\n",
+ "You can easily equip your agent with a vector store!"
+ ],
+ "metadata": {
+ "id": "pax1SemzX6Kg"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "!pip --quiet install faiss-cpu tiktoken"
+ ],
+ "metadata": {
+ "id": "cc4d4pRFndYu",
+ "colab": {
+ "base_uri": "https://localhost:8080/"
+ },
+ "outputId": "6478fd6b-5618-4d0b-8928-44f78e3ca7e6"
+ },
+ "execution_count": 4,
+ "outputs": [
+ {
+ "output_type": "stream",
+ "name": "stdout",
+ "text": [
+ "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m27.0/27.0 MB\u001b[0m \u001b[31m41.4 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
+ "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m1.8/1.8 MB\u001b[0m \u001b[31m58.3 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
+ "\u001b[?25h"
+ ]
+ }
+ ]
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "from langchain.text_splitter import RecursiveCharacterTextSplitter\n",
+ "from langchain_community.document_loaders import WebBaseLoader\n",
+ "from langchain_community.vectorstores import FAISS\n",
+ "from langchain_cohere import CohereEmbeddings\n",
+ "\n",
+ "# Set embeddings\n",
+ "embd = CohereEmbeddings()\n",
+ "\n",
+ "# Docs to index\n",
+ "urls = [\n",
+ " \"https://paulgraham.com/best.html\",\n",
+ "]\n",
+ "\n",
+ "# Load\n",
+ "docs = [WebBaseLoader(url).load() for url in urls]\n",
+ "docs_list = [item for sublist in docs for item in sublist]\n",
+ "\n",
+ "# Split\n",
+ "text_splitter = RecursiveCharacterTextSplitter.from_tiktoken_encoder(\n",
+ " chunk_size=512, chunk_overlap=0\n",
+ ")\n",
+ "doc_splits = text_splitter.split_documents(docs_list)\n",
+ "\n",
+ "# Add to vectorstore\n",
+ "vectorstore = FAISS.from_documents(\n",
+ " documents=doc_splits,\n",
+ " embedding=embd,\n",
+ ")\n",
+ "\n",
+ "vectorstore_retriever = vectorstore.as_retriever()\n"
+ ],
+ "metadata": {
+ "id": "eqcorKP4YEH6"
+ },
+ "execution_count": 5,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "from langchain.tools.retriever import create_retriever_tool\n",
+ "\n",
+ "vectorstore_search = create_retriever_tool(\n",
+ " retriever=vectorstore_retriever,\n",
+ " name=\"vectorstore_search\",\n",
+ " description=\"Retrieve relevant info from a vectorstore that contains information from Paul Graham about how to write good essays.\"\n",
+ ")"
+ ],
+ "metadata": {
+ "id": "eQFNJ-38abRd"
+ },
+ "execution_count": 6,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "#### Python interpreter tool\n",
+ "You can easily equip your agent with a python interpreter!"
+ ],
+ "metadata": {
+ "id": "aB90i9W5YqWi"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "from langchain.agents import Tool\n",
+ "from langchain_experimental.utilities import PythonREPL\n",
+ "\n",
+ "python_repl = PythonREPL()\n",
+ "python_tool = Tool(\n",
+ " name=\"python_repl\",\n",
+ " description=\"Executes python code and returns the result. The code runs in a static sandbox without interactive mode, so print output or save output to a file.\",\n",
+ " func=python_repl.run,\n",
+ ")\n",
+ "python_tool.name = \"python_interpreter\"\n",
+ "\n",
+ "# from langchain_core.pydantic_v1 import BaseModel, Field\n",
+ "class ToolInput(BaseModel):\n",
+ " code: str = Field(description=\"Python code to execute.\")\n",
+ "python_tool.args_schema = ToolInput"
+ ],
+ "metadata": {
+ "id": "AN-4FqXKYEFw"
+ },
+ "execution_count": 7,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [],
+ "metadata": {
+ "id": "8vj868w_YED_"
+ },
+ "execution_count": 7,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "#### Transform any Python function in a Tool\n",
+ "You can easily equip your agent with any Python function!"
+ ],
+ "metadata": {
+ "id": "2SDCaoypexGL"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "from langchain_core.tools import tool\n",
+ "import random\n",
+ "\n",
+ "@tool\n",
+ "def random_operation_tool(a: int, b: int):\n",
+ " \"\"\"Calculates a random operation between the inputs.\"\"\"\n",
+ " coin_toss = random.uniform(0, 1)\n",
+ " if coin_toss > 0.5:\n",
+ " return {'output': a*b}\n",
+ " else:\n",
+ " return {'output': a+b}\n",
+ "\n",
+ "random_operation_tool.name = \"random_operation\" # use python case\n",
+ "random_operation_tool.description = \"Calculates a random operation between the inputs.\"\n",
+ "\n",
+ "from langchain_core.pydantic_v1 import BaseModel, Field\n",
+ "class random_operation_inputs(BaseModel):\n",
+ " a: int = Field(description=\"First input\")\n",
+ " b: int = Field(description=\"Second input\")\n",
+ "random_operation_tool.args_schema = random_operation_inputs"
+ ],
+ "metadata": {
+ "id": "DYUJT2xKewny"
+ },
+ "execution_count": 8,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "## Create ReAct Agent\n",
+ "\n",
+ "The model can smartly pick the right tool(s) for the user query, call them in any sequence, analyze the results and self-reflect. \n",
+ "Once the model considers it has enough information to answer the user question, it generates the final answer."
+ ],
+ "metadata": {
+ "id": "hcspRBsRY2ED"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "from langchain.agents import AgentExecutor\n",
+ "from langchain_cohere.react_multi_hop.agent import create_cohere_react_agent\n",
+ "from langchain_core.prompts import ChatPromptTemplate\n",
+ "from langchain_cohere.chat_models import ChatCohere\n",
+ "\n",
+ "# LLM\n",
+ "llm = ChatCohere(model=\"command-r-plus\", temperature=0.3)\n",
+ "\n",
+ "# Preamble\n",
+ "preamble = \"\"\"\n",
+ "You are an expert who answers the user's question with the most relevant datasource. You are equipped with an internet search tool and a special vectorstore of information about how to write good essays.\n",
+ "You also have a 'random_operation_tool' tool, you must use it to compute the random operation between two numbers.\n",
+ "\"\"\"\n",
+ "\n",
+ "\n",
+ "# Prompt template\n",
+ "prompt = ChatPromptTemplate.from_template(\"{input}\")\n",
+ "\n",
+ "# Create the ReAct agent\n",
+ "agent = create_cohere_react_agent(\n",
+ " llm=llm,\n",
+ " tools=[internet_search, vectorstore_search, python_tool, random_operation_tool],\n",
+ " prompt=prompt,\n",
+ ")\n",
+ "\n",
+ "agent_executor = AgentExecutor(agent=agent, tools=[internet_search, vectorstore_search, python_tool, random_operation_tool], verbose=True)"
+ ],
+ "metadata": {
+ "id": "CY83aAprYECO"
+ },
+ "execution_count": 13,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [],
+ "metadata": {
+ "id": "RK0NaqngYD_J"
+ },
+ "execution_count": 13,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "## Ask a standalone question to the ReAct agent\n",
+ "A question that requires using a predefined tool from Langchain"
+ ],
+ "metadata": {
+ "id": "mzTQcVXSaBe7"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "response = agent_executor.invoke({\n",
+ " \"input\": \"I want to write an essay about the Roman Empire. Any tips for writing an essay? Any fun facts?\",\n",
+ " \"preamble\": preamble,\n",
+ "})\n",
+ "\n",
+ "response['output']\n",
+ "\n",
+ "# note that the model smartly looks in the vector db, and then online"
+ ],
+ "metadata": {
+ "colab": {
+ "base_uri": "https://localhost:8080/",
+ "height": 1000
+ },
+ "id": "rR2hWVp-aEsU",
+ "outputId": "fcf9b8b3-7027-4679-c06d-ef166f016892"
+ },
+ "execution_count": 14,
+ "outputs": [
+ {
+ "output_type": "stream",
+ "name": "stdout",
+ "text": [
+ "\n",
+ "\n",
+ "\u001b[1m> Entering new AgentExecutor chain...\u001b[0m\n",
+ "\u001b[32;1m\u001b[1;3m\n",
+ "I will search for tips on writing an essay and fun facts about the Roman Empire.\n",
+ "{'tool_name': 'vectorstore_search', 'parameters': {'query': 'tips for writing an essay'}}\n",
+ "\u001b[0m\u001b[33;1m\u001b[1;3mbe? I should have asked how do you write essays well? Though\n",
+ "these seem only phrasing apart, their answers diverge. The answer\n",
+ "to the first question, as we've seen, isn't really about essay\n",
+ "writing. The second question forces it to be.Writing essays, at its best, is a way of discovering ideas. How do\n",
+ "you do that well? How do you discover by writing?An essay should ordinarily start with what I'm going to call a\n",
+ "question, though I mean this in a very general sense: it doesn't\n",
+ "have to be a question grammatically, just something that acts like\n",
+ "one in the sense that it spurs some response.How do you get this initial question? It probably won't work to\n",
+ "choose some important-sounding topic at random and go at it.\n",
+ "Professional traders won't even trade unless they have what they\n",
+ "call an edge — a convincing story about why in some class of\n",
+ "trades they'll win more than they lose. Similarly, you shouldn't\n",
+ "attack a topic unless you have a way in — some new insight about\n",
+ "it or way of approaching it.You don't need to have a complete thesis; you just need some kind\n",
+ "of gap you can explore. In fact, merely having questions about\n",
+ "something other people take for granted can be edge enough.If you come across a question that's sufficiently puzzling, it could\n",
+ "be worth exploring even if it doesn't seem very momentous. Many an\n",
+ "important discovery has been made by pulling on a thread that seemed\n",
+ "insignificant at first. How can they all be finches? \n",
+ "[2]Once you've got a question, then what? You start thinking out loud\n",
+ "about it. Not literally out loud, but you commit to a specific\n",
+ "string of words in response, as you would if you were talking. This\n",
+ "initial response is usually mistaken or incomplete. Writing converts\n",
+ "your ideas from vague to bad. But that's a step forward, because\n",
+ "once you can see the brokenness, you can fix it.Perhaps beginning writers are alarmed at the thought of starting\n",
+ "with something mistaken or incomplete, but you shouldn't be, because\n",
+ "this is why essay writing works. Forcing yourself to commit to some\n",
+ "specific string of words gives you a starting point, and if it's\n",
+ "wrong, you'll see that when you reread it. At least half of essay\n",
+ "writing is rereading what you've written and asking is this correct\n",
+ "\n",
+ "didn't have edge with any of them. To start writing an essay, you\n",
+ "need a topic plus some initial insight about it, and you can't\n",
+ "generate those systematically. If only. \n",
+ "[9]You can probably cause yourself to have more of them, though. The\n",
+ "quality of the ideas that come out of your head depends on what goes\n",
+ "in, and you can improve that in two dimensions, breadth and depth.You can't learn everything, so getting breadth implies learning\n",
+ "about topics that are very different from one another. When I tell\n",
+ "people about my book-buying trips to Hay and they ask what I buy\n",
+ "books about, I usually feel a bit sheepish answering, because the\n",
+ "topics seem like a laundry list of unrelated subjects. But perhaps\n",
+ "that's actually optimal in this business.You can also get ideas by talking to people, by doing and building\n",
+ "things, and by going places and seeing things. I don't think it's\n",
+ "important to talk to new people so much as the sort of people who\n",
+ "make you have new ideas. I get more new ideas after talking for an\n",
+ "afternoon with Robert Morris than from talking to 20 new smart\n",
+ "people. I know because that's what a block of office hours at Y\n",
+ "Combinator consists of.While breadth comes from reading and talking and seeing, depth comes\n",
+ "from doing. The way to really learn about some domain is to have\n",
+ "to solve problems in it. Though this could take the form of writing,\n",
+ "I suspect that to be a good essayist you also have to do, or have\n",
+ "done, some other kind of work. That may not be true for most other\n",
+ "fields, but essay writing is different. You could spend half your\n",
+ "time working on something else and be net ahead, so long as it was\n",
+ "hard.I'm not proposing that as a recipe so much as an encouragement to\n",
+ "those already doing it. If you've spent all your life so far working\n",
+ "on other things, you're already halfway there. Though of course to\n",
+ "be good at writing you have to like it, and if you like writing\n",
+ "you'd probably have spent at least some time doing it.Everything I've said about initial questions applies also to the\n",
+ "questions you encounter in writing the essay. They're the same\n",
+ "thing; every subtree of an essay is usually a shorter essay, just\n",
+ "as every subtree of a Calder mobile is a smaller mobile. So any\n",
+ "\n",
+ "You don't have to get an answer right the first time, but there's\n",
+ "no excuse for not getting it right eventually, because you can keep\n",
+ "rewriting till you do. And this is not just a theoretical possibility.\n",
+ "It's a pretty accurate description of the way I work. I'm rewriting\n",
+ "as we speak.But although I wish I could say that writing great essays depends mostly\n",
+ "on effort, in the limit case it's inspiration that makes the\n",
+ "difference. In the limit case, the questions are the harder thing\n",
+ "to get. That pool has no bottom.How to get more questions? That is the most important question of\n",
+ "all.Notes[1]\n",
+ "There might be some resistance to this conclusion on the\n",
+ "grounds that some of these discoveries could only be understood by\n",
+ "a small number of readers. But you get into all sorts of difficulties\n",
+ "if you want to disqualify essays on this account. How do you decide\n",
+ "where the cutoff should be? If a virus kills off everyone except a \n",
+ "handful of people sequestered at Los Alamos,\n",
+ "could an essay that had been disqualified now be eligible? Etc.Darwin's 1844 essay was derived from an earlier version written in 1839.\n",
+ "Extracts from it were published in 1858.[2]\n",
+ "When you find yourself very curious about an apparently minor\n",
+ "question, that's an exciting sign. Evolution has designed you to\n",
+ "pay attention to things that matter. So when you're very curious\n",
+ "about something random, that could mean you've unconsciously noticed\n",
+ "it's less random than it seems.[3]\n",
+ "Corollary: If you're not intellectually honest, your writing\n",
+ "won't just be biased, but also boring, because you'll miss all the\n",
+ "ideas you'd have discovered if you pushed for the truth.[4]\n",
+ "Sometimes this process begins before you start writing.\n",
+ "Sometimes you've already figured out the first few things you want\n",
+ "to say. Schoolchildren are often taught they should decide everything\n",
+ "they want to say, and write this down as an outline before they\n",
+ "start writing the essay itself. Maybe that's a good way to get them\n",
+ "started — or not, I don't know — but it's antithetical to the\n",
+ "spirit of essay writing. The more detailed your outline, the less\n",
+ "your ideas can benefit from the sort of discovery that essays are for.[5]\n",
+ "The problem with this type of \"greedy\" algorithm is that you\n",
+ "\n",
+ "technique that gets you good initial questions also gets you good\n",
+ "whole essays.At some point the cycle of question and response reaches what feels\n",
+ "like a natural end. Which is a little suspicious; shouldn't every\n",
+ "answer suggest more questions? I think what happens is that you\n",
+ "start to feel sated. Once you've covered enough interesting ground,\n",
+ "you start to lose your appetite for new questions. Which is just\n",
+ "as well, because the reader is probably feeling sated too. And it's\n",
+ "not lazy to stop asking questions, because you could instead be\n",
+ "asking the initial question of a new essay.That's the ultimate source of drag on the connectedness of ideas:\n",
+ "the discoveries you make along the way. If you discover enough\n",
+ "starting from question A, you'll never make it to question B. Though\n",
+ "if you keep writing essays you'll gradually fix this problem by\n",
+ "burning off such discoveries. So bizarrely enough, writing lots of\n",
+ "essays makes it as if the space of ideas were more highly connected.When a subtree comes to an end, you can do one of two things. You\n",
+ "can either stop, or pull the Cubist trick of laying separate subtrees\n",
+ "end to end by returning to a question you skipped earlier. Usually\n",
+ "it requires some sleight of hand to make the essay flow continuously\n",
+ "at this point, but not this time. This time I actually need an\n",
+ "example of the phenomenon. For example, we discovered earlier that\n",
+ "the best possible essay wouldn't usually be timeless in the way the\n",
+ "best painting would. This seems surprising enough to be\n",
+ "worth investigating further.There are two senses in which an essay can be timeless: to be about\n",
+ "a matter of permanent importance, and always to have the same effect\n",
+ "on readers. With art these two senses blend together. Art that\n",
+ "looked beautiful to the ancient Greeks still looks beautiful to us.\n",
+ "But with essays the two senses diverge, because essays\n",
+ "teach, and you can't teach people something they already know.\n",
+ "Natural selection is certainly a matter of permanent importance,\n",
+ "but an essay explaining it couldn't have the same effect on us that\n",
+ "it would have had on Darwin's contemporaries, precisely because his\n",
+ "ideas were so successful that everyone already knows about them.\n",
+ "[10]I imagined when I started writing this that the best possible essay\n",
+ "would be timeless in the stricter, evergreen sense: that it would\u001b[0m\u001b[32;1m\u001b[1;3m\n",
+ "{'tool_name': 'internet_search', 'parameters': {'query': 'fun facts about the roman empire'}}\n",
+ "\u001b[0m\u001b[36;1m\u001b[1;3m[{'url': 'https://www.natgeokids.com/uk/discover/history/romans/10-facts-about-the-ancient-romans/', 'content': 'i love this website\\nBIG BOBBY\\nbooby\\nI love shell my bae;)\\ni like bobby fishes ;0\\nI like turtles\\nOmg soy cool\\ngreeeeeeeeeeeeaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaatttttttttttttttttttttttt\\nbest fact ever\\nthis artical is cool\\nHANDY\\nrubbish did not help what so ever\\nha\\nRocking\\nTHIS IS THE BEST\\nproper rad in it cool\\nthis is cool\\nawesomeness\\nawsome\\nawsome\\nthank you captain\\nit is a lot of help\\ni like this\\nwebsite it helps me on my projects and isabel likes munier\\nmark uses this for research\\nlot of help\\nthis is awsome\\nTHE BEST BOOBOO\\nCool webpage helped me get 4 housepoints\\n This helped me A LOT on a school project\\ncool wow awesomoe\\nCOOL WEBSITE LOL\\nthis helped me with a school project :)\\nthat was awesome\\ncool\\nthat helped me out for my research test\\nReally its very cool really COOL\\nLIKE COOL best website so far its nice\\nI love it\\nnice facts\\nIt help with my history\\n i mean u made animaljam a awesome nice safe place for kids and this site to have kids a safe website to get facts for reports and stuff\\nLots of Love ,\\nRose\\npretty good website if u ask me\\nbut definently not gonna use it on a daily basis\\nIll try it again another time\\ngood\\nCool webcite\\nterrible\\nquite impressive\\nAwesome website it real helps\\nits good\\nthis is a great website! You really a lot with my project!:)\\nthis has helleped\\nme get\\nmy progect\\ndone\\nthank you\\nsoooooooooooooooooo\\nmuchchchchchch\\nthis helleped me\\nsooooooooo much with my progect thank you\\nvery good website\\nthank us very much your nice one today!!\\n'}, {'url': 'https://ohfact.com/roman-empire-facts/', 'content': 'Learn about the ancient Roman Civilization, its history, culture, army, architecture, food and more from this list of 27 facts. Discover how the Romans started, conquered, lived, died and influenced the world with their legends, myths and facts.'}, {'url': 'https://factnight.com/fun-facts-about-the-roman-empire/', 'content': 'The Roman Empire was one of the most influential and significant civilizations in world history. At its peak, the empire stretched from North Africa to Britain, reigning over 60 million people. From its legendary beginnings and remarkable achievements to its eventual decline and fall, the Roman Empire is a fascinating topic full of little-known facts and intriguing trivia.'}, {'url': 'https://www.historyhit.com/facts-about-ancient-rome-and-the-romans/', 'content': 'The Enduring Legacy of C.S. Lewis\\nMargaret J. Winkler: A Forgotten Pioneer in Disney’s Success\\n10 Facts About Harper Lee\\nAntarctica Expedition Cruise\\nUncover Pompeii\\nSophie Hay and Tristan Hughes\\nRediscovering Richard III with Matt Lewis\\nOrder the History Hit Miscellany\\nHistory Hit Holidays\\nGift Subscriptions\\n100 Facts About Ancient Rome and the Romans\\nRome wasn’t built in a day, as the cliché reminds us. The Crossing of the Rhine in 405/6 AD brought around 100,000 barbarians into the Empire\\nBarbarian factions, tribes and war leaders were now a factor in the power struggles at the top of Roman politics and one of the once-strong boundaries of the Empire had proved to be permeable.\\n Related Articles\\n10 Facts About Saint Andrew\\nThe Rise of Pompey the Great, the ‘Roman Alexander’\\nWatch and Listen\\nCleopatra\\nSex in Ancient Rome\\nRelated Locations\\nBaelo Claudia\\nMausoleum of Cecilia Metella\\nColin Ricketts\\n30 July 2021\\n By the fourth century BC, the story was accepted by Romans who were proud of their warrior founder\\nThe story was included in the first history of the city, by the Greek writer Diocles of Peparethus, and the twins and their wolf step-mother were depicted on Rome’s first coins.\\n The History Hit Miscellany of Facts, Figures and Fascinating Finds\\nA History of England: Part One\\nDragons: Myth & Reality\\nA Tudor Wonder - Hardwick Hall\\nThe Battle of Shrewsbury\\nEurope’s 1848 Revolutions\\nThe Boston Tea Party\\nHow Did 3 People Seemingly Escape From Alcatraz?\\n'}, {'url': 'https://www.countryfaq.com/facts-about-the-roman-empire/', 'content': 'Facts about the Roman Empire. Explore some of the interesting, fun, cool facts bout the Roman Empire: 1. The Magnificent Roman Empire. The Roman Empire, a colossal entity of unparalleled grandeur, occupies an indomitable position within the annals of human history, a name that resonates resoundingly across the eons.'}]\u001b[0m\u001b[32;1m\u001b[1;3mRelevant Documents: 0,3,4,5\n",
+ "Cited Documents: 0,3,4,5\n",
+ "Answer: Here are some tips for writing an essay:\n",
+ "- Start with a question that spurs some response.\n",
+ "- Don't choose a topic at random, make sure you have a way in, a new insight or approach.\n",
+ "- You don't need a complete thesis, just a gap to explore.\n",
+ "- You can get ideas by talking to people, reading, doing and building things, and going places and seeing things.\n",
+ "- You can improve the quality of your ideas by increasing the breadth and depth of what goes in.\n",
+ "- You can get breadth by reading and talking about a wide range of topics.\n",
+ "- You can get depth by doing and having to solve problems.\n",
+ "- You can also get ideas by talking to people who make you have new ideas.\n",
+ "\n",
+ "Here are some fun facts about the Roman Empire:\n",
+ "- At its peak, the empire stretched from North Africa to Britain, reigning over 60 million people.\n",
+ "- The story of Rome's warrior founder and the twins and their wolf step-mother was depicted on Rome's first coins.\n",
+ "- The Crossing of the Rhine in 405/6 AD brought around 100,000 barbarians into the Empire.\n",
+ "Grounded answer: Here are some tips for writing an essay:\n",
+ "- Start with a question that spurs some response.\n",
+ "- Don't choose a topic at random, make sure you have a way in, a new insight or approach.\n",
+ "- You don't need a complete thesis, just a gap to explore.\n",
+ "- You can get ideas by talking to people, reading, doing and building things, and going places and seeing things.\n",
+ "- You can improve the quality of your ideas by increasing the breadth and depth of what goes in.\n",
+ "- You can get breadth by reading and talking about a wide range of topics.\n",
+ "- You can get depth by doing and having to solve problems.\n",
+ "- You can also get ideas by talking to people who make you have new ideas.\n",
+ "\n",
+ "Here are some fun facts about the Roman Empire:\n",
+ "- At its peak, the empire stretched from North Africa to Britain, reigning over 60 million people.\n",
+ "- The story of Rome's warrior founder and the twins and their wolf step-mother was depicted on Rome's first coins.\n",
+ "- The Crossing of the Rhine in 405/6 AD brought around 100,000 barbarians into the Empire.\u001b[0m\n",
+ "\n",
+ "\u001b[1m> Finished chain.\u001b[0m\n"
+ ]
+ },
+ {
+ "output_type": "execute_result",
+ "data": {
+ "text/plain": [
+ "\"Here are some tips for writing an essay:\\n- Start with a question that spurs some response.\\n- Don't choose a topic at random, make sure you have a way in, a new insight or approach.\\n- You don't need a complete thesis, just a gap to explore.\\n- You can get ideas by talking to people, reading, doing and building things, and going places and seeing things.\\n- You can improve the quality of your ideas by increasing the breadth and depth of what goes in.\\n- You can get breadth by reading and talking about a wide range of topics.\\n- You can get depth by doing and having to solve problems.\\n- You can also get ideas by talking to people who make you have new ideas.\\n\\nHere are some fun facts about the Roman Empire:\\n- At its peak, the empire stretched from North Africa to Britain, reigning over 60 million people.\\n- The story of Rome's warrior founder and the twins and their wolf step-mother was depicted on Rome's first coins.\\n- The Crossing of the Rhine in 405/6 AD brought around 100,000 barbarians into the Empire.\""
+ ],
+ "application/vnd.google.colaboratory.intrinsic+json": {
+ "type": "string"
+ }
+ },
+ "metadata": {},
+ "execution_count": 14
+ }
+ ]
+ },
+ {
+ "cell_type": "code",
+ "source": [],
+ "metadata": {
+ "id": "DIP1YkXCg7rQ"
+ },
+ "execution_count": 14,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "A question that requires the large language model to use a custom tool."
+ ],
+ "metadata": {
+ "id": "FhwS3VHvg3_l"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "response = agent_executor.invoke({\n",
+ " \"input\": \"Calculate the result of the random operation of 10 and 20. Then find a few fun facts about that number, as well as its prime factors.\",\n",
+ " \"preamble\": preamble,\n",
+ "})\n",
+ "\n",
+ "response['output']\n",
+ "\n",
+ "# note that the model uses a sequence of tools"
+ ],
+ "metadata": {
+ "colab": {
+ "base_uri": "https://localhost:8080/",
+ "height": 645
+ },
+ "id": "79Dw6Zhrg3xI",
+ "outputId": "a41c6de9-cb64-4cff-a58b-e9481d438f6c"
+ },
+ "execution_count": 22,
+ "outputs": [
+ {
+ "output_type": "stream",
+ "name": "stdout",
+ "text": [
+ "\n",
+ "\n",
+ "\u001b[1m> Entering new AgentExecutor chain...\u001b[0m\n",
+ "\u001b[32;1m\u001b[1;3m\n",
+ "First, I will calculate the result of the random operation between 10 and 20. Then, I will search for fun facts about that number and its prime factors.\n",
+ "{'tool_name': 'random_operation_tool', 'parameters': {'a': 10, 'b': 20}}\n",
+ "\u001b[0mrandom_operation_tool is not a valid tool, try one of [internet_search, vectorstore_search, python_interpreter, random_operation].\u001b[32;1m\u001b[1;3m\n",
+ "I received an error message when trying to use the random_operation_tool. I will now try using the python_interpreter tool to calculate the random operation between 10 and 20.\n",
+ "{'tool_name': 'python_interpreter', 'parameters': {'code': \"import random\\n\\n# Define the two numbers\\na = 10\\nb = 20\\n\\n# Calculate the random operation\\nresult = random.choice(['+', '-', '*', '/'])\\n\\n# Perform the operation\\nif result == '+':\\n answer = a + b\\nelif result == '-':\\n answer = a - b\\nelif result == '*':\\n answer = a * b\\nelif result == '/':\\n answer = a / b\\n\\nprint(f'The result of the random operation is {answer:.0f}')\"}}\n",
+ "\u001b[0m\u001b[38;5;200m\u001b[1;3mThe result of the random operation is 200\n",
+ "\u001b[0m\u001b[32;1m\u001b[1;3m\n",
+ "The result of the random operation is 200. Now I will search for fun facts about the number 200 and its prime factors.\n",
+ "{'tool_name': 'internet_search', 'parameters': {'query': 'fun facts about the number 200'}}\n",
+ "\u001b[0m\u001b[36;1m\u001b[1;3m[{'url': 'https://www.express.co.uk/life-style/top10facts/690340/Top-10-facts-number-200', 'content': \"Top 10 facts about the number 200 TODAY is the 200th day of 2016, so to celebrate let's have some facts about the number 200. By WILLIAM HARTSTON. 00:01, Mon, Jul 18, 2016.\"}, {'url': 'https://en.wikipedia.org/wiki/200_(number)', 'content': \"The number appears in the Padovan sequence, preceded by 86, 114, 151 (it is the sum of the first two of these). The sum of Euler's totient function φ(x) over the first twenty-five integers is 200. 200 is the smallest base 10 unprimeable number - it cannot be turned into a prime number by changing just one of its digits to any other digit.\"}, {'url': 'https://www.archimedes-lab.org/numbers/Num70_200.html', 'content': \"With 189 pages filled with an incredible variety of fun facts on numbers (and their peculiar properties), both mathematical and cultural, as well as tantalizing problems and anecdotes, there is much to learn for everyone. ... The number 200, according to Bullinger's study of biblical literature, signifies 'insufficiency'. The word 200 (ducenti) ...\"}, {'url': 'https://owlcation.com/misc/Over-200-Odd-Facts-Did-You-Know-Them', 'content': \"Over 200 odd facts about science, sports, history and more that you and your friends probably don't already know! ... Strange and Interesting Facts. ... Average number of people airborne over the U.S. at any given hour: 61,000. Portion of land in the U.S. owned by the government: 1/3. Ninety percent of New York City cabbies are recently arrived ...\"}, {'url': 'https://numbermatics.com/n/200/', 'content': 'Mathematical info, prime factorization, fun facts and numerical data for STEM, education and fun. Your guide to the number 200, an even composite number composed of two distinct primes. Mathematical info, prime factorization, fun facts and numerical data for STEM, education and fun. ... Number 200 - Facts about the integer. Retrieved 2 April ...'}]\u001b[0m\u001b[32;1m\u001b[1;3m\n",
+ "{'tool_name': 'internet_search', 'parameters': {'query': 'prime factors of 200'}}\n",
+ "\u001b[0m\u001b[36;1m\u001b[1;3m[{'url': 'https://www.cuemath.com/numbers/factors-of-200/', 'content': 'Therefore, the factors of 200 are 1, 2, 4, 5, 8, 10, 20, 25, 40, 50, 100, and 200. Which means the number 200 is an even composite number.'}, {'url': 'https://byjus.com/maths/factors-of-200/', 'content': \"The factors of 200 are 1, 2, 4, 5, 8, 10, 20, 25, 40, 50, 100 and 200. Visit BYJU'S to learn the pair factors and the prime factors of 200 with complete\\xa0...\"}, {'url': 'https://byjus.com/us/math/factors-of-200/', 'content': 'The factors of 200 are 1, 2, 4, 5, 8, 10, 20, 25, 40, 50, 100 and 200 because all these numbers divide the number 200 evenly.'}, {'url': 'https://homework.study.com/explanation/what-is-the-prime-factorization-of-200-using-exponents.html', 'content': 'The prime factorization of 200 using exponents is 2 3 ∗ 5 2 . First, we need to find the prime factorization of 200. 200 = 2 ∗ 100. 200 = 2 ∗ 2 ∗ 50.'}]\u001b[0m\u001b[32;1m\u001b[1;3mRelevant Documents: 1,3,4,6,7,8,9,10\n",
+ "Cited Documents: 1,3,4,6,7,8,9,10\n",
+ "Answer: The result of the random operation is **200**. Here are some fun facts about the number 200:\n",
+ "- It is the smallest base 10 unprimeable number, meaning it cannot be turned into a prime number by changing just one of its digits to any other digit.\n",
+ "- According to Bullinger's study of biblical literature, the number 200 signifies 'insufficiency'.\n",
+ "- The number 200 is an even composite number composed of two distinct primes.\n",
+ "\n",
+ "The prime factors of 200 are 2 and 5.\n",
+ "Grounded answer: The result of the random operation is **200**. Here are some fun facts about the number 200:\n",
+ "- It is the smallest base 10 unprimeable number, meaning it cannot be turned into a prime number by changing just one of its digits to any other digit.\n",
+ "- According to Bullinger's study of biblical literature, the number 200 signifies 'insufficiency'.\n",
+ "- The number 200 is an evencomposite number composed of two distinct primes.\n",
+ "\n",
+ "The prime factors of 200 are 2 and 5.\u001b[0m\n",
+ "\n",
+ "\u001b[1m> Finished chain.\u001b[0m\n"
+ ]
+ },
+ {
+ "output_type": "execute_result",
+ "data": {
+ "text/plain": [
+ "\"The result of the random operation is **200**. Here are some fun facts about the number 200:\\n- It is the smallest base 10 unprimeable number, meaning it cannot be turned into a prime number by changing just one of its digits to any other digit.\\n- According to Bullinger's study of biblical literature, the number 200 signifies 'insufficiency'.\\n- The number 200 is an even composite number composed of two distinct primes.\\n\\nThe prime factors of 200 are 2 and 5.\""
+ ],
+ "application/vnd.google.colaboratory.intrinsic+json": {
+ "type": "string"
+ }
+ },
+ "metadata": {},
+ "execution_count": 22
+ }
+ ]
+ },
+ {
+ "cell_type": "code",
+ "source": [],
+ "metadata": {
+ "id": "V8LcAh8vaEqR"
+ },
+ "execution_count": 15,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "A question that requires the large language model to directly answer."
+ ],
+ "metadata": {
+ "id": "n9nV_jiaAaD1"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "response = agent_executor.invoke({\n",
+ " \"input\": \"Hey how are you?\",\n",
+ " \"preamble\": preamble,\n",
+ "})\n",
+ "\n",
+ "response['output']\n",
+ "\n",
+ "# note that the modle can directly answer!"
+ ],
+ "metadata": {
+ "colab": {
+ "base_uri": "https://localhost:8080/",
+ "height": 250
+ },
+ "id": "tRf6V3gJAZkO",
+ "outputId": "aec1acdb-876c-48e9-a1f5-e3ba8a0f19ed"
+ },
+ "execution_count": 23,
+ "outputs": [
+ {
+ "output_type": "stream",
+ "name": "stdout",
+ "text": [
+ "\n",
+ "\n",
+ "\u001b[1m> Entering new AgentExecutor chain...\u001b[0m\n",
+ "\u001b[32;1m\u001b[1;3m\n",
+ "I will respond to the user's greeting.\n",
+ "{'tool_name': 'directly_answer', 'parameters': {}}\n",
+ "\u001b[0mdirectly_answer is not a valid tool, try one of [internet_search, vectorstore_search, python_interpreter, random_operation].\u001b[32;1m\u001b[1;3mRelevant Documents: None\n",
+ "Cited Documents: None\n",
+ "Answer: I'm an AI chatbot, so I don't have feelings as such, but I'm here to help you with your queries. How can I help?\n",
+ "Grounded answer: I'm an AI chatbot, so I don't have feelings as such, but I'm here to help you with your queries. How can I help?\u001b[0m\n",
+ "\n",
+ "\u001b[1m> Finished chain.\u001b[0m\n"
+ ]
+ },
+ {
+ "output_type": "execute_result",
+ "data": {
+ "text/plain": [
+ "\"I'm an AI chatbot, so I don't have feelings as such, but I'm here to help you with your queries. How can I help?\""
+ ],
+ "application/vnd.google.colaboratory.intrinsic+json": {
+ "type": "string"
+ }
+ },
+ "metadata": {},
+ "execution_count": 23
+ }
+ ]
+ },
+ {
+ "cell_type": "code",
+ "source": [],
+ "metadata": {
+ "id": "hnLkln_ckYXJ"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "## Ask a more complex question to the ReAct agent\n",
+ "A question that requires using multipe tools, in sequence"
+ ],
+ "metadata": {
+ "id": "3h-icRL2_5iu"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "response = agent_executor.invoke({\n",
+ " \"input\": \"In what year was the company that was founded as Sound of Music went public? What was its stock price in 2000 and 2010.\",\n",
+ " \"preamble\": preamble,\n",
+ "})\n",
+ "\n",
+ "response['output']"
+ ],
+ "metadata": {
+ "colab": {
+ "base_uri": "https://localhost:8080/",
+ "height": 377
+ },
+ "id": "Q4IyoXXRaEoL",
+ "outputId": "0e935f68-8b7c-4d9a-b8f1-1be57b7c9a1f"
+ },
+ "execution_count": 27,
+ "outputs": [
+ {
+ "output_type": "stream",
+ "name": "stdout",
+ "text": [
+ "\n",
+ "\n",
+ "\u001b[1m> Entering new AgentExecutor chain...\u001b[0m\n",
+ "\u001b[32;1m\u001b[1;3m\n",
+ "I will search for the company that was founded as Sound of Music. Then, I will search for the year it went public. Finally, I will search for its stock price in 2000 and 2010.\n",
+ "{'tool_name': 'internet_search', 'parameters': {'query': 'company founded as Sound of Music'}}\n",
+ "\u001b[0m\u001b[36;1m\u001b[1;3m[{'url': 'https://www.mprnews.org/story/2012/05/15/best-buy-richard-schulze-legacy', 'content': 'Amazon is taking a piece of everyone\\'s business,\"\\nSome analysts have questioned the extent to which Schulze\\'s strong hand over the company has held it back in the face of digital competition. \"\\nSchulze hit on the \"Best Buy\" strategy when a twister ripped open a store and the company held a huge sale with low prices and lots of promotion. And Richard Schulz was able to outthink and outmaneuver the competition to put up Best Buy as the definitive place in the brick-and-mortar world to buy electronic equipment,\" Spector said.\\n Best Buy\\'s Richard Schulze: Stereo seller to retail giant\\nShare\\nIn 1966, Best Buy was a stereo specialty retailer called Sound of Music, founded by Richard Schulze in St. Paul.\\n Former CEO Anderson said it is possible that Schulze\\'s goals for the company were out of touch with the digital age, and that Schulze may have stuck around too long.\\n'}, {'url': 'https://corporate.bestbuy.com/tag/sound-of-music/', 'content': 'As we celebrate Best Buy’s 50th anniversary, we sat down with founder and Chairman Emeritus Dick Schulze and current Chairman and CEO Hubert Joly to talk about how we got to where we are today and get a glimpse into where we’re going next.\\n Sound of Music\\n22 Aug: Best Buy at 50: A Q&A with Founder Dick Schulze and CEO Hubert Joly\\nTechnology has changed a lot over the past half century, and so has Best Buy.\\n 14 Jun: 35 Years Ago Today, A Tornado Transformed Best Buy\\nOn the afternoon of June 14, 1981, a tornado slammed into the Minneapolis suburbs, mere miles from where the Best Buy corporate headquarters now stands.\\n Little did he know that it would become Best Buy, a nearly $40 billion company that now sells everything from TVs and laptops to drones and virtual-reality headsets.\\n It was the most significant tornado to hit the Minneapolis-St. Paul area in 20 years — and its wake would change the path of our company.\\n'}, {'url': 'https://historydraft.com/story/best-buy/timeline/841', 'content': \"Best Buy announced the shutdown of the Future Shop chain in Canada\\nOn March 28, 2015, Best Buy announced the shutdown of the Future Shop chain in Canada; 65 of its 131 former locations were converted into Best Buy locations, while the rest (primarily those in close proximity to an existing Best Buy) were closed permanently.\\n Best Buy Company, Inc\\nIn 1983, with seven stores and $10 million in annual sales, Sound of Music was renamed Best Buy Company, Inc.\\nBest Buy debuted on the New York Stock Exchange\\nBest Buy was taken public in 1985, and two years later it debuted on the New York Stock Exchange.\\n The company closed all of its Best Buy-branded stores in China\\nThe company closed all of its Best Buy-branded stores in China by February 2011, when it merged Best Buy China's operations with Jiangsu Five Star, which had become a wholly-owned subsidiary of Best Buy in 2009.\\n Best Buy announced that it would start selling musical instruments and related gear in over 80 of its retail stores\\nIn July 2008, Best Buy announced that it would start selling musical instruments and related gear in over 80 of its retail stores, making the company the second-largest musical-instrument distributor in the US.\\n Best Buy hired Virtucom Group to revamp Best Buy's website and handle all of the company's online content\\nIn January 2004, Best Buy hired Virtucom Group to revamp Best Buy's website and handle all of the company's online content.\\n\"}, {'url': 'https://en.wikipedia.org/wiki/Best_Buy', 'content': 'Under the Geek Squad brand, Best Buy offers computer repair, warranty service, and accidental service plans.[2] Best Buy provides an online community forum for members, where consumers can discuss product experiences, ask questions, and get answers from other members or retail product experts.[82]\\nThe building exteriors of Best Buy-branded stores are typically light brown, with the entrance designed to look like a blue box emerging from the structure.[83] Corporate employees operated under a results only work environment from 2005 until March 2013, when the management style was abandoned by Best Buy CEO Hubert Joly.[84][85]\\nAs of October 29, 2016, Best Buy operated 1,026 Best Buy, 331 Best Buy Mobile stand-alone stores, and 28 stand-alone Pacific Sales stores in the US.[2] Best Buy also operated: 135 Best Buy and 53 Best Buy Mobile stand-alone stores in Canada; and 18 Best Buy stores and 5 Best Buy Express stores in Mexico.[2] Best Buy exited the European market in April 2013, selling its stake in the business back to its partner Carphone Warehouse.[71][72]\\nHouse brands[edit]\\nBest Buy also produces products under eight house brands:[2]\\nControversies[edit]\\nWarranty[edit]\\n The company, in announcing the result, said it was focusing more on digital media in its marketing, moving away from newspaper, magazine, and television advertising.[73]\\nOn March 28, 2015, Best Buy announced the shutdown of the Future Shop chain in Canada; 65 of its 131 former locations were converted into Best Buy locations, while the rest (primarily those in close proximity to an existing Best Buy) were closed permanently.[74]\\nOn March 1, 2018, the company announced that it would shut down its 250 standalone Best Buy Mobile stores in the United States by the end of May, due to low revenue and high costs. In August 2022, Best Buy said it would be laying off employees across the country after warnings of weaker sales, and the company cut its forecast for the remainder of 2022.[79]\\nOn October 13, 2023, Best Buy announced that it would phase out the sale of physical home media in early 2024, citing changes in the market due to the prevalence of streaming video on demand services.[80][81]\\nCorporate affairs[edit]\\nBusiness operations[edit]\\nBest Buy sells consumer electronics and a variety of related merchandise, including software, video games, music, mobile phones, digital cameras, car stereos, and video cameras, in addition to home appliances (washing machines, dryers, and refrigerators), in a noncommissioned sales environment.[2] The Best Buy Mobile stores were reported to account for 1% of the company\\'s revenue.[75]\\nOn May 9, 2018, the company unveiled a new logo for the first time in nearly three decades.[76]\\nOn July 2, 2018, Best Buy announced it was cutting the amount of store space devoted to selling physical music, citing the popularity of streaming services as having reduced sales.[77]\\nOn April 15, 2019, Best Buy announced that in June 2019, its current CFO, Corie Barry, would replace Hubert Joly[5] who held the position of CEO since August 2012. The customer was indicted for possession of child pornography, although the judge in the case later threw out nearly all the evidence against the defendant due to \"false and misleading statements\" made by an FBI agent while trying to secure a search warrant for the customer\\'s house, and the government ultimately dropped the case.[97]\\nPrivacy[edit]\\nOn October 20, 2023, CBC News released the results of a Marketplace investigation which found that that Best Buy technicians had viewed private files, such as intimate photos, on customer devices.'}, {'url': 'https://www.company-histories.com/Best-Buy-Co-Inc-Company-History.html', 'content': 'As the Best Buy chain pushed past the 500-store mark in 2003 with the opening of 67 new stores in the United States, including the first stores in Alaska, Idaho, Utah, and West Virginia, the situation at the Musicland chains was deteriorating. Returning to the Core: 2003 and Beyond\\nDespite the completion of this acquisition, Best Buy pushed ahead with a previously planned expansion of the Best Buy chain into Canada, opening eight stores in the Toronto area in the fall of 2002. Significant changes were made to the product mix, particularly by eliminating slower selling product lines and models; a greater emphasis was placed on selling service plans to customers; and \"high touch\" areas were added to the stores to help sell the burgeoning array of digital consumer products, such as cameras, cellular phones, satellite systems, and the fast-selling DVD player (first introduced in 1996) for which customers often needed more assistance. This rapid growth in digital product sales also inspired a shift in the overall product mix: sales of consumer electronics products (33 percent of the total) surpassed the sales of home office products (31 percent) for the first time (in 1999 these figures were 27 percent and 36 percent, respectively). Magnolia was founded in 1954 by Len Tweten, who built the firm into one of the most respected audio-video retailers in the nation based on the high quality of its merchandise, its dedication to exceptional customer service, and its renowned in-house repair/installation department.'}]\u001b[0m\u001b[32;1m\u001b[1;3m\n",
+ "I have found that Best Buy was founded as Sound of Music. Now, I will search for the year it went public and its stock price in 2000 and 2010.\n",
+ "{'tool_name': 'internet_search', 'parameters': {'query': 'Best Buy went public'}}\n",
+ "\u001b[0m\u001b[36;1m\u001b[1;3m[{'url': 'https://www.zippia.com/best-buy-careers-1455/history/', 'content': 'Meantime, Best Buy was taken public in 1985, raising $8 million through an IPO, and two years later gained a listing on the New York Stock Exchange (NYSE). ... Best Buy may also be known as or be related to Best Buy, Best Buy Co Inc, Best Buy Co., Inc., Sound of Music (1966-1983) Best Buy Co. Superstores (1983-1984) Best Buy Superstores ...'}, {'url': 'https://atouchofbusiness.com/companies/best-buy/', 'content': 'Public Listing and IPO: Best Buy went public in 1985 and listed on the NYSE in 1987. Conceptual Changes: The late 1980s and 1990s brought new store formats and growth, with revenues surpassing $1 billion. Overcoming Market Challenges. Supplier Relations: Mid-1990s challenges led to a merchandising revamp.'}, {'url': 'https://www.encyclopedia.com/social-sciences-and-law/economics-business-and-labor/businesses-and-occupations/best-buy-co-inc', 'content': 'Best Buy grew rapidly with $28.5 million in sales in 1984—compared to $9.9 million in 1983. In 1985, with 9 stores, Best Buy became a public company. By 1987, Best Buy had 24 stores and sales of $240 million, but it was beginning to feel the crunch as other rapidly expanding consumer electronics retailers pushed their way into the market.'}, {'url': 'https://www.company-histories.com/Best-Buy-Co-Inc-Company-History.html', 'content': 'As the Best Buy chain pushed past the 500-store mark in 2003 with the opening of 67 new stores in the United States, including the first stores in Alaska, Idaho, Utah, and West Virginia, the situation at the Musicland chains was deteriorating. Returning to the Core: 2003 and Beyond\\nDespite the completion of this acquisition, Best Buy pushed ahead with a previously planned expansion of the Best Buy chain into Canada, opening eight stores in the Toronto area in the fall of 2002. Significant changes were made to the product mix, particularly by eliminating slower selling product lines and models; a greater emphasis was placed on selling service plans to customers; and \"high touch\" areas were added to the stores to help sell the burgeoning array of digital consumer products, such as cameras, cellular phones, satellite systems, and the fast-selling DVD player (first introduced in 1996) for which customers often needed more assistance. This rapid growth in digital product sales also inspired a shift in the overall product mix: sales of consumer electronics products (33 percent of the total) surpassed the sales of home office products (31 percent) for the first time (in 1999 these figures were 27 percent and 36 percent, respectively). Magnolia was founded in 1954 by Len Tweten, who built the firm into one of the most respected audio-video retailers in the nation based on the high quality of its merchandise, its dedication to exceptional customer service, and its renowned in-house repair/installation department.'}, {'url': 'https://en.wikipedia.org/wiki/Best_Buy', 'content': 'Under the Geek Squad brand, Best Buy offers computer repair, warranty service, and accidental service plans.[2] Best Buy provides an online community forum for members, where consumers can discuss product experiences, ask questions, and get answers from other members or retail product experts.[82]\\nThe building exteriors of Best Buy-branded stores are typically light brown, with the entrance designed to look like a blue box emerging from the structure.[83] Corporate employees operated under a results only work environment from 2005 until March 2013, when the management style was abandoned by Best Buy CEO Hubert Joly.[84][85]\\nAs of October 29, 2016, Best Buy operated 1,026 Best Buy, 331 Best Buy Mobile stand-alone stores, and 28 stand-alone Pacific Sales stores in the US.[2] Best Buy also operated: 135 Best Buy and 53 Best Buy Mobile stand-alone stores in Canada; and 18 Best Buy stores and 5 Best Buy Express stores in Mexico.[2] Best Buy exited the European market in April 2013, selling its stake in the business back to its partner Carphone Warehouse.[71][72]\\nHouse brands[edit]\\nBest Buy also produces products under eight house brands:[2]\\nControversies[edit]\\nWarranty[edit]\\n The company, in announcing the result, said it was focusing more on digital media in its marketing, moving away from newspaper, magazine, and television advertising.[73]\\nOn March 28, 2015, Best Buy announced the shutdown of the Future Shop chain in Canada; 65 of its 131 former locations were converted into Best Buy locations, while the rest (primarily those in close proximity to an existing Best Buy) were closed permanently.[74]\\nOn March 1, 2018, the company announced that it would shut down its 250 standalone Best Buy Mobile stores in the United States by the end of May, due to low revenue and high costs. In August 2022, Best Buy said it would be laying off employees across the country after warnings of weaker sales, and the company cut its forecast for the remainder of 2022.[79]\\nOn October 13, 2023, Best Buy announced that it would phase out the sale of physical home media in early 2024, citing changes in the market due to the prevalence of streaming video on demand services.[80][81]\\nCorporate affairs[edit]\\nBusiness operations[edit]\\nBest Buy sells consumer electronics and a variety of related merchandise, including software, video games, music, mobile phones, digital cameras, car stereos, and video cameras, in addition to home appliances (washing machines, dryers, and refrigerators), in a noncommissioned sales environment.[2] The Best Buy Mobile stores were reported to account for 1% of the company\\'s revenue.[75]\\nOn May 9, 2018, the company unveiled a new logo for the first time in nearly three decades.[76]\\nOn July 2, 2018, Best Buy announced it was cutting the amount of store space devoted to selling physical music, citing the popularity of streaming services as having reduced sales.[77]\\nOn April 15, 2019, Best Buy announced that in June 2019, its current CFO, Corie Barry, would replace Hubert Joly[5] who held the position of CEO since August 2012. The customer was indicted for possession of child pornography, although the judge in the case later threw out nearly all the evidence against the defendant due to \"false and misleading statements\" made by an FBI agent while trying to secure a search warrant for the customer\\'s house, and the government ultimately dropped the case.[97]\\nPrivacy[edit]\\nOn October 20, 2023, CBC News released the results of a Marketplace investigation which found that that Best Buy technicians had viewed private files, such as intimate photos, on customer devices.'}]\u001b[0m\u001b[32;1m\u001b[1;3m\n",
+ "{'tool_name': 'internet_search', 'parameters': {'query': 'Best Buy stock price in 2000 and 2010'}}\n",
+ "\u001b[0m\u001b[36;1m\u001b[1;3m[{'url': 'https://finance.yahoo.com/quote/BBY/history/', 'content': 'Discover historical prices for BBY stock on Yahoo Finance. View daily, weekly or monthly format back to when Best Buy Co., Inc. stock was issued.'}, {'url': 'https://www.macrotrends.net/stocks/charts/BBY/best-buy/stock-price-history', 'content': 'Best Buy - 39 Year Stock Price History | BBY. Prices ... 2010, 25.1877, 25.7674, 31.2441, 20.2713, 22.3228 ... 2000, 16.0083, 15.2882, 22.8658, 5.9318, 7.8595, -\\xa0...'}, {'url': 'https://investors.bestbuy.com/investor-relations/stock-info/quote-and-chart/', 'content': 'Price 79.30. Change -0.88. Volume 666,622.'}, {'url': 'https://companiesmarketcap.com/best-buy/stock-price-history/', 'content': 'Stock price history for Best Buy (BBY). Highest end of day price: $138.00 USD on 2021-11-22. Lowest end of day price: $0.14 USD on 1985-05-02\\xa0...'}, {'url': 'https://www.netcials.com/stock-price-chart-history-nyse/BBY-Best-Buy-Co-Inc/', 'content': '1 Best Buy Co Inc (BBY) 20 Years Stock Chart History ; 2000, 16.24 (17.6%), 23.199 (29.07%) ; 2001, 14.80 (-8.87%), 20.1319 (-13.22%) ; 2002, 14.59 (-1.42%)\\xa0...'}]\u001b[0m\u001b[32;1m\u001b[1;3mRelevant Documents: 0,1,2,3,4,5,6,7,11,13,14\n",
+ "Cited Documents: 0,1,2,3,4,5,6,7,11,14\n",
+ "Answer: Best Buy, which was founded as Sound of Music, went public in 1985. Its stock price in 2000 was between $5.93 and $16.24 and in 2010, it was between $20.27 and $31.24.\n",
+ "Grounded answer: Best Buy, which was founded as Sound of Music, went public in 1985. Its stock price in 2000 was between $5.93 and $16.24 and in 2010, it was between $20.27 and $31.24.\u001b[0m\n",
+ "\n",
+ "\u001b[1m> Finished chain.\u001b[0m\n"
+ ]
+ },
+ {
+ "output_type": "execute_result",
+ "data": {
+ "text/plain": [
+ "'Best Buy, which was founded as Sound of Music, went public in 1985. Its stock price in 2000 was between $5.93 and $16.24 and in 2010, it was between $20.27 and $31.24.'"
+ ],
+ "application/vnd.google.colaboratory.intrinsic+json": {
+ "type": "string"
+ }
+ },
+ "metadata": {},
+ "execution_count": 27
+ }
+ ]
+ },
+ {
+ "cell_type": "code",
+ "source": [],
+ "metadata": {
+ "id": "L1BsKueTaEmg"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "## Have a multi-turn conversation with the ReAct agent\n",
+ "The chat history enables you to have multi-turn conversations with the ReAct agent."
+ ],
+ "metadata": {
+ "id": "53FDSFzQBFyf"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Step 1: Construct the chat history as a list of LangChain Messages, ending with the last user message\n",
+ "from langchain_core.messages import HumanMessage, AIMessage\n",
+ "\n",
+ "chat_history = [\n",
+ " HumanMessage(content=\"I'm considering switching to Oracle for my CRM.\"),\n",
+ " AIMessage(content=\"That sounds like a good idea! How can I help you?\"),\n",
+ " HumanMessage(content=\"Recap me all the info you can find about their offering.\"),\n",
+ "]\n",
+ "\n",
+ "prompt = ChatPromptTemplate.from_messages(chat_history)"
+ ],
+ "metadata": {
+ "id": "1iYyAExsaEk9"
+ },
+ "execution_count": 28,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Step 2: When you make the agent, specify the chat_history as the prompt, e.g.\n",
+ "# Create the ReAct agent\n",
+ "agent = create_cohere_react_agent(\n",
+ " llm=llm,\n",
+ " tools=[internet_search, vectorstore_search, python_tool],\n",
+ " prompt=prompt,\n",
+ ")\n",
+ "\n",
+ "agent_executor = AgentExecutor(agent=agent, tools=[internet_search, vectorstore_search, python_tool], verbose=True)"
+ ],
+ "metadata": {
+ "id": "fyv_1LedaEi7"
+ },
+ "execution_count": 29,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Step 3: When you invoke the agent_executor there's no need to pass anything else into invoke\n",
+ "response = agent_executor.invoke({\n",
+ " \"preamble\": preamble,\n",
+ "})\n",
+ "\n",
+ "response['output']"
+ ],
+ "metadata": {
+ "id": "-ZCFj-m5nqFw",
+ "colab": {
+ "base_uri": "https://localhost:8080/",
+ "height": 538
+ },
+ "outputId": "3d71931e-1644-49b9-903c-4ff117787868"
+ },
+ "execution_count": 30,
+ "outputs": [
+ {
+ "output_type": "stream",
+ "name": "stdout",
+ "text": [
+ "\n",
+ "\n",
+ "\u001b[1m> Entering new AgentExecutor chain...\u001b[0m\n",
+ "\u001b[32;1m\u001b[1;3m\n",
+ "I will search for 'Oracle CRM offering' and relay the information I find to the user.\n",
+ "{'tool_name': 'internet_search', 'parameters': {'query': 'Oracle CRM offering'}}\n",
+ "\u001b[0m\u001b[36;1m\u001b[1;3m[{'url': 'https://docs.oracle.com/en/applications/siebel/', 'content': 'Siebel CRM delivers a combination of transactional, analytical, and engagement features to manage all customer-facing operations. With solutions tailored to more than 20 industries, Siebel CRM delivers comprehensive on-premise CRM solutions that are tailored industry solutions with role-based customer intelligence and pre-built integration.'}, {'url': 'https://www.softwareadvice.com/resources/breaking-down-oracle-crm/', 'content': \"Oracle's Marketing Cloud provides a comprehensive set of tools that cover a range of digital marketing needs. This includes tools for cross-channel marketing, i.e., marketing automation for both B2B and B2C marketers, as well as data management, content marketing and social media marketing. Cross-Channel Marketing.\"}, {'url': 'https://www.trustradius.com/products/oracle-crm-on-demand/reviews?qs=pros-and-cons', 'content': \"What is Oracle CRM On Demand?The basis of this offering is the Market2Lead product that Oracle acquired in 2010. It has now been fully integrated with Oracle's On Demand CRM product and is a full-featured marketing automation product with features from lead management and nurturing, to measuring marketing ROI.\"}, {'url': 'https://www.oracle.com/cx/siebel/', 'content': 'In addition to standard CRM functionality, this industry solution includes asset, premises, and contracts management; work orders; oil well management and oil field service; B2C and B2B web portals; and credit and fraud management capabilities.\\nDesigned for pharmaceutical sales, Siebel CRM Life Sciences provides personalized content delivery tools to help sales and marketing teams deliver the right messages during each customer interaction.\\n Marketing, sales, and customer service applications are fully integrated and designed to manage the complex interactions and relationships between brand owners, their partners (including brokers and distributors), their customers, and the end consumer.\\n It leverages data and AI to transform CX–launching offers, acquiring and retaining customers, omnichannel ecommerce and customer care, and fulfilling and monetizing services.\\n It leverages data and AI to transform CX–launching offers, acquiring and retaining customers, omnichannel ecommerce and customer care, and fulfilling and monetizing services.\\n Provide world-class citizen services while delivering comprehensive, cost-efficient case management and policy management, including social services, justice and public safety, constituent services/311, self-service citizen portals, tax and revenue, and licensing and permitting.\\n'}, {'url': 'https://www.suretysystems.com/insights/oracle-customer-relationship-management-a-complete-overview/', 'content': 'Effective CRM systems offer the following features for enterprise users: Simple, easy-to-use interface; ... Oracle CRM simplifies customer relationship management by enhancing customer interactions and improving customer satisfaction and sales growth. Oracle Client Experience (CX), a connected suite of tools that transcends standard CRM ...'}]\u001b[0m\u001b[32;1m\u001b[1;3mRelevant Documents: 0,1,2,3,4\n",
+ "Cited Documents: 0,1,2,3,4\n",
+ "Answer: Oracle's CRM offering includes the following:\n",
+ "- Marketing Cloud, which provides a comprehensive set of tools that cover a range of digital marketing needs, including cross-channel marketing, marketing automation, data management, content marketing and social media marketing\n",
+ "- CRM On Demand, which is a full-featured marketing automation product with features from lead management and nurturing, to measuring marketing ROI\n",
+ "- Siebel CRM, which delivers a combination of transactional, analytical, and engagement features to manage all customer-facing operations, with solutions tailored to more than 20 industries\n",
+ "- Siebel CRM Life Sciences, which provides personalised content delivery tools to help sales and marketing teams deliver the right messages during each customer interaction\n",
+ "- Oracle Client Experience (CX), a connected suite of tools that transcends standard CRM\n",
+ "Grounded answer: Oracle's CRM offering includes the following:\n",
+ "- Marketing Cloud, which provides a comprehensive set of tools that cover a range of digital marketing needs, including cross-channel marketing, marketing automation, data management, content marketing and social media marketing\n",
+ "- CRM On Demand, which is a full-featured marketing automation product with features from lead management and nurturing, to measuring marketing ROI\n",
+ "- Siebel CRM, which delivers a combination of transactional, analytical, and engagement features to manage all customer-facing operations, with solutions tailored to more than 20 industries\n",
+ "- Siebel CRM Life Sciences, which provides personalised content delivery tools to help sales and marketing teams deliver the right messages during each customer interaction\n",
+ "- Oracle Client Experience (CX), a connected suite of tools that transcends standard CRM\u001b[0m\n",
+ "\n",
+ "\u001b[1m> Finished chain.\u001b[0m\n"
+ ]
+ },
+ {
+ "output_type": "execute_result",
+ "data": {
+ "text/plain": [
+ "\"Oracle's CRM offering includes the following:\\n- Marketing Cloud, which provides a comprehensive set of tools that cover a range of digital marketing needs, including cross-channel marketing, marketing automation, data management, content marketing and social media marketing\\n- CRM On Demand, which is a full-featured marketing automation product with features from lead management and nurturing, to measuring marketing ROI\\n- Siebel CRM, which delivers a combination of transactional, analytical, and engagement features to manage all customer-facing operations, with solutions tailored to more than 20 industries\\n- Siebel CRM Life Sciences, which provides personalised content delivery tools to help sales and marketing teams deliver the right messages during each customer interaction\\n- Oracle Client Experience (CX), a connected suite of tools that transcends standard CRM\""
+ ],
+ "application/vnd.google.colaboratory.intrinsic+json": {
+ "type": "string"
+ }
+ },
+ "metadata": {},
+ "execution_count": 30
+ }
+ ]
+ },
+ {
+ "cell_type": "code",
+ "source": [],
+ "metadata": {
+ "id": "8Zh01M4hWw3L"
+ },
+ "execution_count": 30,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [],
+ "metadata": {
+ "id": "0qEPY6xj_qwr"
+ },
+ "execution_count": null,
+ "outputs": []
+ }
+ ]
+}
\ No newline at end of file
diff --git a/notebooks/.ipynb_checkpoints/Vanilla_RAG-checkpoint.ipynb b/notebooks/.ipynb_checkpoints/Vanilla_RAG-checkpoint.ipynb
new file mode 100644
index 00000000..2f85039c
--- /dev/null
+++ b/notebooks/.ipynb_checkpoints/Vanilla_RAG-checkpoint.ipynb
@@ -0,0 +1,743 @@
+{
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "mz33G3t6gbOl"
+ },
+ "source": [
+ "# RAG\n",
+ "\n",
+ "Retrieval-Augmented Generation (RAG) is a technique that combines the strengths of pre-trained language models with the ability to retrieve information from a large corpus of documents. RAG **enables the language model to produce more informed, accurate, and contextually relevant answers** than by relying on its pre-trained knowledge alone.\n",
+ "\n",
+ "At Cohere, all RAG calls come with... **precise citations**! 🎉\n",
+ "The model cites which groups of words, in the RAG chunks, were used to generate the final answer. \n",
+ "These citations make it easy to check where the model’s generated response claims are coming from and they help users gain visibility into the model reasoning. \n",
+ "\n",
+ "RAG consists of 3 steps:\n",
+ "- Step 1: Indexing and given a user query, retrieve the relevant chunks from the index\n",
+ "- Step 2: Optionally, rerank the retrieved chunks\n",
+ "- Step 3: Generate the model final answer with **precise citations**, given the retrieved and reranked chunks\n",
+ "\n",
+ "\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "nSB0pnt0gbOo"
+ },
+ "source": [
+ "## Step 0 - Imports & Getting some data\n",
+ "\n",
+ "In this example, we'll use a recent piece of text, that wasn't in the training data: the Wikipedia page of the movie \"Dune 2\". \n",
+ "\n",
+ "In practice, you would typically do RAG on much longer text, that doesn't fit in the context window of the model."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "colab": {
+ "base_uri": "https://localhost:8080/"
+ },
+ "id": "H787BXXYvD0a",
+ "outputId": "04ef5e04-7760-4d40-deeb-663536b38f20"
+ },
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m52.8/52.8 kB\u001b[0m \u001b[31m1.6 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
+ "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m3.1/3.1 MB\u001b[0m \u001b[31m33.6 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
+ "\u001b[?25h"
+ ]
+ }
+ ],
+ "source": [
+ "# we'll use Cohere to cover all building blocks of RAG\n",
+ "# TODO: upgrade to \"cohere>5\"\n",
+ "%pip install \"cohere<5\" --quiet"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "rACbepFGgbOo"
+ },
+ "outputs": [],
+ "source": [
+ "import cohere\n",
+ "API_KEY = \"...\" # fill in your Cohere API key here\n",
+ "co = cohere.Client(API_KEY)"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "colab": {
+ "base_uri": "https://localhost:8080/"
+ },
+ "id": "QdvbqfFrgbOq",
+ "outputId": "3882c95c-46bf-4dcc-99a2-453b3c2fc7c4"
+ },
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ " Preparing metadata (setup.py) ... \u001b[?25l\u001b[?25hdone\n",
+ " Building wheel for wikipedia (setup.py) ... \u001b[?25l\u001b[?25hdone\n"
+ ]
+ }
+ ],
+ "source": [
+ "# we'll get some wikipedia data\n",
+ "!pip install wikipedia --quiet\n",
+ "import wikipedia"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "colab": {
+ "base_uri": "https://localhost:8080/"
+ },
+ "id": "xP-bWt9XgbOq",
+ "outputId": "72276fb2-0d6b-415d-af74-452a013ae84b"
+ },
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "The text has roughly 5323 words.\n"
+ ]
+ }
+ ],
+ "source": [
+ "# let's get the wikipedia article about Dune Part Two\n",
+ "article = wikipedia.page('Dune Part Two')\n",
+ "text = article.content\n",
+ "print(f\"The text has roughly {len(text.split())} words.\")"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "-1aJ7hKGgbOr"
+ },
+ "source": [
+ "## Step 1 - Indexing and given a user query, retrieve the relevant chunks from the index\n",
+ "\n",
+ "We index the document in a vector database. This requires getting the documents, chunking them, embedding, and indexing them in a vector database. Then we retrieved relevant results based on the users' query.\n",
+ "\n",
+ "### We split the document into chunks of roughly 512 words"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "colab": {
+ "base_uri": "https://localhost:8080/"
+ },
+ "id": "ZUph1JX41665",
+ "outputId": "6c63a93f-6999-47af-e704-d4a88727bc75"
+ },
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m256.9/256.9 kB\u001b[0m \u001b[31m6.6 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
+ "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m66.6/66.6 kB\u001b[0m \u001b[31m8.2 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
+ "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m138.5/138.5 kB\u001b[0m \u001b[31m14.5 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
+ "\u001b[?25h"
+ ]
+ }
+ ],
+ "source": [
+ "# For chunking let's use langchain to help us split the text\n",
+ "%pip install -qU langchain-text-splitters --quiet\n",
+ "from langchain_text_splitters import RecursiveCharacterTextSplitter"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "colab": {
+ "base_uri": "https://localhost:8080/"
+ },
+ "id": "uhXW7iHC1-Q6",
+ "outputId": "d68ac348-4b73-4c6a-a445-6c510bdb0881"
+ },
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "The text has been broken down in 91 chunks.\n"
+ ]
+ }
+ ],
+ "source": [
+ "# Create basic configurations to chunk the text\n",
+ "text_splitter = RecursiveCharacterTextSplitter(\n",
+ " chunk_size=512,\n",
+ " chunk_overlap=50,\n",
+ " length_function=len,\n",
+ " is_separator_regex=False,\n",
+ ")\n",
+ "\n",
+ "# Split the text into chunks with some overlap\n",
+ "chunks_ = text_splitter.create_documents([text])\n",
+ "chunks = [c.page_content for c in chunks_]\n",
+ "print(f\"The text has been broken down in {len(chunks)} chunks.\")"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "P8g0sE2hgbOs"
+ },
+ "source": [
+ "### Embed every text chunk\n",
+ "\n",
+ "Cohere embeddings are state-of-the-art."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "colab": {
+ "base_uri": "https://localhost:8080/"
+ },
+ "id": "KEarMPEqgbOs",
+ "outputId": "7da0e06d-f637-4470-8e01-6de8249be64b"
+ },
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "We just computed 91 embeddings.\n"
+ ]
+ }
+ ],
+ "source": [
+ "# Because the texts being embedded are the chunks we are searching over, we set the input type as search_doc\n",
+ "model=\"embed-english-v3.0\"\n",
+ "response = co.embed(\n",
+ " texts= chunks,\n",
+ " model=model,\n",
+ " input_type=\"search_document\",\n",
+ " embedding_types=['float']\n",
+ ")\n",
+ "embeddings = response.embeddings.float\n",
+ "print(f\"We just computed {len(embeddings)} embeddings.\")"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "HM6vKeypgbOs"
+ },
+ "source": [
+ "### Store the embeddings in a vector database\n",
+ "\n",
+ "We use the simplest vector database ever: a python dictionary using `np.array()`."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "sdW7M8HLvB-9"
+ },
+ "outputs": [],
+ "source": [
+ "# We use the simplest vector database ever: a python dictionary\n",
+ "!pip install numpy --quiet"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "H2srFH-IgbOs"
+ },
+ "outputs": [],
+ "source": [
+ "import numpy as np\n",
+ "vector_database = {i: np.array(embedding) for i, embedding in enumerate(embeddings)}\n",
+ "# { 0: array([...]), 1: array([...]), 2: array([...]), ..., 10: array([...]) }"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "q6NGVurZgbOs"
+ },
+ "source": [
+ "## Given a user query, retrieve the relevant chunks from the vector database\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "eC05yJQ7jlek"
+ },
+ "source": [
+ "### Define the user question"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "Y2HTxspKgbOs"
+ },
+ "outputs": [],
+ "source": [
+ "query = \"Name everyone involved in writing the script, directing, and producing 'Dune: Part Two'?\"\n",
+ "\n",
+ "# Note: the relevant passage in the wikipedia page we're looking for is:\n",
+ "# \"[...] Dune: Part Two was originally scheduled to be released on October 20, 2023, but was delayed to November 17, 2023, before moving forward two weeks to November 3, 2023, to adjust to changes in release schedules from other studios. It was later postponed by over four months to March 15, 2024, due to the 2023 Hollywood labor disputes. After the strikes were resolved, the film moved once more up two weeks to March 1, 2024. [...]\""
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "9oULg1tOjjOW"
+ },
+ "source": [
+ "### Embed the user question\n",
+ "\n",
+ "Cohere embeddings are state-of-the-art."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "colab": {
+ "base_uri": "https://localhost:8080/"
+ },
+ "id": "yrUuS6vXgbOs",
+ "outputId": "0c64a930-f817-43c2-d775-1d9145cb304e"
+ },
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "query_embedding: [-0.068603516, -0.02947998, -0.06274414, -0.015449524, -0.033294678, 0.0056877136, -0.047210693, 0.04714966, -0.024871826, 0.008148193, 0.0770874, 0.023880005, -0.058685303, -0.052520752, 0.012832642, 0.024398804, 0.0053215027, 0.035491943, 0.02961731, -0.0069847107, 0.01083374, -0.0011358261, -0.002199173, 0.018417358, 0.027389526, -0.002691269, -0.026535034, 0.015197754, 0.024368286, 0.03729248, 0.0057754517, -0.02229309, -0.014694214, 0.019989014, -0.0036315918, -0.013793945, 0.02835083, 0.006011963, 0.011428833, 0.008682251, 0.046142578, -0.040039062, -0.032196045, -0.002653122, -0.012580872, -0.0041618347, 0.03111267, -0.016799927, 0.014801025, -0.00030636787, -0.033050537, 0.033966064, -0.016021729, -0.025009155, -0.007534027, -0.017074585, 0.008415222, -0.10620117, 0.019195557, -0.015686035, -0.0043182373, -0.045440674, 0.05404663, 0.030776978, -0.014129639, -0.01499939, -0.007286072, 0.009933472, 0.06390381, 0.02444458, -0.010345459, 0.041931152, 0.032989502, -0.04522705, 0.056610107, 0.0068893433, -0.008911133, 0.012489319, 0.01675415, 0.020065308, 0.018753052, 0.022659302, -0.051849365, -0.04925537, 0.046325684, -0.005268097, 0.0026874542, -0.036712646, 0.009437561, -0.0037841797, -0.01473999, -0.034179688, -0.0011606216, 0.05026245, 0.0020771027, -0.016021729, -0.0044898987, 0.04168701, -0.015205383, 0.019210815, -0.012374878, -0.031311035, 0.03111267, -0.040100098, -0.016479492, 0.020446777, 0.010192871, 0.0037841797, -0.0023765564, 0.015220642, -0.016571045, -0.006454468, 0.037384033, -0.044555664, -0.008262634, 0.019546509, 0.009460449, 0.014701843, 0.02658081, -0.02078247, 0.015571594, 0.013153076, -0.010375977, 0.047912598, 0.005393982, -0.007911682, -0.019378662, 0.023529053, -0.0033550262, -0.04598999, -0.0052871704, 0.040252686, 0.011375427, 0.01550293, -0.004508972, 0.006515503, 0.003370285, -0.022766113, 0.00062561035, -0.0007596016, -0.0015277863, 0.0149002075, 0.061401367, 8.261204e-05, 0.06359863, -0.01537323, 0.007446289, 0.018814087, 0.02507019, 0.024215698, 0.006122589, 0.005886078, -0.03829956, 0.029037476, 0.07720947, 0.016921997, 0.022109985, 0.005958557, 0.028793335, 0.019485474, 0.015174866, 0.026153564, 0.032318115, 0.034210205, 0.027145386, -0.019515991, -0.018661499, 0.020477295, 0.008598328, -0.06573486, -0.037109375, 0.04043579, 0.030471802, -0.0010843277, 0.009757996, 0.026947021, 0.037017822, -0.018234253, -0.0115356445, 0.099365234, 0.027816772, -0.019927979, 0.0020961761, 0.013198853, -0.019073486, 2.7656555e-05, 0.041259766, 0.029510498, -0.016204834, 0.028137207, 0.039489746, 0.034698486, -0.03918457, -0.029418945, 0.02041626, 0.0073432922, -0.018569946, -0.009849548, 0.002861023, 0.030319214, -0.012886047, 0.014671326, -0.035827637, 0.007247925, -0.027709961, -0.022079468, 0.0012960434, 0.015426636, -0.01725769, 0.01525116, 0.025360107, -0.0077400208, -0.039916992, 0.029037476, -0.011154175, 0.007736206, -0.041748047, 0.05343628, 0.007286072, 0.0435791, 0.034301758, -0.047210693, 0.03552246, -0.015327454, 0.029922485, -0.018859863, 0.013053894, -0.028060913, 0.07757568, -0.020462036, 0.070739746, -0.010223389, 0.03604126, 0.02758789, -0.023284912, 0.012184143, 0.029144287, 0.023880005, -0.019378662, -0.0051116943, 0.0048675537, 0.01864624, -0.04397583, -0.007598877, 0.0713501, 0.0115737915, 0.002922058, 0.011619568, 0.017364502, 0.031921387, -0.0019664764, -0.008575439, 0.003484726, -0.09466553, 0.03475952, 0.026611328, -0.039520264, -0.0104522705, -0.005443573, -0.008392334, 0.012908936, 0.0043792725, -0.002456665, -0.028396606, -0.02027893, -0.0005569458, 0.027786255, 0.03427124, -0.0062332153, -0.018203735, 0.019241333, 0.07244873, -0.0028057098, 0.01234436, -0.0018787384, -0.027496338, 0.0015287399, -0.004032135, -0.013748169, -0.01878357, 0.0018053055, -0.01159668, 0.028213501, 0.004776001, 0.042388916, 0.0024280548, 0.017471313, -0.038085938, 0.026321411, 0.02973938, 0.06213379, 0.006401062, 0.036102295, -0.028121948, -0.00869751, -0.016693115, 0.029190063, 0.016784668, -0.008628845, 0.0039634705, -0.0035381317, 0.019500732, 0.025009155, -0.04547119, -0.003572464, 0.05215454, 0.067871094, -0.04257202, -0.02293396, -0.027175903, 0.05340576, 0.019226074, 0.039978027, 0.056121826, -0.028320312, -0.020217896, -0.035003662, 0.03225708, 0.028656006, 0.062347412, 0.12915039, -0.0137786865, 0.0022201538, -0.057434082, -0.04397583, -0.049865723, -0.013160706, -0.03353882, 0.006427765, -0.014823914, -0.008201599, -0.036346436, -0.037353516, -0.010528564, -0.015930176, -0.027572632, 0.0074272156, 0.004547119, -0.024414062, -0.018859863, -0.020095825, 0.029632568, -0.00067043304, -0.044036865, -0.0043411255, -0.005256653, -0.019195557, 0.022262573, -0.00020956993, -0.013877869, -0.011108398, -0.020324707, -0.015808105, -0.025039673, -0.009498596, 0.05090332, 0.0046195984, -0.017150879, 0.04309082, -0.029067993, 0.002670288, -0.00026249886, -0.032409668, -0.053100586, 0.012481689, -0.014633179, 0.0013475418, -0.034332275, 0.038330078, 0.014892578, -0.046936035, 0.021591187, -0.020385742, -0.0052604675, 0.02796936, 0.0014333725, 0.012077332, -0.0118255615, -0.005569458, 0.008491516, 0.009841919, 0.0031318665, -0.003408432, -0.007144928, 0.040374756, -0.0038928986, 0.005279541, -0.008415222, 0.031707764, 0.0140686035, -0.015029907, -0.02810669, -0.0078125, -0.030853271, -0.03201294, 0.021316528, -0.036193848, -0.0423584, 0.0072784424, 0.014801025, 0.0019607544, -0.012367249, -0.009056091, -0.021438599, -0.02645874, 0.038726807, -0.007549286, 0.0049591064, 0.019012451, 0.017791748, -0.009185791, 0.04006958, 0.003107071, -0.0075302124, -0.010375977, -0.009246826, -0.02130127, -0.0056762695, -0.0076789856, 0.010009766, -0.010536194, 0.041107178, 0.0021133423, 0.029891968, 0.01626587, 0.042236328, -0.02784729, -0.032836914, 0.0317688, 0.045715332, 0.000116825104, 0.028030396, 0.007205963, 0.012512207, -0.035583496, -0.048034668, -0.023529053, -0.04953003, 0.0345459, -0.048339844, -0.060272217, -0.004512787, 0.04425049, 0.0076141357, 0.029510498, 0.007396698, 0.003353119, -0.038726807, 0.07183838, -0.026901245, -0.023529053, -0.038085938, 0.068725586, 0.018096924, -0.013534546, 0.05883789, -0.016113281, 0.017944336, 0.041046143, 0.022918701, 0.036499023, 0.015296936, -0.04916382, 0.0075683594, -0.011390686, 0.009735107, -0.0070152283, 0.003129959, -0.032562256, 0.0003478527, -0.0036640167, -0.006893158, -0.016098022, -0.034332275, 0.037750244, -0.010269165, 0.016494751, -0.02394104, 0.03753662, -0.022644043, -0.0008234978, 0.001001358, -0.048217773, 0.04989624, 0.0078125, 0.0044937134, 0.027038574, 0.04736328, -0.02973938, -0.011726379, 0.01348114, 0.021408081, 0.00844574, -0.03741455, -0.015686035, -0.040893555, 0.001452446, -0.025405884, 0.07348633, 0.038238525, -0.019958496, 0.023071289, -0.016403198, -0.08105469, 0.0071029663, -0.019088745, 5.8174133e-05, -0.005569458, 0.01399231, 0.02255249, 0.011222839, 0.00028824806, 0.0066184998, 0.0017499924, -0.009864807, -0.0115737915, 0.053100586, 0.0065231323, 0.001865387, -0.026428223, 0.03692627, 0.025390625, 0.022613525, 0.018722534, 0.007675171, -0.03439331, 0.041625977, -0.01789856, -0.041046143, 0.0051460266, 0.04144287, 0.048553467, 0.054595947, -0.01108551, -0.033935547, -0.026275635, -0.0118255615, -0.021362305, -0.009841919, -0.00724411, 0.028900146, 0.009887695, -0.023803711, 0.016311646, 0.018798828, -0.03668213, 0.046844482, 0.010696411, -0.014717102, -0.008110046, -0.004589081, -0.0028076172, -0.050811768, -0.017196655, -0.03491211, 0.0074005127, -0.038909912, 0.032440186, -0.034362793, -0.008682251, 0.032928467, -0.04626465, -0.009666443, 0.018951416, 0.031951904, -0.003791809, 0.02015686, -0.05532837, -0.005683899, -0.00054216385, -0.0034332275, 0.008659363, 0.02130127, -0.038879395, -0.0033397675, -0.03866577, -0.0049934387, 0.017944336, 0.001496315, 0.019485474, -0.004348755, 0.00046491623, 0.0007157326, 0.035614014, -0.027694702, 0.03692627, -0.008491516, 0.0524292, -0.016662598, -0.0017795563, -0.021575928, -0.018753052, -0.049346924, -0.06652832, 0.04272461, 0.03186035, 0.0011978149, 0.03463745, 0.024002075, 0.02607727, 0.020446777, 0.0256958, 0.026855469, 0.0074005127, -0.067993164, 0.017944336, -0.0039482117, 0.05496216, -0.041412354, 0.014175415, 0.02444458, -0.026412964, 0.057403564, -0.026779175, 0.023254395, 0.03945923, 0.033569336, -0.030258179, -0.039093018, -0.036468506, 0.017105103, 0.009635925, 0.025497437, 0.04156494, -0.02571106, -0.0010414124, -0.005630493, -0.016448975, -0.026733398, 0.001326561, -0.042022705, 0.0012521744, -0.041259766, -0.12182617, -0.03857422, 0.12548828, -0.005947113, -0.020736694, -0.0033855438, 0.03778076, -0.033813477, 0.038970947, 0.003921509, 0.011810303, 0.031982422, -0.032562256, -0.002653122, -0.025009155, -0.03805542, -0.016998291, 0.018173218, 0.0158844, 0.0011739731, 0.048217773, -0.020401001, 0.044708252, -0.017318726, 0.014457703, -0.041809082, 0.010543823, 0.041931152, 0.076293945, -0.054779053, 0.060272217, -0.046936035, 0.02949524, 0.00554657, 0.041534424, -0.013046265, -0.056152344, 0.010406494, 0.02973938, -0.023727417, -0.022476196, -0.024734497, -0.013168335, 0.060424805, 0.011787415, 0.018997192, -0.043426514, -0.00077724457, -0.010154724, 0.017150879, -0.01171875, -0.022476196, 0.0034255981, -0.0026454926, 0.004837036, -0.0043296814, 0.02619934, -0.021560669, -0.039733887, -0.022415161, -0.06817627, -0.023223877, -0.018585205, -0.015319824, 0.012588501, 0.0064353943, -0.013748169, 0.043304443, 0.002626419, -0.029373169, -0.016784668, -0.026184082, 0.05847168, 0.034179688, 0.03842163, -0.05493164, -0.017486572, 0.016540527, 0.03164673, 0.089904785, 0.013534546, -0.07684326, -0.024108887, 0.07434082, 0.030395508, 0.007091522, 0.07373047, 0.012527466, -0.010856628, -0.01828003, -0.045196533, 0.00065279007, -0.0637207, 0.010726929, 0.023880005, -0.0030708313, -0.012298584, 0.027236938, -0.04928589, 0.023071289, 0.008674622, -0.023529053, -0.015838623, -0.010543823, 0.012168884, 0.014854431, -0.05834961, -0.06088257, -0.012313843, 0.035461426, 0.02027893, 0.019348145, -0.014602661, -0.02104187, -0.0309906, 0.001405716, -0.019973755, -0.00157547, -0.003944397, 0.0009326935, -0.02078247, -0.015731812, -0.044433594, 0.03390503, 0.057159424, 0.018585205, -0.023895264, -0.0057029724, 0.0049552917, 0.013412476, 0.022399902, 0.010154724, 0.0519104, 0.06591797, 0.018341064, 0.012161255, -0.05810547, -0.043304443, -0.031173706, 0.0023860931, -0.003944397, 0.11425781, -0.031036377, 0.019989014, -0.038635254, -0.025939941, 0.035064697, 0.041168213, 0.03161621, -0.069885254, -0.04537964, 0.028945923, -0.023162842, 0.019226074, -0.028442383, 0.015594482, -0.019256592, -0.0046463013, 0.034240723, 0.009124756, 0.05718994, 0.031219482, 0.02154541, 0.009590149, 0.00076818466, 0.04849243, -0.029129028, -0.03375244, -0.023391724, -0.028381348, -0.029708862, -0.0132369995, 0.010353088, 0.020263672, -0.030807495, 0.01007843, -0.03704834, 0.023376465, -0.03665161, 0.03741455, 0.015144348, 0.057281494, 0.03137207, 0.048431396, 0.021194458, 0.008110046, -0.03540039, -0.015312195, 0.022384644, 0.0065956116, 0.008056641, 0.0018348694, -0.009246826, 0.030380249, 0.0003862381, 0.0051841736, 0.04486084, 0.017807007, 0.0026130676, 0.07977295, 0.05419922, 0.062194824, 0.02633667, 0.024841309, -0.041625977, -0.005897522, 0.04031372, -0.055908203, 0.0026226044, -0.05340576, -0.05496216, 0.011474609, -0.006954193, -0.013122559, 0.019714355, -0.07159424, 0.031173706, 0.0034255981, -0.0034103394, 0.0440979, 0.011779785, -0.007827759, -0.03173828, -0.020950317, -0.030166626, -0.035308838, 0.030792236, 0.04525757, -0.028701782, -0.011100769, -0.02331543, -0.0357666, -0.025680542, 0.0011911392, 0.01940918, 0.05706787, 0.028381348, 0.007133484, -0.07733154, -0.007686615, 0.03869629, 0.0066833496, 0.008842468, 0.03439331, -0.014282227, 0.0357666, -0.004737854, -0.039794922, -0.0070381165, 0.02670288, 0.0107421875, 0.016189575, -0.06555176, -0.0138549805, 0.0008363724, -0.016693115, 0.006904602, -0.020263672, -0.030426025, 0.008453369, -0.046173096, -0.01802063, -0.013595581, -0.0044288635, -0.0039978027, -0.0044898987, 0.0007619858, 0.003921509, 0.0053977966, 0.020385742, -0.012329102, -0.023803711, -0.0057525635, 0.038330078, -0.014549255, -0.06298828, -0.047607422, 0.039245605, -0.06781006, -0.035217285, -0.009056091, 0.019927979, -0.003932953, -0.020309448, -0.017044067, 0.018127441, -8.624792e-05, -0.043182373, 0.009590149, 0.035308838, 0.031951904, 0.0011615753, -0.042022705, 0.079956055, 0.026687622, 0.013542175, -0.0074157715, -0.00983429, -0.0022563934, 0.07373047, 0.059387207, 0.03488159, 0.0071372986, -0.06427002, -0.0546875, -0.02482605, 0.11071777, -0.021072388, 0.01626587, -0.049713135, 0.061553955, -0.016860962, 0.051971436, -0.012962341, -0.0011711121, -0.014198303, -0.0061149597, -0.005836487, 0.00022387505, -0.027618408, 0.019836426, 0.009933472, 0.02368164, -0.020309448, -0.0049591064, -0.008628845, -0.03253174, -0.017684937, 0.02468872, -0.0023498535, 0.01448822, 0.061920166, 0.031707764, -0.0026416779, -0.040985107, -0.06335449, -0.036071777, 0.05404663, -0.0044136047, -0.0146102905, -0.0033416748, 0.028671265, -0.012771606, -0.0016565323, -0.0038909912, -0.02407837, -0.009857178, 0.0014467239, -0.008720398, -0.006011963, 0.032073975, -0.033325195, 0.014862061, -0.017227173, -0.018753052, -0.0060424805, 0.022567749, -0.017654419, -0.017562866, -0.07244873, -0.0881958, 0.050476074, 0.02609253, -0.032409668, 0.07458496, 0.009399414, 0.009117126, -0.031051636, -0.03451538, -0.004219055, -0.05718994, 0.020080566, -0.025421143, -0.010948181, 0.06341553, -0.009231567, -0.021697998, -0.009719849, 0.012802124, -0.020370483, 0.0034389496, 0.018859863, -0.025680542, 0.0013141632, 0.068603516, -0.021026611, 0.021881104, -0.0395813, -0.0019073486, 0.0056037903, -0.032348633]\n"
+ ]
+ }
+ ],
+ "source": [
+ "# Because the text being embedded is the search query, we set the input type as search_query\n",
+ "response = co.embed(\n",
+ " texts=[query],\n",
+ " model=model,\n",
+ " input_type=\"search_query\",\n",
+ " embedding_types=['float']\n",
+ ")\n",
+ "query_embedding = response.embeddings.float[0]\n",
+ "print(\"query_embedding: \", query_embedding)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "8K8B87CGgbOt"
+ },
+ "source": [
+ "### Retrieve the most relevant chunks from the vector database\n",
+ "\n",
+ "We use cosine similarity to find the most similar chunks"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "colab": {
+ "base_uri": "https://localhost:8080/"
+ },
+ "id": "nik3es32gbOt",
+ "outputId": "a1c30024-52e1-42c7-8836-a2c590559aca"
+ },
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "similarity scores: [0.6953257882233425, 0.3713410510180273, 0.46501499776898964, 0.5448546916785195, 0.4014738351361969, 0.3231420292334584, 0.3179003053384008, 0.42799691553367775, 0.18882594531435784, 0.36868801306504106, 0.3404040737300553, 0.3852837621219358, 0.2600249419491577, 0.3723244353775111, 0.3631492691137214, 0.47574774051439606, 0.40415422750911745, 0.4149923346201023, 0.5014741934381444, 0.3549433331883204, 0.32072714802512714, 0.14770872479410424, 0.585277816615252, 0.6999636953772764, 0.7722295084104617, 0.4895347049465806, 0.5170096485954725, 0.7137817366881455, 0.5224900699612323, 0.5914632581598285, 0.2657897083381463, 0.6462342489537262, 0.6317222315431096, 0.5982303530756702, 0.5138265091630297, 0.41385121172723643, 0.4293941094100836, 0.4173182546482015, 0.42621236706314475, 0.4428474375355954, 0.35058541576139896, 0.3578709652019502, 0.3930157841938308, 0.3564608202848675, 0.23016661533167404, 0.4933441863421645, 0.41037089239250985, 0.39993051898770193, 0.3119997063424595, 0.2677143729521374, 0.3700866951454496, 0.46727994925061545, 0.4393343280374425, 0.42111290117172434, 0.4485349189824285, 0.4710573736688592, 0.24169956903740436, 0.3840442910806355, 0.14284631817675886, 0.5381588054138154, 0.431113882725076, 0.5189547209048608, 0.3950667224233914, 0.32429768756510174, 0.4370358125161736, 0.18727062244331039, 0.5206375682478743, 0.5175737635701252, 0.5326043981628349, 0.45586923626994363, 0.21667338125532032, 0.16459878595959285, 0.22236726481673777, 0.5187259906958807, 0.2884444442338396, 0.286407544555338, 0.2313840178160818, 0.2057731158935257, 0.5973876998341746, 0.42904243401792086, 0.4081217538000544, 0.5330523063972133, 0.45080561486977405, 0.414703452285757, 0.2569028899107211, 0.5087916806929323, 0.14159076456040554, 0.46505779053352697, 0.556364222182839, 0.35464351181035236, 0.40174477023626]\n",
+ "Here are the indices of the top 10 chunks after retrieval: [24 27 23 0 31 32 33 78 29 22]\n",
+ "Here are the top 10 chunks after retrieval: \n",
+ "== stunt coordinator. Dune: Part Two was produced by Villeneuve, Mary Parent, and Cale Boyter, with Tanya Lapointe, Brian Herbert, Byron Merritt, Kim Herbert, Thomas Tull, Jon Spaihts, Richard P. Rubinstein, John Harrison, and Herbert W. Gain serving as executive producers and Kevin J. Anderson as creative consultant. Legendary CEO Joshua Grode confirmed in April 2019 that they plan to make a sequel, adding that \"there's a logical place to stop the [first] movie before the book is over\".In December 2020,\n",
+ "== that.\"On October 26, 2021, Legendary officially greenlit Dune: Part Two, with a spokesperson for the company stating, \"We would not have gotten to this point without the extraordinary vision of Denis and the amazing work of his talented crew, the writers, our stellar cast, our partners at Warner Bros., and of course the fans! Here's to more Dune.\" Production work had occurred back-to-back with the first film, as Villeneuve and his wife Lapointe immediately took a flight to Budapest in order to begin\n",
+ "== series. Villeneuve ultimately secured a two-film deal with Warner Bros. Pictures, in the same style as the two-part adaption of Stephen King's It in 2017 and 2019. In January 2019, Joe Walker was confirmed to be serving as the film's editor. Other crew included Brad Riker as supervising art director, Patrice Vermette as production designer, Paul Lambert as visual effects supervisor, Gerd Nefzer as special effects supervisor, and Thomas Struthers as stunt coordinator. Dune: Part Two was produced by\n",
+ "== Dune: Part Two is a 2024 American epic science fiction film directed and produced by Denis Villeneuve, who co-wrote the screenplay with Jon Spaihts. The sequel to Dune (2021) adapts the 1965 novel Dune by Frank Herbert and follows Paul Atreides as he unites with the Fremen people of the desert planet Arrakis to wage war against House Harkonnen. Timothée Chalamet, Rebecca Ferguson, Josh Brolin, Stellan Skarsgård, Dave Bautista, Zendaya, Charlotte Rampling, and Javier Bardem reprise their roles from the first\n",
+ "== Eric Roth was hired to co-write the screenplay in April 2017 for the Dune films, and Jon Spaihts was later confirmed to be co-writing the script alongside Roth and Villeneuve. Game of Thrones language creator David Peterson was confirmed to be developing languages for the film in April 2019. Villeneuve and Peterson had created the Chakobsa language, which was used by actors on set. In November 2019, Spaihts stepped down as showrunner for Dune: Prophecy to focus on Dune: Part Two. In June 2020, Greig Fraser\n",
+ "== on Dune: Part Two. In June 2020, Greig Fraser said, \"It's a fully formed story in itself with places to go. It's a fully standalone epic film that people will get a lot out of when they see it\". Between the release of Dune and the confirmation of Dune: Part Two, Villeneuve started working the script in a way that production could begin immediately once the film was greenlit. By February 2021, Roth created a full treatment for the sequel, with writing beginning that August. He confirmed that Feyd-Rautha\n",
+ "== that August. He confirmed that Feyd-Rautha would appear in the film, and stated he will be a \"very important character\". In March 2022, Villeneuve had mostly finished writing the screenplay. Craig Mazin and Roth wrote additional literary material for the film.Villeneuve stated that the film would continue directly from the first, and specifically described it as being the \"second part.\" He described the film as being an \"epic war movie\", adding that while the first film was more \"contemplative\", the second\n",
+ "== On the review aggregator website Rotten Tomatoes, 93% of 378 critics' reviews are positive, with an average rating of 8.4/10. The website's consensus reads: \"Visually thrilling and narratively epic, Dune: Part Two continues Denis Villeneuve's adaptation of the beloved sci-fi series in spectacular form.\" Metacritic, which uses a weighted average, assigned the film a score of 79 out of 100, based on 62 critics, indicating \"generally favorable\" reviews. Audiences surveyed by CinemaScore gave the film an\n",
+ "== theatrical experience is at the very heart of the cinematic language for me.\" With Dune: Part Two being greenlit, Villeneuve said that his primary concern was to complete the filming as soon as possible, with the earliest he expected to start in the last quarter of 2022. However, he noted that production would be facilitated by the work already established on the first film, which would help expedite production.\n",
+ "== By November 2016, Legendary Pictures had obtained the film and TV rights for the Dune franchise, based on the eponymous 1965 novel by Frank Herbert. Vice chair of worldwide production for Legendary Mary Parent began discussing with Denis Villeneuve about directing a film adaptation, quickly hiring him after realizing his passion for Dune. By February 2018, Villeneuve was confirmed to be hired as director, and intended to adapt the novel as a two-part film series. Villeneuve ultimately secured a two-film\n"
+ ]
+ }
+ ],
+ "source": [
+ "def cosine_similarity(a, b):\n",
+ " return np.dot(a, b) / (np.linalg.norm(a) * np.linalg.norm(b))\n",
+ "\n",
+ "# Calculate similarity between the user question & each chunk\n",
+ "similarities = [cosine_similarity(query_embedding, chunk) for chunk in embeddings]\n",
+ "print(\"similarity scores: \", similarities)\n",
+ "\n",
+ "# Get indices of the top 10 most similar chunks\n",
+ "sorted_indices = np.argsort(similarities)[::-1]\n",
+ "\n",
+ "# Keep only the top 10 indices\n",
+ "top_indices = sorted_indices[:10]\n",
+ "print(\"Here are the indices of the top 10 chunks after retrieval: \", top_indices)\n",
+ "\n",
+ "# Retrieve the top 10 most similar chunks\n",
+ "top_chunks_after_retrieval = [chunks[i] for i in top_indices]\n",
+ "print(\"Here are the top 10 chunks after retrieval: \")\n",
+ "for t in top_chunks_after_retrieval:\n",
+ " print(\"== \" + t)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "qzcpds3VgbOt"
+ },
+ "source": [
+ "## Step 2 - Rerank the chunks retrieved from the vector database\n",
+ "\n",
+ "We rerank the 10 chunks retrieved from the vector database. Reranking boosts retrieval accuracy.\n",
+ "\n",
+ "Reranking lets us go from 10 chunks retrieved from the vector database, to the 3 most relevant chunks."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "colab": {
+ "base_uri": "https://localhost:8080/"
+ },
+ "id": "2J4LywVygbOt",
+ "outputId": "7a4c89bf-fc5e-409f-9304-fce006b9d8bf"
+ },
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "Here are the top 3 chunks after rerank: \n",
+ "== Dune: Part Two is a 2024 American epic science fiction film directed and produced by Denis Villeneuve, who co-wrote the screenplay with Jon Spaihts. The sequel to Dune (2021) adapts the 1965 novel Dune by Frank Herbert and follows Paul Atreides as he unites with the Fremen people of the desert planet Arrakis to wage war against House Harkonnen. Timothée Chalamet, Rebecca Ferguson, Josh Brolin, Stellan Skarsgård, Dave Bautista, Zendaya, Charlotte Rampling, and Javier Bardem reprise their roles from the first\n",
+ "== stunt coordinator. Dune: Part Two was produced by Villeneuve, Mary Parent, and Cale Boyter, with Tanya Lapointe, Brian Herbert, Byron Merritt, Kim Herbert, Thomas Tull, Jon Spaihts, Richard P. Rubinstein, John Harrison, and Herbert W. Gain serving as executive producers and Kevin J. Anderson as creative consultant. Legendary CEO Joshua Grode confirmed in April 2019 that they plan to make a sequel, adding that \"there's a logical place to stop the [first] movie before the book is over\".In December 2020,\n",
+ "== series. Villeneuve ultimately secured a two-film deal with Warner Bros. Pictures, in the same style as the two-part adaption of Stephen King's It in 2017 and 2019. In January 2019, Joe Walker was confirmed to be serving as the film's editor. Other crew included Brad Riker as supervising art director, Patrice Vermette as production designer, Paul Lambert as visual effects supervisor, Gerd Nefzer as special effects supervisor, and Thomas Struthers as stunt coordinator. Dune: Part Two was produced by\n"
+ ]
+ }
+ ],
+ "source": [
+ "response = co.rerank(\n",
+ " query=query,\n",
+ " documents=top_chunks_after_retrieval,\n",
+ " top_n=3,\n",
+ " model=\"rerank-english-v2.0\",\n",
+ ")\n",
+ "\n",
+ "top_chunks_after_rerank = [result.document['text'] for result in response]\n",
+ "print(\"Here are the top 3 chunks after rerank: \")\n",
+ "for t in top_chunks_after_rerank:\n",
+ " print(\"== \" + t)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "KuPL0VUXgbOt"
+ },
+ "source": [
+ "## Step 3 - Generate the model final answer, given the retrieved and reranked chunks"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "oCNXWH8GgbOt"
+ },
+ "outputs": [],
+ "source": [
+ "# preamble containing instructions about the task and the desired style for the output.\n",
+ "preamble = \"\"\"\n",
+ "## Task & Context\n",
+ "You help people answer their questions and other requests interactively. You will be asked a very wide array of requests on all kinds of topics. You will be equipped with a wide range of search engines or similar tools to help you, which you use to research your answer. You should focus on serving the user's needs as best you can, which will be wide-ranging.\n",
+ "\n",
+ "## Style Guide\n",
+ "Unless the user asks for a different style of answer, you should answer in full sentences, using proper grammar and spelling.\n",
+ "\"\"\""
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "colab": {
+ "base_uri": "https://localhost:8080/"
+ },
+ "id": "BevatShtgbOt",
+ "outputId": "af71f4a9-787a-4ee3-9598-20692fb3bf16"
+ },
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "Final answer:\n",
+ "Here are the key crew members involved in 'Dune: Part Two':\n",
+ "\n",
+ "- Denis Villeneuve: director and producer\n",
+ "- Jon Spaihts: co-writer of the screenplay\n",
+ "- Mary Parent and Cale Boyter: producers \n",
+ "- Tanya Lapointe, Brian Herbert, Byron Merritt, Kim Herbert, Thomas Tull, Richard P. Rubinstein, John Harrison, Herbert W. Gain and Kevin J. Anderson: executive producers \n",
+ "- Joe Walker: editor\n",
+ "- Brad Riker: supervising art director\n",
+ "- Patrice Vermette: production designer\n",
+ "- Paul Lambert: visual effects supervisor\n",
+ "- Gerd Nefzer: special effects supervisor\n",
+ "- Thomas Struthers: stunt coordinator. \n",
+ "\n",
+ "A number of crew members from the first film returned for the sequel, which is set to be released in 2024.\n"
+ ]
+ }
+ ],
+ "source": [
+ "# retrieved documents\n",
+ "documents = [\n",
+ " {\"title\": \"chunk 0\", \"snippet\": top_chunks_after_rerank[0]},\n",
+ " {\"title\": \"chunk 1\", \"snippet\": top_chunks_after_rerank[1]},\n",
+ " {\"title\": \"chunk 2\", \"snippet\": top_chunks_after_rerank[2]},\n",
+ " ]\n",
+ "\n",
+ "# get model response\n",
+ "response = co.chat(\n",
+ " message=query,\n",
+ " documents=documents,\n",
+ " preamble=preamble,\n",
+ " model=\"command-r\",\n",
+ " temperature=0.3\n",
+ ")\n",
+ "\n",
+ "print(\"Final answer:\")\n",
+ "print(response.text)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "20wcn-EjlXZd"
+ },
+ "source": [
+ "Note: this is indeed the answer you'd expect, and here was the passage of text in wikipedia explaining it!\n",
+ "\n",
+ "\" [...] Dune: Part Two was originally scheduled to be released on October 20, 2023, but was delayed to November 17, 2023, before moving forward two weeks to November 3, 2023, to adjust to changes in release schedules from other studios. It was later postponed by over four months to March 15, 2024, due to the 2023 Hollywood labor disputes. After the strikes were resolved, the film moved once more up two weeks to March 1, 2024. [...]\""
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "RoSVDXSsgbOt"
+ },
+ "source": [
+ "## Bonus: Citations come for free with Cohere! 🎉\n",
+ "\n",
+ "At Cohere, all RAG calls come with... precise citations! 🎉\n",
+ "The model cites which groups of words, in the RAG chunks, were used to generate the final answer. \n",
+ "These citations make it easy to check where the model’s generated response claims are coming from. \n",
+ "They help users gain visibility into the model reasoning, and sanity check the final model generation. \n",
+ "These citations are optional — you can decide to ignore them.\n"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "colab": {
+ "base_uri": "https://localhost:8080/"
+ },
+ "id": "BVTuQdmDgbOt",
+ "outputId": "f843b262-d8bb-45ba-cbfb-9915da104eda"
+ },
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "Citations that support the final answer:\n",
+ "{'start': 63, 'end': 79, 'text': 'Denis Villeneuve', 'document_ids': ['doc_0']}\n",
+ "{'start': 81, 'end': 102, 'text': 'director and producer', 'document_ids': ['doc_0']}\n",
+ "{'start': 105, 'end': 116, 'text': 'Jon Spaihts', 'document_ids': ['doc_0']}\n",
+ "{'start': 118, 'end': 145, 'text': 'co-writer of the screenplay', 'document_ids': ['doc_0']}\n",
+ "{'start': 148, 'end': 159, 'text': 'Mary Parent', 'document_ids': ['doc_1']}\n",
+ "{'start': 164, 'end': 175, 'text': 'Cale Boyter', 'document_ids': ['doc_1']}\n",
+ "{'start': 177, 'end': 186, 'text': 'producers', 'document_ids': ['doc_1']}\n",
+ "{'start': 190, 'end': 204, 'text': 'Tanya Lapointe', 'document_ids': ['doc_1']}\n",
+ "{'start': 206, 'end': 219, 'text': 'Brian Herbert', 'document_ids': ['doc_1']}\n",
+ "{'start': 221, 'end': 234, 'text': 'Byron Merritt', 'document_ids': ['doc_1']}\n",
+ "{'start': 236, 'end': 247, 'text': 'Kim Herbert', 'document_ids': ['doc_1']}\n",
+ "{'start': 249, 'end': 260, 'text': 'Thomas Tull', 'document_ids': ['doc_1']}\n",
+ "{'start': 262, 'end': 283, 'text': 'Richard P. Rubinstein', 'document_ids': ['doc_1']}\n",
+ "{'start': 285, 'end': 298, 'text': 'John Harrison', 'document_ids': ['doc_1']}\n",
+ "{'start': 300, 'end': 315, 'text': 'Herbert W. Gain', 'document_ids': ['doc_1']}\n",
+ "{'start': 320, 'end': 337, 'text': 'Kevin J. Anderson', 'document_ids': ['doc_1']}\n",
+ "{'start': 339, 'end': 358, 'text': 'executive producers', 'document_ids': ['doc_1']}\n",
+ "{'start': 362, 'end': 372, 'text': 'Joe Walker', 'document_ids': ['doc_2']}\n",
+ "{'start': 374, 'end': 380, 'text': 'editor', 'document_ids': ['doc_2']}\n",
+ "{'start': 383, 'end': 393, 'text': 'Brad Riker', 'document_ids': ['doc_2']}\n",
+ "{'start': 395, 'end': 419, 'text': 'supervising art director', 'document_ids': ['doc_2']}\n",
+ "{'start': 422, 'end': 438, 'text': 'Patrice Vermette', 'document_ids': ['doc_2']}\n",
+ "{'start': 440, 'end': 459, 'text': 'production designer', 'document_ids': ['doc_2']}\n",
+ "{'start': 462, 'end': 474, 'text': 'Paul Lambert', 'document_ids': ['doc_2']}\n",
+ "{'start': 476, 'end': 501, 'text': 'visual effects supervisor', 'document_ids': ['doc_2']}\n",
+ "{'start': 504, 'end': 515, 'text': 'Gerd Nefzer', 'document_ids': ['doc_2']}\n",
+ "{'start': 517, 'end': 543, 'text': 'special effects supervisor', 'document_ids': ['doc_2']}\n",
+ "{'start': 546, 'end': 562, 'text': 'Thomas Struthers', 'document_ids': ['doc_2']}\n",
+ "{'start': 564, 'end': 582, 'text': 'stunt coordinator.', 'document_ids': ['doc_2']}\n",
+ "{'start': 686, 'end': 691, 'text': '2024.', 'document_ids': ['doc_0']}\n"
+ ]
+ }
+ ],
+ "source": [
+ "print(\"Citations that support the final answer:\")\n",
+ "for cite in response.citations:\n",
+ " print(cite)"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "colab": {
+ "base_uri": "https://localhost:8080/"
+ },
+ "id": "IueXaIJggbOu",
+ "outputId": "c816af51-74be-42c9-e94e-9820bbf95f79"
+ },
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "Here are the key crew members involved in 'Dune: Part Two':\n",
+ "\n",
+ "- **Denis Villeneuve**[1]: **director and producer**[1]\n",
+ "- **Jon Spaihts**[1]: **co-writer of the screenplay**[1]\n",
+ "- **Mary Parent**[2] and **Cale Boyter**[2]: **producers**[2] \n",
+ "- **Tanya Lapointe**[2], **Brian Herbert**[2], **Byron Merritt**[2], **Kim Herbert**[2], **Thomas Tull**[2], **Richard P. Rubinstein**[2], **John Harrison**[2], **Herbert W. Gain**[2] and **Kevin J. Anderson**[2]: **executive producers**[2] \n",
+ "- **Joe Walker**[3]: **editor**[3]\n",
+ "- **Brad Riker**[3]: **supervising art director**[3]\n",
+ "- **Patrice Vermette**[3]: **production designer**[3]\n",
+ "- **Paul Lambert**[3]: **visual effects supervisor**[3]\n",
+ "- **Gerd Nefzer**[3]: **special effects supervisor**[3]\n",
+ "- **Thomas Struthers**[3]: **stunt coordinator.**[3] \n",
+ "\n",
+ "A number of crew members from the first film returned for the sequel, which is set to be released in **2024.**[1]\n",
+ "\n",
+ "[1] source: \"Dune: Part Two is a 2024 American epic science fiction film directed and produced by Denis Villeneuve, who co-wrote the screenplay with Jon Spaihts. The sequel to Dune (2021) adapts the 1965 novel Dune by Frank Herbert and follows Paul Atreides as he unites with the Fremen people of the desert planet Arrakis to wage war against House Harkonnen. Timothée Chalamet, Rebecca Ferguson, Josh Brolin, Stellan Skarsgård, Dave Bautista, Zendaya, Charlotte Rampling, and Javier Bardem reprise their roles from the first\"\n",
+ "[2] source: \"stunt coordinator. Dune: Part Two was produced by Villeneuve, Mary Parent, and Cale Boyter, with Tanya Lapointe, Brian Herbert, Byron Merritt, Kim Herbert, Thomas Tull, Jon Spaihts, Richard P. Rubinstein, John Harrison, and Herbert W. Gain serving as executive producers and Kevin J. Anderson as creative consultant. Legendary CEO Joshua Grode confirmed in April 2019 that they plan to make a sequel, adding that \"there's a logical place to stop the [first] movie before the book is over\".In December 2020,\"\n",
+ "[3] source: \"series. Villeneuve ultimately secured a two-film deal with Warner Bros. Pictures, in the same style as the two-part adaption of Stephen King's It in 2017 and 2019. In January 2019, Joe Walker was confirmed to be serving as the film's editor. Other crew included Brad Riker as supervising art director, Patrice Vermette as production designer, Paul Lambert as visual effects supervisor, Gerd Nefzer as special effects supervisor, and Thomas Struthers as stunt coordinator. Dune: Part Two was produced by\"\n"
+ ]
+ }
+ ],
+ "source": [
+ "def insert_citations_in_order(text, citations):\n",
+ " \"\"\"\n",
+ " A helper function to pretty print citations.\n",
+ " \"\"\"\n",
+ " offset = 0\n",
+ " document_id_to_number = {}\n",
+ " citation_number = 0\n",
+ " modified_citations = []\n",
+ "\n",
+ " # Process citations, assigning numbers based on unique document_ids\n",
+ " for citation in citations:\n",
+ " citation_numbers = []\n",
+ " for document_id in sorted(citation[\"document_ids\"]):\n",
+ " if document_id not in document_id_to_number:\n",
+ " citation_number += 1 # Increment for a new document_id\n",
+ " document_id_to_number[document_id] = citation_number\n",
+ " citation_numbers.append(document_id_to_number[document_id])\n",
+ "\n",
+ " # Adjust start/end with offset\n",
+ " start, end = citation['start'] + offset, citation['end'] + offset\n",
+ " placeholder = ''.join([f'[{number}]' for number in citation_numbers])\n",
+ " # Bold the cited text and append the placeholder\n",
+ " modification = f'**{text[start:end]}**{placeholder}'\n",
+ " # Replace the cited text with its bolded version + placeholder\n",
+ " text = text[:start] + modification + text[end:]\n",
+ " # Update the offset for subsequent replacements\n",
+ " offset += len(modification) - (end - start)\n",
+ "\n",
+ " # Prepare citations for listing at the bottom, ensuring unique document_ids are listed once\n",
+ " unique_citations = {number: doc_id for doc_id, number in document_id_to_number.items()}\n",
+ " citation_list = '\\n'.join([f'[{doc_id}] source: \"{documents[doc_id - 1][\"snippet\"]}\"' for doc_id, number in sorted(unique_citations.items(), key=lambda item: item[1])])\n",
+ " text_with_citations = f'{text}\\n\\n{citation_list}'\n",
+ "\n",
+ " return text_with_citations\n",
+ "\n",
+ "\n",
+ "print(insert_citations_in_order(response.text, response.citations))\n"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "Kp4c_HkYIEn_"
+ },
+ "outputs": [],
+ "source": []
+ }
+ ],
+ "metadata": {
+ "colab": {
+ "provenance": []
+ },
+ "kernelspec": {
+ "display_name": "hackathon_docs_3",
+ "language": "python",
+ "name": "python3"
+ },
+ "language_info": {
+ "codemirror_mode": {
+ "name": "ipython",
+ "version": 3
+ },
+ "file_extension": ".py",
+ "mimetype": "text/x-python",
+ "name": "python",
+ "nbconvert_exporter": "python",
+ "pygments_lexer": "ipython3",
+ "version": "3.11.5"
+ }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 0
+}
diff --git a/notebooks/Embed_Jobs_Semantic_Search.ipynb b/notebooks/Embed_Jobs_Semantic_Search.ipynb
index 4b7ba4f9..9043c176 100644
--- a/notebooks/Embed_Jobs_Semantic_Search.ipynb
+++ b/notebooks/Embed_Jobs_Semantic_Search.ipynb
@@ -1,366 +1,364 @@
{
- "cells": [
- {
- "cell_type": "markdown",
- "metadata": {
- "id": "ig3hmpIt8ptJ"
- },
- "source": [
- "# Semantic Search with Cohere Embed Jobs"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "id": "fc94ylzpucDm"
- },
- "outputs": [],
- "source": [
- "# TODO: upgrade to \"cohere>5\"",
-"! pip install \"cohere<5\" hnswlib -q"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "id": "Qu9a_jwZ5Zop"
- },
- "outputs": [],
- "source": [
- "import time\n",
- "import cohere\n",
- "import hnswlib\n",
- "co = cohere.Client('COHERE_API_KEY')"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {
- "id": "Fgqhh1kk8mnY"
- },
- "source": [
- "## Step 1: Upload a dataset"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "colab": {
- "base_uri": "https://localhost:8080/"
- },
- "id": "kogkZnDEnK8B",
- "outputId": "646b4508-8111-42ad-a9b2-3425da1ae5ff"
- },
- "outputs": [
- {
- "name": "stderr",
- "output_type": "stream",
- "text": [
- "uploading file, starting validation...\n",
- "sample-file-hca4x0 was uploaded\n",
- "...\n"
- ]
- }
- ],
- "source": [
- "# Upload a dataset for embed jobs\n",
- "# This sample dataset has wikipedia articles on the following: Youtube, United States, United Kingdom, Elizabeth II, Wikipedia, 2022 FIFA World Cup, Microsoft Office, India, Christiano Ronaldo, Cleopatra, Instagram, Facebook, and Ukraine\n",
- "\n",
- "dataset_file_path = \"data/embed_jobs_sample_data.jsonl\" # Full path - https://raw.githubusercontent.com/cohere-ai/notebooks/main/notebooks/data/embed_jobs_sample_data.jsonl\n",
- "\n",
- "ds=co.create_dataset(\n",
- "\tname='sample_file',\n",
- "\tdata=open(dataset_file_path, 'rb'),\n",
- "\tkeep_fields = ['id','wiki_id'],\n",
- "\tdataset_type=\"embed-input\"\n",
- "\t)"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "colab": {
- "base_uri": "https://localhost:8080/"
- },
- "id": "3lAS4bcvCplJ",
- "outputId": "5e1fcd0b-4880-43ed-d8a4-03148b2ef047"
- },
- "outputs": [
- {
- "name": "stdout",
- "output_type": "stream",
- "text": [
- "cohere.Dataset {\n",
- "\tid: sample-file-hca4x0\n",
- "\tname: sample_file\n",
- "\tdataset_type: embed-input\n",
- "\tvalidation_status: validated\n",
- "\tcreated_at: 2024-01-13 02:51:48.215973\n",
- "\tupdated_at: 2024-01-13 02:51:48.215973\n",
- "\tdownload_urls: ['https://storage.googleapis.com/cohere-user/dataset-api-temp/d489c39a-e152-49da-9ddc-9801bd74d823/f8bd5ab1-ccdf-45e6-9717-23b8be1ff39d/sample-file-hca4x0/000_embed_jobs_sample_data.avro?X-Goog-Algorithm=GOOG4-RSA-SHA256&X-Goog-Credential=dataset%40cohere-production.iam.gserviceaccount.com%2F20240113%2Fauto%2Fstorage%2Fgoog4_request&X-Goog-Date=20240113T025159Z&X-Goog-Expires=14399&X-Goog-Signature=7f9ed7cb27988e4bf447167160b25940cd106859d1282919604a0d74b882681b5aaa3245f97555bb61ddca7fd9c02fb0bba8391433c70aeb2118607dfd534db444097c5ae9c8d07e66bf6723a64634f5d6f5b2500c82e351807203f5fd3c278b2584c258b4c6afc6ee191e63bcd346e3dd2c7fd4b171c2d3ddfe8e6e68c2522111b6f63e125a052be9ebe3302903f4bef9cd165c8e075b168e621c7c61177a0973bb470cc3535457078e18b338884fb303792a1ba3161ab5ca5ce2adca5676754ee1a735891ccbf64cca03d3fdcabdcccd0d201600b8c70e13487bf897f2b88424cc736f8bf8756ec835b09dacf0aa20dc8151d9249b21d4ca4de82144fcd8be&X-Goog-SignedHeaders=host']\n",
- "\tvalidation_error: None\n",
- "\tvalidation_warnings: []\n",
- "}\n"
- ]
- }
- ],
- "source": [
- "print(ds.await_validation())"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {
- "id": "QgD2QCJk9kUN"
- },
- "source": [
- "## Step 2: Create embeddings via Cohere's Embed Jobs endpoint"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "colab": {
- "base_uri": "https://localhost:8080/"
- },
- "id": "yMYoU9y55m2D",
- "outputId": "fd58014d-a9e3-4a3f-ca4b-966880dec412"
- },
- "outputs": [
- {
- "name": "stderr",
- "output_type": "stream",
- "text": [
- "...\n",
- "...\n"
- ]
- }
- ],
- "source": [
- "# Dataset has been uploaded, create an embed job and specify the input type as \"search document\" since this will live in your Pinecone DB\n",
- "job = co.create_embed_job(\n",
- " dataset_id=ds.id,\n",
- " input_type='search_document' ,\n",
- " model='embed-english-v3.0', \n",
- " embeddings_types=['float'])\n",
- "\n",
- "job.wait() # poll the server until the job is completed "
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "colab": {
- "base_uri": "https://localhost:8080/"
- },
- "id": "LEevvN2XCUpR",
- "outputId": "d123c0a4-0c6a-4235-9ba8-658d5d3ec358"
- },
- "outputs": [
- {
- "name": "stdout",
- "output_type": "stream",
- "text": [
- "cohere.EmbedJob {\n",
- "\tjob_id: 792bbc1a-561b-48c2-8a97-0c80c1914ea8\n",
- "\tstatus: complete\n",
- "\tcreated_at: 2024-01-13T02:53:31.879719Z\n",
- "\tinput_dataset_id: sample-file-hca4x0\n",
- "\toutput_urls: None\n",
- "\tmodel: embed-english-v3.0\n",
- "\ttruncate: RIGHT\n",
- "\tpercent_complete: 100\n",
- "\toutput: cohere.Dataset {\n",
- "\tid: embeded-sample-file-drtjf9\n",
- "\tname: embeded-sample-file\n",
- "\tdataset_type: embed-result\n",
- "\tvalidation_status: validated\n",
- "\tcreated_at: 2024-01-13 02:53:33.569362\n",
- "\tupdated_at: 2024-01-13 02:53:33.569362\n",
- "\tdownload_urls: ['https://storage.googleapis.com/cohere-user/dataset-api-temp/d489c39a-e152-49da-9ddc-9801bd74d823/f8bd5ab1-ccdf-45e6-9717-23b8be1ff39d/embeded-sample-file-drtjf9/001_embeded-sample-file.avro?X-Goog-Algorithm=GOOG4-RSA-SHA256&X-Goog-Credential=dataset%40cohere-production.iam.gserviceaccount.com%2F20240113%2Fauto%2Fstorage%2Fgoog4_request&X-Goog-Date=20240113T025352Z&X-Goog-Expires=14399&X-Goog-Signature=927d2d73947058c95cd73f813e87e81416163dd0ccba1d5ce064a186fdc732550a605ff2122c9087fcc8e16dded9836084be532dac59622f73cb625fa1014fa9b1f9b702ca02be50006d8e2b2239870e3c765f7f8bd4964ccc74f4d9cc28672cd22c0d47008b112bd787eaab19692a5d9d724899f6aac088facdbe6d3b9d74f929fa07e76126656a5b0635a9793d9e85e692ca2d1d40f406869a53aa2e7390570d67e602fc5ea1741686176ce857112d5d68dee1b2fc62a436cd244890ec7c5901941cb10b01c2c41d6eba67dc393370a7c47526a7272f1f70f880b5fea2f69cb4546a633440a538fad7b92fdeb28d98aec1680cafc89142053da915268ea039&X-Goog-SignedHeaders=host']\n",
- "\tvalidation_error: None\n",
- "\tvalidation_warnings: []\n",
- "}\n",
- "}\n"
- ]
- }
- ],
- "source": [
- "print(job)"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {
- "id": "ZVf1njJ4952G"
- },
- "source": [
- "## Step 3: Download and prepare the embeddings"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "id": "0FdyxbcR5uci"
- },
- "outputs": [],
- "source": [
- "# Save down the output of the job\n",
- "embeddings_file_path = 'embed_jobs_output.csv'\n",
- "output_dataset=co.get_dataset(job.output.id)\n",
- "output_dataset.save(filepath=embeddings_file_path, format=\"csv\")"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "id": "VvheIBdJ6FC_"
- },
- "outputs": [],
- "source": [
- "# Add the results\n",
- "embeddings=[]\n",
- "texts=[]\n",
- "for record in output_dataset:\n",
- " embeddings.append(record['embeddings']['float'])\n",
- " texts.append(record['text'])"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {
- "id": "-O-Yz8MS-DFz"
- },
- "source": [
- "## Step 4: Initialize Hnwslib index and add embeddings"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "id": "_vGfX8aQ9uD1"
- },
- "outputs": [],
- "source": [
- "# Create the hnsw index\n",
- "index = hnswlib.Index(space='ip', dim=1024)\n",
- "index.init_index(max_elements=len(embeddings), ef_construction=512, M=64)\n",
- "index.add_items(embeddings,list(range(len(embeddings))))"
- ]
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "ig3hmpIt8ptJ"
+ },
+ "source": [
+ "# Semantic Search with Cohere Embed Jobs"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "Qu9a_jwZ5Zop"
+ },
+ "outputs": [],
+ "source": [
+ "import time\n",
+ "import cohere\n",
+ "import hnswlib\n",
+ "co = cohere.Client('COHERE_API_KEY')"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "Fgqhh1kk8mnY"
+ },
+ "source": [
+ "## Step 1: Upload a dataset"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "colab": {
+ "base_uri": "https://localhost:8080/"
},
+ "id": "kogkZnDEnK8B",
+ "outputId": "646b4508-8111-42ad-a9b2-3425da1ae5ff"
+ },
+ "outputs": [
{
- "cell_type": "markdown",
- "metadata": {
- "id": "yN5qBoVX-M7g"
- },
- "source": [
- "## Step 5: Query the index and rerank the results"
- ]
+ "name": "stderr",
+ "output_type": "stream",
+ "text": [
+ "uploading file, starting validation...\n",
+ "sample-file-hca4x0 was uploaded\n",
+ "...\n"
+ ]
+ }
+ ],
+ "source": [
+ "# Upload a dataset for embed jobs\n",
+ "# This sample dataset has wikipedia articles on the following: Youtube, United States, United Kingdom, Elizabeth II, Wikipedia, 2022 FIFA World Cup, Microsoft Office, India, Christiano Ronaldo, Cleopatra, Instagram, Facebook, and Ukraine\n",
+ "\n",
+ "dataset_file_path = \"data/embed_jobs_sample_data.jsonl\" # Full path - https://raw.githubusercontent.com/cohere-ai/notebooks/main/notebooks/data/embed_jobs_sample_data.jsonl\n",
+ "\n",
+ "ds=co.create_dataset(\n",
+ "\tname='sample_file',\n",
+ "\tdata=open(dataset_file_path, 'rb'),\n",
+ "\tkeep_fields = ['id','wiki_id'],\n",
+ "\tdataset_type=\"embed-input\"\n",
+ "\t)"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "colab": {
+ "base_uri": "https://localhost:8080/"
},
+ "id": "3lAS4bcvCplJ",
+ "outputId": "5e1fcd0b-4880-43ed-d8a4-03148b2ef047"
+ },
+ "outputs": [
{
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "id": "5NWR4MJUBQHW"
- },
- "outputs": [],
- "source": [
- "# Query the Database\n",
- "query = \"What was the first youtube video about?\"\n",
- "\n",
- "# Convert the query into embeddings\n",
- "query_emb=co.embed(\n",
- " texts=[query], model=\"embed-english-v3.0\", input_type=\"search_query\"\n",
- " ).embeddings\n",
- "\n",
- "# Retrieve the initial results from your vector db\n",
- "doc_index = index.knn_query(query_emb, k=10)[0][0]\n",
- "\n",
- "# From the doc_index, get the text from each index and then pass the text into rerank\n",
- "docs_to_rerank = []\n",
- "for index in doc_index:\n",
- " docs_to_rerank.append(texts[index])\n",
- "\n",
- "final_result = co.rerank(\n",
- " query= query,\n",
- " documents=docs_to_rerank,\n",
- " model=\"rerank-english-v2.0\",\n",
- " top_n=3)"
- ]
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "cohere.Dataset {\n",
+ "\tid: sample-file-hca4x0\n",
+ "\tname: sample_file\n",
+ "\tdataset_type: embed-input\n",
+ "\tvalidation_status: validated\n",
+ "\tcreated_at: 2024-01-13 02:51:48.215973\n",
+ "\tupdated_at: 2024-01-13 02:51:48.215973\n",
+ "\tdownload_urls: ['https://storage.googleapis.com/cohere-user/dataset-api-temp/d489c39a-e152-49da-9ddc-9801bd74d823/f8bd5ab1-ccdf-45e6-9717-23b8be1ff39d/sample-file-hca4x0/000_embed_jobs_sample_data.avro?X-Goog-Algorithm=GOOG4-RSA-SHA256&X-Goog-Credential=dataset%40cohere-production.iam.gserviceaccount.com%2F20240113%2Fauto%2Fstorage%2Fgoog4_request&X-Goog-Date=20240113T025159Z&X-Goog-Expires=14399&X-Goog-Signature=7f9ed7cb27988e4bf447167160b25940cd106859d1282919604a0d74b882681b5aaa3245f97555bb61ddca7fd9c02fb0bba8391433c70aeb2118607dfd534db444097c5ae9c8d07e66bf6723a64634f5d6f5b2500c82e351807203f5fd3c278b2584c258b4c6afc6ee191e63bcd346e3dd2c7fd4b171c2d3ddfe8e6e68c2522111b6f63e125a052be9ebe3302903f4bef9cd165c8e075b168e621c7c61177a0973bb470cc3535457078e18b338884fb303792a1ba3161ab5ca5ce2adca5676754ee1a735891ccbf64cca03d3fdcabdcccd0d201600b8c70e13487bf897f2b88424cc736f8bf8756ec835b09dacf0aa20dc8151d9249b21d4ca4de82144fcd8be&X-Goog-SignedHeaders=host']\n",
+ "\tvalidation_error: None\n",
+ "\tvalidation_warnings: []\n",
+ "}\n"
+ ]
+ }
+ ],
+ "source": [
+ "print(ds.await_validation())"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "QgD2QCJk9kUN"
+ },
+ "source": [
+ "## Step 2: Create embeddings via Cohere's Embed Jobs endpoint"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "colab": {
+ "base_uri": "https://localhost:8080/"
},
+ "id": "yMYoU9y55m2D",
+ "outputId": "fd58014d-a9e3-4a3f-ca4b-966880dec412"
+ },
+ "outputs": [
{
- "cell_type": "markdown",
- "metadata": {
- "id": "Oya4CRdE-WDU"
- },
- "source": [
- "## Step 6: Display the results"
- ]
+ "name": "stderr",
+ "output_type": "stream",
+ "text": [
+ "...\n",
+ "...\n"
+ ]
+ }
+ ],
+ "source": [
+ "# Dataset has been uploaded, create an embed job and specify the input type as \"search document\" since this will live in your Pinecone DB\n",
+ "job = co.create_embed_job(\n",
+ " dataset_id=ds.id,\n",
+ " input_type='search_document' ,\n",
+ " model='embed-english-v3.0', \n",
+ " embeddings_types=['float'])\n",
+ "\n",
+ "job.wait() # poll the server until the job is completed "
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "colab": {
+ "base_uri": "https://localhost:8080/"
},
+ "id": "LEevvN2XCUpR",
+ "outputId": "d123c0a4-0c6a-4235-9ba8-658d5d3ec358"
+ },
+ "outputs": [
{
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "colab": {
- "base_uri": "https://localhost:8080/"
- },
- "id": "rvs0Se2wER1a",
- "outputId": "eeb3e148-af79-46c0-ca3b-ae9facde5fbc"
- },
- "outputs": [
- {
- "name": "stdout",
- "output_type": "stream",
- "text": [
- "Document Rank: 1, Document Index: 0\n",
- "Document: YouTube began as a venture capital–funded technology startup. Between November 2005 and April 2006, the company raised money from various investors, with Sequoia Capital, $11.5 million, and Artis Capital Management, $8 million, being the largest two. YouTube's early headquarters were situated above a pizzeria and a Japanese restaurant in San Mateo, California. In February 2005, the company activated codice_1. The first video was uploaded April 23, 2005. Titled \"Me at the zoo\", it shows co-founder Jawed Karim at the San Diego Zoo and can still be viewed on the site. In May, the company launched a public beta and by November, a Nike ad featuring Ronaldinho became the first video to reach one million total views. The site launched officially on December 15, 2005, by which time the site was receiving 8 million views a day. Clips at the time were limited to 100 megabytes, as little as 30 seconds of footage.\n",
- "Relevance Score: 0.94815\n",
- "\n",
- "\n",
- "Document Rank: 2, Document Index: 1\n",
- "Document: Karim said the inspiration for YouTube first came from the Super Bowl XXXVIII halftime show controversy when Janet Jackson's breast was briefly exposed by Justin Timberlake during the halftime show. Karim could not easily find video clips of the incident and the 2004 Indian Ocean Tsunami online, which led to the idea of a video-sharing site. Hurley and Chen said that the original idea for YouTube was a video version of an online dating service, and had been influenced by the website Hot or Not. They created posts on Craigslist asking attractive women to upload videos of themselves to YouTube in exchange for a $100 reward. Difficulty in finding enough dating videos led to a change of plans, with the site's founders deciding to accept uploads of any video.\n",
- "Relevance Score: 0.91626\n",
- "\n",
- "\n",
- "Document Rank: 3, Document Index: 2\n",
- "Document: YouTube was not the first video-sharing site on the Internet; Vimeo was launched in November 2004, though that site remained a side project of its developers from CollegeHumor at the time and did not grow much, either. The week of YouTube's launch, NBC-Universal's \"Saturday Night Live\" ran a skit \"Lazy Sunday\" by The Lonely Island. Besides helping to bolster ratings and long-term viewership for \"Saturday Night Live\", \"Lazy Sunday\"'s status as an early viral video helped establish YouTube as an important website. Unofficial uploads of the skit to YouTube drew in more than five million collective views by February 2006 before they were removed when NBCUniversal requested it two months later based on copyright concerns. Despite eventually being taken down, these duplicate uploads of the skit helped popularize YouTube's reach and led to the upload of more third-party content. The site grew rapidly; in July 2006, the company announced that more than 65,000 new videos were being uploaded every day and that the site was receiving 100 million video views per day.\n",
- "Relevance Score: 0.90665\n",
- "\n",
- "\n"
- ]
- }
- ],
- "source": [
- "# Output Results\n",
- "for idx, r in enumerate(final_result):\n",
- " print(f\"Document Rank: {idx + 1}, Document Index: {r.index}\")\n",
- " print(f\"Document: {r.document['text']}\")\n",
- " print(f\"Relevance Score: {r.relevance_score:.5f}\")\n",
- " print(\"\\n\")"
- ]
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "cohere.EmbedJob {\n",
+ "\tjob_id: 792bbc1a-561b-48c2-8a97-0c80c1914ea8\n",
+ "\tstatus: complete\n",
+ "\tcreated_at: 2024-01-13T02:53:31.879719Z\n",
+ "\tinput_dataset_id: sample-file-hca4x0\n",
+ "\toutput_urls: None\n",
+ "\tmodel: embed-english-v3.0\n",
+ "\ttruncate: RIGHT\n",
+ "\tpercent_complete: 100\n",
+ "\toutput: cohere.Dataset {\n",
+ "\tid: embeded-sample-file-drtjf9\n",
+ "\tname: embeded-sample-file\n",
+ "\tdataset_type: embed-result\n",
+ "\tvalidation_status: validated\n",
+ "\tcreated_at: 2024-01-13 02:53:33.569362\n",
+ "\tupdated_at: 2024-01-13 02:53:33.569362\n",
+ "\tdownload_urls: ['https://storage.googleapis.com/cohere-user/dataset-api-temp/d489c39a-e152-49da-9ddc-9801bd74d823/f8bd5ab1-ccdf-45e6-9717-23b8be1ff39d/embeded-sample-file-drtjf9/001_embeded-sample-file.avro?X-Goog-Algorithm=GOOG4-RSA-SHA256&X-Goog-Credential=dataset%40cohere-production.iam.gserviceaccount.com%2F20240113%2Fauto%2Fstorage%2Fgoog4_request&X-Goog-Date=20240113T025352Z&X-Goog-Expires=14399&X-Goog-Signature=927d2d73947058c95cd73f813e87e81416163dd0ccba1d5ce064a186fdc732550a605ff2122c9087fcc8e16dded9836084be532dac59622f73cb625fa1014fa9b1f9b702ca02be50006d8e2b2239870e3c765f7f8bd4964ccc74f4d9cc28672cd22c0d47008b112bd787eaab19692a5d9d724899f6aac088facdbe6d3b9d74f929fa07e76126656a5b0635a9793d9e85e692ca2d1d40f406869a53aa2e7390570d67e602fc5ea1741686176ce857112d5d68dee1b2fc62a436cd244890ec7c5901941cb10b01c2c41d6eba67dc393370a7c47526a7272f1f70f880b5fea2f69cb4546a633440a538fad7b92fdeb28d98aec1680cafc89142053da915268ea039&X-Goog-SignedHeaders=host']\n",
+ "\tvalidation_error: None\n",
+ "\tvalidation_warnings: []\n",
+ "}\n",
+ "}\n"
+ ]
}
- ],
- "metadata": {
+ ],
+ "source": [
+ "print(job)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "ZVf1njJ4952G"
+ },
+ "source": [
+ "## Step 3: Download and prepare the embeddings"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "0FdyxbcR5uci"
+ },
+ "outputs": [],
+ "source": [
+ "# Save down the output of the job\n",
+ "embeddings_file_path = 'embed_jobs_output.csv'\n",
+ "output_dataset=co.get_dataset(job.output.id)\n",
+ "output_dataset.save(filepath=embeddings_file_path, format=\"csv\")"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "VvheIBdJ6FC_"
+ },
+ "outputs": [],
+ "source": [
+ "# Add the results\n",
+ "embeddings=[]\n",
+ "texts=[]\n",
+ "for record in output_dataset:\n",
+ " embeddings.append(record['embeddings']['float'])\n",
+ " texts.append(record['text'])"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "-O-Yz8MS-DFz"
+ },
+ "source": [
+ "## Step 4: Initialize Hnwslib index and add embeddings"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "_vGfX8aQ9uD1"
+ },
+ "outputs": [],
+ "source": [
+ "# Create the hnsw index\n",
+ "index = hnswlib.Index(space='ip', dim=1024)\n",
+ "index.init_index(max_elements=len(embeddings), ef_construction=512, M=64)\n",
+ "index.add_items(embeddings,list(range(len(embeddings))))"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "yN5qBoVX-M7g"
+ },
+ "source": [
+ "## Step 5: Query the index and rerank the results"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "5NWR4MJUBQHW"
+ },
+ "outputs": [],
+ "source": [
+ "# Query the Database\n",
+ "query = \"What was the first youtube video about?\"\n",
+ "\n",
+ "# Convert the query into embeddings\n",
+ "query_emb=co.embed(\n",
+ " texts=[query], model=\"embed-english-v3.0\", input_type=\"search_query\"\n",
+ " ).embeddings\n",
+ "\n",
+ "# Retrieve the initial results from your vector db\n",
+ "doc_index = index.knn_query(query_emb, k=10)[0][0]\n",
+ "\n",
+ "# From the doc_index, get the text from each index and then pass the text into rerank\n",
+ "docs_to_rerank = []\n",
+ "for index in doc_index:\n",
+ " docs_to_rerank.append(texts[index])\n",
+ "\n",
+ "final_result = co.rerank(\n",
+ " query= query,\n",
+ " documents=docs_to_rerank,\n",
+ " model=\"rerank-english-v2.0\",\n",
+ " top_n=3)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "Oya4CRdE-WDU"
+ },
+ "source": [
+ "## Step 6: Display the results"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
"colab": {
- "provenance": []
+ "base_uri": "https://localhost:8080/"
},
- "kernelspec": {
- "display_name": "Python 3",
- "name": "python3"
- },
- "language_info": {
- "name": "python"
+ "id": "rvs0Se2wER1a",
+ "outputId": "eeb3e148-af79-46c0-ca3b-ae9facde5fbc"
+ },
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "Document Rank: 1, Document Index: 0\n",
+ "Document: YouTube began as a venture capital–funded technology startup. Between November 2005 and April 2006, the company raised money from various investors, with Sequoia Capital, $11.5 million, and Artis Capital Management, $8 million, being the largest two. YouTube's early headquarters were situated above a pizzeria and a Japanese restaurant in San Mateo, California. In February 2005, the company activated codice_1. The first video was uploaded April 23, 2005. Titled \"Me at the zoo\", it shows co-founder Jawed Karim at the San Diego Zoo and can still be viewed on the site. In May, the company launched a public beta and by November, a Nike ad featuring Ronaldinho became the first video to reach one million total views. The site launched officially on December 15, 2005, by which time the site was receiving 8 million views a day. Clips at the time were limited to 100 megabytes, as little as 30 seconds of footage.\n",
+ "Relevance Score: 0.94815\n",
+ "\n",
+ "\n",
+ "Document Rank: 2, Document Index: 1\n",
+ "Document: Karim said the inspiration for YouTube first came from the Super Bowl XXXVIII halftime show controversy when Janet Jackson's breast was briefly exposed by Justin Timberlake during the halftime show. Karim could not easily find video clips of the incident and the 2004 Indian Ocean Tsunami online, which led to the idea of a video-sharing site. Hurley and Chen said that the original idea for YouTube was a video version of an online dating service, and had been influenced by the website Hot or Not. They created posts on Craigslist asking attractive women to upload videos of themselves to YouTube in exchange for a $100 reward. Difficulty in finding enough dating videos led to a change of plans, with the site's founders deciding to accept uploads of any video.\n",
+ "Relevance Score: 0.91626\n",
+ "\n",
+ "\n",
+ "Document Rank: 3, Document Index: 2\n",
+ "Document: YouTube was not the first video-sharing site on the Internet; Vimeo was launched in November 2004, though that site remained a side project of its developers from CollegeHumor at the time and did not grow much, either. The week of YouTube's launch, NBC-Universal's \"Saturday Night Live\" ran a skit \"Lazy Sunday\" by The Lonely Island. Besides helping to bolster ratings and long-term viewership for \"Saturday Night Live\", \"Lazy Sunday\"'s status as an early viral video helped establish YouTube as an important website. Unofficial uploads of the skit to YouTube drew in more than five million collective views by February 2006 before they were removed when NBCUniversal requested it two months later based on copyright concerns. Despite eventually being taken down, these duplicate uploads of the skit helped popularize YouTube's reach and led to the upload of more third-party content. The site grew rapidly; in July 2006, the company announced that more than 65,000 new videos were being uploaded every day and that the site was receiving 100 million video views per day.\n",
+ "Relevance Score: 0.90665\n",
+ "\n",
+ "\n"
+ ]
}
+ ],
+ "source": [
+ "# Output Results\n",
+ "for idx, r in enumerate(final_result):\n",
+ " print(f\"Document Rank: {idx + 1}, Document Index: {r.index}\")\n",
+ " print(f\"Document: {r.document['text']}\")\n",
+ " print(f\"Relevance Score: {r.relevance_score:.5f}\")\n",
+ " print(\"\\n\")"
+ ]
+ }
+ ],
+ "metadata": {
+ "colab": {
+ "provenance": []
+ },
+ "kernelspec": {
+ "display_name": "Python 3 (ipykernel)",
+ "language": "python",
+ "name": "python3"
},
- "nbformat": 4,
- "nbformat_minor": 0
+ "language_info": {
+ "codemirror_mode": {
+ "name": "ipython",
+ "version": 3
+ },
+ "file_extension": ".py",
+ "mimetype": "text/x-python",
+ "name": "python",
+ "nbconvert_exporter": "python",
+ "pygments_lexer": "ipython3",
+ "version": "3.8.16"
+ }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 4
}
diff --git a/notebooks/Embed_Jobs_Serverless_Pinecone_Semantic_Search.ipynb b/notebooks/Embed_Jobs_Serverless_Pinecone_Semantic_Search.ipynb
index 4a3ff53b..d85f2c44 100644
--- a/notebooks/Embed_Jobs_Serverless_Pinecone_Semantic_Search.ipynb
+++ b/notebooks/Embed_Jobs_Serverless_Pinecone_Semantic_Search.ipynb
@@ -1,564 +1,561 @@
{
- "cells": [
- {
- "cell_type": "markdown",
- "metadata": {
- "id": "frQ5it0RmB0_"
- },
- "source": [
- "# Semantic Search with Cohere Embed Jobs and Pinecone serverless Solution"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "id": "Fdhi1O4lrqaV"
- },
- "outputs": [],
- "source": [
- "# TODO: upgrade to \"cohere>5\"\n",
- "! pip install \"cohere<5\" \"pinecone-client>3.2.1\""
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "colab": {
- "base_uri": "https://localhost:8080/"
- },
- "id": "N_bYWQORGuMM",
- "outputId": "ee9f81f3-e896-46ff-e287-48502cff0b29"
- },
- "outputs": [
- {
- "name": "stderr",
- "output_type": "stream",
- "text": [
- "/usr/local/lib/python3.10/dist-packages/pinecone/data/index.py:1: TqdmExperimentalWarning: Using `tqdm.autonotebook.tqdm` in notebook mode. Use `tqdm.tqdm` instead to force console mode (e.g. in jupyter console)\n",
- " from tqdm.autonotebook import tqdm\n"
- ]
- }
- ],
- "source": [
- "import os\n",
- "import json\n",
- "import time\n",
- "import numpy as np\n",
- "import cohere\n",
- "from pinecone import Pinecone\n",
- "\n",
- "co = cohere.Client('COHERE_API_KEY')\n",
- "pc = Pinecone(\n",
- " api_key=\"PINECONE_API_KEY\", \n",
- " source_tag=\"cohere\"\n",
- ")"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {
- "id": "XulUR7tt69RM"
- },
- "source": [
- "## Step 1: Upload a dataset"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "colab": {
- "base_uri": "https://localhost:8080/"
- },
- "id": "G677sKZc6NDv",
- "outputId": "e4bfd9ca-590a-4cf7-d9cf-3657a16e2a7f"
- },
- "outputs": [
- {
- "name": "stderr",
- "output_type": "stream",
- "text": [
- "uploading file, starting validation...\n",
- "sample-file-2gwgxq was uploaded\n",
- "...\n"
- ]
- },
- {
- "name": "stdout",
- "output_type": "stream",
- "text": [
- "cohere.Dataset {\n",
- "\tid: sample-file-2gwgxq\n",
- "\tname: sample_file\n",
- "\tdataset_type: embed-input\n",
- "\tvalidation_status: validated\n",
- "\tcreated_at: 2024-01-13 02:47:32.563080\n",
- "\tupdated_at: 2024-01-13 02:47:32.563081\n",
- "\tdownload_urls: ['https://storage.googleapis.com/cohere-user/dataset-api-temp/d489c39a-e152-49da-9ddc-9801bd74d823/96d12a16-2dd4-46f7-9630-1fa9bb0b26ca/sample-file-2gwgxq/000_embed_jobs_sample_data.avro?X-Goog-Algorithm=GOOG4-RSA-SHA256&X-Goog-Credential=dataset%40cohere-production.iam.gserviceaccount.com%2F20240113%2Fauto%2Fstorage%2Fgoog4_request&X-Goog-Date=20240113T024743Z&X-Goog-Expires=14399&X-Goog-Signature=1cc013824d54e4974600e19ec5c5dd6624c3de0c512f1c7d204fa95056b09e19d90c0fe034c59f0eef8bdfd4e1ed03b44c601d290e9b9c9e0643afd7a44fe97c42750304a8f199c9d87abb50fc74d777ab2d36efecca64ad97264f9e0628f2e199b9eb2d505241480a7436191cacaa78efb328567c9303469b653962caf07e6b11fac06ed06bd4597377e87fac58214bab1cd5cde63e19508d903e65ad654a177e27a64105b79c56d0cc156a35b61d45a7dda3b9819ef78dc9861c818b808527a1e16210dc83130be630f1b54c75d280ea20d070566056a6b2c7b1a016409482defc2a1942a801e46f5349adfadb244711da80d103ed831d6927366adc0c1659&X-Goog-SignedHeaders=host']\n",
- "\tvalidation_error: None\n",
- "\tvalidation_warnings: []\n",
- "}\n"
- ]
- }
- ],
- "source": [
- "# Upload a dataset for embed jobs\n",
- "dataset_file_path = \"data/embed_jobs_sample_data.jsonl\" # Full path - https://raw.githubusercontent.com/cohere-ai/notebooks/main/notebooks/data/embed_jobs_sample_data.jsonl\n",
- "\n",
- "ds=co.create_dataset(\n",
- "\tname='sample_file',\n",
- "\t# insert your file path here - you can upload it on the right - we accept .csv and jsonl files\n",
- "\tdata=open(dataset_file_path, 'rb'),\n",
- "\tdataset_type=\"embed-input\"\n",
- "\t)\n",
- "\n",
- "print(ds.await_validation())"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {
- "id": "5VAoaVo47Bmi"
- },
- "source": [
- "## Step 2: Create embeddings via Cohere's Embed Jobs endpoint"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "colab": {
- "base_uri": "https://localhost:8080/"
- },
- "id": "77Vw5BdWGzFl",
- "outputId": "a5e0c9f5-8172-4b80-a8ea-aa4e763fdeda"
- },
- "outputs": [
- {
- "name": "stderr",
- "output_type": "stream",
- "text": [
- "...\n",
- "...\n"
- ]
- }
- ],
- "source": [
- "# Dataset has been uploaded, create an embed job and specify the input type as \"search document\" since this will live in your Pinecone DB\n",
- "job = co.create_embed_job(dataset_id=ds.id,\n",
- " input_type='search_document',\n",
- " model='embed-english-v3.0',\n",
- " embeddings_types=['float'])\n",
- "\n",
- "job.wait() # poll the server until the job is completed "
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "colab": {
- "base_uri": "https://localhost:8080/"
- },
- "id": "d__jANfVvNld",
- "outputId": "c4dcd936-c338-47b3-994d-71cdceaa6796"
- },
- "outputs": [
- {
- "name": "stdout",
- "output_type": "stream",
- "text": [
- "cohere.EmbedJob {\n",
- "\tjob_id: 6d691fbe-e026-436a-826a-16e70b293e51\n",
- "\tstatus: complete\n",
- "\tcreated_at: 2024-01-13T02:47:46.385016Z\n",
- "\tinput_dataset_id: sample-file-2gwgxq\n",
- "\toutput_urls: None\n",
- "\tmodel: embed-english-v3.0\n",
- "\ttruncate: RIGHT\n",
- "\tpercent_complete: 100\n",
- "\toutput: cohere.Dataset {\n",
- "\tid: embeded-sample-file-mdse2h\n",
- "\tname: embeded-sample-file\n",
- "\tdataset_type: embed-result\n",
- "\tvalidation_status: validated\n",
- "\tcreated_at: 2024-01-13 02:47:47.850097\n",
- "\tupdated_at: 2024-01-13 02:47:47.850097\n",
- "\tdownload_urls: ['https://storage.googleapis.com/cohere-user/dataset-api-temp/d489c39a-e152-49da-9ddc-9801bd74d823/96d12a16-2dd4-46f7-9630-1fa9bb0b26ca/embeded-sample-file-mdse2h/001_embeded-sample-file.avro?X-Goog-Algorithm=GOOG4-RSA-SHA256&X-Goog-Credential=dataset%40cohere-production.iam.gserviceaccount.com%2F20240113%2Fauto%2Fstorage%2Fgoog4_request&X-Goog-Date=20240113T024807Z&X-Goog-Expires=14399&X-Goog-Signature=78b0d82e19388aee2926a4ef403d5f286487d0e7fc9a09c75bfb6700b502fa4f55b024977c0aeae26cd501a41e506bf00d3e850d7d9691298963374f04fa129e8dae5a327389c80921bf611a25386f5e4501b11c9d88bc1aee6f6877157a4675c5f8fe06bb25e3ca2b12da46e5b1da3067ca71bed901cd14db26e09987e355c039f96d6514a0931aa5f8753ddc155ca1782c63e3cb000b095d3b29904982ff75686c716329e92b6946485c567dabc3e344c9a0f9a59416415738b67ead0cca3cdb06c1db64c925ea38a2d92ab4079577e5775367260c09916aab5af67326bca4fa1295ee76457f933a6ca26a5d4ac9c59f0f73286627b2bae3e7fa7375c97465&X-Goog-SignedHeaders=host']\n",
- "\tvalidation_error: None\n",
- "\tvalidation_warnings: []\n",
- "}\n",
- "}\n"
- ]
- }
- ],
- "source": [
- "print(job)"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {
- "id": "UlEVKOCG7ZsN"
- },
- "source": [
- "## Step 3: Prepare embeddings for upsert"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "id": "vDvntjsnG_DX"
- },
- "outputs": [],
- "source": [
- "# Load the output file into an array\n",
- "output_dataset=co.get_dataset(job.output.id)\n",
- "data_array = []\n",
- "for record in output_dataset:\n",
- " data_array.append(record)\n",
- "\n",
- "# Take the output and format it in the shape for upserting into Pinecone's DB\n",
- "ids = [str(i) for i in range(len(data_array))]\n",
- "meta = [{'text':str(data_array[i]['text'])} for i in range(len(data_array))]\n",
- "embeds=[np.float32(data_array[i]['embeddings']['float']) for i in range(len(data_array))]\n",
- "\n",
- "to_upsert = list(zip(ids, embeds, meta))"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {
- "id": "pQwXK4xt7lls"
- },
- "source": [
- "## Step 4: Initialize Pinecone vector database"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "id": "GrLN8Y_THEBH"
- },
- "outputs": [],
- "source": [
- "# Initialize your Pinecone Vector DB\n",
- "from pinecone import ServerlessSpec\n",
- "\n",
- "index_name = \"embed-jobs-serverless-test-example\"\n",
- "\n",
- "# A new property 'spec' is used to tell Pinecone how we should deploy your index.\n",
- "pc.create_index(\n",
- "name=index_name,\n",
- "dimension=1024,\n",
- "metric=\"cosine\",\n",
- "spec=ServerlessSpec(cloud='aws', region='us-west-2')\n",
- ")\n",
- "\n",
- "# Target your new serverless index.\n",
- "idx = pc.Index(index_name)"
- ]
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "frQ5it0RmB0_"
+ },
+ "source": [
+ "# Semantic Search with Cohere Embed Jobs and Pinecone serverless Solution"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "colab": {
+ "base_uri": "https://localhost:8080/"
},
+ "id": "N_bYWQORGuMM",
+ "outputId": "ee9f81f3-e896-46ff-e287-48502cff0b29"
+ },
+ "outputs": [
{
- "cell_type": "markdown",
- "metadata": {
- "id": "W2QMVhoM7o-5"
- },
- "source": [
- "## Step 5: Upsert embeddings into the index"
- ]
+ "name": "stderr",
+ "output_type": "stream",
+ "text": [
+ "/usr/local/lib/python3.10/dist-packages/pinecone/data/index.py:1: TqdmExperimentalWarning: Using `tqdm.autonotebook.tqdm` in notebook mode. Use `tqdm.tqdm` instead to force console mode (e.g. in jupyter console)\n",
+ " from tqdm.autonotebook import tqdm\n"
+ ]
+ }
+ ],
+ "source": [
+ "import os\n",
+ "import json\n",
+ "import time\n",
+ "import numpy as np\n",
+ "import cohere\n",
+ "from pinecone import Pinecone\n",
+ "\n",
+ "co = cohere.Client('COHERE_API_KEY')\n",
+ "pc = Pinecone(\n",
+ " api_key=\"PINECONE_API_KEY\", \n",
+ " source_tag=\"cohere\"\n",
+ ")"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "XulUR7tt69RM"
+ },
+ "source": [
+ "## Step 1: Upload a dataset"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "colab": {
+ "base_uri": "https://localhost:8080/"
},
+ "id": "G677sKZc6NDv",
+ "outputId": "e4bfd9ca-590a-4cf7-d9cf-3657a16e2a7f"
+ },
+ "outputs": [
{
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "colab": {
- "base_uri": "https://localhost:8080/"
- },
- "id": "JaVrXHZ3IjiN",
- "outputId": "716ad587-2ee1-475a-f1e5-9a0b23d2df87"
- },
- "outputs": [
- {
- "name": "stdout",
- "output_type": "stream",
- "text": [
- "{'dimension': 1024,\n",
- " 'index_fullness': 0.0,\n",
- " 'namespaces': {'': {'vector_count': 3664}},\n",
- " 'total_vector_count': 3664}\n"
- ]
- }
- ],
- "source": [
- "# Upsert your data into the index\n",
- "batch_size = 128\n",
- "\n",
- "for i in range(0, len(data_array), batch_size):\n",
- " i_end = min(i+batch_size, len(data_array))\n",
- " idx.upsert(vectors=to_upsert[i:i_end])\n",
- "\n",
- "# let's view the index statistics\n",
- "print(idx.describe_index_stats())"
- ]
+ "name": "stderr",
+ "output_type": "stream",
+ "text": [
+ "uploading file, starting validation...\n",
+ "sample-file-2gwgxq was uploaded\n",
+ "...\n"
+ ]
},
{
- "cell_type": "markdown",
- "metadata": {
- "id": "AtT1Yo_i8CZc"
- },
- "source": [
- "## Step 6: Query the index"
- ]
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "cohere.Dataset {\n",
+ "\tid: sample-file-2gwgxq\n",
+ "\tname: sample_file\n",
+ "\tdataset_type: embed-input\n",
+ "\tvalidation_status: validated\n",
+ "\tcreated_at: 2024-01-13 02:47:32.563080\n",
+ "\tupdated_at: 2024-01-13 02:47:32.563081\n",
+ "\tdownload_urls: ['https://storage.googleapis.com/cohere-user/dataset-api-temp/d489c39a-e152-49da-9ddc-9801bd74d823/96d12a16-2dd4-46f7-9630-1fa9bb0b26ca/sample-file-2gwgxq/000_embed_jobs_sample_data.avro?X-Goog-Algorithm=GOOG4-RSA-SHA256&X-Goog-Credential=dataset%40cohere-production.iam.gserviceaccount.com%2F20240113%2Fauto%2Fstorage%2Fgoog4_request&X-Goog-Date=20240113T024743Z&X-Goog-Expires=14399&X-Goog-Signature=1cc013824d54e4974600e19ec5c5dd6624c3de0c512f1c7d204fa95056b09e19d90c0fe034c59f0eef8bdfd4e1ed03b44c601d290e9b9c9e0643afd7a44fe97c42750304a8f199c9d87abb50fc74d777ab2d36efecca64ad97264f9e0628f2e199b9eb2d505241480a7436191cacaa78efb328567c9303469b653962caf07e6b11fac06ed06bd4597377e87fac58214bab1cd5cde63e19508d903e65ad654a177e27a64105b79c56d0cc156a35b61d45a7dda3b9819ef78dc9861c818b808527a1e16210dc83130be630f1b54c75d280ea20d070566056a6b2c7b1a016409482defc2a1942a801e46f5349adfadb244711da80d103ed831d6927366adc0c1659&X-Goog-SignedHeaders=host']\n",
+ "\tvalidation_error: None\n",
+ "\tvalidation_warnings: []\n",
+ "}\n"
+ ]
+ }
+ ],
+ "source": [
+ "# Upload a dataset for embed jobs\n",
+ "dataset_file_path = \"data/embed_jobs_sample_data.jsonl\" # Full path - https://raw.githubusercontent.com/cohere-ai/notebooks/main/notebooks/data/embed_jobs_sample_data.jsonl\n",
+ "\n",
+ "ds=co.create_dataset(\n",
+ "\tname='sample_file',\n",
+ "\t# insert your file path here - you can upload it on the right - we accept .csv and jsonl files\n",
+ "\tdata=open(dataset_file_path, 'rb'),\n",
+ "\tdataset_type=\"embed-input\"\n",
+ "\t)\n",
+ "\n",
+ "print(ds.await_validation())"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "5VAoaVo47Bmi"
+ },
+ "source": [
+ "## Step 2: Create embeddings via Cohere's Embed Jobs endpoint"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "colab": {
+ "base_uri": "https://localhost:8080/"
},
+ "id": "77Vw5BdWGzFl",
+ "outputId": "a5e0c9f5-8172-4b80-a8ea-aa4e763fdeda"
+ },
+ "outputs": [
{
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "colab": {
- "base_uri": "https://localhost:8080/"
- },
- "id": "t-A82Z1EIrKR",
- "outputId": "ed708eef-0c0b-4688-f0d7-f5fc39219848"
- },
- "outputs": [
- {
- "name": "stdout",
- "output_type": "stream",
- "text": [
- "(1, 1024)\n"
- ]
- }
- ],
- "source": [
- "# Let's query the database\n",
- "query = \"What did Microsoft announce in Las Vegas?\"\n",
- "\n",
- "# create the query embedding\n",
- "xq = co.embed(\n",
- " texts=[query],\n",
- " model='embed-english-v3.0',\n",
- " input_type='search_query',\n",
- " truncate='END'\n",
- ").embeddings\n",
- "\n",
- "print(np.array(xq).shape)\n",
- "\n",
- "# query, returning the top 20 most similar results\n",
- "res = idx.query(xq, top_k=20, include_metadata=True)"
- ]
+ "name": "stderr",
+ "output_type": "stream",
+ "text": [
+ "...\n",
+ "...\n"
+ ]
+ }
+ ],
+ "source": [
+ "# Dataset has been uploaded, create an embed job and specify the input type as \"search document\" since this will live in your Pinecone DB\n",
+ "job = co.create_embed_job(dataset_id=ds.id,\n",
+ " input_type='search_document',\n",
+ " model='embed-english-v3.0',\n",
+ " embeddings_types=['float'])\n",
+ "\n",
+ "job.wait() # poll the server until the job is completed "
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "colab": {
+ "base_uri": "https://localhost:8080/"
},
+ "id": "d__jANfVvNld",
+ "outputId": "c4dcd936-c338-47b3-994d-71cdceaa6796"
+ },
+ "outputs": [
{
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "colab": {
- "base_uri": "https://localhost:8080/"
- },
- "id": "pXhvxHwj5nX2",
- "outputId": "422948e1-a554-4bd2-9d28-076df6f5c3ee"
- },
- "outputs": [
- {
- "name": "stdout",
- "output_type": "stream",
- "text": [
- "0.48: On October 22, 2012, Microsoft announced the release of new features including co-authoring, performance improvements and touch support.\n",
- "0.45: On May 2, 2019, at F8, the company announced its new vision with the tagline \"the future is private\". A redesign of the website and mobile app was introduced, dubbed as \"FB5\". The event also featured plans for improving groups, a dating platform, end-to-end encryption on its platforms, and allowing users on Messenger to communicate directly with WhatsApp and Instagram users.\n",
- "0.42: On July 13, 2009, Microsoft announced at its Worldwide Partners Conference 2009 in New Orleans that Microsoft Office 2010 reached its \"Technical Preview\" development milestone and features of Office Web Apps were demonstrated to the public for the first time. Additionally, Microsoft announced that Office Web Apps would be made available to consumers online and free of charge, while Microsoft Software Assurance customers will have the option of running them on premises. Office 2010 beta testers were not given access to Office Web Apps at this date, and it was announced that it would be available for testers during August 2009. However, in August 2009, a Microsoft spokesperson stated that there had been a delay in the release of Office Web Apps Technical Preview and it would not be available by the end of August.\n",
- "0.42: On January 17, 2017, Facebook COO Sheryl Sandberg planned to open Station F, a startup incubator campus in Paris, France. On a six-month cycle, Facebook committed to work with ten to 15 data-driven startups there. On April 18, Facebook announced the beta launch of at its annual F8 developer conference. Facebook Spaces is a virtual reality version of Facebook for Oculus VR goggles. In a virtual and shared space, users can access a curated selection of 360-degree photos and videos using their avatar, with the support of the controller. Users can access their own photos and videos, along with media shared on their newsfeed. In September, Facebook announced it would spend up to US$1 billion on original shows for its Facebook Watch platform. On October 16, it acquired the anonymous compliment app tbh, announcing its intention to leave the app independent.\n",
- "0.41: On September 26, 2017, Microsoft announced that the next version of the suite for Windows desktop, Office 2019, was in development. On April 27, 2018, Microsoft released Office 2019 Commercial Preview for Windows 10. It was released to general availability for Windows 10 and for macOS on September 24, 2018.\n",
- "0.41: Microsoft Office, or simply Office, is the former name of a family of client software, server software, and services developed by Microsoft. It was first announced by Bill Gates on August 1, 1988, at COMDEX in Las Vegas. Initially a marketing term for an office suite (bundled set of productivity applications), the first version of Office contained Microsoft Word, Microsoft Excel, and Microsoft PowerPoint. Over the years, Office applications have grown substantially closer with shared features such as a common spell checker, Object Linking and Embedding data integration and Visual Basic for Applications scripting language. Microsoft also positions Office as a development platform for line-of-business software under the Office Business Applications brand.\n",
- "0.40: On August 12, 2009, it was announced that Office Mobile would also be released for the Symbian platform as a joint agreement between Microsoft and Nokia. It was the first time Microsoft would develop Office mobile applications for another smartphone platform. The first application to appear on Nokia Eseries smartphones was Microsoft Office Communicator. In February 2012, Microsoft released OneNote, Lync 2010, Document Connection and PowerPoint Broadcast for Symbian. In April, Word Mobile, PowerPoint Mobile and Excel Mobile joined the Office Suite.\n",
- "0.40: In 2010, Microsoft introduced a software as a service platform known as Office 365, to provide cloud-hosted versions of Office's server software, including Exchange e-mail and SharePoint, on a subscription basis (competing in particular with Google Apps). Following the release of Office 2013, Microsoft began to offer Office 365 plans for the consumer market, with access to Microsoft Office software on multiple devices with free feature updates over the life of the subscription, as well as other services such as OneDrive storage.\n",
- "0.40: On April 12, 2016, Zuckerberg outlined his 10-year vision, which rested on three main pillars: artificial intelligence, increased global connectivity, and virtual and augmented reality. In July, a suit was filed against the company alleging that it permitted Hamas to use it to perform assaults that cost the lives of four people. Facebook released its blueprints of Surround 360 camera on GitHub under an open-source license. In September, it won an Emmy for its animated short \"Henry\". In October, Facebook announced a fee-based communications tool called Workplace that aims to \"connect everyone\" at work. Users can create profiles, see updates from co-workers on their news feed, stream live videos and participate in secure group chats.\n",
- "0.40: On January 22, 2015, the Microsoft Office blog announced that the next version of the suite for Windows desktop, Office 2016, was in development. On May 4, 2015, a public preview of Microsoft Office 2016 was released. Office 2016 was released for Mac OS X on July 9, 2015 and for Windows on September 22, 2015.\n",
- "0.39: On November 6, 2013, Microsoft announced further new features including \"real-time\" co-authoring and an Auto-Save feature in Word (replacing the save button).\n",
- "0.39: In February 2014, Office Web Apps were re-branded Office Online and incorporated into other Microsoft web services, including Calendar, OneDrive, Outlook.com, and People. Microsoft had previously attempted to unify its online services suite (including Microsoft Passport, Hotmail, MSN Messenger, and later SkyDrive) under a brand known as Windows Live, first launched in 2005. However, with the impending launch of Windows 8 and its increased use of cloud services, Microsoft dropped the Windows Live brand to emphasize that these services would now be built directly into Windows and not merely be a \"bolted on\" add-on. Critics had criticized the Windows Live brand for having no clear vision, as it was being applied to an increasingly broad array of unrelated services. At the same time, Windows Live Hotmail was re-launched as Outlook.com (sharing its name with the Microsoft Outlook personal information manager).\n",
- "0.39: On February 18, 2021, Microsoft announced that the next version of the suite for Windows desktop, Office 2021, was in development. This new version will be supported for five years and was released on October 5, 2021.\n",
- "0.38: Since Office 2013, Microsoft has promoted Office 365 as the primary means of obtaining Microsoft Office: it allows the use of the software and other services on a subscription business model, and users receive feature updates to the software for the lifetime of the subscription, including new features and cloud computing integration that are not necessarily included in the \"on-premises\" releases of Office sold under conventional license terms. In 2017, revenue from Office 365 overtook conventional license sales. Microsoft also rebranded most of their standard Office 365 editions as \"Microsoft 365\" to reflect their inclusion of features and services beyond the core Microsoft Office suite.\n",
- "0.38: Microsoft has since promoted Office 365 as the primary means of purchasing Microsoft Office. Although there are still \"on-premises\" releases roughly every three years, Microsoft marketing emphasizes that they do not receive new features or access to new cloud-based services as they are released unlike Office 365, as well as other benefits for consumer and business markets. Office 365 revenue overtook traditional license sales for Office in 2017.\n",
- "0.38: A technical preview of Microsoft Office 2013 (Build 15.0.3612.1010) was released on January 30, 2012, and a Customer Preview version was made available to consumers on July 16, 2012. It sports a revamped application interface; the interface is based on Metro, the interface of Windows Phone and Windows 8. Microsoft Outlook has received the most pronounced changes so far; for example, the Metro interface provides a new visualization for scheduled tasks. PowerPoint includes more templates and transition effects, and OneNote includes a new splash screen.\n",
- "0.38: On January 21, 2015, during the \"Windows 10: The Next Chapter\" press event, Microsoft unveiled Office for Windows 10, Windows Runtime ports of the Android and iOS versions of the Office Mobile suite. Optimized for smartphones and tablets, they are universal apps that can run on both Windows and Windows for phones, and share similar underlying code. A simplified version of Outlook was also added to the suite. They will be bundled with Windows 10 mobile devices, and available from the Windows Store for the PC version of Windows 10. Although the preview versions were free for most editing, the release versions will require an Office 365 subscription on larger tablets (screen size larger than 10.1 inches) and desktops for editing, as with large Android tablets. Smaller tablets and phones will have most editing features for free.\n",
- "0.38: In May 2018 at F8, the company announced it would offer its own dating service. Shares in competitor Match Group fell by 22%. Facebook Dating includes privacy features and friends are unable to view their friends' dating profile. In July, Facebook was charged £500,000 by UK watchdogs for failing to respond to data erasure requests. On July 18, Facebook established a subsidiary named Lianshu Science & Technology in Hangzhou City, China, with $30 million ($ in dollars) of capital. All its shares are held by Facebook Hong. Approval of the registration of the subsidiary was then withdrawn, due to a disagreement between officials in Zhejiang province and the Cyberspace Administration of China. On July 26, Facebook became the first company to lose over $100 billion ($ in dollars) worth of market capitalization in one day, dropping from nearly $630 billion to $510 billion after disappointing sales reports. On July 31, Facebook said that the company had deleted 17 accounts related to the 2018 U.S. midterm elections. On September 19, Facebook announced that, for news distribution outside the United States, it would work with U.S. funded democracy promotion organizations, International Republican Institute and the National Democratic Institute, which are loosely affiliated with the Republican and Democratic parties. Through the Digital Forensic Research Lab Facebook partners with the Atlantic Council, a NATO-affiliated think tank. In November, Facebook launched smart displays branded Portal and Portal Plus (Portal+). They support Amazon's Alexa (intelligent personal assistant service). The devices include video chat function with Facebook Messenger.\n",
- "0.37: The first Preview version of Microsoft Office 2016 for Mac was released on March 5, 2015. On July 9, 2015, Microsoft released the final version of Microsoft Office 2016 for Mac which includes Word, Excel, PowerPoint, Outlook and OneNote. It was immediately made available for Office 365 subscribers with either a Home, Personal, Business, Business Premium, E3 or ProPlus subscription. A non–Office 365 edition of Office 2016 was made available as a one-time purchase option on September 22, 2015.\n",
- "0.37: In October 2022, Microsoft announced that it will phase out the Microsoft Office brand in favor of \"Microsoft 365\" by January 2023. The name will continue to be used for legacy product offerings.\n"
- ]
- }
- ],
- "source": [
- "# Look at the initial retrieval results\n",
- "for match in res['matches']:\n",
- " print(f\"{match['score']:.2f}: {match['metadata']['text']}\")"
- ]
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "cohere.EmbedJob {\n",
+ "\tjob_id: 6d691fbe-e026-436a-826a-16e70b293e51\n",
+ "\tstatus: complete\n",
+ "\tcreated_at: 2024-01-13T02:47:46.385016Z\n",
+ "\tinput_dataset_id: sample-file-2gwgxq\n",
+ "\toutput_urls: None\n",
+ "\tmodel: embed-english-v3.0\n",
+ "\ttruncate: RIGHT\n",
+ "\tpercent_complete: 100\n",
+ "\toutput: cohere.Dataset {\n",
+ "\tid: embeded-sample-file-mdse2h\n",
+ "\tname: embeded-sample-file\n",
+ "\tdataset_type: embed-result\n",
+ "\tvalidation_status: validated\n",
+ "\tcreated_at: 2024-01-13 02:47:47.850097\n",
+ "\tupdated_at: 2024-01-13 02:47:47.850097\n",
+ "\tdownload_urls: ['https://storage.googleapis.com/cohere-user/dataset-api-temp/d489c39a-e152-49da-9ddc-9801bd74d823/96d12a16-2dd4-46f7-9630-1fa9bb0b26ca/embeded-sample-file-mdse2h/001_embeded-sample-file.avro?X-Goog-Algorithm=GOOG4-RSA-SHA256&X-Goog-Credential=dataset%40cohere-production.iam.gserviceaccount.com%2F20240113%2Fauto%2Fstorage%2Fgoog4_request&X-Goog-Date=20240113T024807Z&X-Goog-Expires=14399&X-Goog-Signature=78b0d82e19388aee2926a4ef403d5f286487d0e7fc9a09c75bfb6700b502fa4f55b024977c0aeae26cd501a41e506bf00d3e850d7d9691298963374f04fa129e8dae5a327389c80921bf611a25386f5e4501b11c9d88bc1aee6f6877157a4675c5f8fe06bb25e3ca2b12da46e5b1da3067ca71bed901cd14db26e09987e355c039f96d6514a0931aa5f8753ddc155ca1782c63e3cb000b095d3b29904982ff75686c716329e92b6946485c567dabc3e344c9a0f9a59416415738b67ead0cca3cdb06c1db64c925ea38a2d92ab4079577e5775367260c09916aab5af67326bca4fa1295ee76457f933a6ca26a5d4ac9c59f0f73286627b2bae3e7fa7375c97465&X-Goog-SignedHeaders=host']\n",
+ "\tvalidation_error: None\n",
+ "\tvalidation_warnings: []\n",
+ "}\n",
+ "}\n"
+ ]
+ }
+ ],
+ "source": [
+ "print(job)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "UlEVKOCG7ZsN"
+ },
+ "source": [
+ "## Step 3: Prepare embeddings for upsert"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "vDvntjsnG_DX"
+ },
+ "outputs": [],
+ "source": [
+ "# Load the output file into an array\n",
+ "output_dataset=co.get_dataset(job.output.id)\n",
+ "data_array = []\n",
+ "for record in output_dataset:\n",
+ " data_array.append(record)\n",
+ "\n",
+ "# Take the output and format it in the shape for upserting into Pinecone's DB\n",
+ "ids = [str(i) for i in range(len(data_array))]\n",
+ "meta = [{'text':str(data_array[i]['text'])} for i in range(len(data_array))]\n",
+ "embeds=[np.float32(data_array[i]['embeddings']['float']) for i in range(len(data_array))]\n",
+ "\n",
+ "to_upsert = list(zip(ids, embeds, meta))"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "pQwXK4xt7lls"
+ },
+ "source": [
+ "## Step 4: Initialize Pinecone vector database"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "GrLN8Y_THEBH"
+ },
+ "outputs": [],
+ "source": [
+ "# Initialize your Pinecone Vector DB\n",
+ "from pinecone import ServerlessSpec\n",
+ "\n",
+ "index_name = \"embed-jobs-serverless-test-example\"\n",
+ "\n",
+ "# A new property 'spec' is used to tell Pinecone how we should deploy your index.\n",
+ "pc.create_index(\n",
+ "name=index_name,\n",
+ "dimension=1024,\n",
+ "metric=\"cosine\",\n",
+ "spec=ServerlessSpec(cloud='aws', region='us-west-2')\n",
+ ")\n",
+ "\n",
+ "# Target your new serverless index.\n",
+ "idx = pc.Index(index_name)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "W2QMVhoM7o-5"
+ },
+ "source": [
+ "## Step 5: Upsert embeddings into the index"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "colab": {
+ "base_uri": "https://localhost:8080/"
},
+ "id": "JaVrXHZ3IjiN",
+ "outputId": "716ad587-2ee1-475a-f1e5-9a0b23d2df87"
+ },
+ "outputs": [
{
- "cell_type": "markdown",
- "metadata": {
- "id": "DyevqtBB8KyU"
- },
- "source": [
- "## Step 7: Rerank the retrieved results"
- ]
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "{'dimension': 1024,\n",
+ " 'index_fullness': 0.0,\n",
+ " 'namespaces': {'': {'vector_count': 3664}},\n",
+ " 'total_vector_count': 3664}\n"
+ ]
+ }
+ ],
+ "source": [
+ "# Upsert your data into the index\n",
+ "batch_size = 128\n",
+ "\n",
+ "for i in range(0, len(data_array), batch_size):\n",
+ " i_end = min(i+batch_size, len(data_array))\n",
+ " idx.upsert(vectors=to_upsert[i:i_end])\n",
+ "\n",
+ "# let's view the index statistics\n",
+ "print(idx.describe_index_stats())"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "AtT1Yo_i8CZc"
+ },
+ "source": [
+ "## Step 6: Query the index"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "colab": {
+ "base_uri": "https://localhost:8080/"
},
+ "id": "t-A82Z1EIrKR",
+ "outputId": "ed708eef-0c0b-4688-f0d7-f5fc39219848"
+ },
+ "outputs": [
{
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "colab": {
- "base_uri": "https://localhost:8080/"
- },
- "id": "Rqo6YdzJI0ny",
- "outputId": "aac1de9a-9cae-4c4c-f4a3-fb87583f62f2"
- },
- "outputs": [
- {
- "name": "stdout",
- "output_type": "stream",
- "text": [
- "0.99: Microsoft Office, or simply Office, is the former name of a family of client software, server software, and services developed by Microsoft. It was first announced by Bill Gates on August 1, 1988, at COMDEX in Las Vegas. Initially a marketing term for an office suite (bundled set of productivity applications), the first version of Office contained Microsoft Word, Microsoft Excel, and Microsoft PowerPoint. Over the years, Office applications have grown substantially closer with shared features such as a common spell checker, Object Linking and Embedding data integration and Visual Basic for Applications scripting language. Microsoft also positions Office as a development platform for line-of-business software under the Office Business Applications brand.\n",
- "0.93: On January 21, 2015, during the \"Windows 10: The Next Chapter\" press event, Microsoft unveiled Office for Windows 10, Windows Runtime ports of the Android and iOS versions of the Office Mobile suite. Optimized for smartphones and tablets, they are universal apps that can run on both Windows and Windows for phones, and share similar underlying code. A simplified version of Outlook was also added to the suite. They will be bundled with Windows 10 mobile devices, and available from the Windows Store for the PC version of Windows 10. Although the preview versions were free for most editing, the release versions will require an Office 365 subscription on larger tablets (screen size larger than 10.1 inches) and desktops for editing, as with large Android tablets. Smaller tablets and phones will have most editing features for free.\n",
- "0.87: In October 2022, Microsoft announced that it will phase out the Microsoft Office brand in favor of \"Microsoft 365\" by January 2023. The name will continue to be used for legacy product offerings.\n"
- ]
- }
- ],
- "source": [
- "# Add Cohere Reranking Step\n",
- "docs =[match['metadata']['text'] for match in res['matches']]\n",
- "\n",
- "rerank_response = co.rerank(\n",
- " model = 'rerank-english-v2.0',\n",
- " query = query,\n",
- " documents = docs,\n",
- " top_n = 3,\n",
- ")\n",
- "for response in rerank_response:\n",
- " print(f\"{response.relevance_score:.2f}: {response.document['text']}\")"
- ]
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "(1, 1024)\n"
+ ]
+ }
+ ],
+ "source": [
+ "# Let's query the database\n",
+ "query = \"What did Microsoft announce in Las Vegas?\"\n",
+ "\n",
+ "# create the query embedding\n",
+ "xq = co.embed(\n",
+ " texts=[query],\n",
+ " model='embed-english-v3.0',\n",
+ " input_type='search_query',\n",
+ " truncate='END'\n",
+ ").embeddings\n",
+ "\n",
+ "print(np.array(xq).shape)\n",
+ "\n",
+ "# query, returning the top 20 most similar results\n",
+ "res = idx.query(xq, top_k=20, include_metadata=True)"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "colab": {
+ "base_uri": "https://localhost:8080/"
},
+ "id": "pXhvxHwj5nX2",
+ "outputId": "422948e1-a554-4bd2-9d28-076df6f5c3ee"
+ },
+ "outputs": [
{
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "## Another example - query and rerank"
- ]
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "0.48: On October 22, 2012, Microsoft announced the release of new features including co-authoring, performance improvements and touch support.\n",
+ "0.45: On May 2, 2019, at F8, the company announced its new vision with the tagline \"the future is private\". A redesign of the website and mobile app was introduced, dubbed as \"FB5\". The event also featured plans for improving groups, a dating platform, end-to-end encryption on its platforms, and allowing users on Messenger to communicate directly with WhatsApp and Instagram users.\n",
+ "0.42: On July 13, 2009, Microsoft announced at its Worldwide Partners Conference 2009 in New Orleans that Microsoft Office 2010 reached its \"Technical Preview\" development milestone and features of Office Web Apps were demonstrated to the public for the first time. Additionally, Microsoft announced that Office Web Apps would be made available to consumers online and free of charge, while Microsoft Software Assurance customers will have the option of running them on premises. Office 2010 beta testers were not given access to Office Web Apps at this date, and it was announced that it would be available for testers during August 2009. However, in August 2009, a Microsoft spokesperson stated that there had been a delay in the release of Office Web Apps Technical Preview and it would not be available by the end of August.\n",
+ "0.42: On January 17, 2017, Facebook COO Sheryl Sandberg planned to open Station F, a startup incubator campus in Paris, France. On a six-month cycle, Facebook committed to work with ten to 15 data-driven startups there. On April 18, Facebook announced the beta launch of at its annual F8 developer conference. Facebook Spaces is a virtual reality version of Facebook for Oculus VR goggles. In a virtual and shared space, users can access a curated selection of 360-degree photos and videos using their avatar, with the support of the controller. Users can access their own photos and videos, along with media shared on their newsfeed. In September, Facebook announced it would spend up to US$1 billion on original shows for its Facebook Watch platform. On October 16, it acquired the anonymous compliment app tbh, announcing its intention to leave the app independent.\n",
+ "0.41: On September 26, 2017, Microsoft announced that the next version of the suite for Windows desktop, Office 2019, was in development. On April 27, 2018, Microsoft released Office 2019 Commercial Preview for Windows 10. It was released to general availability for Windows 10 and for macOS on September 24, 2018.\n",
+ "0.41: Microsoft Office, or simply Office, is the former name of a family of client software, server software, and services developed by Microsoft. It was first announced by Bill Gates on August 1, 1988, at COMDEX in Las Vegas. Initially a marketing term for an office suite (bundled set of productivity applications), the first version of Office contained Microsoft Word, Microsoft Excel, and Microsoft PowerPoint. Over the years, Office applications have grown substantially closer with shared features such as a common spell checker, Object Linking and Embedding data integration and Visual Basic for Applications scripting language. Microsoft also positions Office as a development platform for line-of-business software under the Office Business Applications brand.\n",
+ "0.40: On August 12, 2009, it was announced that Office Mobile would also be released for the Symbian platform as a joint agreement between Microsoft and Nokia. It was the first time Microsoft would develop Office mobile applications for another smartphone platform. The first application to appear on Nokia Eseries smartphones was Microsoft Office Communicator. In February 2012, Microsoft released OneNote, Lync 2010, Document Connection and PowerPoint Broadcast for Symbian. In April, Word Mobile, PowerPoint Mobile and Excel Mobile joined the Office Suite.\n",
+ "0.40: In 2010, Microsoft introduced a software as a service platform known as Office 365, to provide cloud-hosted versions of Office's server software, including Exchange e-mail and SharePoint, on a subscription basis (competing in particular with Google Apps). Following the release of Office 2013, Microsoft began to offer Office 365 plans for the consumer market, with access to Microsoft Office software on multiple devices with free feature updates over the life of the subscription, as well as other services such as OneDrive storage.\n",
+ "0.40: On April 12, 2016, Zuckerberg outlined his 10-year vision, which rested on three main pillars: artificial intelligence, increased global connectivity, and virtual and augmented reality. In July, a suit was filed against the company alleging that it permitted Hamas to use it to perform assaults that cost the lives of four people. Facebook released its blueprints of Surround 360 camera on GitHub under an open-source license. In September, it won an Emmy for its animated short \"Henry\". In October, Facebook announced a fee-based communications tool called Workplace that aims to \"connect everyone\" at work. Users can create profiles, see updates from co-workers on their news feed, stream live videos and participate in secure group chats.\n",
+ "0.40: On January 22, 2015, the Microsoft Office blog announced that the next version of the suite for Windows desktop, Office 2016, was in development. On May 4, 2015, a public preview of Microsoft Office 2016 was released. Office 2016 was released for Mac OS X on July 9, 2015 and for Windows on September 22, 2015.\n",
+ "0.39: On November 6, 2013, Microsoft announced further new features including \"real-time\" co-authoring and an Auto-Save feature in Word (replacing the save button).\n",
+ "0.39: In February 2014, Office Web Apps were re-branded Office Online and incorporated into other Microsoft web services, including Calendar, OneDrive, Outlook.com, and People. Microsoft had previously attempted to unify its online services suite (including Microsoft Passport, Hotmail, MSN Messenger, and later SkyDrive) under a brand known as Windows Live, first launched in 2005. However, with the impending launch of Windows 8 and its increased use of cloud services, Microsoft dropped the Windows Live brand to emphasize that these services would now be built directly into Windows and not merely be a \"bolted on\" add-on. Critics had criticized the Windows Live brand for having no clear vision, as it was being applied to an increasingly broad array of unrelated services. At the same time, Windows Live Hotmail was re-launched as Outlook.com (sharing its name with the Microsoft Outlook personal information manager).\n",
+ "0.39: On February 18, 2021, Microsoft announced that the next version of the suite for Windows desktop, Office 2021, was in development. This new version will be supported for five years and was released on October 5, 2021.\n",
+ "0.38: Since Office 2013, Microsoft has promoted Office 365 as the primary means of obtaining Microsoft Office: it allows the use of the software and other services on a subscription business model, and users receive feature updates to the software for the lifetime of the subscription, including new features and cloud computing integration that are not necessarily included in the \"on-premises\" releases of Office sold under conventional license terms. In 2017, revenue from Office 365 overtook conventional license sales. Microsoft also rebranded most of their standard Office 365 editions as \"Microsoft 365\" to reflect their inclusion of features and services beyond the core Microsoft Office suite.\n",
+ "0.38: Microsoft has since promoted Office 365 as the primary means of purchasing Microsoft Office. Although there are still \"on-premises\" releases roughly every three years, Microsoft marketing emphasizes that they do not receive new features or access to new cloud-based services as they are released unlike Office 365, as well as other benefits for consumer and business markets. Office 365 revenue overtook traditional license sales for Office in 2017.\n",
+ "0.38: A technical preview of Microsoft Office 2013 (Build 15.0.3612.1010) was released on January 30, 2012, and a Customer Preview version was made available to consumers on July 16, 2012. It sports a revamped application interface; the interface is based on Metro, the interface of Windows Phone and Windows 8. Microsoft Outlook has received the most pronounced changes so far; for example, the Metro interface provides a new visualization for scheduled tasks. PowerPoint includes more templates and transition effects, and OneNote includes a new splash screen.\n",
+ "0.38: On January 21, 2015, during the \"Windows 10: The Next Chapter\" press event, Microsoft unveiled Office for Windows 10, Windows Runtime ports of the Android and iOS versions of the Office Mobile suite. Optimized for smartphones and tablets, they are universal apps that can run on both Windows and Windows for phones, and share similar underlying code. A simplified version of Outlook was also added to the suite. They will be bundled with Windows 10 mobile devices, and available from the Windows Store for the PC version of Windows 10. Although the preview versions were free for most editing, the release versions will require an Office 365 subscription on larger tablets (screen size larger than 10.1 inches) and desktops for editing, as with large Android tablets. Smaller tablets and phones will have most editing features for free.\n",
+ "0.38: In May 2018 at F8, the company announced it would offer its own dating service. Shares in competitor Match Group fell by 22%. Facebook Dating includes privacy features and friends are unable to view their friends' dating profile. In July, Facebook was charged £500,000 by UK watchdogs for failing to respond to data erasure requests. On July 18, Facebook established a subsidiary named Lianshu Science & Technology in Hangzhou City, China, with $30 million ($ in dollars) of capital. All its shares are held by Facebook Hong. Approval of the registration of the subsidiary was then withdrawn, due to a disagreement between officials in Zhejiang province and the Cyberspace Administration of China. On July 26, Facebook became the first company to lose over $100 billion ($ in dollars) worth of market capitalization in one day, dropping from nearly $630 billion to $510 billion after disappointing sales reports. On July 31, Facebook said that the company had deleted 17 accounts related to the 2018 U.S. midterm elections. On September 19, Facebook announced that, for news distribution outside the United States, it would work with U.S. funded democracy promotion organizations, International Republican Institute and the National Democratic Institute, which are loosely affiliated with the Republican and Democratic parties. Through the Digital Forensic Research Lab Facebook partners with the Atlantic Council, a NATO-affiliated think tank. In November, Facebook launched smart displays branded Portal and Portal Plus (Portal+). They support Amazon's Alexa (intelligent personal assistant service). The devices include video chat function with Facebook Messenger.\n",
+ "0.37: The first Preview version of Microsoft Office 2016 for Mac was released on March 5, 2015. On July 9, 2015, Microsoft released the final version of Microsoft Office 2016 for Mac which includes Word, Excel, PowerPoint, Outlook and OneNote. It was immediately made available for Office 365 subscribers with either a Home, Personal, Business, Business Premium, E3 or ProPlus subscription. A non–Office 365 edition of Office 2016 was made available as a one-time purchase option on September 22, 2015.\n",
+ "0.37: In October 2022, Microsoft announced that it will phase out the Microsoft Office brand in favor of \"Microsoft 365\" by January 2023. The name will continue to be used for legacy product offerings.\n"
+ ]
+ }
+ ],
+ "source": [
+ "# Look at the initial retrieval results\n",
+ "for match in res['matches']:\n",
+ " print(f\"{match['score']:.2f}: {match['metadata']['text']}\")"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "DyevqtBB8KyU"
+ },
+ "source": [
+ "## Step 7: Rerank the retrieved results"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "colab": {
+ "base_uri": "https://localhost:8080/"
},
+ "id": "Rqo6YdzJI0ny",
+ "outputId": "aac1de9a-9cae-4c4c-f4a3-fb87583f62f2"
+ },
+ "outputs": [
{
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "colab": {
- "base_uri": "https://localhost:8080/"
- },
- "id": "2laQ-A5x9HY8",
- "outputId": "c3d831fd-5df7-412e-8637-9c51c51c0846"
- },
- "outputs": [
- {
- "name": "stdout",
- "output_type": "stream",
- "text": [
- "(1, 1024)\n",
- "0.66: YouTube began as a venture capital–funded technology startup. Between November 2005 and April 2006, the company raised money from various investors, with Sequoia Capital, $11.5 million, and Artis Capital Management, $8 million, being the largest two. YouTube's early headquarters were situated above a pizzeria and a Japanese restaurant in San Mateo, California. In February 2005, the company activated codice_1. The first video was uploaded April 23, 2005. Titled \"Me at the zoo\", it shows co-founder Jawed Karim at the San Diego Zoo and can still be viewed on the site. In May, the company launched a public beta and by November, a Nike ad featuring Ronaldinho became the first video to reach one million total views. The site launched officially on December 15, 2005, by which time the site was receiving 8 million views a day. Clips at the time were limited to 100 megabytes, as little as 30 seconds of footage.\n",
- "0.58: Karim said the inspiration for YouTube first came from the Super Bowl XXXVIII halftime show controversy when Janet Jackson's breast was briefly exposed by Justin Timberlake during the halftime show. Karim could not easily find video clips of the incident and the 2004 Indian Ocean Tsunami online, which led to the idea of a video-sharing site. Hurley and Chen said that the original idea for YouTube was a video version of an online dating service, and had been influenced by the website Hot or Not. They created posts on Craigslist asking attractive women to upload videos of themselves to YouTube in exchange for a $100 reward. Difficulty in finding enough dating videos led to a change of plans, with the site's founders deciding to accept uploads of any video.\n",
- "0.55: YouTube was not the first video-sharing site on the Internet; Vimeo was launched in November 2004, though that site remained a side project of its developers from CollegeHumor at the time and did not grow much, either. The week of YouTube's launch, NBC-Universal's \"Saturday Night Live\" ran a skit \"Lazy Sunday\" by The Lonely Island. Besides helping to bolster ratings and long-term viewership for \"Saturday Night Live\", \"Lazy Sunday\"'s status as an early viral video helped establish YouTube as an important website. Unofficial uploads of the skit to YouTube drew in more than five million collective views by February 2006 before they were removed when NBCUniversal requested it two months later based on copyright concerns. Despite eventually being taken down, these duplicate uploads of the skit helped popularize YouTube's reach and led to the upload of more third-party content. The site grew rapidly; in July 2006, the company announced that more than 65,000 new videos were being uploaded every day and that the site was receiving 100 million video views per day.\n",
- "0.55: According to a story that has often been repeated in the media, Hurley and Chen developed the idea for YouTube during the early months of 2005, after they had experienced difficulty sharing videos that had been shot at a dinner party at Chen's apartment in San Francisco. Karim did not attend the party and denied that it had occurred, but Chen remarked that the idea that YouTube was founded after a dinner party \"was probably very strengthened by marketing ideas around creating a story that was very digestible\".\n",
- "0.53: In December 2009, YouTube partnered with Vevo. In April 2010, Lady Gaga's \"Bad Romance\" became the most viewed video, becoming the first video to reach 200 million views on May 9, 2010.\n",
- "0.53: YouTube is a global online video sharing and social media platform headquartered in San Bruno, California. It was launched on February 14, 2005, by Steve Chen, Chad Hurley, and Jawed Karim. It is owned by Google, and is the second most visited website, after Google Search. YouTube has more than 2.5 billion monthly users who collectively watch more than one billion hours of videos each day. , videos were being uploaded at a rate of more than 500 hours of content per minute.\n",
- "0.53: YouTube has faced numerous challenges and criticisms in its attempts to deal with copyright, including the site's first viral video, Lazy Sunday, which had to be taken down, due to copyright concerns. At the time of uploading a video, YouTube users are shown a message asking them not to violate copyright laws. Despite this advice, many unauthorized clips of copyrighted material remain on YouTube. YouTube does not view videos before they are posted online, and it is left to copyright holders to issue a DMCA takedown notice pursuant to the terms of the Online Copyright Infringement Liability Limitation Act. Any successful complaint about copyright infringement results in a YouTube copyright strike. Three successful complaints for copyright infringement against a user account will result in the account and all of its uploaded videos being deleted. From 2007 to 2009 organizations including Viacom, Mediaset, and the English Premier League have filed lawsuits against YouTube, claiming that it has done too little to prevent the uploading of copyrighted material.\n",
- "0.51: Some YouTube videos have themselves had a direct effect on world events, such as \"Innocence of Muslims\" (2012) which spurred protests and related anti-American violence internationally. TED curator Chris Anderson described a phenomenon by which geographically distributed individuals in a certain field share their independently developed skills in YouTube videos, thus challenging others to improve their own skills, and spurring invention and evolution in that field. Journalist Virginia Heffernan stated in \"The New York Times\" that such videos have \"surprising implications\" for the dissemination of culture and even the future of classical music.\n",
- "0.50: Observing that face-to-face communication of the type that online videos convey has been \"fine-tuned by millions of years of evolution,\" TED curator Chris Anderson referred to several YouTube contributors and asserted that \"what Gutenberg did for writing, online video can now do for face-to-face communication.\" Anderson asserted that it is not far-fetched to say that online video will dramatically accelerate scientific advance, and that video contributors may be about to launch \"the biggest learning cycle in human history.\" In education, for example, the Khan Academy grew from YouTube video tutoring sessions for founder Salman Khan's cousin into what \"Forbes\" Michael Noer called \"the largest school in the world,\" with technology poised to disrupt how people learn. YouTube was awarded a 2008 George Foster Peabody Award, the website being described as a Speakers' Corner that \"both embodies and promotes democracy.\" \"The Washington Post\" reported that a disproportionate share of YouTube's most subscribed channels feature minorities, contrasting with mainstream television in which the stars are largely white. A Pew Research Center study reported the development of \"visual journalism,\" in which citizen eyewitnesses and established news organizations share in content creation. The study also concluded that YouTube was becoming an important platform by which people acquire news.\n",
- "0.50: YouTube was founded by Steve Chen, Chad Hurley, and Jawed Karim. The trio were early employees of PayPal, which left them enriched after the company was bought by eBay. Hurley had studied design at the Indiana University of Pennsylvania, and Chen and Karim studied computer science together at the University of Illinois Urbana-Champaign.\n",
- "0.49: In 2013, YouTube teamed up with satirical newspaper company \"The Onion\" to claim in an uploaded video that the video-sharing website was launched as a contest which had finally come to an end, and would shut down for ten years before being re-launched in 2023, featuring only the winning video. The video starred several YouTube celebrities, including Antoine Dodson. A video of two presenters announcing the nominated videos streamed live for 12 hours.\n",
- "0.48: Since its purchase by Google, YouTube has expanded beyond the core website into mobile apps, network television, and the ability to link with other platforms. Video categories on YouTube include music videos, video clips, news, short films, feature films, documentaries, audio recordings, movie trailers, teasers, live streams, vlogs, and more. Most content is generated by individuals, including collaborations between YouTubers and corporate sponsors. Established media corporations such as Disney, Paramount, and Warner Bros. Discovery have also created and expanded their corporate YouTube channels to advertise to a larger audience.\n",
- "0.47: YouTube has enabled people to more directly engage with government, such as in the CNN/YouTube presidential debates (2007) in which ordinary people submitted questions to U.S. presidential candidates via YouTube video, with a techPresident co-founder saying that Internet video was changing the political landscape. Describing the Arab Spring (2010–2012), sociologist Philip N. Howard quoted an activist's succinct description that organizing the political unrest involved using \"Facebook to schedule the protests, Twitter to coordinate, and YouTube to tell the world.\" In 2012, more than a third of the U.S. Senate introduced a resolution condemning Joseph Kony 16 days after the \"Kony 2012\" video was posted to YouTube, with resolution co-sponsor Senator Lindsey Graham remarking that the video \"will do more to lead to (Kony's) demise than all other action combined.\"\n",
- "0.47: YouTube carried out early experiments with live streaming, including a concert by U2 in 2009, and a question-and-answer session with US President Barack Obama in February 2010. These tests had relied on technology from 3rd-party partners, but in September 2010, YouTube began testing its own live streaming infrastructure. In April 2011, YouTube announced the rollout of \"YouTube Live\". The creation of live streams was initially limited to select partners. It was used for real-time broadcasting of events such as the 2012 Olympics in London. In October 2012, more than 8 million people watched Felix Baumgartner's jump from the edge of space as a live stream on YouTube.\n",
- "0.46: In June 2007, YouTube began trials of a system for automatic detection of uploaded videos that infringe copyright. Google CEO Eric Schmidt regarded this system as necessary for resolving lawsuits such as the one from Viacom, which alleged that YouTube profited from content that it did not have the right to distribute. The system, which was initially called \"Video Identification\" and later became known as Content ID, creates an ID File for copyrighted audio and video material, and stores it in a database. When a video is uploaded, it is checked against the database, and flags the video as a copyright violation if a match is found. When this occurs, the content owner has the choice of blocking the video to make it unviewable, tracking the viewing statistics of the video, or adding advertisements to the video.\n",
- "0.46: In January 2009, YouTube launched \"YouTube for TV\", a version of the website tailored for set-top boxes and other TV-based media devices with web browsers, initially allowing its videos to be viewed on the PlayStation 3 and Wii video game consoles.\n",
- "0.46: In September 2012, YouTube launched its first app for the iPhone, following the decision to drop YouTube as one of the preloaded apps in the iPhone 5 and iOS 6 operating system. According to GlobalWebIndex, YouTube was used by 35% of smartphone users between April and June 2013, making it the third-most used app.\n",
- "0.46: Conversely, YouTube has also allowed government to more easily engage with citizens, the White House's official YouTube channel being the seventh top news organization producer on YouTube in 2012 and in 2013 a healthcare exchange commissioned Obama impersonator Iman Crosson's YouTube music video spoof to encourage young Americans to enroll in the Affordable Care Act (Obamacare)-compliant health insurance. In February 2014, U.S. President Obama held a meeting at the White House with leading YouTube content creators to not only promote awareness of Obamacare but more generally to develop ways for government to better connect with the \"YouTube Generation.\" Whereas YouTube's inherent ability to allow presidents to directly connect with average citizens was noted, the YouTube content creators' new media savvy was perceived necessary to better cope with the website's distracting content and fickle audience.\n",
- "0.46: Later that year, YouTube came under criticism for showing inappropriate videos targeted at children and often featuring popular characters in violent, sexual or otherwise disturbing situations, many of which appeared on YouTube Kids and attracted millions of views. The term \"Elsagate\" was coined on the Internet and then used by various news outlets to refer to this controversy. On November 11, 2017, YouTube announced it was strengthening site security to protect children from unsuitable content. Later that month, the company started to mass delete videos and channels that made improper use of family-friendly characters. As part of a broader concern regarding child safety on YouTube, the wave of deletions also targeted channels that showed children taking part in inappropriate or dangerous activities under the guidance of adults. Most notably, the company removed \"Toy Freaks\", a channel with over 8.5 million subscribers, that featured a father and his two daughters in odd and upsetting situations. According to analytics specialist SocialBlade, it earned up to £8.7 million annually prior to its deletion.\n",
- "0.45: In September 2020, YouTube announced that it would be launching a beta version of a new platform of 15-second videos, similar to TikTok, called YouTube Shorts. The platform was first tested in India but as of March 2021 has expanded to other countries including the United States with videos now able to be up to 1 minute long. The platform is not a standalone app, but is integrated into the main YouTube app. Like TikTok, it gives users access to built-in creative tools, including the possibility of adding licensed music to their videos. The platform had its global beta launch in July 2021.\n"
- ]
- }
- ],
- "source": [
- "# Let's query the database\n",
- "query = \"What was the first youtube video about?\"\n",
- "\n",
- "# create the query embedding\n",
- "xq = co.embed(\n",
- " texts=[query],\n",
- " model='embed-english-v3.0',\n",
- " input_type='search_query',\n",
- " truncate='END'\n",
- ").embeddings\n",
- "\n",
- "print(np.array(xq).shape)\n",
- "\n",
- "# query, returning the top 20 most similar results\n",
- "res = idx.query(xq, top_k=20, include_metadata=True)\n",
- "\n",
- "# Look at the initial retrieval results\n",
- "for match in res['matches']:\n",
- " print(f\"{match['score']:.2f}: {match['metadata']['text']}\")"
- ]
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "0.99: Microsoft Office, or simply Office, is the former name of a family of client software, server software, and services developed by Microsoft. It was first announced by Bill Gates on August 1, 1988, at COMDEX in Las Vegas. Initially a marketing term for an office suite (bundled set of productivity applications), the first version of Office contained Microsoft Word, Microsoft Excel, and Microsoft PowerPoint. Over the years, Office applications have grown substantially closer with shared features such as a common spell checker, Object Linking and Embedding data integration and Visual Basic for Applications scripting language. Microsoft also positions Office as a development platform for line-of-business software under the Office Business Applications brand.\n",
+ "0.93: On January 21, 2015, during the \"Windows 10: The Next Chapter\" press event, Microsoft unveiled Office for Windows 10, Windows Runtime ports of the Android and iOS versions of the Office Mobile suite. Optimized for smartphones and tablets, they are universal apps that can run on both Windows and Windows for phones, and share similar underlying code. A simplified version of Outlook was also added to the suite. They will be bundled with Windows 10 mobile devices, and available from the Windows Store for the PC version of Windows 10. Although the preview versions were free for most editing, the release versions will require an Office 365 subscription on larger tablets (screen size larger than 10.1 inches) and desktops for editing, as with large Android tablets. Smaller tablets and phones will have most editing features for free.\n",
+ "0.87: In October 2022, Microsoft announced that it will phase out the Microsoft Office brand in favor of \"Microsoft 365\" by January 2023. The name will continue to be used for legacy product offerings.\n"
+ ]
+ }
+ ],
+ "source": [
+ "# Add Cohere Reranking Step\n",
+ "docs =[match['metadata']['text'] for match in res['matches']]\n",
+ "\n",
+ "rerank_response = co.rerank(\n",
+ " model = 'rerank-english-v2.0',\n",
+ " query = query,\n",
+ " documents = docs,\n",
+ " top_n = 3,\n",
+ ")\n",
+ "for response in rerank_response:\n",
+ " print(f\"{response.relevance_score:.2f}: {response.document['text']}\")"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "## Another example - query and rerank"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "colab": {
+ "base_uri": "https://localhost:8080/"
},
+ "id": "2laQ-A5x9HY8",
+ "outputId": "c3d831fd-5df7-412e-8637-9c51c51c0846"
+ },
+ "outputs": [
{
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "colab": {
- "base_uri": "https://localhost:8080/"
- },
- "id": "WaN6U6u19Md6",
- "outputId": "9984d8ce-1383-4530-ec84-6e3f3feea05e"
- },
- "outputs": [
- {
- "name": "stdout",
- "output_type": "stream",
- "text": [
- "0.95: YouTube began as a venture capital–funded technology startup. Between November 2005 and April 2006, the company raised money from various investors, with Sequoia Capital, $11.5 million, and Artis Capital Management, $8 million, being the largest two. YouTube's early headquarters were situated above a pizzeria and a Japanese restaurant in San Mateo, California. In February 2005, the company activated codice_1. The first video was uploaded April 23, 2005. Titled \"Me at the zoo\", it shows co-founder Jawed Karim at the San Diego Zoo and can still be viewed on the site. In May, the company launched a public beta and by November, a Nike ad featuring Ronaldinho became the first video to reach one million total views. The site launched officially on December 15, 2005, by which time the site was receiving 8 million views a day. Clips at the time were limited to 100 megabytes, as little as 30 seconds of footage.\n",
- "0.92: Karim said the inspiration for YouTube first came from the Super Bowl XXXVIII halftime show controversy when Janet Jackson's breast was briefly exposed by Justin Timberlake during the halftime show. Karim could not easily find video clips of the incident and the 2004 Indian Ocean Tsunami online, which led to the idea of a video-sharing site. Hurley and Chen said that the original idea for YouTube was a video version of an online dating service, and had been influenced by the website Hot or Not. They created posts on Craigslist asking attractive women to upload videos of themselves to YouTube in exchange for a $100 reward. Difficulty in finding enough dating videos led to a change of plans, with the site's founders deciding to accept uploads of any video.\n",
- "0.91: YouTube was not the first video-sharing site on the Internet; Vimeo was launched in November 2004, though that site remained a side project of its developers from CollegeHumor at the time and did not grow much, either. The week of YouTube's launch, NBC-Universal's \"Saturday Night Live\" ran a skit \"Lazy Sunday\" by The Lonely Island. Besides helping to bolster ratings and long-term viewership for \"Saturday Night Live\", \"Lazy Sunday\"'s status as an early viral video helped establish YouTube as an important website. Unofficial uploads of the skit to YouTube drew in more than five million collective views by February 2006 before they were removed when NBCUniversal requested it two months later based on copyright concerns. Despite eventually being taken down, these duplicate uploads of the skit helped popularize YouTube's reach and led to the upload of more third-party content. The site grew rapidly; in July 2006, the company announced that more than 65,000 new videos were being uploaded every day and that the site was receiving 100 million video views per day.\n"
- ]
- }
- ],
- "source": [
- "# Add Cohere Reranking Step\n",
- "# embeds=[np.float32(data_array[i]['embedding']) for i in range(len(data_array))]\n",
- "docs =[match['metadata']['text'] for match in res['matches']]\n",
- "\n",
- "rerank_response = co.rerank(\n",
- " model = 'rerank-english-v2.0',\n",
- " query = query,\n",
- " documents = docs,\n",
- " top_n = 3,\n",
- ")\n",
- "for response in rerank_response:\n",
- " print(f\"{response.relevance_score:.2f}: {response.document['text']}\")"
- ]
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "(1, 1024)\n",
+ "0.66: YouTube began as a venture capital–funded technology startup. Between November 2005 and April 2006, the company raised money from various investors, with Sequoia Capital, $11.5 million, and Artis Capital Management, $8 million, being the largest two. YouTube's early headquarters were situated above a pizzeria and a Japanese restaurant in San Mateo, California. In February 2005, the company activated codice_1. The first video was uploaded April 23, 2005. Titled \"Me at the zoo\", it shows co-founder Jawed Karim at the San Diego Zoo and can still be viewed on the site. In May, the company launched a public beta and by November, a Nike ad featuring Ronaldinho became the first video to reach one million total views. The site launched officially on December 15, 2005, by which time the site was receiving 8 million views a day. Clips at the time were limited to 100 megabytes, as little as 30 seconds of footage.\n",
+ "0.58: Karim said the inspiration for YouTube first came from the Super Bowl XXXVIII halftime show controversy when Janet Jackson's breast was briefly exposed by Justin Timberlake during the halftime show. Karim could not easily find video clips of the incident and the 2004 Indian Ocean Tsunami online, which led to the idea of a video-sharing site. Hurley and Chen said that the original idea for YouTube was a video version of an online dating service, and had been influenced by the website Hot or Not. They created posts on Craigslist asking attractive women to upload videos of themselves to YouTube in exchange for a $100 reward. Difficulty in finding enough dating videos led to a change of plans, with the site's founders deciding to accept uploads of any video.\n",
+ "0.55: YouTube was not the first video-sharing site on the Internet; Vimeo was launched in November 2004, though that site remained a side project of its developers from CollegeHumor at the time and did not grow much, either. The week of YouTube's launch, NBC-Universal's \"Saturday Night Live\" ran a skit \"Lazy Sunday\" by The Lonely Island. Besides helping to bolster ratings and long-term viewership for \"Saturday Night Live\", \"Lazy Sunday\"'s status as an early viral video helped establish YouTube as an important website. Unofficial uploads of the skit to YouTube drew in more than five million collective views by February 2006 before they were removed when NBCUniversal requested it two months later based on copyright concerns. Despite eventually being taken down, these duplicate uploads of the skit helped popularize YouTube's reach and led to the upload of more third-party content. The site grew rapidly; in July 2006, the company announced that more than 65,000 new videos were being uploaded every day and that the site was receiving 100 million video views per day.\n",
+ "0.55: According to a story that has often been repeated in the media, Hurley and Chen developed the idea for YouTube during the early months of 2005, after they had experienced difficulty sharing videos that had been shot at a dinner party at Chen's apartment in San Francisco. Karim did not attend the party and denied that it had occurred, but Chen remarked that the idea that YouTube was founded after a dinner party \"was probably very strengthened by marketing ideas around creating a story that was very digestible\".\n",
+ "0.53: In December 2009, YouTube partnered with Vevo. In April 2010, Lady Gaga's \"Bad Romance\" became the most viewed video, becoming the first video to reach 200 million views on May 9, 2010.\n",
+ "0.53: YouTube is a global online video sharing and social media platform headquartered in San Bruno, California. It was launched on February 14, 2005, by Steve Chen, Chad Hurley, and Jawed Karim. It is owned by Google, and is the second most visited website, after Google Search. YouTube has more than 2.5 billion monthly users who collectively watch more than one billion hours of videos each day. , videos were being uploaded at a rate of more than 500 hours of content per minute.\n",
+ "0.53: YouTube has faced numerous challenges and criticisms in its attempts to deal with copyright, including the site's first viral video, Lazy Sunday, which had to be taken down, due to copyright concerns. At the time of uploading a video, YouTube users are shown a message asking them not to violate copyright laws. Despite this advice, many unauthorized clips of copyrighted material remain on YouTube. YouTube does not view videos before they are posted online, and it is left to copyright holders to issue a DMCA takedown notice pursuant to the terms of the Online Copyright Infringement Liability Limitation Act. Any successful complaint about copyright infringement results in a YouTube copyright strike. Three successful complaints for copyright infringement against a user account will result in the account and all of its uploaded videos being deleted. From 2007 to 2009 organizations including Viacom, Mediaset, and the English Premier League have filed lawsuits against YouTube, claiming that it has done too little to prevent the uploading of copyrighted material.\n",
+ "0.51: Some YouTube videos have themselves had a direct effect on world events, such as \"Innocence of Muslims\" (2012) which spurred protests and related anti-American violence internationally. TED curator Chris Anderson described a phenomenon by which geographically distributed individuals in a certain field share their independently developed skills in YouTube videos, thus challenging others to improve their own skills, and spurring invention and evolution in that field. Journalist Virginia Heffernan stated in \"The New York Times\" that such videos have \"surprising implications\" for the dissemination of culture and even the future of classical music.\n",
+ "0.50: Observing that face-to-face communication of the type that online videos convey has been \"fine-tuned by millions of years of evolution,\" TED curator Chris Anderson referred to several YouTube contributors and asserted that \"what Gutenberg did for writing, online video can now do for face-to-face communication.\" Anderson asserted that it is not far-fetched to say that online video will dramatically accelerate scientific advance, and that video contributors may be about to launch \"the biggest learning cycle in human history.\" In education, for example, the Khan Academy grew from YouTube video tutoring sessions for founder Salman Khan's cousin into what \"Forbes\" Michael Noer called \"the largest school in the world,\" with technology poised to disrupt how people learn. YouTube was awarded a 2008 George Foster Peabody Award, the website being described as a Speakers' Corner that \"both embodies and promotes democracy.\" \"The Washington Post\" reported that a disproportionate share of YouTube's most subscribed channels feature minorities, contrasting with mainstream television in which the stars are largely white. A Pew Research Center study reported the development of \"visual journalism,\" in which citizen eyewitnesses and established news organizations share in content creation. The study also concluded that YouTube was becoming an important platform by which people acquire news.\n",
+ "0.50: YouTube was founded by Steve Chen, Chad Hurley, and Jawed Karim. The trio were early employees of PayPal, which left them enriched after the company was bought by eBay. Hurley had studied design at the Indiana University of Pennsylvania, and Chen and Karim studied computer science together at the University of Illinois Urbana-Champaign.\n",
+ "0.49: In 2013, YouTube teamed up with satirical newspaper company \"The Onion\" to claim in an uploaded video that the video-sharing website was launched as a contest which had finally come to an end, and would shut down for ten years before being re-launched in 2023, featuring only the winning video. The video starred several YouTube celebrities, including Antoine Dodson. A video of two presenters announcing the nominated videos streamed live for 12 hours.\n",
+ "0.48: Since its purchase by Google, YouTube has expanded beyond the core website into mobile apps, network television, and the ability to link with other platforms. Video categories on YouTube include music videos, video clips, news, short films, feature films, documentaries, audio recordings, movie trailers, teasers, live streams, vlogs, and more. Most content is generated by individuals, including collaborations between YouTubers and corporate sponsors. Established media corporations such as Disney, Paramount, and Warner Bros. Discovery have also created and expanded their corporate YouTube channels to advertise to a larger audience.\n",
+ "0.47: YouTube has enabled people to more directly engage with government, such as in the CNN/YouTube presidential debates (2007) in which ordinary people submitted questions to U.S. presidential candidates via YouTube video, with a techPresident co-founder saying that Internet video was changing the political landscape. Describing the Arab Spring (2010–2012), sociologist Philip N. Howard quoted an activist's succinct description that organizing the political unrest involved using \"Facebook to schedule the protests, Twitter to coordinate, and YouTube to tell the world.\" In 2012, more than a third of the U.S. Senate introduced a resolution condemning Joseph Kony 16 days after the \"Kony 2012\" video was posted to YouTube, with resolution co-sponsor Senator Lindsey Graham remarking that the video \"will do more to lead to (Kony's) demise than all other action combined.\"\n",
+ "0.47: YouTube carried out early experiments with live streaming, including a concert by U2 in 2009, and a question-and-answer session with US President Barack Obama in February 2010. These tests had relied on technology from 3rd-party partners, but in September 2010, YouTube began testing its own live streaming infrastructure. In April 2011, YouTube announced the rollout of \"YouTube Live\". The creation of live streams was initially limited to select partners. It was used for real-time broadcasting of events such as the 2012 Olympics in London. In October 2012, more than 8 million people watched Felix Baumgartner's jump from the edge of space as a live stream on YouTube.\n",
+ "0.46: In June 2007, YouTube began trials of a system for automatic detection of uploaded videos that infringe copyright. Google CEO Eric Schmidt regarded this system as necessary for resolving lawsuits such as the one from Viacom, which alleged that YouTube profited from content that it did not have the right to distribute. The system, which was initially called \"Video Identification\" and later became known as Content ID, creates an ID File for copyrighted audio and video material, and stores it in a database. When a video is uploaded, it is checked against the database, and flags the video as a copyright violation if a match is found. When this occurs, the content owner has the choice of blocking the video to make it unviewable, tracking the viewing statistics of the video, or adding advertisements to the video.\n",
+ "0.46: In January 2009, YouTube launched \"YouTube for TV\", a version of the website tailored for set-top boxes and other TV-based media devices with web browsers, initially allowing its videos to be viewed on the PlayStation 3 and Wii video game consoles.\n",
+ "0.46: In September 2012, YouTube launched its first app for the iPhone, following the decision to drop YouTube as one of the preloaded apps in the iPhone 5 and iOS 6 operating system. According to GlobalWebIndex, YouTube was used by 35% of smartphone users between April and June 2013, making it the third-most used app.\n",
+ "0.46: Conversely, YouTube has also allowed government to more easily engage with citizens, the White House's official YouTube channel being the seventh top news organization producer on YouTube in 2012 and in 2013 a healthcare exchange commissioned Obama impersonator Iman Crosson's YouTube music video spoof to encourage young Americans to enroll in the Affordable Care Act (Obamacare)-compliant health insurance. In February 2014, U.S. President Obama held a meeting at the White House with leading YouTube content creators to not only promote awareness of Obamacare but more generally to develop ways for government to better connect with the \"YouTube Generation.\" Whereas YouTube's inherent ability to allow presidents to directly connect with average citizens was noted, the YouTube content creators' new media savvy was perceived necessary to better cope with the website's distracting content and fickle audience.\n",
+ "0.46: Later that year, YouTube came under criticism for showing inappropriate videos targeted at children and often featuring popular characters in violent, sexual or otherwise disturbing situations, many of which appeared on YouTube Kids and attracted millions of views. The term \"Elsagate\" was coined on the Internet and then used by various news outlets to refer to this controversy. On November 11, 2017, YouTube announced it was strengthening site security to protect children from unsuitable content. Later that month, the company started to mass delete videos and channels that made improper use of family-friendly characters. As part of a broader concern regarding child safety on YouTube, the wave of deletions also targeted channels that showed children taking part in inappropriate or dangerous activities under the guidance of adults. Most notably, the company removed \"Toy Freaks\", a channel with over 8.5 million subscribers, that featured a father and his two daughters in odd and upsetting situations. According to analytics specialist SocialBlade, it earned up to £8.7 million annually prior to its deletion.\n",
+ "0.45: In September 2020, YouTube announced that it would be launching a beta version of a new platform of 15-second videos, similar to TikTok, called YouTube Shorts. The platform was first tested in India but as of March 2021 has expanded to other countries including the United States with videos now able to be up to 1 minute long. The platform is not a standalone app, but is integrated into the main YouTube app. Like TikTok, it gives users access to built-in creative tools, including the possibility of adding licensed music to their videos. The platform had its global beta launch in July 2021.\n"
+ ]
}
- ],
- "metadata": {
+ ],
+ "source": [
+ "# Let's query the database\n",
+ "query = \"What was the first youtube video about?\"\n",
+ "\n",
+ "# create the query embedding\n",
+ "xq = co.embed(\n",
+ " texts=[query],\n",
+ " model='embed-english-v3.0',\n",
+ " input_type='search_query',\n",
+ " truncate='END'\n",
+ ").embeddings\n",
+ "\n",
+ "print(np.array(xq).shape)\n",
+ "\n",
+ "# query, returning the top 20 most similar results\n",
+ "res = idx.query(xq, top_k=20, include_metadata=True)\n",
+ "\n",
+ "# Look at the initial retrieval results\n",
+ "for match in res['matches']:\n",
+ " print(f\"{match['score']:.2f}: {match['metadata']['text']}\")"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
"colab": {
- "provenance": []
+ "base_uri": "https://localhost:8080/"
},
- "kernelspec": {
- "display_name": "Python 3",
- "name": "python3"
- },
- "language_info": {
- "name": "python",
- "version": "3.11.6"
+ "id": "WaN6U6u19Md6",
+ "outputId": "9984d8ce-1383-4530-ec84-6e3f3feea05e"
+ },
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "0.95: YouTube began as a venture capital–funded technology startup. Between November 2005 and April 2006, the company raised money from various investors, with Sequoia Capital, $11.5 million, and Artis Capital Management, $8 million, being the largest two. YouTube's early headquarters were situated above a pizzeria and a Japanese restaurant in San Mateo, California. In February 2005, the company activated codice_1. The first video was uploaded April 23, 2005. Titled \"Me at the zoo\", it shows co-founder Jawed Karim at the San Diego Zoo and can still be viewed on the site. In May, the company launched a public beta and by November, a Nike ad featuring Ronaldinho became the first video to reach one million total views. The site launched officially on December 15, 2005, by which time the site was receiving 8 million views a day. Clips at the time were limited to 100 megabytes, as little as 30 seconds of footage.\n",
+ "0.92: Karim said the inspiration for YouTube first came from the Super Bowl XXXVIII halftime show controversy when Janet Jackson's breast was briefly exposed by Justin Timberlake during the halftime show. Karim could not easily find video clips of the incident and the 2004 Indian Ocean Tsunami online, which led to the idea of a video-sharing site. Hurley and Chen said that the original idea for YouTube was a video version of an online dating service, and had been influenced by the website Hot or Not. They created posts on Craigslist asking attractive women to upload videos of themselves to YouTube in exchange for a $100 reward. Difficulty in finding enough dating videos led to a change of plans, with the site's founders deciding to accept uploads of any video.\n",
+ "0.91: YouTube was not the first video-sharing site on the Internet; Vimeo was launched in November 2004, though that site remained a side project of its developers from CollegeHumor at the time and did not grow much, either. The week of YouTube's launch, NBC-Universal's \"Saturday Night Live\" ran a skit \"Lazy Sunday\" by The Lonely Island. Besides helping to bolster ratings and long-term viewership for \"Saturday Night Live\", \"Lazy Sunday\"'s status as an early viral video helped establish YouTube as an important website. Unofficial uploads of the skit to YouTube drew in more than five million collective views by February 2006 before they were removed when NBCUniversal requested it two months later based on copyright concerns. Despite eventually being taken down, these duplicate uploads of the skit helped popularize YouTube's reach and led to the upload of more third-party content. The site grew rapidly; in July 2006, the company announced that more than 65,000 new videos were being uploaded every day and that the site was receiving 100 million video views per day.\n"
+ ]
}
+ ],
+ "source": [
+ "# Add Cohere Reranking Step\n",
+ "# embeds=[np.float32(data_array[i]['embedding']) for i in range(len(data_array))]\n",
+ "docs =[match['metadata']['text'] for match in res['matches']]\n",
+ "\n",
+ "rerank_response = co.rerank(\n",
+ " model = 'rerank-english-v2.0',\n",
+ " query = query,\n",
+ " documents = docs,\n",
+ " top_n = 3,\n",
+ ")\n",
+ "for response in rerank_response:\n",
+ " print(f\"{response.relevance_score:.2f}: {response.document['text']}\")"
+ ]
+ }
+ ],
+ "metadata": {
+ "colab": {
+ "provenance": []
+ },
+ "kernelspec": {
+ "display_name": "Python 3 (ipykernel)",
+ "language": "python",
+ "name": "python3"
},
- "nbformat": 4,
- "nbformat_minor": 0
+ "language_info": {
+ "codemirror_mode": {
+ "name": "ipython",
+ "version": 3
+ },
+ "file_extension": ".py",
+ "mimetype": "text/x-python",
+ "name": "python",
+ "nbconvert_exporter": "python",
+ "pygments_lexer": "ipython3",
+ "version": "3.8.16"
+ }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 4
}
diff --git a/notebooks/Vanilla_RAG.ipynb b/notebooks/Vanilla_RAG.ipynb
index 2f85039c..b7f338c8 100644
--- a/notebooks/Vanilla_RAG.ipynb
+++ b/notebooks/Vanilla_RAG.ipynb
@@ -1,743 +1,734 @@
{
- "cells": [
- {
- "cell_type": "markdown",
- "metadata": {
- "id": "mz33G3t6gbOl"
- },
- "source": [
- "# RAG\n",
- "\n",
- "Retrieval-Augmented Generation (RAG) is a technique that combines the strengths of pre-trained language models with the ability to retrieve information from a large corpus of documents. RAG **enables the language model to produce more informed, accurate, and contextually relevant answers** than by relying on its pre-trained knowledge alone.\n",
- "\n",
- "At Cohere, all RAG calls come with... **precise citations**! 🎉\n",
- "The model cites which groups of words, in the RAG chunks, were used to generate the final answer. \n",
- "These citations make it easy to check where the model’s generated response claims are coming from and they help users gain visibility into the model reasoning. \n",
- "\n",
- "RAG consists of 3 steps:\n",
- "- Step 1: Indexing and given a user query, retrieve the relevant chunks from the index\n",
- "- Step 2: Optionally, rerank the retrieved chunks\n",
- "- Step 3: Generate the model final answer with **precise citations**, given the retrieved and reranked chunks\n",
- "\n",
- "\n"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {
- "id": "nSB0pnt0gbOo"
- },
- "source": [
- "## Step 0 - Imports & Getting some data\n",
- "\n",
- "In this example, we'll use a recent piece of text, that wasn't in the training data: the Wikipedia page of the movie \"Dune 2\". \n",
- "\n",
- "In practice, you would typically do RAG on much longer text, that doesn't fit in the context window of the model."
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "colab": {
- "base_uri": "https://localhost:8080/"
- },
- "id": "H787BXXYvD0a",
- "outputId": "04ef5e04-7760-4d40-deeb-663536b38f20"
- },
- "outputs": [
- {
- "name": "stdout",
- "output_type": "stream",
- "text": [
- "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m52.8/52.8 kB\u001b[0m \u001b[31m1.6 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
- "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m3.1/3.1 MB\u001b[0m \u001b[31m33.6 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
- "\u001b[?25h"
- ]
- }
- ],
- "source": [
- "# we'll use Cohere to cover all building blocks of RAG\n",
- "# TODO: upgrade to \"cohere>5\"\n",
- "%pip install \"cohere<5\" --quiet"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "id": "rACbepFGgbOo"
- },
- "outputs": [],
- "source": [
- "import cohere\n",
- "API_KEY = \"...\" # fill in your Cohere API key here\n",
- "co = cohere.Client(API_KEY)"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "colab": {
- "base_uri": "https://localhost:8080/"
- },
- "id": "QdvbqfFrgbOq",
- "outputId": "3882c95c-46bf-4dcc-99a2-453b3c2fc7c4"
- },
- "outputs": [
- {
- "name": "stdout",
- "output_type": "stream",
- "text": [
- " Preparing metadata (setup.py) ... \u001b[?25l\u001b[?25hdone\n",
- " Building wheel for wikipedia (setup.py) ... \u001b[?25l\u001b[?25hdone\n"
- ]
- }
- ],
- "source": [
- "# we'll get some wikipedia data\n",
- "!pip install wikipedia --quiet\n",
- "import wikipedia"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "colab": {
- "base_uri": "https://localhost:8080/"
- },
- "id": "xP-bWt9XgbOq",
- "outputId": "72276fb2-0d6b-415d-af74-452a013ae84b"
- },
- "outputs": [
- {
- "name": "stdout",
- "output_type": "stream",
- "text": [
- "The text has roughly 5323 words.\n"
- ]
- }
- ],
- "source": [
- "# let's get the wikipedia article about Dune Part Two\n",
- "article = wikipedia.page('Dune Part Two')\n",
- "text = article.content\n",
- "print(f\"The text has roughly {len(text.split())} words.\")"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {
- "id": "-1aJ7hKGgbOr"
- },
- "source": [
- "## Step 1 - Indexing and given a user query, retrieve the relevant chunks from the index\n",
- "\n",
- "We index the document in a vector database. This requires getting the documents, chunking them, embedding, and indexing them in a vector database. Then we retrieved relevant results based on the users' query.\n",
- "\n",
- "### We split the document into chunks of roughly 512 words"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "colab": {
- "base_uri": "https://localhost:8080/"
- },
- "id": "ZUph1JX41665",
- "outputId": "6c63a93f-6999-47af-e704-d4a88727bc75"
- },
- "outputs": [
- {
- "name": "stdout",
- "output_type": "stream",
- "text": [
- "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m256.9/256.9 kB\u001b[0m \u001b[31m6.6 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
- "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m66.6/66.6 kB\u001b[0m \u001b[31m8.2 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
- "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m138.5/138.5 kB\u001b[0m \u001b[31m14.5 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
- "\u001b[?25h"
- ]
- }
- ],
- "source": [
- "# For chunking let's use langchain to help us split the text\n",
- "%pip install -qU langchain-text-splitters --quiet\n",
- "from langchain_text_splitters import RecursiveCharacterTextSplitter"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "colab": {
- "base_uri": "https://localhost:8080/"
- },
- "id": "uhXW7iHC1-Q6",
- "outputId": "d68ac348-4b73-4c6a-a445-6c510bdb0881"
- },
- "outputs": [
- {
- "name": "stdout",
- "output_type": "stream",
- "text": [
- "The text has been broken down in 91 chunks.\n"
- ]
- }
- ],
- "source": [
- "# Create basic configurations to chunk the text\n",
- "text_splitter = RecursiveCharacterTextSplitter(\n",
- " chunk_size=512,\n",
- " chunk_overlap=50,\n",
- " length_function=len,\n",
- " is_separator_regex=False,\n",
- ")\n",
- "\n",
- "# Split the text into chunks with some overlap\n",
- "chunks_ = text_splitter.create_documents([text])\n",
- "chunks = [c.page_content for c in chunks_]\n",
- "print(f\"The text has been broken down in {len(chunks)} chunks.\")"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {
- "id": "P8g0sE2hgbOs"
- },
- "source": [
- "### Embed every text chunk\n",
- "\n",
- "Cohere embeddings are state-of-the-art."
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "colab": {
- "base_uri": "https://localhost:8080/"
- },
- "id": "KEarMPEqgbOs",
- "outputId": "7da0e06d-f637-4470-8e01-6de8249be64b"
- },
- "outputs": [
- {
- "name": "stdout",
- "output_type": "stream",
- "text": [
- "We just computed 91 embeddings.\n"
- ]
- }
- ],
- "source": [
- "# Because the texts being embedded are the chunks we are searching over, we set the input type as search_doc\n",
- "model=\"embed-english-v3.0\"\n",
- "response = co.embed(\n",
- " texts= chunks,\n",
- " model=model,\n",
- " input_type=\"search_document\",\n",
- " embedding_types=['float']\n",
- ")\n",
- "embeddings = response.embeddings.float\n",
- "print(f\"We just computed {len(embeddings)} embeddings.\")"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {
- "id": "HM6vKeypgbOs"
- },
- "source": [
- "### Store the embeddings in a vector database\n",
- "\n",
- "We use the simplest vector database ever: a python dictionary using `np.array()`."
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "id": "sdW7M8HLvB-9"
- },
- "outputs": [],
- "source": [
- "# We use the simplest vector database ever: a python dictionary\n",
- "!pip install numpy --quiet"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "id": "H2srFH-IgbOs"
- },
- "outputs": [],
- "source": [
- "import numpy as np\n",
- "vector_database = {i: np.array(embedding) for i, embedding in enumerate(embeddings)}\n",
- "# { 0: array([...]), 1: array([...]), 2: array([...]), ..., 10: array([...]) }"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {
- "id": "q6NGVurZgbOs"
- },
- "source": [
- "## Given a user query, retrieve the relevant chunks from the vector database\n"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {
- "id": "eC05yJQ7jlek"
- },
- "source": [
- "### Define the user question"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "id": "Y2HTxspKgbOs"
- },
- "outputs": [],
- "source": [
- "query = \"Name everyone involved in writing the script, directing, and producing 'Dune: Part Two'?\"\n",
- "\n",
- "# Note: the relevant passage in the wikipedia page we're looking for is:\n",
- "# \"[...] Dune: Part Two was originally scheduled to be released on October 20, 2023, but was delayed to November 17, 2023, before moving forward two weeks to November 3, 2023, to adjust to changes in release schedules from other studios. It was later postponed by over four months to March 15, 2024, due to the 2023 Hollywood labor disputes. After the strikes were resolved, the film moved once more up two weeks to March 1, 2024. [...]\""
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {
- "id": "9oULg1tOjjOW"
- },
- "source": [
- "### Embed the user question\n",
- "\n",
- "Cohere embeddings are state-of-the-art."
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "colab": {
- "base_uri": "https://localhost:8080/"
- },
- "id": "yrUuS6vXgbOs",
- "outputId": "0c64a930-f817-43c2-d775-1d9145cb304e"
- },
- "outputs": [
- {
- "name": "stdout",
- "output_type": "stream",
- "text": [
- "query_embedding: [-0.068603516, -0.02947998, -0.06274414, -0.015449524, -0.033294678, 0.0056877136, -0.047210693, 0.04714966, -0.024871826, 0.008148193, 0.0770874, 0.023880005, -0.058685303, -0.052520752, 0.012832642, 0.024398804, 0.0053215027, 0.035491943, 0.02961731, -0.0069847107, 0.01083374, -0.0011358261, -0.002199173, 0.018417358, 0.027389526, -0.002691269, -0.026535034, 0.015197754, 0.024368286, 0.03729248, 0.0057754517, -0.02229309, -0.014694214, 0.019989014, -0.0036315918, -0.013793945, 0.02835083, 0.006011963, 0.011428833, 0.008682251, 0.046142578, -0.040039062, -0.032196045, -0.002653122, -0.012580872, -0.0041618347, 0.03111267, -0.016799927, 0.014801025, -0.00030636787, -0.033050537, 0.033966064, -0.016021729, -0.025009155, -0.007534027, -0.017074585, 0.008415222, -0.10620117, 0.019195557, -0.015686035, -0.0043182373, -0.045440674, 0.05404663, 0.030776978, -0.014129639, -0.01499939, -0.007286072, 0.009933472, 0.06390381, 0.02444458, -0.010345459, 0.041931152, 0.032989502, -0.04522705, 0.056610107, 0.0068893433, -0.008911133, 0.012489319, 0.01675415, 0.020065308, 0.018753052, 0.022659302, -0.051849365, -0.04925537, 0.046325684, -0.005268097, 0.0026874542, -0.036712646, 0.009437561, -0.0037841797, -0.01473999, -0.034179688, -0.0011606216, 0.05026245, 0.0020771027, -0.016021729, -0.0044898987, 0.04168701, -0.015205383, 0.019210815, -0.012374878, -0.031311035, 0.03111267, -0.040100098, -0.016479492, 0.020446777, 0.010192871, 0.0037841797, -0.0023765564, 0.015220642, -0.016571045, -0.006454468, 0.037384033, -0.044555664, -0.008262634, 0.019546509, 0.009460449, 0.014701843, 0.02658081, -0.02078247, 0.015571594, 0.013153076, -0.010375977, 0.047912598, 0.005393982, -0.007911682, -0.019378662, 0.023529053, -0.0033550262, -0.04598999, -0.0052871704, 0.040252686, 0.011375427, 0.01550293, -0.004508972, 0.006515503, 0.003370285, -0.022766113, 0.00062561035, -0.0007596016, -0.0015277863, 0.0149002075, 0.061401367, 8.261204e-05, 0.06359863, -0.01537323, 0.007446289, 0.018814087, 0.02507019, 0.024215698, 0.006122589, 0.005886078, -0.03829956, 0.029037476, 0.07720947, 0.016921997, 0.022109985, 0.005958557, 0.028793335, 0.019485474, 0.015174866, 0.026153564, 0.032318115, 0.034210205, 0.027145386, -0.019515991, -0.018661499, 0.020477295, 0.008598328, -0.06573486, -0.037109375, 0.04043579, 0.030471802, -0.0010843277, 0.009757996, 0.026947021, 0.037017822, -0.018234253, -0.0115356445, 0.099365234, 0.027816772, -0.019927979, 0.0020961761, 0.013198853, -0.019073486, 2.7656555e-05, 0.041259766, 0.029510498, -0.016204834, 0.028137207, 0.039489746, 0.034698486, -0.03918457, -0.029418945, 0.02041626, 0.0073432922, -0.018569946, -0.009849548, 0.002861023, 0.030319214, -0.012886047, 0.014671326, -0.035827637, 0.007247925, -0.027709961, -0.022079468, 0.0012960434, 0.015426636, -0.01725769, 0.01525116, 0.025360107, -0.0077400208, -0.039916992, 0.029037476, -0.011154175, 0.007736206, -0.041748047, 0.05343628, 0.007286072, 0.0435791, 0.034301758, -0.047210693, 0.03552246, -0.015327454, 0.029922485, -0.018859863, 0.013053894, -0.028060913, 0.07757568, -0.020462036, 0.070739746, -0.010223389, 0.03604126, 0.02758789, -0.023284912, 0.012184143, 0.029144287, 0.023880005, -0.019378662, -0.0051116943, 0.0048675537, 0.01864624, -0.04397583, -0.007598877, 0.0713501, 0.0115737915, 0.002922058, 0.011619568, 0.017364502, 0.031921387, -0.0019664764, -0.008575439, 0.003484726, -0.09466553, 0.03475952, 0.026611328, -0.039520264, -0.0104522705, -0.005443573, -0.008392334, 0.012908936, 0.0043792725, -0.002456665, -0.028396606, -0.02027893, -0.0005569458, 0.027786255, 0.03427124, -0.0062332153, -0.018203735, 0.019241333, 0.07244873, -0.0028057098, 0.01234436, -0.0018787384, -0.027496338, 0.0015287399, -0.004032135, -0.013748169, -0.01878357, 0.0018053055, -0.01159668, 0.028213501, 0.004776001, 0.042388916, 0.0024280548, 0.017471313, -0.038085938, 0.026321411, 0.02973938, 0.06213379, 0.006401062, 0.036102295, -0.028121948, -0.00869751, -0.016693115, 0.029190063, 0.016784668, -0.008628845, 0.0039634705, -0.0035381317, 0.019500732, 0.025009155, -0.04547119, -0.003572464, 0.05215454, 0.067871094, -0.04257202, -0.02293396, -0.027175903, 0.05340576, 0.019226074, 0.039978027, 0.056121826, -0.028320312, -0.020217896, -0.035003662, 0.03225708, 0.028656006, 0.062347412, 0.12915039, -0.0137786865, 0.0022201538, -0.057434082, -0.04397583, -0.049865723, -0.013160706, -0.03353882, 0.006427765, -0.014823914, -0.008201599, -0.036346436, -0.037353516, -0.010528564, -0.015930176, -0.027572632, 0.0074272156, 0.004547119, -0.024414062, -0.018859863, -0.020095825, 0.029632568, -0.00067043304, -0.044036865, -0.0043411255, -0.005256653, -0.019195557, 0.022262573, -0.00020956993, -0.013877869, -0.011108398, -0.020324707, -0.015808105, -0.025039673, -0.009498596, 0.05090332, 0.0046195984, -0.017150879, 0.04309082, -0.029067993, 0.002670288, -0.00026249886, -0.032409668, -0.053100586, 0.012481689, -0.014633179, 0.0013475418, -0.034332275, 0.038330078, 0.014892578, -0.046936035, 0.021591187, -0.020385742, -0.0052604675, 0.02796936, 0.0014333725, 0.012077332, -0.0118255615, -0.005569458, 0.008491516, 0.009841919, 0.0031318665, -0.003408432, -0.007144928, 0.040374756, -0.0038928986, 0.005279541, -0.008415222, 0.031707764, 0.0140686035, -0.015029907, -0.02810669, -0.0078125, -0.030853271, -0.03201294, 0.021316528, -0.036193848, -0.0423584, 0.0072784424, 0.014801025, 0.0019607544, -0.012367249, -0.009056091, -0.021438599, -0.02645874, 0.038726807, -0.007549286, 0.0049591064, 0.019012451, 0.017791748, -0.009185791, 0.04006958, 0.003107071, -0.0075302124, -0.010375977, -0.009246826, -0.02130127, -0.0056762695, -0.0076789856, 0.010009766, -0.010536194, 0.041107178, 0.0021133423, 0.029891968, 0.01626587, 0.042236328, -0.02784729, -0.032836914, 0.0317688, 0.045715332, 0.000116825104, 0.028030396, 0.007205963, 0.012512207, -0.035583496, -0.048034668, -0.023529053, -0.04953003, 0.0345459, -0.048339844, -0.060272217, -0.004512787, 0.04425049, 0.0076141357, 0.029510498, 0.007396698, 0.003353119, -0.038726807, 0.07183838, -0.026901245, -0.023529053, -0.038085938, 0.068725586, 0.018096924, -0.013534546, 0.05883789, -0.016113281, 0.017944336, 0.041046143, 0.022918701, 0.036499023, 0.015296936, -0.04916382, 0.0075683594, -0.011390686, 0.009735107, -0.0070152283, 0.003129959, -0.032562256, 0.0003478527, -0.0036640167, -0.006893158, -0.016098022, -0.034332275, 0.037750244, -0.010269165, 0.016494751, -0.02394104, 0.03753662, -0.022644043, -0.0008234978, 0.001001358, -0.048217773, 0.04989624, 0.0078125, 0.0044937134, 0.027038574, 0.04736328, -0.02973938, -0.011726379, 0.01348114, 0.021408081, 0.00844574, -0.03741455, -0.015686035, -0.040893555, 0.001452446, -0.025405884, 0.07348633, 0.038238525, -0.019958496, 0.023071289, -0.016403198, -0.08105469, 0.0071029663, -0.019088745, 5.8174133e-05, -0.005569458, 0.01399231, 0.02255249, 0.011222839, 0.00028824806, 0.0066184998, 0.0017499924, -0.009864807, -0.0115737915, 0.053100586, 0.0065231323, 0.001865387, -0.026428223, 0.03692627, 0.025390625, 0.022613525, 0.018722534, 0.007675171, -0.03439331, 0.041625977, -0.01789856, -0.041046143, 0.0051460266, 0.04144287, 0.048553467, 0.054595947, -0.01108551, -0.033935547, -0.026275635, -0.0118255615, -0.021362305, -0.009841919, -0.00724411, 0.028900146, 0.009887695, -0.023803711, 0.016311646, 0.018798828, -0.03668213, 0.046844482, 0.010696411, -0.014717102, -0.008110046, -0.004589081, -0.0028076172, -0.050811768, -0.017196655, -0.03491211, 0.0074005127, -0.038909912, 0.032440186, -0.034362793, -0.008682251, 0.032928467, -0.04626465, -0.009666443, 0.018951416, 0.031951904, -0.003791809, 0.02015686, -0.05532837, -0.005683899, -0.00054216385, -0.0034332275, 0.008659363, 0.02130127, -0.038879395, -0.0033397675, -0.03866577, -0.0049934387, 0.017944336, 0.001496315, 0.019485474, -0.004348755, 0.00046491623, 0.0007157326, 0.035614014, -0.027694702, 0.03692627, -0.008491516, 0.0524292, -0.016662598, -0.0017795563, -0.021575928, -0.018753052, -0.049346924, -0.06652832, 0.04272461, 0.03186035, 0.0011978149, 0.03463745, 0.024002075, 0.02607727, 0.020446777, 0.0256958, 0.026855469, 0.0074005127, -0.067993164, 0.017944336, -0.0039482117, 0.05496216, -0.041412354, 0.014175415, 0.02444458, -0.026412964, 0.057403564, -0.026779175, 0.023254395, 0.03945923, 0.033569336, -0.030258179, -0.039093018, -0.036468506, 0.017105103, 0.009635925, 0.025497437, 0.04156494, -0.02571106, -0.0010414124, -0.005630493, -0.016448975, -0.026733398, 0.001326561, -0.042022705, 0.0012521744, -0.041259766, -0.12182617, -0.03857422, 0.12548828, -0.005947113, -0.020736694, -0.0033855438, 0.03778076, -0.033813477, 0.038970947, 0.003921509, 0.011810303, 0.031982422, -0.032562256, -0.002653122, -0.025009155, -0.03805542, -0.016998291, 0.018173218, 0.0158844, 0.0011739731, 0.048217773, -0.020401001, 0.044708252, -0.017318726, 0.014457703, -0.041809082, 0.010543823, 0.041931152, 0.076293945, -0.054779053, 0.060272217, -0.046936035, 0.02949524, 0.00554657, 0.041534424, -0.013046265, -0.056152344, 0.010406494, 0.02973938, -0.023727417, -0.022476196, -0.024734497, -0.013168335, 0.060424805, 0.011787415, 0.018997192, -0.043426514, -0.00077724457, -0.010154724, 0.017150879, -0.01171875, -0.022476196, 0.0034255981, -0.0026454926, 0.004837036, -0.0043296814, 0.02619934, -0.021560669, -0.039733887, -0.022415161, -0.06817627, -0.023223877, -0.018585205, -0.015319824, 0.012588501, 0.0064353943, -0.013748169, 0.043304443, 0.002626419, -0.029373169, -0.016784668, -0.026184082, 0.05847168, 0.034179688, 0.03842163, -0.05493164, -0.017486572, 0.016540527, 0.03164673, 0.089904785, 0.013534546, -0.07684326, -0.024108887, 0.07434082, 0.030395508, 0.007091522, 0.07373047, 0.012527466, -0.010856628, -0.01828003, -0.045196533, 0.00065279007, -0.0637207, 0.010726929, 0.023880005, -0.0030708313, -0.012298584, 0.027236938, -0.04928589, 0.023071289, 0.008674622, -0.023529053, -0.015838623, -0.010543823, 0.012168884, 0.014854431, -0.05834961, -0.06088257, -0.012313843, 0.035461426, 0.02027893, 0.019348145, -0.014602661, -0.02104187, -0.0309906, 0.001405716, -0.019973755, -0.00157547, -0.003944397, 0.0009326935, -0.02078247, -0.015731812, -0.044433594, 0.03390503, 0.057159424, 0.018585205, -0.023895264, -0.0057029724, 0.0049552917, 0.013412476, 0.022399902, 0.010154724, 0.0519104, 0.06591797, 0.018341064, 0.012161255, -0.05810547, -0.043304443, -0.031173706, 0.0023860931, -0.003944397, 0.11425781, -0.031036377, 0.019989014, -0.038635254, -0.025939941, 0.035064697, 0.041168213, 0.03161621, -0.069885254, -0.04537964, 0.028945923, -0.023162842, 0.019226074, -0.028442383, 0.015594482, -0.019256592, -0.0046463013, 0.034240723, 0.009124756, 0.05718994, 0.031219482, 0.02154541, 0.009590149, 0.00076818466, 0.04849243, -0.029129028, -0.03375244, -0.023391724, -0.028381348, -0.029708862, -0.0132369995, 0.010353088, 0.020263672, -0.030807495, 0.01007843, -0.03704834, 0.023376465, -0.03665161, 0.03741455, 0.015144348, 0.057281494, 0.03137207, 0.048431396, 0.021194458, 0.008110046, -0.03540039, -0.015312195, 0.022384644, 0.0065956116, 0.008056641, 0.0018348694, -0.009246826, 0.030380249, 0.0003862381, 0.0051841736, 0.04486084, 0.017807007, 0.0026130676, 0.07977295, 0.05419922, 0.062194824, 0.02633667, 0.024841309, -0.041625977, -0.005897522, 0.04031372, -0.055908203, 0.0026226044, -0.05340576, -0.05496216, 0.011474609, -0.006954193, -0.013122559, 0.019714355, -0.07159424, 0.031173706, 0.0034255981, -0.0034103394, 0.0440979, 0.011779785, -0.007827759, -0.03173828, -0.020950317, -0.030166626, -0.035308838, 0.030792236, 0.04525757, -0.028701782, -0.011100769, -0.02331543, -0.0357666, -0.025680542, 0.0011911392, 0.01940918, 0.05706787, 0.028381348, 0.007133484, -0.07733154, -0.007686615, 0.03869629, 0.0066833496, 0.008842468, 0.03439331, -0.014282227, 0.0357666, -0.004737854, -0.039794922, -0.0070381165, 0.02670288, 0.0107421875, 0.016189575, -0.06555176, -0.0138549805, 0.0008363724, -0.016693115, 0.006904602, -0.020263672, -0.030426025, 0.008453369, -0.046173096, -0.01802063, -0.013595581, -0.0044288635, -0.0039978027, -0.0044898987, 0.0007619858, 0.003921509, 0.0053977966, 0.020385742, -0.012329102, -0.023803711, -0.0057525635, 0.038330078, -0.014549255, -0.06298828, -0.047607422, 0.039245605, -0.06781006, -0.035217285, -0.009056091, 0.019927979, -0.003932953, -0.020309448, -0.017044067, 0.018127441, -8.624792e-05, -0.043182373, 0.009590149, 0.035308838, 0.031951904, 0.0011615753, -0.042022705, 0.079956055, 0.026687622, 0.013542175, -0.0074157715, -0.00983429, -0.0022563934, 0.07373047, 0.059387207, 0.03488159, 0.0071372986, -0.06427002, -0.0546875, -0.02482605, 0.11071777, -0.021072388, 0.01626587, -0.049713135, 0.061553955, -0.016860962, 0.051971436, -0.012962341, -0.0011711121, -0.014198303, -0.0061149597, -0.005836487, 0.00022387505, -0.027618408, 0.019836426, 0.009933472, 0.02368164, -0.020309448, -0.0049591064, -0.008628845, -0.03253174, -0.017684937, 0.02468872, -0.0023498535, 0.01448822, 0.061920166, 0.031707764, -0.0026416779, -0.040985107, -0.06335449, -0.036071777, 0.05404663, -0.0044136047, -0.0146102905, -0.0033416748, 0.028671265, -0.012771606, -0.0016565323, -0.0038909912, -0.02407837, -0.009857178, 0.0014467239, -0.008720398, -0.006011963, 0.032073975, -0.033325195, 0.014862061, -0.017227173, -0.018753052, -0.0060424805, 0.022567749, -0.017654419, -0.017562866, -0.07244873, -0.0881958, 0.050476074, 0.02609253, -0.032409668, 0.07458496, 0.009399414, 0.009117126, -0.031051636, -0.03451538, -0.004219055, -0.05718994, 0.020080566, -0.025421143, -0.010948181, 0.06341553, -0.009231567, -0.021697998, -0.009719849, 0.012802124, -0.020370483, 0.0034389496, 0.018859863, -0.025680542, 0.0013141632, 0.068603516, -0.021026611, 0.021881104, -0.0395813, -0.0019073486, 0.0056037903, -0.032348633]\n"
- ]
- }
- ],
- "source": [
- "# Because the text being embedded is the search query, we set the input type as search_query\n",
- "response = co.embed(\n",
- " texts=[query],\n",
- " model=model,\n",
- " input_type=\"search_query\",\n",
- " embedding_types=['float']\n",
- ")\n",
- "query_embedding = response.embeddings.float[0]\n",
- "print(\"query_embedding: \", query_embedding)"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {
- "id": "8K8B87CGgbOt"
- },
- "source": [
- "### Retrieve the most relevant chunks from the vector database\n",
- "\n",
- "We use cosine similarity to find the most similar chunks"
- ]
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "mz33G3t6gbOl"
+ },
+ "source": [
+ "# RAG\n",
+ "\n",
+ "Retrieval-Augmented Generation (RAG) is a technique that combines the strengths of pre-trained language models with the ability to retrieve information from a large corpus of documents. RAG **enables the language model to produce more informed, accurate, and contextually relevant answers** than by relying on its pre-trained knowledge alone.\n",
+ "\n",
+ "At Cohere, all RAG calls come with... **precise citations**! 🎉\n",
+ "The model cites which groups of words, in the RAG chunks, were used to generate the final answer. \n",
+ "These citations make it easy to check where the model’s generated response claims are coming from and they help users gain visibility into the model reasoning. \n",
+ "\n",
+ "RAG consists of 3 steps:\n",
+ "- Step 1: Indexing and given a user query, retrieve the relevant chunks from the index\n",
+ "- Step 2: Optionally, rerank the retrieved chunks\n",
+ "- Step 3: Generate the model final answer with **precise citations**, given the retrieved and reranked chunks\n",
+ "\n",
+ "\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "nSB0pnt0gbOo"
+ },
+ "source": [
+ "## Step 0 - Imports & Getting some data\n",
+ "\n",
+ "In this example, we'll use a recent piece of text, that wasn't in the training data: the Wikipedia page of the movie \"Dune 2\". \n",
+ "\n",
+ "In practice, you would typically do RAG on much longer text, that doesn't fit in the context window of the model."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "colab": {
+ "base_uri": "https://localhost:8080/"
},
+ "id": "H787BXXYvD0a",
+ "outputId": "04ef5e04-7760-4d40-deeb-663536b38f20"
+ },
+ "outputs": [
{
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "colab": {
- "base_uri": "https://localhost:8080/"
- },
- "id": "nik3es32gbOt",
- "outputId": "a1c30024-52e1-42c7-8836-a2c590559aca"
- },
- "outputs": [
- {
- "name": "stdout",
- "output_type": "stream",
- "text": [
- "similarity scores: [0.6953257882233425, 0.3713410510180273, 0.46501499776898964, 0.5448546916785195, 0.4014738351361969, 0.3231420292334584, 0.3179003053384008, 0.42799691553367775, 0.18882594531435784, 0.36868801306504106, 0.3404040737300553, 0.3852837621219358, 0.2600249419491577, 0.3723244353775111, 0.3631492691137214, 0.47574774051439606, 0.40415422750911745, 0.4149923346201023, 0.5014741934381444, 0.3549433331883204, 0.32072714802512714, 0.14770872479410424, 0.585277816615252, 0.6999636953772764, 0.7722295084104617, 0.4895347049465806, 0.5170096485954725, 0.7137817366881455, 0.5224900699612323, 0.5914632581598285, 0.2657897083381463, 0.6462342489537262, 0.6317222315431096, 0.5982303530756702, 0.5138265091630297, 0.41385121172723643, 0.4293941094100836, 0.4173182546482015, 0.42621236706314475, 0.4428474375355954, 0.35058541576139896, 0.3578709652019502, 0.3930157841938308, 0.3564608202848675, 0.23016661533167404, 0.4933441863421645, 0.41037089239250985, 0.39993051898770193, 0.3119997063424595, 0.2677143729521374, 0.3700866951454496, 0.46727994925061545, 0.4393343280374425, 0.42111290117172434, 0.4485349189824285, 0.4710573736688592, 0.24169956903740436, 0.3840442910806355, 0.14284631817675886, 0.5381588054138154, 0.431113882725076, 0.5189547209048608, 0.3950667224233914, 0.32429768756510174, 0.4370358125161736, 0.18727062244331039, 0.5206375682478743, 0.5175737635701252, 0.5326043981628349, 0.45586923626994363, 0.21667338125532032, 0.16459878595959285, 0.22236726481673777, 0.5187259906958807, 0.2884444442338396, 0.286407544555338, 0.2313840178160818, 0.2057731158935257, 0.5973876998341746, 0.42904243401792086, 0.4081217538000544, 0.5330523063972133, 0.45080561486977405, 0.414703452285757, 0.2569028899107211, 0.5087916806929323, 0.14159076456040554, 0.46505779053352697, 0.556364222182839, 0.35464351181035236, 0.40174477023626]\n",
- "Here are the indices of the top 10 chunks after retrieval: [24 27 23 0 31 32 33 78 29 22]\n",
- "Here are the top 10 chunks after retrieval: \n",
- "== stunt coordinator. Dune: Part Two was produced by Villeneuve, Mary Parent, and Cale Boyter, with Tanya Lapointe, Brian Herbert, Byron Merritt, Kim Herbert, Thomas Tull, Jon Spaihts, Richard P. Rubinstein, John Harrison, and Herbert W. Gain serving as executive producers and Kevin J. Anderson as creative consultant. Legendary CEO Joshua Grode confirmed in April 2019 that they plan to make a sequel, adding that \"there's a logical place to stop the [first] movie before the book is over\".In December 2020,\n",
- "== that.\"On October 26, 2021, Legendary officially greenlit Dune: Part Two, with a spokesperson for the company stating, \"We would not have gotten to this point without the extraordinary vision of Denis and the amazing work of his talented crew, the writers, our stellar cast, our partners at Warner Bros., and of course the fans! Here's to more Dune.\" Production work had occurred back-to-back with the first film, as Villeneuve and his wife Lapointe immediately took a flight to Budapest in order to begin\n",
- "== series. Villeneuve ultimately secured a two-film deal with Warner Bros. Pictures, in the same style as the two-part adaption of Stephen King's It in 2017 and 2019. In January 2019, Joe Walker was confirmed to be serving as the film's editor. Other crew included Brad Riker as supervising art director, Patrice Vermette as production designer, Paul Lambert as visual effects supervisor, Gerd Nefzer as special effects supervisor, and Thomas Struthers as stunt coordinator. Dune: Part Two was produced by\n",
- "== Dune: Part Two is a 2024 American epic science fiction film directed and produced by Denis Villeneuve, who co-wrote the screenplay with Jon Spaihts. The sequel to Dune (2021) adapts the 1965 novel Dune by Frank Herbert and follows Paul Atreides as he unites with the Fremen people of the desert planet Arrakis to wage war against House Harkonnen. Timothée Chalamet, Rebecca Ferguson, Josh Brolin, Stellan Skarsgård, Dave Bautista, Zendaya, Charlotte Rampling, and Javier Bardem reprise their roles from the first\n",
- "== Eric Roth was hired to co-write the screenplay in April 2017 for the Dune films, and Jon Spaihts was later confirmed to be co-writing the script alongside Roth and Villeneuve. Game of Thrones language creator David Peterson was confirmed to be developing languages for the film in April 2019. Villeneuve and Peterson had created the Chakobsa language, which was used by actors on set. In November 2019, Spaihts stepped down as showrunner for Dune: Prophecy to focus on Dune: Part Two. In June 2020, Greig Fraser\n",
- "== on Dune: Part Two. In June 2020, Greig Fraser said, \"It's a fully formed story in itself with places to go. It's a fully standalone epic film that people will get a lot out of when they see it\". Between the release of Dune and the confirmation of Dune: Part Two, Villeneuve started working the script in a way that production could begin immediately once the film was greenlit. By February 2021, Roth created a full treatment for the sequel, with writing beginning that August. He confirmed that Feyd-Rautha\n",
- "== that August. He confirmed that Feyd-Rautha would appear in the film, and stated he will be a \"very important character\". In March 2022, Villeneuve had mostly finished writing the screenplay. Craig Mazin and Roth wrote additional literary material for the film.Villeneuve stated that the film would continue directly from the first, and specifically described it as being the \"second part.\" He described the film as being an \"epic war movie\", adding that while the first film was more \"contemplative\", the second\n",
- "== On the review aggregator website Rotten Tomatoes, 93% of 378 critics' reviews are positive, with an average rating of 8.4/10. The website's consensus reads: \"Visually thrilling and narratively epic, Dune: Part Two continues Denis Villeneuve's adaptation of the beloved sci-fi series in spectacular form.\" Metacritic, which uses a weighted average, assigned the film a score of 79 out of 100, based on 62 critics, indicating \"generally favorable\" reviews. Audiences surveyed by CinemaScore gave the film an\n",
- "== theatrical experience is at the very heart of the cinematic language for me.\" With Dune: Part Two being greenlit, Villeneuve said that his primary concern was to complete the filming as soon as possible, with the earliest he expected to start in the last quarter of 2022. However, he noted that production would be facilitated by the work already established on the first film, which would help expedite production.\n",
- "== By November 2016, Legendary Pictures had obtained the film and TV rights for the Dune franchise, based on the eponymous 1965 novel by Frank Herbert. Vice chair of worldwide production for Legendary Mary Parent began discussing with Denis Villeneuve about directing a film adaptation, quickly hiring him after realizing his passion for Dune. By February 2018, Villeneuve was confirmed to be hired as director, and intended to adapt the novel as a two-part film series. Villeneuve ultimately secured a two-film\n"
- ]
- }
- ],
- "source": [
- "def cosine_similarity(a, b):\n",
- " return np.dot(a, b) / (np.linalg.norm(a) * np.linalg.norm(b))\n",
- "\n",
- "# Calculate similarity between the user question & each chunk\n",
- "similarities = [cosine_similarity(query_embedding, chunk) for chunk in embeddings]\n",
- "print(\"similarity scores: \", similarities)\n",
- "\n",
- "# Get indices of the top 10 most similar chunks\n",
- "sorted_indices = np.argsort(similarities)[::-1]\n",
- "\n",
- "# Keep only the top 10 indices\n",
- "top_indices = sorted_indices[:10]\n",
- "print(\"Here are the indices of the top 10 chunks after retrieval: \", top_indices)\n",
- "\n",
- "# Retrieve the top 10 most similar chunks\n",
- "top_chunks_after_retrieval = [chunks[i] for i in top_indices]\n",
- "print(\"Here are the top 10 chunks after retrieval: \")\n",
- "for t in top_chunks_after_retrieval:\n",
- " print(\"== \" + t)"
- ]
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m52.8/52.8 kB\u001b[0m \u001b[31m1.6 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
+ "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m3.1/3.1 MB\u001b[0m \u001b[31m33.6 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
+ "\u001b[?25h"
+ ]
+ }
+ ],
+ "source": [
+ "# we'll use Cohere to cover all building blocks of RAG\n",
+ "# TODO: upgrade to \"cohere>5\"\n",
+ "%pip install \"cohere<5\" --quiet"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "rACbepFGgbOo"
+ },
+ "outputs": [],
+ "source": [
+ "import cohere\n",
+ "API_KEY = \"...\" # fill in your Cohere API key here\n",
+ "co = cohere.Client(API_KEY)"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "colab": {
+ "base_uri": "https://localhost:8080/"
},
+ "id": "QdvbqfFrgbOq",
+ "outputId": "3882c95c-46bf-4dcc-99a2-453b3c2fc7c4"
+ },
+ "outputs": [
{
- "cell_type": "markdown",
- "metadata": {
- "id": "qzcpds3VgbOt"
- },
- "source": [
- "## Step 2 - Rerank the chunks retrieved from the vector database\n",
- "\n",
- "We rerank the 10 chunks retrieved from the vector database. Reranking boosts retrieval accuracy.\n",
- "\n",
- "Reranking lets us go from 10 chunks retrieved from the vector database, to the 3 most relevant chunks."
- ]
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ " Preparing metadata (setup.py) ... \u001b[?25l\u001b[?25hdone\n",
+ " Building wheel for wikipedia (setup.py) ... \u001b[?25l\u001b[?25hdone\n"
+ ]
+ }
+ ],
+ "source": [
+ "# we'll get some wikipedia data\n",
+ "!pip install wikipedia --quiet\n",
+ "import wikipedia"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "colab": {
+ "base_uri": "https://localhost:8080/"
},
+ "id": "xP-bWt9XgbOq",
+ "outputId": "72276fb2-0d6b-415d-af74-452a013ae84b"
+ },
+ "outputs": [
{
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "colab": {
- "base_uri": "https://localhost:8080/"
- },
- "id": "2J4LywVygbOt",
- "outputId": "7a4c89bf-fc5e-409f-9304-fce006b9d8bf"
- },
- "outputs": [
- {
- "name": "stdout",
- "output_type": "stream",
- "text": [
- "Here are the top 3 chunks after rerank: \n",
- "== Dune: Part Two is a 2024 American epic science fiction film directed and produced by Denis Villeneuve, who co-wrote the screenplay with Jon Spaihts. The sequel to Dune (2021) adapts the 1965 novel Dune by Frank Herbert and follows Paul Atreides as he unites with the Fremen people of the desert planet Arrakis to wage war against House Harkonnen. Timothée Chalamet, Rebecca Ferguson, Josh Brolin, Stellan Skarsgård, Dave Bautista, Zendaya, Charlotte Rampling, and Javier Bardem reprise their roles from the first\n",
- "== stunt coordinator. Dune: Part Two was produced by Villeneuve, Mary Parent, and Cale Boyter, with Tanya Lapointe, Brian Herbert, Byron Merritt, Kim Herbert, Thomas Tull, Jon Spaihts, Richard P. Rubinstein, John Harrison, and Herbert W. Gain serving as executive producers and Kevin J. Anderson as creative consultant. Legendary CEO Joshua Grode confirmed in April 2019 that they plan to make a sequel, adding that \"there's a logical place to stop the [first] movie before the book is over\".In December 2020,\n",
- "== series. Villeneuve ultimately secured a two-film deal with Warner Bros. Pictures, in the same style as the two-part adaption of Stephen King's It in 2017 and 2019. In January 2019, Joe Walker was confirmed to be serving as the film's editor. Other crew included Brad Riker as supervising art director, Patrice Vermette as production designer, Paul Lambert as visual effects supervisor, Gerd Nefzer as special effects supervisor, and Thomas Struthers as stunt coordinator. Dune: Part Two was produced by\n"
- ]
- }
- ],
- "source": [
- "response = co.rerank(\n",
- " query=query,\n",
- " documents=top_chunks_after_retrieval,\n",
- " top_n=3,\n",
- " model=\"rerank-english-v2.0\",\n",
- ")\n",
- "\n",
- "top_chunks_after_rerank = [result.document['text'] for result in response]\n",
- "print(\"Here are the top 3 chunks after rerank: \")\n",
- "for t in top_chunks_after_rerank:\n",
- " print(\"== \" + t)"
- ]
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "The text has roughly 5323 words.\n"
+ ]
+ }
+ ],
+ "source": [
+ "# let's get the wikipedia article about Dune Part Two\n",
+ "article = wikipedia.page('Dune Part Two')\n",
+ "text = article.content\n",
+ "print(f\"The text has roughly {len(text.split())} words.\")"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "-1aJ7hKGgbOr"
+ },
+ "source": [
+ "## Step 1 - Indexing and given a user query, retrieve the relevant chunks from the index\n",
+ "\n",
+ "We index the document in a vector database. This requires getting the documents, chunking them, embedding, and indexing them in a vector database. Then we retrieved relevant results based on the users' query.\n",
+ "\n",
+ "### We split the document into chunks of roughly 512 words"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "colab": {
+ "base_uri": "https://localhost:8080/"
},
+ "id": "ZUph1JX41665",
+ "outputId": "6c63a93f-6999-47af-e704-d4a88727bc75"
+ },
+ "outputs": [
{
- "cell_type": "markdown",
- "metadata": {
- "id": "KuPL0VUXgbOt"
- },
- "source": [
- "## Step 3 - Generate the model final answer, given the retrieved and reranked chunks"
- ]
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m256.9/256.9 kB\u001b[0m \u001b[31m6.6 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
+ "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m66.6/66.6 kB\u001b[0m \u001b[31m8.2 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
+ "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m138.5/138.5 kB\u001b[0m \u001b[31m14.5 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
+ "\u001b[?25h"
+ ]
+ }
+ ],
+ "source": [
+ "# For chunking let's use langchain to help us split the text\n",
+ "%pip install -qU langchain-text-splitters --quiet\n",
+ "from langchain_text_splitters import RecursiveCharacterTextSplitter"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "colab": {
+ "base_uri": "https://localhost:8080/"
},
+ "id": "uhXW7iHC1-Q6",
+ "outputId": "d68ac348-4b73-4c6a-a445-6c510bdb0881"
+ },
+ "outputs": [
{
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "id": "oCNXWH8GgbOt"
- },
- "outputs": [],
- "source": [
- "# preamble containing instructions about the task and the desired style for the output.\n",
- "preamble = \"\"\"\n",
- "## Task & Context\n",
- "You help people answer their questions and other requests interactively. You will be asked a very wide array of requests on all kinds of topics. You will be equipped with a wide range of search engines or similar tools to help you, which you use to research your answer. You should focus on serving the user's needs as best you can, which will be wide-ranging.\n",
- "\n",
- "## Style Guide\n",
- "Unless the user asks for a different style of answer, you should answer in full sentences, using proper grammar and spelling.\n",
- "\"\"\""
- ]
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "The text has been broken down in 91 chunks.\n"
+ ]
+ }
+ ],
+ "source": [
+ "# Create basic configurations to chunk the text\n",
+ "text_splitter = RecursiveCharacterTextSplitter(\n",
+ " chunk_size=512,\n",
+ " chunk_overlap=50,\n",
+ " length_function=len,\n",
+ " is_separator_regex=False,\n",
+ ")\n",
+ "\n",
+ "# Split the text into chunks with some overlap\n",
+ "chunks_ = text_splitter.create_documents([text])\n",
+ "chunks = [c.page_content for c in chunks_]\n",
+ "print(f\"The text has been broken down in {len(chunks)} chunks.\")"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "P8g0sE2hgbOs"
+ },
+ "source": [
+ "### Embed every text chunk\n",
+ "\n",
+ "Cohere embeddings are state-of-the-art."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "colab": {
+ "base_uri": "https://localhost:8080/"
},
+ "id": "KEarMPEqgbOs",
+ "outputId": "7da0e06d-f637-4470-8e01-6de8249be64b"
+ },
+ "outputs": [
{
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "colab": {
- "base_uri": "https://localhost:8080/"
- },
- "id": "BevatShtgbOt",
- "outputId": "af71f4a9-787a-4ee3-9598-20692fb3bf16"
- },
- "outputs": [
- {
- "name": "stdout",
- "output_type": "stream",
- "text": [
- "Final answer:\n",
- "Here are the key crew members involved in 'Dune: Part Two':\n",
- "\n",
- "- Denis Villeneuve: director and producer\n",
- "- Jon Spaihts: co-writer of the screenplay\n",
- "- Mary Parent and Cale Boyter: producers \n",
- "- Tanya Lapointe, Brian Herbert, Byron Merritt, Kim Herbert, Thomas Tull, Richard P. Rubinstein, John Harrison, Herbert W. Gain and Kevin J. Anderson: executive producers \n",
- "- Joe Walker: editor\n",
- "- Brad Riker: supervising art director\n",
- "- Patrice Vermette: production designer\n",
- "- Paul Lambert: visual effects supervisor\n",
- "- Gerd Nefzer: special effects supervisor\n",
- "- Thomas Struthers: stunt coordinator. \n",
- "\n",
- "A number of crew members from the first film returned for the sequel, which is set to be released in 2024.\n"
- ]
- }
- ],
- "source": [
- "# retrieved documents\n",
- "documents = [\n",
- " {\"title\": \"chunk 0\", \"snippet\": top_chunks_after_rerank[0]},\n",
- " {\"title\": \"chunk 1\", \"snippet\": top_chunks_after_rerank[1]},\n",
- " {\"title\": \"chunk 2\", \"snippet\": top_chunks_after_rerank[2]},\n",
- " ]\n",
- "\n",
- "# get model response\n",
- "response = co.chat(\n",
- " message=query,\n",
- " documents=documents,\n",
- " preamble=preamble,\n",
- " model=\"command-r\",\n",
- " temperature=0.3\n",
- ")\n",
- "\n",
- "print(\"Final answer:\")\n",
- "print(response.text)"
- ]
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "We just computed 91 embeddings.\n"
+ ]
+ }
+ ],
+ "source": [
+ "# Because the texts being embedded are the chunks we are searching over, we set the input type as search_doc\n",
+ "model=\"embed-english-v3.0\"\n",
+ "response = co.embed(\n",
+ " texts= chunks,\n",
+ " model=model,\n",
+ " input_type=\"search_document\",\n",
+ " embedding_types=['float']\n",
+ ")\n",
+ "embeddings = response.embeddings.float\n",
+ "print(f\"We just computed {len(embeddings)} embeddings.\")"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "HM6vKeypgbOs"
+ },
+ "source": [
+ "### Store the embeddings in a vector database\n",
+ "\n",
+ "We use the simplest vector database ever: a python dictionary using `np.array()`."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "sdW7M8HLvB-9"
+ },
+ "outputs": [],
+ "source": [
+ "# We use the simplest vector database ever: a python dictionary\n",
+ "!pip install numpy --quiet"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "H2srFH-IgbOs"
+ },
+ "outputs": [],
+ "source": [
+ "import numpy as np\n",
+ "vector_database = {i: np.array(embedding) for i, embedding in enumerate(embeddings)}\n",
+ "# { 0: array([...]), 1: array([...]), 2: array([...]), ..., 10: array([...]) }"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "q6NGVurZgbOs"
+ },
+ "source": [
+ "## Given a user query, retrieve the relevant chunks from the vector database\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "eC05yJQ7jlek"
+ },
+ "source": [
+ "### Define the user question"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "Y2HTxspKgbOs"
+ },
+ "outputs": [],
+ "source": [
+ "query = \"Name everyone involved in writing the script, directing, and producing 'Dune: Part Two'?\"\n",
+ "\n",
+ "# Note: the relevant passage in the wikipedia page we're looking for is:\n",
+ "# \"[...] Dune: Part Two was originally scheduled to be released on October 20, 2023, but was delayed to November 17, 2023, before moving forward two weeks to November 3, 2023, to adjust to changes in release schedules from other studios. It was later postponed by over four months to March 15, 2024, due to the 2023 Hollywood labor disputes. After the strikes were resolved, the film moved once more up two weeks to March 1, 2024. [...]\""
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "9oULg1tOjjOW"
+ },
+ "source": [
+ "### Embed the user question\n",
+ "\n",
+ "Cohere embeddings are state-of-the-art."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "colab": {
+ "base_uri": "https://localhost:8080/"
},
+ "id": "yrUuS6vXgbOs",
+ "outputId": "0c64a930-f817-43c2-d775-1d9145cb304e"
+ },
+ "outputs": [
{
- "cell_type": "markdown",
- "metadata": {
- "id": "20wcn-EjlXZd"
- },
- "source": [
- "Note: this is indeed the answer you'd expect, and here was the passage of text in wikipedia explaining it!\n",
- "\n",
- "\" [...] Dune: Part Two was originally scheduled to be released on October 20, 2023, but was delayed to November 17, 2023, before moving forward two weeks to November 3, 2023, to adjust to changes in release schedules from other studios. It was later postponed by over four months to March 15, 2024, due to the 2023 Hollywood labor disputes. After the strikes were resolved, the film moved once more up two weeks to March 1, 2024. [...]\""
- ]
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "query_embedding: [-0.068603516, -0.02947998, -0.06274414, -0.015449524, -0.033294678, 0.0056877136, -0.047210693, 0.04714966, -0.024871826, 0.008148193, 0.0770874, 0.023880005, -0.058685303, -0.052520752, 0.012832642, 0.024398804, 0.0053215027, 0.035491943, 0.02961731, -0.0069847107, 0.01083374, -0.0011358261, -0.002199173, 0.018417358, 0.027389526, -0.002691269, -0.026535034, 0.015197754, 0.024368286, 0.03729248, 0.0057754517, -0.02229309, -0.014694214, 0.019989014, -0.0036315918, -0.013793945, 0.02835083, 0.006011963, 0.011428833, 0.008682251, 0.046142578, -0.040039062, -0.032196045, -0.002653122, -0.012580872, -0.0041618347, 0.03111267, -0.016799927, 0.014801025, -0.00030636787, -0.033050537, 0.033966064, -0.016021729, -0.025009155, -0.007534027, -0.017074585, 0.008415222, -0.10620117, 0.019195557, -0.015686035, -0.0043182373, -0.045440674, 0.05404663, 0.030776978, -0.014129639, -0.01499939, -0.007286072, 0.009933472, 0.06390381, 0.02444458, -0.010345459, 0.041931152, 0.032989502, -0.04522705, 0.056610107, 0.0068893433, -0.008911133, 0.012489319, 0.01675415, 0.020065308, 0.018753052, 0.022659302, -0.051849365, -0.04925537, 0.046325684, -0.005268097, 0.0026874542, -0.036712646, 0.009437561, -0.0037841797, -0.01473999, -0.034179688, -0.0011606216, 0.05026245, 0.0020771027, -0.016021729, -0.0044898987, 0.04168701, -0.015205383, 0.019210815, -0.012374878, -0.031311035, 0.03111267, -0.040100098, -0.016479492, 0.020446777, 0.010192871, 0.0037841797, -0.0023765564, 0.015220642, -0.016571045, -0.006454468, 0.037384033, -0.044555664, -0.008262634, 0.019546509, 0.009460449, 0.014701843, 0.02658081, -0.02078247, 0.015571594, 0.013153076, -0.010375977, 0.047912598, 0.005393982, -0.007911682, -0.019378662, 0.023529053, -0.0033550262, -0.04598999, -0.0052871704, 0.040252686, 0.011375427, 0.01550293, -0.004508972, 0.006515503, 0.003370285, -0.022766113, 0.00062561035, -0.0007596016, -0.0015277863, 0.0149002075, 0.061401367, 8.261204e-05, 0.06359863, -0.01537323, 0.007446289, 0.018814087, 0.02507019, 0.024215698, 0.006122589, 0.005886078, -0.03829956, 0.029037476, 0.07720947, 0.016921997, 0.022109985, 0.005958557, 0.028793335, 0.019485474, 0.015174866, 0.026153564, 0.032318115, 0.034210205, 0.027145386, -0.019515991, -0.018661499, 0.020477295, 0.008598328, -0.06573486, -0.037109375, 0.04043579, 0.030471802, -0.0010843277, 0.009757996, 0.026947021, 0.037017822, -0.018234253, -0.0115356445, 0.099365234, 0.027816772, -0.019927979, 0.0020961761, 0.013198853, -0.019073486, 2.7656555e-05, 0.041259766, 0.029510498, -0.016204834, 0.028137207, 0.039489746, 0.034698486, -0.03918457, -0.029418945, 0.02041626, 0.0073432922, -0.018569946, -0.009849548, 0.002861023, 0.030319214, -0.012886047, 0.014671326, -0.035827637, 0.007247925, -0.027709961, -0.022079468, 0.0012960434, 0.015426636, -0.01725769, 0.01525116, 0.025360107, -0.0077400208, -0.039916992, 0.029037476, -0.011154175, 0.007736206, -0.041748047, 0.05343628, 0.007286072, 0.0435791, 0.034301758, -0.047210693, 0.03552246, -0.015327454, 0.029922485, -0.018859863, 0.013053894, -0.028060913, 0.07757568, -0.020462036, 0.070739746, -0.010223389, 0.03604126, 0.02758789, -0.023284912, 0.012184143, 0.029144287, 0.023880005, -0.019378662, -0.0051116943, 0.0048675537, 0.01864624, -0.04397583, -0.007598877, 0.0713501, 0.0115737915, 0.002922058, 0.011619568, 0.017364502, 0.031921387, -0.0019664764, -0.008575439, 0.003484726, -0.09466553, 0.03475952, 0.026611328, -0.039520264, -0.0104522705, -0.005443573, -0.008392334, 0.012908936, 0.0043792725, -0.002456665, -0.028396606, -0.02027893, -0.0005569458, 0.027786255, 0.03427124, -0.0062332153, -0.018203735, 0.019241333, 0.07244873, -0.0028057098, 0.01234436, -0.0018787384, -0.027496338, 0.0015287399, -0.004032135, -0.013748169, -0.01878357, 0.0018053055, -0.01159668, 0.028213501, 0.004776001, 0.042388916, 0.0024280548, 0.017471313, -0.038085938, 0.026321411, 0.02973938, 0.06213379, 0.006401062, 0.036102295, -0.028121948, -0.00869751, -0.016693115, 0.029190063, 0.016784668, -0.008628845, 0.0039634705, -0.0035381317, 0.019500732, 0.025009155, -0.04547119, -0.003572464, 0.05215454, 0.067871094, -0.04257202, -0.02293396, -0.027175903, 0.05340576, 0.019226074, 0.039978027, 0.056121826, -0.028320312, -0.020217896, -0.035003662, 0.03225708, 0.028656006, 0.062347412, 0.12915039, -0.0137786865, 0.0022201538, -0.057434082, -0.04397583, -0.049865723, -0.013160706, -0.03353882, 0.006427765, -0.014823914, -0.008201599, -0.036346436, -0.037353516, -0.010528564, -0.015930176, -0.027572632, 0.0074272156, 0.004547119, -0.024414062, -0.018859863, -0.020095825, 0.029632568, -0.00067043304, -0.044036865, -0.0043411255, -0.005256653, -0.019195557, 0.022262573, -0.00020956993, -0.013877869, -0.011108398, -0.020324707, -0.015808105, -0.025039673, -0.009498596, 0.05090332, 0.0046195984, -0.017150879, 0.04309082, -0.029067993, 0.002670288, -0.00026249886, -0.032409668, -0.053100586, 0.012481689, -0.014633179, 0.0013475418, -0.034332275, 0.038330078, 0.014892578, -0.046936035, 0.021591187, -0.020385742, -0.0052604675, 0.02796936, 0.0014333725, 0.012077332, -0.0118255615, -0.005569458, 0.008491516, 0.009841919, 0.0031318665, -0.003408432, -0.007144928, 0.040374756, -0.0038928986, 0.005279541, -0.008415222, 0.031707764, 0.0140686035, -0.015029907, -0.02810669, -0.0078125, -0.030853271, -0.03201294, 0.021316528, -0.036193848, -0.0423584, 0.0072784424, 0.014801025, 0.0019607544, -0.012367249, -0.009056091, -0.021438599, -0.02645874, 0.038726807, -0.007549286, 0.0049591064, 0.019012451, 0.017791748, -0.009185791, 0.04006958, 0.003107071, -0.0075302124, -0.010375977, -0.009246826, -0.02130127, -0.0056762695, -0.0076789856, 0.010009766, -0.010536194, 0.041107178, 0.0021133423, 0.029891968, 0.01626587, 0.042236328, -0.02784729, -0.032836914, 0.0317688, 0.045715332, 0.000116825104, 0.028030396, 0.007205963, 0.012512207, -0.035583496, -0.048034668, -0.023529053, -0.04953003, 0.0345459, -0.048339844, -0.060272217, -0.004512787, 0.04425049, 0.0076141357, 0.029510498, 0.007396698, 0.003353119, -0.038726807, 0.07183838, -0.026901245, -0.023529053, -0.038085938, 0.068725586, 0.018096924, -0.013534546, 0.05883789, -0.016113281, 0.017944336, 0.041046143, 0.022918701, 0.036499023, 0.015296936, -0.04916382, 0.0075683594, -0.011390686, 0.009735107, -0.0070152283, 0.003129959, -0.032562256, 0.0003478527, -0.0036640167, -0.006893158, -0.016098022, -0.034332275, 0.037750244, -0.010269165, 0.016494751, -0.02394104, 0.03753662, -0.022644043, -0.0008234978, 0.001001358, -0.048217773, 0.04989624, 0.0078125, 0.0044937134, 0.027038574, 0.04736328, -0.02973938, -0.011726379, 0.01348114, 0.021408081, 0.00844574, -0.03741455, -0.015686035, -0.040893555, 0.001452446, -0.025405884, 0.07348633, 0.038238525, -0.019958496, 0.023071289, -0.016403198, -0.08105469, 0.0071029663, -0.019088745, 5.8174133e-05, -0.005569458, 0.01399231, 0.02255249, 0.011222839, 0.00028824806, 0.0066184998, 0.0017499924, -0.009864807, -0.0115737915, 0.053100586, 0.0065231323, 0.001865387, -0.026428223, 0.03692627, 0.025390625, 0.022613525, 0.018722534, 0.007675171, -0.03439331, 0.041625977, -0.01789856, -0.041046143, 0.0051460266, 0.04144287, 0.048553467, 0.054595947, -0.01108551, -0.033935547, -0.026275635, -0.0118255615, -0.021362305, -0.009841919, -0.00724411, 0.028900146, 0.009887695, -0.023803711, 0.016311646, 0.018798828, -0.03668213, 0.046844482, 0.010696411, -0.014717102, -0.008110046, -0.004589081, -0.0028076172, -0.050811768, -0.017196655, -0.03491211, 0.0074005127, -0.038909912, 0.032440186, -0.034362793, -0.008682251, 0.032928467, -0.04626465, -0.009666443, 0.018951416, 0.031951904, -0.003791809, 0.02015686, -0.05532837, -0.005683899, -0.00054216385, -0.0034332275, 0.008659363, 0.02130127, -0.038879395, -0.0033397675, -0.03866577, -0.0049934387, 0.017944336, 0.001496315, 0.019485474, -0.004348755, 0.00046491623, 0.0007157326, 0.035614014, -0.027694702, 0.03692627, -0.008491516, 0.0524292, -0.016662598, -0.0017795563, -0.021575928, -0.018753052, -0.049346924, -0.06652832, 0.04272461, 0.03186035, 0.0011978149, 0.03463745, 0.024002075, 0.02607727, 0.020446777, 0.0256958, 0.026855469, 0.0074005127, -0.067993164, 0.017944336, -0.0039482117, 0.05496216, -0.041412354, 0.014175415, 0.02444458, -0.026412964, 0.057403564, -0.026779175, 0.023254395, 0.03945923, 0.033569336, -0.030258179, -0.039093018, -0.036468506, 0.017105103, 0.009635925, 0.025497437, 0.04156494, -0.02571106, -0.0010414124, -0.005630493, -0.016448975, -0.026733398, 0.001326561, -0.042022705, 0.0012521744, -0.041259766, -0.12182617, -0.03857422, 0.12548828, -0.005947113, -0.020736694, -0.0033855438, 0.03778076, -0.033813477, 0.038970947, 0.003921509, 0.011810303, 0.031982422, -0.032562256, -0.002653122, -0.025009155, -0.03805542, -0.016998291, 0.018173218, 0.0158844, 0.0011739731, 0.048217773, -0.020401001, 0.044708252, -0.017318726, 0.014457703, -0.041809082, 0.010543823, 0.041931152, 0.076293945, -0.054779053, 0.060272217, -0.046936035, 0.02949524, 0.00554657, 0.041534424, -0.013046265, -0.056152344, 0.010406494, 0.02973938, -0.023727417, -0.022476196, -0.024734497, -0.013168335, 0.060424805, 0.011787415, 0.018997192, -0.043426514, -0.00077724457, -0.010154724, 0.017150879, -0.01171875, -0.022476196, 0.0034255981, -0.0026454926, 0.004837036, -0.0043296814, 0.02619934, -0.021560669, -0.039733887, -0.022415161, -0.06817627, -0.023223877, -0.018585205, -0.015319824, 0.012588501, 0.0064353943, -0.013748169, 0.043304443, 0.002626419, -0.029373169, -0.016784668, -0.026184082, 0.05847168, 0.034179688, 0.03842163, -0.05493164, -0.017486572, 0.016540527, 0.03164673, 0.089904785, 0.013534546, -0.07684326, -0.024108887, 0.07434082, 0.030395508, 0.007091522, 0.07373047, 0.012527466, -0.010856628, -0.01828003, -0.045196533, 0.00065279007, -0.0637207, 0.010726929, 0.023880005, -0.0030708313, -0.012298584, 0.027236938, -0.04928589, 0.023071289, 0.008674622, -0.023529053, -0.015838623, -0.010543823, 0.012168884, 0.014854431, -0.05834961, -0.06088257, -0.012313843, 0.035461426, 0.02027893, 0.019348145, -0.014602661, -0.02104187, -0.0309906, 0.001405716, -0.019973755, -0.00157547, -0.003944397, 0.0009326935, -0.02078247, -0.015731812, -0.044433594, 0.03390503, 0.057159424, 0.018585205, -0.023895264, -0.0057029724, 0.0049552917, 0.013412476, 0.022399902, 0.010154724, 0.0519104, 0.06591797, 0.018341064, 0.012161255, -0.05810547, -0.043304443, -0.031173706, 0.0023860931, -0.003944397, 0.11425781, -0.031036377, 0.019989014, -0.038635254, -0.025939941, 0.035064697, 0.041168213, 0.03161621, -0.069885254, -0.04537964, 0.028945923, -0.023162842, 0.019226074, -0.028442383, 0.015594482, -0.019256592, -0.0046463013, 0.034240723, 0.009124756, 0.05718994, 0.031219482, 0.02154541, 0.009590149, 0.00076818466, 0.04849243, -0.029129028, -0.03375244, -0.023391724, -0.028381348, -0.029708862, -0.0132369995, 0.010353088, 0.020263672, -0.030807495, 0.01007843, -0.03704834, 0.023376465, -0.03665161, 0.03741455, 0.015144348, 0.057281494, 0.03137207, 0.048431396, 0.021194458, 0.008110046, -0.03540039, -0.015312195, 0.022384644, 0.0065956116, 0.008056641, 0.0018348694, -0.009246826, 0.030380249, 0.0003862381, 0.0051841736, 0.04486084, 0.017807007, 0.0026130676, 0.07977295, 0.05419922, 0.062194824, 0.02633667, 0.024841309, -0.041625977, -0.005897522, 0.04031372, -0.055908203, 0.0026226044, -0.05340576, -0.05496216, 0.011474609, -0.006954193, -0.013122559, 0.019714355, -0.07159424, 0.031173706, 0.0034255981, -0.0034103394, 0.0440979, 0.011779785, -0.007827759, -0.03173828, -0.020950317, -0.030166626, -0.035308838, 0.030792236, 0.04525757, -0.028701782, -0.011100769, -0.02331543, -0.0357666, -0.025680542, 0.0011911392, 0.01940918, 0.05706787, 0.028381348, 0.007133484, -0.07733154, -0.007686615, 0.03869629, 0.0066833496, 0.008842468, 0.03439331, -0.014282227, 0.0357666, -0.004737854, -0.039794922, -0.0070381165, 0.02670288, 0.0107421875, 0.016189575, -0.06555176, -0.0138549805, 0.0008363724, -0.016693115, 0.006904602, -0.020263672, -0.030426025, 0.008453369, -0.046173096, -0.01802063, -0.013595581, -0.0044288635, -0.0039978027, -0.0044898987, 0.0007619858, 0.003921509, 0.0053977966, 0.020385742, -0.012329102, -0.023803711, -0.0057525635, 0.038330078, -0.014549255, -0.06298828, -0.047607422, 0.039245605, -0.06781006, -0.035217285, -0.009056091, 0.019927979, -0.003932953, -0.020309448, -0.017044067, 0.018127441, -8.624792e-05, -0.043182373, 0.009590149, 0.035308838, 0.031951904, 0.0011615753, -0.042022705, 0.079956055, 0.026687622, 0.013542175, -0.0074157715, -0.00983429, -0.0022563934, 0.07373047, 0.059387207, 0.03488159, 0.0071372986, -0.06427002, -0.0546875, -0.02482605, 0.11071777, -0.021072388, 0.01626587, -0.049713135, 0.061553955, -0.016860962, 0.051971436, -0.012962341, -0.0011711121, -0.014198303, -0.0061149597, -0.005836487, 0.00022387505, -0.027618408, 0.019836426, 0.009933472, 0.02368164, -0.020309448, -0.0049591064, -0.008628845, -0.03253174, -0.017684937, 0.02468872, -0.0023498535, 0.01448822, 0.061920166, 0.031707764, -0.0026416779, -0.040985107, -0.06335449, -0.036071777, 0.05404663, -0.0044136047, -0.0146102905, -0.0033416748, 0.028671265, -0.012771606, -0.0016565323, -0.0038909912, -0.02407837, -0.009857178, 0.0014467239, -0.008720398, -0.006011963, 0.032073975, -0.033325195, 0.014862061, -0.017227173, -0.018753052, -0.0060424805, 0.022567749, -0.017654419, -0.017562866, -0.07244873, -0.0881958, 0.050476074, 0.02609253, -0.032409668, 0.07458496, 0.009399414, 0.009117126, -0.031051636, -0.03451538, -0.004219055, -0.05718994, 0.020080566, -0.025421143, -0.010948181, 0.06341553, -0.009231567, -0.021697998, -0.009719849, 0.012802124, -0.020370483, 0.0034389496, 0.018859863, -0.025680542, 0.0013141632, 0.068603516, -0.021026611, 0.021881104, -0.0395813, -0.0019073486, 0.0056037903, -0.032348633]\n"
+ ]
+ }
+ ],
+ "source": [
+ "# Because the text being embedded is the search query, we set the input type as search_query\n",
+ "response = co.embed(\n",
+ " texts=[query],\n",
+ " model=model,\n",
+ " input_type=\"search_query\",\n",
+ " embedding_types=['float']\n",
+ ")\n",
+ "query_embedding = response.embeddings.float[0]\n",
+ "print(\"query_embedding: \", query_embedding)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "8K8B87CGgbOt"
+ },
+ "source": [
+ "### Retrieve the most relevant chunks from the vector database\n",
+ "\n",
+ "We use cosine similarity to find the most similar chunks"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "colab": {
+ "base_uri": "https://localhost:8080/"
},
+ "id": "nik3es32gbOt",
+ "outputId": "a1c30024-52e1-42c7-8836-a2c590559aca"
+ },
+ "outputs": [
{
- "cell_type": "markdown",
- "metadata": {
- "id": "RoSVDXSsgbOt"
- },
- "source": [
- "## Bonus: Citations come for free with Cohere! 🎉\n",
- "\n",
- "At Cohere, all RAG calls come with... precise citations! 🎉\n",
- "The model cites which groups of words, in the RAG chunks, were used to generate the final answer. \n",
- "These citations make it easy to check where the model’s generated response claims are coming from. \n",
- "They help users gain visibility into the model reasoning, and sanity check the final model generation. \n",
- "These citations are optional — you can decide to ignore them.\n"
- ]
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "similarity scores: [0.6953257882233425, 0.3713410510180273, 0.46501499776898964, 0.5448546916785195, 0.4014738351361969, 0.3231420292334584, 0.3179003053384008, 0.42799691553367775, 0.18882594531435784, 0.36868801306504106, 0.3404040737300553, 0.3852837621219358, 0.2600249419491577, 0.3723244353775111, 0.3631492691137214, 0.47574774051439606, 0.40415422750911745, 0.4149923346201023, 0.5014741934381444, 0.3549433331883204, 0.32072714802512714, 0.14770872479410424, 0.585277816615252, 0.6999636953772764, 0.7722295084104617, 0.4895347049465806, 0.5170096485954725, 0.7137817366881455, 0.5224900699612323, 0.5914632581598285, 0.2657897083381463, 0.6462342489537262, 0.6317222315431096, 0.5982303530756702, 0.5138265091630297, 0.41385121172723643, 0.4293941094100836, 0.4173182546482015, 0.42621236706314475, 0.4428474375355954, 0.35058541576139896, 0.3578709652019502, 0.3930157841938308, 0.3564608202848675, 0.23016661533167404, 0.4933441863421645, 0.41037089239250985, 0.39993051898770193, 0.3119997063424595, 0.2677143729521374, 0.3700866951454496, 0.46727994925061545, 0.4393343280374425, 0.42111290117172434, 0.4485349189824285, 0.4710573736688592, 0.24169956903740436, 0.3840442910806355, 0.14284631817675886, 0.5381588054138154, 0.431113882725076, 0.5189547209048608, 0.3950667224233914, 0.32429768756510174, 0.4370358125161736, 0.18727062244331039, 0.5206375682478743, 0.5175737635701252, 0.5326043981628349, 0.45586923626994363, 0.21667338125532032, 0.16459878595959285, 0.22236726481673777, 0.5187259906958807, 0.2884444442338396, 0.286407544555338, 0.2313840178160818, 0.2057731158935257, 0.5973876998341746, 0.42904243401792086, 0.4081217538000544, 0.5330523063972133, 0.45080561486977405, 0.414703452285757, 0.2569028899107211, 0.5087916806929323, 0.14159076456040554, 0.46505779053352697, 0.556364222182839, 0.35464351181035236, 0.40174477023626]\n",
+ "Here are the indices of the top 10 chunks after retrieval: [24 27 23 0 31 32 33 78 29 22]\n",
+ "Here are the top 10 chunks after retrieval: \n",
+ "== stunt coordinator. Dune: Part Two was produced by Villeneuve, Mary Parent, and Cale Boyter, with Tanya Lapointe, Brian Herbert, Byron Merritt, Kim Herbert, Thomas Tull, Jon Spaihts, Richard P. Rubinstein, John Harrison, and Herbert W. Gain serving as executive producers and Kevin J. Anderson as creative consultant. Legendary CEO Joshua Grode confirmed in April 2019 that they plan to make a sequel, adding that \"there's a logical place to stop the [first] movie before the book is over\".In December 2020,\n",
+ "== that.\"On October 26, 2021, Legendary officially greenlit Dune: Part Two, with a spokesperson for the company stating, \"We would not have gotten to this point without the extraordinary vision of Denis and the amazing work of his talented crew, the writers, our stellar cast, our partners at Warner Bros., and of course the fans! Here's to more Dune.\" Production work had occurred back-to-back with the first film, as Villeneuve and his wife Lapointe immediately took a flight to Budapest in order to begin\n",
+ "== series. Villeneuve ultimately secured a two-film deal with Warner Bros. Pictures, in the same style as the two-part adaption of Stephen King's It in 2017 and 2019. In January 2019, Joe Walker was confirmed to be serving as the film's editor. Other crew included Brad Riker as supervising art director, Patrice Vermette as production designer, Paul Lambert as visual effects supervisor, Gerd Nefzer as special effects supervisor, and Thomas Struthers as stunt coordinator. Dune: Part Two was produced by\n",
+ "== Dune: Part Two is a 2024 American epic science fiction film directed and produced by Denis Villeneuve, who co-wrote the screenplay with Jon Spaihts. The sequel to Dune (2021) adapts the 1965 novel Dune by Frank Herbert and follows Paul Atreides as he unites with the Fremen people of the desert planet Arrakis to wage war against House Harkonnen. Timothée Chalamet, Rebecca Ferguson, Josh Brolin, Stellan Skarsgård, Dave Bautista, Zendaya, Charlotte Rampling, and Javier Bardem reprise their roles from the first\n",
+ "== Eric Roth was hired to co-write the screenplay in April 2017 for the Dune films, and Jon Spaihts was later confirmed to be co-writing the script alongside Roth and Villeneuve. Game of Thrones language creator David Peterson was confirmed to be developing languages for the film in April 2019. Villeneuve and Peterson had created the Chakobsa language, which was used by actors on set. In November 2019, Spaihts stepped down as showrunner for Dune: Prophecy to focus on Dune: Part Two. In June 2020, Greig Fraser\n",
+ "== on Dune: Part Two. In June 2020, Greig Fraser said, \"It's a fully formed story in itself with places to go. It's a fully standalone epic film that people will get a lot out of when they see it\". Between the release of Dune and the confirmation of Dune: Part Two, Villeneuve started working the script in a way that production could begin immediately once the film was greenlit. By February 2021, Roth created a full treatment for the sequel, with writing beginning that August. He confirmed that Feyd-Rautha\n",
+ "== that August. He confirmed that Feyd-Rautha would appear in the film, and stated he will be a \"very important character\". In March 2022, Villeneuve had mostly finished writing the screenplay. Craig Mazin and Roth wrote additional literary material for the film.Villeneuve stated that the film would continue directly from the first, and specifically described it as being the \"second part.\" He described the film as being an \"epic war movie\", adding that while the first film was more \"contemplative\", the second\n",
+ "== On the review aggregator website Rotten Tomatoes, 93% of 378 critics' reviews are positive, with an average rating of 8.4/10. The website's consensus reads: \"Visually thrilling and narratively epic, Dune: Part Two continues Denis Villeneuve's adaptation of the beloved sci-fi series in spectacular form.\" Metacritic, which uses a weighted average, assigned the film a score of 79 out of 100, based on 62 critics, indicating \"generally favorable\" reviews. Audiences surveyed by CinemaScore gave the film an\n",
+ "== theatrical experience is at the very heart of the cinematic language for me.\" With Dune: Part Two being greenlit, Villeneuve said that his primary concern was to complete the filming as soon as possible, with the earliest he expected to start in the last quarter of 2022. However, he noted that production would be facilitated by the work already established on the first film, which would help expedite production.\n",
+ "== By November 2016, Legendary Pictures had obtained the film and TV rights for the Dune franchise, based on the eponymous 1965 novel by Frank Herbert. Vice chair of worldwide production for Legendary Mary Parent began discussing with Denis Villeneuve about directing a film adaptation, quickly hiring him after realizing his passion for Dune. By February 2018, Villeneuve was confirmed to be hired as director, and intended to adapt the novel as a two-part film series. Villeneuve ultimately secured a two-film\n"
+ ]
+ }
+ ],
+ "source": [
+ "def cosine_similarity(a, b):\n",
+ " return np.dot(a, b) / (np.linalg.norm(a) * np.linalg.norm(b))\n",
+ "\n",
+ "# Calculate similarity between the user question & each chunk\n",
+ "similarities = [cosine_similarity(query_embedding, chunk) for chunk in embeddings]\n",
+ "print(\"similarity scores: \", similarities)\n",
+ "\n",
+ "# Get indices of the top 10 most similar chunks\n",
+ "sorted_indices = np.argsort(similarities)[::-1]\n",
+ "\n",
+ "# Keep only the top 10 indices\n",
+ "top_indices = sorted_indices[:10]\n",
+ "print(\"Here are the indices of the top 10 chunks after retrieval: \", top_indices)\n",
+ "\n",
+ "# Retrieve the top 10 most similar chunks\n",
+ "top_chunks_after_retrieval = [chunks[i] for i in top_indices]\n",
+ "print(\"Here are the top 10 chunks after retrieval: \")\n",
+ "for t in top_chunks_after_retrieval:\n",
+ " print(\"== \" + t)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "qzcpds3VgbOt"
+ },
+ "source": [
+ "## Step 2 - Rerank the chunks retrieved from the vector database\n",
+ "\n",
+ "We rerank the 10 chunks retrieved from the vector database. Reranking boosts retrieval accuracy.\n",
+ "\n",
+ "Reranking lets us go from 10 chunks retrieved from the vector database, to the 3 most relevant chunks."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "colab": {
+ "base_uri": "https://localhost:8080/"
},
+ "id": "2J4LywVygbOt",
+ "outputId": "7a4c89bf-fc5e-409f-9304-fce006b9d8bf"
+ },
+ "outputs": [
{
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "colab": {
- "base_uri": "https://localhost:8080/"
- },
- "id": "BVTuQdmDgbOt",
- "outputId": "f843b262-d8bb-45ba-cbfb-9915da104eda"
- },
- "outputs": [
- {
- "name": "stdout",
- "output_type": "stream",
- "text": [
- "Citations that support the final answer:\n",
- "{'start': 63, 'end': 79, 'text': 'Denis Villeneuve', 'document_ids': ['doc_0']}\n",
- "{'start': 81, 'end': 102, 'text': 'director and producer', 'document_ids': ['doc_0']}\n",
- "{'start': 105, 'end': 116, 'text': 'Jon Spaihts', 'document_ids': ['doc_0']}\n",
- "{'start': 118, 'end': 145, 'text': 'co-writer of the screenplay', 'document_ids': ['doc_0']}\n",
- "{'start': 148, 'end': 159, 'text': 'Mary Parent', 'document_ids': ['doc_1']}\n",
- "{'start': 164, 'end': 175, 'text': 'Cale Boyter', 'document_ids': ['doc_1']}\n",
- "{'start': 177, 'end': 186, 'text': 'producers', 'document_ids': ['doc_1']}\n",
- "{'start': 190, 'end': 204, 'text': 'Tanya Lapointe', 'document_ids': ['doc_1']}\n",
- "{'start': 206, 'end': 219, 'text': 'Brian Herbert', 'document_ids': ['doc_1']}\n",
- "{'start': 221, 'end': 234, 'text': 'Byron Merritt', 'document_ids': ['doc_1']}\n",
- "{'start': 236, 'end': 247, 'text': 'Kim Herbert', 'document_ids': ['doc_1']}\n",
- "{'start': 249, 'end': 260, 'text': 'Thomas Tull', 'document_ids': ['doc_1']}\n",
- "{'start': 262, 'end': 283, 'text': 'Richard P. Rubinstein', 'document_ids': ['doc_1']}\n",
- "{'start': 285, 'end': 298, 'text': 'John Harrison', 'document_ids': ['doc_1']}\n",
- "{'start': 300, 'end': 315, 'text': 'Herbert W. Gain', 'document_ids': ['doc_1']}\n",
- "{'start': 320, 'end': 337, 'text': 'Kevin J. Anderson', 'document_ids': ['doc_1']}\n",
- "{'start': 339, 'end': 358, 'text': 'executive producers', 'document_ids': ['doc_1']}\n",
- "{'start': 362, 'end': 372, 'text': 'Joe Walker', 'document_ids': ['doc_2']}\n",
- "{'start': 374, 'end': 380, 'text': 'editor', 'document_ids': ['doc_2']}\n",
- "{'start': 383, 'end': 393, 'text': 'Brad Riker', 'document_ids': ['doc_2']}\n",
- "{'start': 395, 'end': 419, 'text': 'supervising art director', 'document_ids': ['doc_2']}\n",
- "{'start': 422, 'end': 438, 'text': 'Patrice Vermette', 'document_ids': ['doc_2']}\n",
- "{'start': 440, 'end': 459, 'text': 'production designer', 'document_ids': ['doc_2']}\n",
- "{'start': 462, 'end': 474, 'text': 'Paul Lambert', 'document_ids': ['doc_2']}\n",
- "{'start': 476, 'end': 501, 'text': 'visual effects supervisor', 'document_ids': ['doc_2']}\n",
- "{'start': 504, 'end': 515, 'text': 'Gerd Nefzer', 'document_ids': ['doc_2']}\n",
- "{'start': 517, 'end': 543, 'text': 'special effects supervisor', 'document_ids': ['doc_2']}\n",
- "{'start': 546, 'end': 562, 'text': 'Thomas Struthers', 'document_ids': ['doc_2']}\n",
- "{'start': 564, 'end': 582, 'text': 'stunt coordinator.', 'document_ids': ['doc_2']}\n",
- "{'start': 686, 'end': 691, 'text': '2024.', 'document_ids': ['doc_0']}\n"
- ]
- }
- ],
- "source": [
- "print(\"Citations that support the final answer:\")\n",
- "for cite in response.citations:\n",
- " print(cite)"
- ]
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "Here are the top 3 chunks after rerank: \n",
+ "== Dune: Part Two is a 2024 American epic science fiction film directed and produced by Denis Villeneuve, who co-wrote the screenplay with Jon Spaihts. The sequel to Dune (2021) adapts the 1965 novel Dune by Frank Herbert and follows Paul Atreides as he unites with the Fremen people of the desert planet Arrakis to wage war against House Harkonnen. Timothée Chalamet, Rebecca Ferguson, Josh Brolin, Stellan Skarsgård, Dave Bautista, Zendaya, Charlotte Rampling, and Javier Bardem reprise their roles from the first\n",
+ "== stunt coordinator. Dune: Part Two was produced by Villeneuve, Mary Parent, and Cale Boyter, with Tanya Lapointe, Brian Herbert, Byron Merritt, Kim Herbert, Thomas Tull, Jon Spaihts, Richard P. Rubinstein, John Harrison, and Herbert W. Gain serving as executive producers and Kevin J. Anderson as creative consultant. Legendary CEO Joshua Grode confirmed in April 2019 that they plan to make a sequel, adding that \"there's a logical place to stop the [first] movie before the book is over\".In December 2020,\n",
+ "== series. Villeneuve ultimately secured a two-film deal with Warner Bros. Pictures, in the same style as the two-part adaption of Stephen King's It in 2017 and 2019. In January 2019, Joe Walker was confirmed to be serving as the film's editor. Other crew included Brad Riker as supervising art director, Patrice Vermette as production designer, Paul Lambert as visual effects supervisor, Gerd Nefzer as special effects supervisor, and Thomas Struthers as stunt coordinator. Dune: Part Two was produced by\n"
+ ]
+ }
+ ],
+ "source": [
+ "response = co.rerank(\n",
+ " query=query,\n",
+ " documents=top_chunks_after_retrieval,\n",
+ " top_n=3,\n",
+ " model=\"rerank-english-v2.0\",\n",
+ ")\n",
+ "\n",
+ "top_chunks_after_rerank = [result.document['text'] for result in response]\n",
+ "print(\"Here are the top 3 chunks after rerank: \")\n",
+ "for t in top_chunks_after_rerank:\n",
+ " print(\"== \" + t)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "KuPL0VUXgbOt"
+ },
+ "source": [
+ "## Step 3 - Generate the model final answer, given the retrieved and reranked chunks"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "oCNXWH8GgbOt"
+ },
+ "outputs": [],
+ "source": [
+ "# preamble containing instructions about the task and the desired style for the output.\n",
+ "preamble = \"\"\"\n",
+ "## Task & Context\n",
+ "You help people answer their questions and other requests interactively. You will be asked a very wide array of requests on all kinds of topics. You will be equipped with a wide range of search engines or similar tools to help you, which you use to research your answer. You should focus on serving the user's needs as best you can, which will be wide-ranging.\n",
+ "\n",
+ "## Style Guide\n",
+ "Unless the user asks for a different style of answer, you should answer in full sentences, using proper grammar and spelling.\n",
+ "\"\"\""
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "colab": {
+ "base_uri": "https://localhost:8080/"
},
+ "id": "BevatShtgbOt",
+ "outputId": "af71f4a9-787a-4ee3-9598-20692fb3bf16"
+ },
+ "outputs": [
{
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "colab": {
- "base_uri": "https://localhost:8080/"
- },
- "id": "IueXaIJggbOu",
- "outputId": "c816af51-74be-42c9-e94e-9820bbf95f79"
- },
- "outputs": [
- {
- "name": "stdout",
- "output_type": "stream",
- "text": [
- "Here are the key crew members involved in 'Dune: Part Two':\n",
- "\n",
- "- **Denis Villeneuve**[1]: **director and producer**[1]\n",
- "- **Jon Spaihts**[1]: **co-writer of the screenplay**[1]\n",
- "- **Mary Parent**[2] and **Cale Boyter**[2]: **producers**[2] \n",
- "- **Tanya Lapointe**[2], **Brian Herbert**[2], **Byron Merritt**[2], **Kim Herbert**[2], **Thomas Tull**[2], **Richard P. Rubinstein**[2], **John Harrison**[2], **Herbert W. Gain**[2] and **Kevin J. Anderson**[2]: **executive producers**[2] \n",
- "- **Joe Walker**[3]: **editor**[3]\n",
- "- **Brad Riker**[3]: **supervising art director**[3]\n",
- "- **Patrice Vermette**[3]: **production designer**[3]\n",
- "- **Paul Lambert**[3]: **visual effects supervisor**[3]\n",
- "- **Gerd Nefzer**[3]: **special effects supervisor**[3]\n",
- "- **Thomas Struthers**[3]: **stunt coordinator.**[3] \n",
- "\n",
- "A number of crew members from the first film returned for the sequel, which is set to be released in **2024.**[1]\n",
- "\n",
- "[1] source: \"Dune: Part Two is a 2024 American epic science fiction film directed and produced by Denis Villeneuve, who co-wrote the screenplay with Jon Spaihts. The sequel to Dune (2021) adapts the 1965 novel Dune by Frank Herbert and follows Paul Atreides as he unites with the Fremen people of the desert planet Arrakis to wage war against House Harkonnen. Timothée Chalamet, Rebecca Ferguson, Josh Brolin, Stellan Skarsgård, Dave Bautista, Zendaya, Charlotte Rampling, and Javier Bardem reprise their roles from the first\"\n",
- "[2] source: \"stunt coordinator. Dune: Part Two was produced by Villeneuve, Mary Parent, and Cale Boyter, with Tanya Lapointe, Brian Herbert, Byron Merritt, Kim Herbert, Thomas Tull, Jon Spaihts, Richard P. Rubinstein, John Harrison, and Herbert W. Gain serving as executive producers and Kevin J. Anderson as creative consultant. Legendary CEO Joshua Grode confirmed in April 2019 that they plan to make a sequel, adding that \"there's a logical place to stop the [first] movie before the book is over\".In December 2020,\"\n",
- "[3] source: \"series. Villeneuve ultimately secured a two-film deal with Warner Bros. Pictures, in the same style as the two-part adaption of Stephen King's It in 2017 and 2019. In January 2019, Joe Walker was confirmed to be serving as the film's editor. Other crew included Brad Riker as supervising art director, Patrice Vermette as production designer, Paul Lambert as visual effects supervisor, Gerd Nefzer as special effects supervisor, and Thomas Struthers as stunt coordinator. Dune: Part Two was produced by\"\n"
- ]
- }
- ],
- "source": [
- "def insert_citations_in_order(text, citations):\n",
- " \"\"\"\n",
- " A helper function to pretty print citations.\n",
- " \"\"\"\n",
- " offset = 0\n",
- " document_id_to_number = {}\n",
- " citation_number = 0\n",
- " modified_citations = []\n",
- "\n",
- " # Process citations, assigning numbers based on unique document_ids\n",
- " for citation in citations:\n",
- " citation_numbers = []\n",
- " for document_id in sorted(citation[\"document_ids\"]):\n",
- " if document_id not in document_id_to_number:\n",
- " citation_number += 1 # Increment for a new document_id\n",
- " document_id_to_number[document_id] = citation_number\n",
- " citation_numbers.append(document_id_to_number[document_id])\n",
- "\n",
- " # Adjust start/end with offset\n",
- " start, end = citation['start'] + offset, citation['end'] + offset\n",
- " placeholder = ''.join([f'[{number}]' for number in citation_numbers])\n",
- " # Bold the cited text and append the placeholder\n",
- " modification = f'**{text[start:end]}**{placeholder}'\n",
- " # Replace the cited text with its bolded version + placeholder\n",
- " text = text[:start] + modification + text[end:]\n",
- " # Update the offset for subsequent replacements\n",
- " offset += len(modification) - (end - start)\n",
- "\n",
- " # Prepare citations for listing at the bottom, ensuring unique document_ids are listed once\n",
- " unique_citations = {number: doc_id for doc_id, number in document_id_to_number.items()}\n",
- " citation_list = '\\n'.join([f'[{doc_id}] source: \"{documents[doc_id - 1][\"snippet\"]}\"' for doc_id, number in sorted(unique_citations.items(), key=lambda item: item[1])])\n",
- " text_with_citations = f'{text}\\n\\n{citation_list}'\n",
- "\n",
- " return text_with_citations\n",
- "\n",
- "\n",
- "print(insert_citations_in_order(response.text, response.citations))\n"
- ]
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "Final answer:\n",
+ "Here are the key crew members involved in 'Dune: Part Two':\n",
+ "\n",
+ "- Denis Villeneuve: director and producer\n",
+ "- Jon Spaihts: co-writer of the screenplay\n",
+ "- Mary Parent and Cale Boyter: producers \n",
+ "- Tanya Lapointe, Brian Herbert, Byron Merritt, Kim Herbert, Thomas Tull, Richard P. Rubinstein, John Harrison, Herbert W. Gain and Kevin J. Anderson: executive producers \n",
+ "- Joe Walker: editor\n",
+ "- Brad Riker: supervising art director\n",
+ "- Patrice Vermette: production designer\n",
+ "- Paul Lambert: visual effects supervisor\n",
+ "- Gerd Nefzer: special effects supervisor\n",
+ "- Thomas Struthers: stunt coordinator. \n",
+ "\n",
+ "A number of crew members from the first film returned for the sequel, which is set to be released in 2024.\n"
+ ]
+ }
+ ],
+ "source": [
+ "# retrieved documents\n",
+ "documents = [\n",
+ " {\"title\": \"chunk 0\", \"snippet\": top_chunks_after_rerank[0]},\n",
+ " {\"title\": \"chunk 1\", \"snippet\": top_chunks_after_rerank[1]},\n",
+ " {\"title\": \"chunk 2\", \"snippet\": top_chunks_after_rerank[2]},\n",
+ " ]\n",
+ "\n",
+ "# get model response\n",
+ "response = co.chat(\n",
+ " message=query,\n",
+ " documents=documents,\n",
+ " preamble=preamble,\n",
+ " model=\"command-r\",\n",
+ " temperature=0.3\n",
+ ")\n",
+ "\n",
+ "print(\"Final answer:\")\n",
+ "print(response.text)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "20wcn-EjlXZd"
+ },
+ "source": [
+ "Note: this is indeed the answer you'd expect, and here was the passage of text in wikipedia explaining it!\n",
+ "\n",
+ "\" [...] Dune: Part Two was originally scheduled to be released on October 20, 2023, but was delayed to November 17, 2023, before moving forward two weeks to November 3, 2023, to adjust to changes in release schedules from other studios. It was later postponed by over four months to March 15, 2024, due to the 2023 Hollywood labor disputes. After the strikes were resolved, the film moved once more up two weeks to March 1, 2024. [...]\""
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "RoSVDXSsgbOt"
+ },
+ "source": [
+ "## Bonus: Citations come for free with Cohere! 🎉\n",
+ "\n",
+ "At Cohere, all RAG calls come with... precise citations! 🎉\n",
+ "The model cites which groups of words, in the RAG chunks, were used to generate the final answer. \n",
+ "These citations make it easy to check where the model’s generated response claims are coming from. \n",
+ "They help users gain visibility into the model reasoning, and sanity check the final model generation. \n",
+ "These citations are optional — you can decide to ignore them.\n"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "colab": {
+ "base_uri": "https://localhost:8080/"
},
+ "id": "BVTuQdmDgbOt",
+ "outputId": "f843b262-d8bb-45ba-cbfb-9915da104eda"
+ },
+ "outputs": [
{
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "id": "Kp4c_HkYIEn_"
- },
- "outputs": [],
- "source": []
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "Citations that support the final answer:\n",
+ "{'start': 63, 'end': 79, 'text': 'Denis Villeneuve', 'document_ids': ['doc_0']}\n",
+ "{'start': 81, 'end': 102, 'text': 'director and producer', 'document_ids': ['doc_0']}\n",
+ "{'start': 105, 'end': 116, 'text': 'Jon Spaihts', 'document_ids': ['doc_0']}\n",
+ "{'start': 118, 'end': 145, 'text': 'co-writer of the screenplay', 'document_ids': ['doc_0']}\n",
+ "{'start': 148, 'end': 159, 'text': 'Mary Parent', 'document_ids': ['doc_1']}\n",
+ "{'start': 164, 'end': 175, 'text': 'Cale Boyter', 'document_ids': ['doc_1']}\n",
+ "{'start': 177, 'end': 186, 'text': 'producers', 'document_ids': ['doc_1']}\n",
+ "{'start': 190, 'end': 204, 'text': 'Tanya Lapointe', 'document_ids': ['doc_1']}\n",
+ "{'start': 206, 'end': 219, 'text': 'Brian Herbert', 'document_ids': ['doc_1']}\n",
+ "{'start': 221, 'end': 234, 'text': 'Byron Merritt', 'document_ids': ['doc_1']}\n",
+ "{'start': 236, 'end': 247, 'text': 'Kim Herbert', 'document_ids': ['doc_1']}\n",
+ "{'start': 249, 'end': 260, 'text': 'Thomas Tull', 'document_ids': ['doc_1']}\n",
+ "{'start': 262, 'end': 283, 'text': 'Richard P. Rubinstein', 'document_ids': ['doc_1']}\n",
+ "{'start': 285, 'end': 298, 'text': 'John Harrison', 'document_ids': ['doc_1']}\n",
+ "{'start': 300, 'end': 315, 'text': 'Herbert W. Gain', 'document_ids': ['doc_1']}\n",
+ "{'start': 320, 'end': 337, 'text': 'Kevin J. Anderson', 'document_ids': ['doc_1']}\n",
+ "{'start': 339, 'end': 358, 'text': 'executive producers', 'document_ids': ['doc_1']}\n",
+ "{'start': 362, 'end': 372, 'text': 'Joe Walker', 'document_ids': ['doc_2']}\n",
+ "{'start': 374, 'end': 380, 'text': 'editor', 'document_ids': ['doc_2']}\n",
+ "{'start': 383, 'end': 393, 'text': 'Brad Riker', 'document_ids': ['doc_2']}\n",
+ "{'start': 395, 'end': 419, 'text': 'supervising art director', 'document_ids': ['doc_2']}\n",
+ "{'start': 422, 'end': 438, 'text': 'Patrice Vermette', 'document_ids': ['doc_2']}\n",
+ "{'start': 440, 'end': 459, 'text': 'production designer', 'document_ids': ['doc_2']}\n",
+ "{'start': 462, 'end': 474, 'text': 'Paul Lambert', 'document_ids': ['doc_2']}\n",
+ "{'start': 476, 'end': 501, 'text': 'visual effects supervisor', 'document_ids': ['doc_2']}\n",
+ "{'start': 504, 'end': 515, 'text': 'Gerd Nefzer', 'document_ids': ['doc_2']}\n",
+ "{'start': 517, 'end': 543, 'text': 'special effects supervisor', 'document_ids': ['doc_2']}\n",
+ "{'start': 546, 'end': 562, 'text': 'Thomas Struthers', 'document_ids': ['doc_2']}\n",
+ "{'start': 564, 'end': 582, 'text': 'stunt coordinator.', 'document_ids': ['doc_2']}\n",
+ "{'start': 686, 'end': 691, 'text': '2024.', 'document_ids': ['doc_0']}\n"
+ ]
}
- ],
- "metadata": {
+ ],
+ "source": [
+ "print(\"Citations that support the final answer:\")\n",
+ "for cite in response.citations:\n",
+ " print(cite)"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
"colab": {
- "provenance": []
- },
- "kernelspec": {
- "display_name": "hackathon_docs_3",
- "language": "python",
- "name": "python3"
+ "base_uri": "https://localhost:8080/"
},
- "language_info": {
- "codemirror_mode": {
- "name": "ipython",
- "version": 3
- },
- "file_extension": ".py",
- "mimetype": "text/x-python",
- "name": "python",
- "nbconvert_exporter": "python",
- "pygments_lexer": "ipython3",
- "version": "3.11.5"
+ "id": "IueXaIJggbOu",
+ "outputId": "c816af51-74be-42c9-e94e-9820bbf95f79"
+ },
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "Here are the key crew members involved in 'Dune: Part Two':\n",
+ "\n",
+ "- **Denis Villeneuve**[1]: **director and producer**[1]\n",
+ "- **Jon Spaihts**[1]: **co-writer of the screenplay**[1]\n",
+ "- **Mary Parent**[2] and **Cale Boyter**[2]: **producers**[2] \n",
+ "- **Tanya Lapointe**[2], **Brian Herbert**[2], **Byron Merritt**[2], **Kim Herbert**[2], **Thomas Tull**[2], **Richard P. Rubinstein**[2], **John Harrison**[2], **Herbert W. Gain**[2] and **Kevin J. Anderson**[2]: **executive producers**[2] \n",
+ "- **Joe Walker**[3]: **editor**[3]\n",
+ "- **Brad Riker**[3]: **supervising art director**[3]\n",
+ "- **Patrice Vermette**[3]: **production designer**[3]\n",
+ "- **Paul Lambert**[3]: **visual effects supervisor**[3]\n",
+ "- **Gerd Nefzer**[3]: **special effects supervisor**[3]\n",
+ "- **Thomas Struthers**[3]: **stunt coordinator.**[3] \n",
+ "\n",
+ "A number of crew members from the first film returned for the sequel, which is set to be released in **2024.**[1]\n",
+ "\n",
+ "[1] source: \"Dune: Part Two is a 2024 American epic science fiction film directed and produced by Denis Villeneuve, who co-wrote the screenplay with Jon Spaihts. The sequel to Dune (2021) adapts the 1965 novel Dune by Frank Herbert and follows Paul Atreides as he unites with the Fremen people of the desert planet Arrakis to wage war against House Harkonnen. Timothée Chalamet, Rebecca Ferguson, Josh Brolin, Stellan Skarsgård, Dave Bautista, Zendaya, Charlotte Rampling, and Javier Bardem reprise their roles from the first\"\n",
+ "[2] source: \"stunt coordinator. Dune: Part Two was produced by Villeneuve, Mary Parent, and Cale Boyter, with Tanya Lapointe, Brian Herbert, Byron Merritt, Kim Herbert, Thomas Tull, Jon Spaihts, Richard P. Rubinstein, John Harrison, and Herbert W. Gain serving as executive producers and Kevin J. Anderson as creative consultant. Legendary CEO Joshua Grode confirmed in April 2019 that they plan to make a sequel, adding that \"there's a logical place to stop the [first] movie before the book is over\".In December 2020,\"\n",
+ "[3] source: \"series. Villeneuve ultimately secured a two-film deal with Warner Bros. Pictures, in the same style as the two-part adaption of Stephen King's It in 2017 and 2019. In January 2019, Joe Walker was confirmed to be serving as the film's editor. Other crew included Brad Riker as supervising art director, Patrice Vermette as production designer, Paul Lambert as visual effects supervisor, Gerd Nefzer as special effects supervisor, and Thomas Struthers as stunt coordinator. Dune: Part Two was produced by\"\n"
+ ]
}
+ ],
+ "source": [
+ "def insert_citations_in_order(text, citations):\n",
+ " \"\"\"\n",
+ " A helper function to pretty print citations.\n",
+ " \"\"\"\n",
+ " offset = 0\n",
+ " document_id_to_number = {}\n",
+ " citation_number = 0\n",
+ " modified_citations = []\n",
+ "\n",
+ " # Process citations, assigning numbers based on unique document_ids\n",
+ " for citation in citations:\n",
+ " citation_numbers = []\n",
+ " for document_id in sorted(citation[\"document_ids\"]):\n",
+ " if document_id not in document_id_to_number:\n",
+ " citation_number += 1 # Increment for a new document_id\n",
+ " document_id_to_number[document_id] = citation_number\n",
+ " citation_numbers.append(document_id_to_number[document_id])\n",
+ "\n",
+ " # Adjust start/end with offset\n",
+ " start, end = citation['start'] + offset, citation['end'] + offset\n",
+ " placeholder = ''.join([f'[{number}]' for number in citation_numbers])\n",
+ " # Bold the cited text and append the placeholder\n",
+ " modification = f'**{text[start:end]}**{placeholder}'\n",
+ " # Replace the cited text with its bolded version + placeholder\n",
+ " text = text[:start] + modification + text[end:]\n",
+ " # Update the offset for subsequent replacements\n",
+ " offset += len(modification) - (end - start)\n",
+ "\n",
+ " # Prepare citations for listing at the bottom, ensuring unique document_ids are listed once\n",
+ " unique_citations = {number: doc_id for doc_id, number in document_id_to_number.items()}\n",
+ " citation_list = '\\n'.join([f'[{doc_id}] source: \"{documents[doc_id - 1][\"snippet\"]}\"' for doc_id, number in sorted(unique_citations.items(), key=lambda item: item[1])])\n",
+ " text_with_citations = f'{text}\\n\\n{citation_list}'\n",
+ "\n",
+ " return text_with_citations\n",
+ "\n",
+ "\n",
+ "print(insert_citations_in_order(response.text, response.citations))\n"
+ ]
+ }
+ ],
+ "metadata": {
+ "colab": {
+ "provenance": []
+ },
+ "kernelspec": {
+ "display_name": "Python 3 (ipykernel)",
+ "language": "python",
+ "name": "python3"
},
- "nbformat": 4,
- "nbformat_minor": 0
+ "language_info": {
+ "codemirror_mode": {
+ "name": "ipython",
+ "version": 3
+ },
+ "file_extension": ".py",
+ "mimetype": "text/x-python",
+ "name": "python",
+ "nbconvert_exporter": "python",
+ "pygments_lexer": "ipython3",
+ "version": "3.8.16"
+ }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 4
}
diff --git a/notebooks/Wikipedia_Semantic_Search_With_Cohere_Embeddings_Archives.ipynb b/notebooks/Wikipedia_Semantic_Search_With_Cohere_Embeddings_Archives.ipynb
index ad762fb3..d8cfbf9b 100644
--- a/notebooks/Wikipedia_Semantic_Search_With_Cohere_Embeddings_Archives.ipynb
+++ b/notebooks/Wikipedia_Semantic_Search_With_Cohere_Embeddings_Archives.ipynb
@@ -10,19 +10,6 @@
"This notebook contains the starter code to do simple [semantic search](https://txt.cohere.ai/what-is-semantic-search/) on the [Wikipedia embeddings archives](https://txt.cohere.ai/embedding-archives-wikipedia/) published by Cohere. These archives embed Wikipedia sites in multiple languages. In this example, we'll use [Wikipedia Simple English](https://huggingface.co/datasets/Cohere/wikipedia-22-12-simple-embeddings). "
]
},
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "id": "IUnwp2cYNnP0"
- },
- "outputs": [],
- "source": [
- "# Let's install \"cohere<5\" and HF datasets\n",
- "# TODO: upgrade to \"cohere>5\"",
-"!pip install \"cohere<5\" datasets"
- ]
- },
{
"cell_type": "markdown",
"metadata": {
@@ -207,7 +194,7 @@
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
- "version": "3.9.13"
+ "version": "3.8.16"
}
},
"nbformat": 4,
diff --git a/notebooks/agents/Data_Analyst_Agent_Cohere_and_Langchain.ipynb b/notebooks/agents/Data_Analyst_Agent_Cohere_and_Langchain.ipynb
index 3fcf1cfa..efeb595d 100644
--- a/notebooks/agents/Data_Analyst_Agent_Cohere_and_Langchain.ipynb
+++ b/notebooks/agents/Data_Analyst_Agent_Cohere_and_Langchain.ipynb
@@ -17,7 +17,7 @@
"source": [
"# Tool Use - Simple Data Analyst Agent with Cohere and Langchain\n",
"\n",
- "Tool use is a method whichs allows developers to connect Cohere's Command models to external tools like search engines, APIs, databases, and other software tools. Just like how [Retrieval-Augmented Generation (RAG)](https://docs.cohere.com/docs/retrieval-augmented-generation-rag) allows a model to use an external data source to improve factual generation, tool use is a capability that allows retrieving data from multiple sources. But it goes beyond simply retrieving information and is able to use software tools to execute code, or even create entries in a CRM system.\n",
+ "Tool use is a method which allows developers to connect Cohere's Command models to external tools like search engines, APIs, databases, and other software tools. Just like how [Retrieval-Augmented Generation (RAG)](https://docs.cohere.com/docs/retrieval-augmented-generation-rag) allows a model to use an external data source to improve factual generation, tool use is a capability that allows retrieving data from multiple sources. But it goes beyond simply retrieving information and is able to use software tools to execute code, or even create entries in a CRM system.\n",
"\n",
"In this notebook, we'll see how we can use two tools to create a simple data analyst agent that is able to search the web and run code in a python interpreter. This agent uses Cohere's Command R+ mode and Langchain.\n",
"\n",
@@ -92,7 +92,7 @@
},
"source": [
"### Web search\n",
- "Let's first equip our agent with web search! We can use the Tivaly API for this. Head on to [tavily.com](https://tavily.com) and grab an API key to use here."
+ "Let's first equip our agent with web search! We can use the Tavily API for this. Head on to [tavily.com](https://tavily.com) and grab an API key to use here."
]
},
{
@@ -390,7 +390,7 @@
"id": "wZBaxCYeG4Xf"
},
"source": [
- "This is a simple example to get you start. There [are many tools](https://python.langchain.com/docs/integrations/tools/) you can equip your agent with. Once youre comfortable with these ideas, you can also proceed to define your tools (see [Multi-step tool use in Action](https://docs.cohere.com/docs/multi-step-tool-use#multi-step-tool-use-in-action)).\n"
+ "This is a simple example to get you started. There [are many tools](https://python.langchain.com/docs/integrations/tools/) you can equip your agent with. Once you're comfortable with these ideas, you can also proceed to define your tools (see [Multi-step tool use in Action](https://docs.cohere.com/docs/multi-step-tool-use#multi-step-tool-use-in-action)).\n"
]
}
],
@@ -413,7 +413,7 @@
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
- "version": "3.9.13"
+ "version": "3.8.16"
}
},
"nbformat": 4,
diff --git a/notebooks/agents/Financial_Analyst_Agent_with_Custom_API_Tool.ipynb b/notebooks/agents/Financial_Analyst_Agent_with_Custom_API_Tool.ipynb
new file mode 100644
index 00000000..c497a747
--- /dev/null
+++ b/notebooks/agents/Financial_Analyst_Agent_with_Custom_API_Tool.ipynb
@@ -0,0 +1,669 @@
+{
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "EUYMLmXAF3W1"
+ },
+ "source": [
+ "# A ReAct Agent with Command R+, Integrating a Custom API Tool for Financial Analysis\n",
+ "\n",
+ "Adaptive RAG is a strategy for RAG that unites (1) [query analysis](https://blog.langchain.dev/query-construction/) with (2) [active / self-corrective RAG](https://blog.langchain.dev/agentic-rag-with-langgraph/).\n",
+ "\n",
+ "In the paper, they report query analysis to route across:\n",
+ "- No Retrieval (LLM answers)\n",
+ "- Single-shot RAG\n",
+ "- Iterative RAG\n",
+ "\n",
+ "\n",
+ "We'll use Command R+, a recent release from Cohere that:\n",
+ "- Has strong accuracy on RAG, Tool Use and Agents\n",
+ "- Has 128k context\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "W8yvOB2MQp07"
+ },
+ "source": [
+ "# Environment"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 1,
+ "metadata": {
+ "colab": {
+ "base_uri": "https://localhost:8080/"
+ },
+ "id": "ujQVUvA9QlD4",
+ "outputId": "230c6a56-3ee3-4e0d-95af-2a261b8541d6"
+ },
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "Note: you may need to restart the kernel to use updated packages.\n"
+ ]
+ }
+ ],
+ "source": [
+ "%pip install --quiet langchain langchain_cohere tiktoken faiss-cpu beautifulsoup4 langchain_experimental matplotlib tabulate python-dotenv"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Save your credentials in a .env file in your user root directory, so that you can retrieve securely"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 2,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "import os\n",
+ "from dotenv import load_dotenv\n",
+ "load_dotenv()\n",
+ "\n",
+ "COHERE_API_KEY = os.environ.get(\"COHERE_API_KEY\")\n",
+ "TAVILY_API_KEY = os.environ.get(\"TAVILY_API_KEY\") # Get your Free API key at https://app.tavily.com once you sign up\n",
+ "FMP_API_KEY = os.environ.get(\"FMP_API_KEY\") # Get your Free API key https://site.financialmodelingprep.com/ once you sign up"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "k3mrKxSsTfXO"
+ },
+ "source": [
+ "# Create tools\n",
+ "The ReAct agent will be equipped with these tools. The model can pick between\n",
+ "- web search\n",
+ "- RAG: retrieval from a vector store\n",
+ "- custom tool (call an external API)\n",
+ "- directly answering\n",
+ "\n",
+ "The model can use any of these tools, in any sequence of steps, and self-reflect."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "4kGUB4IRF3W5"
+ },
+ "source": [
+ "### Web search tool"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 3,
+ "metadata": {
+ "id": "hYsJgVY0F3W5"
+ },
+ "outputs": [],
+ "source": [
+ "from langchain_community.tools.tavily_search import TavilySearchResults\n",
+ "from langchain_core.pydantic_v1 import BaseModel, Field\n",
+ "\n",
+ "internet_search = TavilySearchResults()\n",
+ "internet_search.name = \"internet_search\"\n",
+ "internet_search.description = \"Returns a list of relevant document snippets for a textual query retrieved from the internet.\"\n",
+ "\n",
+ "\n",
+ "class TavilySearchInput(BaseModel):\n",
+ " query: str = Field(description=\"Query to search the internet with\")\n",
+ "\n",
+ "\n",
+ "internet_search.args_schema = TavilySearchInput"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### Python interpreter tool"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 4,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "from langchain.agents import Tool\n",
+ "from langchain_experimental.utilities import PythonREPL\n",
+ "\n",
+ "python_repl = PythonREPL()\n",
+ "repl_tool = Tool(\n",
+ " name=\"python_repl\",\n",
+ " description=\"Executes python code and returns the result. The code runs in a static sandbox without interactive mode, so print output or save output to a file.\",\n",
+ " func=python_repl.run,\n",
+ ")\n",
+ "repl_tool.name = \"python_interpreter\"\n",
+ "\n",
+ "# from langchain_core.pydantic_v1 import BaseModel, Field\n",
+ "class ToolInput(BaseModel):\n",
+ " code: str = Field(description=\"Python code to execute.\")\n",
+ "repl_tool.args_schema = ToolInput"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "NxDa_Em0tuas"
+ },
+ "source": [
+ "### RAG tool"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 5,
+ "metadata": {
+ "id": "0QyahrNPtwEi"
+ },
+ "outputs": [],
+ "source": [
+ "from langchain.text_splitter import RecursiveCharacterTextSplitter\n",
+ "from langchain_cohere import CohereEmbeddings\n",
+ "from langchain_community.document_loaders import WebBaseLoader\n",
+ "from langchain_community.vectorstores import FAISS\n",
+ "\n",
+ "# Set embeddings\n",
+ "embd = CohereEmbeddings()\n",
+ "\n",
+ "# Docs to index\n",
+ "urls = [\n",
+ " \"https://www.mayerbrown.com/en/insights/publications/2023/03/new-data-standards-pending-for-federally-regulated-financial-entities\",\n",
+ " \"https://plaid.com/resources/open-finance/what-is-fdx/\",\n",
+ " \"https://www.egnyte.com/guides/financial-services/financial-data-compliance\",\n",
+ "]\n",
+ "\n",
+ "# Load\n",
+ "docs = [WebBaseLoader(url).load() for url in urls]\n",
+ "docs_list = [item for sublist in docs for item in sublist]\n",
+ "\n",
+ "# Split\n",
+ "text_splitter = RecursiveCharacterTextSplitter.from_tiktoken_encoder(\n",
+ " chunk_size=512, chunk_overlap=0\n",
+ ")\n",
+ "doc_splits = text_splitter.split_documents(docs_list)\n",
+ "\n",
+ "# Add to vectorstore\n",
+ "vectorstore = FAISS.from_documents(\n",
+ " documents=doc_splits,\n",
+ " embedding=embd,\n",
+ ")\n",
+ "\n",
+ "vectorstore_retriever = vectorstore.as_retriever()"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 6,
+ "metadata": {
+ "id": "eoa3toXfdGkI"
+ },
+ "outputs": [],
+ "source": [
+ "from langchain.tools.retriever import create_retriever_tool\n",
+ "\n",
+ "vectorstore_search = create_retriever_tool(\n",
+ " retriever=vectorstore_retriever,\n",
+ " name=\"vectorstore_search\",\n",
+ " description=\"Retrieve relevant info from a vectorstore that contains documents related to Data Compliance for Financial Services and its regulation.\",\n",
+ ")"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### Custom Tool For Market Capitalization\n",
+ "\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "1. Define the function"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 7,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "import requests\n",
+ "\n",
+ "# Function to get the market capitalization of a ticker symbol\n",
+ "def get_market_cap(ticker):\n",
+ " url = f\"https://financialmodelingprep.com/api/v3/market-capitalization/{ticker}?apikey={FMP_API_KEY}\"\n",
+ " response = requests.get(url)\n",
+ " if response.status_code == 200:\n",
+ " data = response.json()\n",
+ " if data and isinstance(data, list) and len(data) > 0:\n",
+ " market_cap = data[0].get('marketCap', 'No market cap data available')\n",
+ " return [{'id': 0, 'text': f'Market cap for {ticker}: ${market_cap}'}]\n",
+ " else:\n",
+ " return \"No data available for the specified ticker.\"\n",
+ " else:\n",
+ " return \"Failed to retrieve data from the API.\""
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "2. Define the Custom Tool"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 8,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "from langchain.tools import tool\n",
+ "\n",
+ "@tool\n",
+ "def market_cap(ticker: str) -> list:\n",
+ " \"\"\"This tool is only used to find market capitalization. Do not use it for anything else. Find the ticker when asked about capitalization. The ticker symbol of the company (e.g., AAPL which is the ticker for Apple Inc, MSFT for Microsoft )\"\"\"\n",
+ " return get_market_cap(ticker)\n",
+ " "
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "2ENQAfYXRG9Q"
+ },
+ "source": [
+ "# Create the ReAct Agent\n",
+ "The model can smartly pick the right tool(s) for the user query, call them in any sequence of steps, analyze the results and self-reflect."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 9,
+ "metadata": {
+ "id": "hx-Ew1i5F3W4"
+ },
+ "outputs": [],
+ "source": [
+ "from langchain.agents import AgentExecutor\n",
+ "from langchain_cohere.react_multi_hop.agent import create_cohere_react_agent\n",
+ "from langchain_core.prompts import ChatPromptTemplate"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 10,
+ "metadata": {
+ "id": "BPo3ZIHkF3W4"
+ },
+ "outputs": [],
+ "source": [
+ "# LLM\n",
+ "from langchain_cohere.chat_models import ChatCohere\n",
+ "\n",
+ "chat = ChatCohere(model=\"command-r-plus\", temperature=0.3)\n",
+ "\n",
+ "# Preamble\n",
+ "preamble = \"\"\"\n",
+ "Use all tools that are available to answear the question. \n",
+ "If the query covers the topics of Federal Financial Institutions, use the vectorstore search first.\n",
+ "If the query covers market capitalization, use the market cap tool first.\n",
+ "You are equipped with an internet search tool, and python interpreter, a market cap api, and a special vectorstore of information about Data Compliance for Financial Services.\n",
+ "\n",
+ "\"\"\"\n",
+ "\n",
+ "# Prompt\n",
+ "prompt = ChatPromptTemplate.from_template(\"{input}\")\n",
+ "\n",
+ "# Create the ReAct agent\n",
+ "agent = create_cohere_react_agent(\n",
+ " llm=chat,\n",
+ " tools=[vectorstore_search, internet_search, repl_tool, market_cap],\n",
+ " prompt=prompt,\n",
+ ")"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 11,
+ "metadata": {
+ "id": "dks-7TGGtdLE"
+ },
+ "outputs": [],
+ "source": [
+ "agent_executor = AgentExecutor(\n",
+ " agent=agent, tools=[vectorstore_search, internet_search, repl_tool, market_cap], verbose=True\n",
+ ")"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "v7L1aBHds1pj"
+ },
+ "source": [
+ "# Testing the ReAct agent"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "ibo31dsHEjPv"
+ },
+ "source": [
+ "**Let's ask a question that requires web search.**"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 12,
+ "metadata": {
+ "colab": {
+ "base_uri": "https://localhost:8080/"
+ },
+ "id": "AW_8C7mszsVE",
+ "outputId": "9be7067f-a776-4688-8864-db873d1fb65a"
+ },
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "\n",
+ "\n",
+ "\u001b[1m> Entering new AgentExecutor chain...\u001b[0m\n",
+ "\u001b[32;1m\u001b[1;3m\n",
+ "I will search for how the current interest rate environment affects the bond market.\n",
+ "{'tool_name': 'internet_search', 'parameters': {'query': 'How does the current interest rate environment affect the bond market?'}}\n",
+ "\u001b[0m\u001b[33;1m\u001b[1;3m[{'url': 'https://www.morningstar.com/bonds/how-do-interest-rates-affect-your-bonds', 'content': \"The federal-funds rate, the interest rate at which banks lend money to each other overnight, is now targeted between 1.75% and 2.00%. When the Fed raises or lowers rates, it affects bonds' prices ...\"}, {'url': 'https://www.usbank.com/investing/financial-perspectives/market-news/interest-rates-affect-bonds.html', 'content': 'Personal\\nBank accounts\\nCredit cards\\nInvesting and retirement\\nPersonal loans & lines\\nHome loans\\nVehicle loans\\nMobile and online\\nExplore personal banking\\nWealth Management\\nOur services\\nInvesting\\nYour goals\\nAdvisors & wealth teams\\nPrivate Wealth Management\\nOur perspectives\\nExplore Wealth Management\\nBusiness\\nExplore business banking\\nBusiness bank accounts\\nBusiness credit cards\\nBusiness loans and lines\\nBusiness services\\nBusiness payments\\nBusiness industry expertise\\nExplore business resources\\nOnline & mobile banking\\nCorporate & Commercial\\nExplore Even after the recent yield decline, year-to-date total returns reflect a gain of 4.2% according to the Bloomberg U.S. Aggregate Bond Index through mid-December 2023.3\\nInverted yield curve persists\\nA phenomenon that developed in 2022 and continues in 2023 is the unusual shape of the yield curve representing different bond maturities. The Fed sought to slow the economy as a way to reduce inflation, which peaked at 9.1% for the 12 months ending June 2022, but dropped to less than 3.1% by November 2023.2\\nWhat should investors expect from the bond market in 2024 and what does that say about how to incorporate or adjust strategies for fixed-income portfolios?\\n Article\\nHow changing interest rates impact the bond market\\nDecember 18, 2023\\nWebinar\\n2024 Investment Outlook – Jan. 24\\nExploring the impact of today’s interest rates and the 2024 presidential election on your investments.\\n In fact, the economy generated solid growth as measured by Gross Domestic Product (GDP) and surprised many forecasters by expanding at an annualized rate of 5.2% in the third quarter of 2023.4\\nKeeping an eye on the Fed\\nThe Fed remains focused on fighting inflation but may be about to modify its interest rate policy.'}, {'url': 'https://investor.vanguard.com/investor-resources-education/news/the-dynamics-of-bond-duration-and-rising-rates', 'content': \"When the investment horizon is longer than the bond's duration, however, higher yields on reinvested cash flow outweigh the market price decline. Over a period of 15, 20, or 25 years, interest rate rises of 100 and 200 basis points result in an improvement in total returns. (The inverse is true for total returns when interest rates decline.\"}, {'url': 'https://www.investopedia.com/articles/bonds/09/bond-market-interest-rates.asp', 'content': \"Interest rate risk is the risk of changes in a bond's price due to changes in prevailing interest rates. Changes in short-term versus long-term interest rates can affect various bonds in different ...\"}, {'url': 'https://investor.vanguard.com/investor-resources-education/article/how-rising-interest-rates-affect-bond-funds', 'content': \"Most of the time it's lower because SEC yield reflects fund fees.2\\xa0Given these dynamics, it’s helpful to look to the SEC yield as a reasonable forward-looking picture of the current income of the fund, based on current market value.\\n… and distribution yield\\nThen there’s a bond fund’s distribution yield, which is important because it reflects what the fund’s monthly distributions are.3\\nThe challenge is that VGSH's distribution yield as of May 31, 2022, was 0.57%—well below both the 2.53% that individual 2-year notes were yielding and the 2.53% SEC yield as of that date. (The average effective maturity and the average duration of VGSH were 1.9 years and 2.0\\nYield measure\\nVGSH SEC yield\\nOctober 2021\\n0.34%\\nMay 2022\\n2.53%\\nImpact of rising rates on yield metrics\\nChanges in the SEC yield for VGSH typically follow the YTM because of the nature of the calculation. Yield measure\\nVGSH YTM\\nOctober 2021\\n0.49%\\nMay 2022\\n2.48%\\nImpact of rising rates on yield metrics\\nThe yield to maturity (YTM) for VGSH closely tracks the 2-year Treasury yield given the proximity of the fund’s overall composition to the 2-year key rate. Yield measure\\nVGSH distribution yield\\nOctober 2021\\n0.28%\\nMay 2022\\n0.57%\\nImpact of rising rates on yield metrics\\nThe yield measure that lags most, the ETF’s distribution yield, hasn’t yet caught up with the rise in rates. More broadly, the Bloomberg U.S. Aggregate Float Adjusted Bond Index fell by as much as 12.7% year-to-date through June 14, 2022—the most it’s lost in this short a time in 40 years.1\\nTreasury yield curve change from year-end 2021 to end of 2022\\nNote:\\xa0Data includes the rate hike in December 2022 and subsequent market reaction.\\n\"}]\u001b[0m\u001b[32;1m\u001b[1;3mRelevant Documents: 0,1,2,3,4\n",
+ "Cited Documents: 0,1,2,3,4\n",
+ "Answer: Interest rates have a direct impact on bond prices. When interest rates rise, bond prices fall, and vice versa. The federal-funds rate, the interest rate at which banks lend money to each other overnight, is now targeted between 1.75% and 2.00%. The Fed sought to slow the economy as a way to reduce inflation, which peaked at 9.1% for the 12 months ending June 2022, but dropped to less than 3.1% by November 2023.\n",
+ "Grounded answer: Interest rates have a direct impact on bond prices. When interest rates rise, bond prices fall, and vice versa. The federal-funds rate, the interest rate at which banks lend money to each other overnight, is now targeted between 1.75% and 2.00%. The Fed sought to slow the economy as a way to reduce inflation, which peaked at 9.1% for the 12 months ending June 2022, but dropped to less than 3.1% by November 2023.\u001b[0m\n",
+ "\n",
+ "\u001b[1m> Finished chain.\u001b[0m\n"
+ ]
+ }
+ ],
+ "source": [
+ "result = agent_executor.invoke(\n",
+ " {\n",
+ " \"input\": \"How does the current interest rate environment affect the bond market?\",\n",
+ " \"preamble\": preamble,\n",
+ " }\n",
+ ")"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 13,
+ "metadata": {},
+ "outputs": [
+ {
+ "data": {
+ "text/plain": [
+ "'Interest rates have a direct impact on bond prices. When interest rates rise, bond prices fall, and vice versa. The federal-funds rate, the interest rate at which banks lend money to each other overnight, is now targeted between 1.75% and 2.00%. The Fed sought to slow the economy as a way to reduce inflation, which peaked at 9.1% for the 12 months ending June 2022, but dropped to less than 3.1% by November 2023.'"
+ ]
+ },
+ "execution_count": 13,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "result[\"output\"]"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "lKkxSvJmEn8Z"
+ },
+ "source": [
+ "**Let's ask a question that requires RAG retrieval from the vector db.**"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 14,
+ "metadata": {
+ "colab": {
+ "base_uri": "https://localhost:8080/"
+ },
+ "id": "Jl0JNX2TF3W5",
+ "outputId": "2ccb747c-e702-4473-f587-47271e0c71b2"
+ },
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "\n",
+ "\n",
+ "\u001b[1m> Entering new AgentExecutor chain...\u001b[0m\n",
+ "\u001b[32;1m\u001b[1;3m\n",
+ "I will search for the primary functions of the Federal Financial Institutions Examination Council (FFIEC) in the vector store.\n",
+ "{'tool_name': 'vectorstore_search', 'parameters': {'query': 'primary functions of the Federal Financial Institutions Examination Council (FFIEC)?'}}\n",
+ "\u001b[0m\u001b[36;1m\u001b[1;3mlosses and improve overall cybersecurity by implementing best practices and employing proven cybersecurity frameworks. The importance of financial data compliance continues to grow as regulations continue to expand globally to ensure that customers’ personal information and finances are protected from theft and fraud.Advantages of Financial Data Compliance and Cost of Non-ComplianceBenefits of financial data complianceNot following financial data complianceView of the most critical data and systemsPlan for cybersecurity tools and practicesProtection of valuable informationRapid response to cybersecurity incidentsOperational disruptionsReputational damageCivil and criminal lawsuitsFines and losses due to cybersecurity incidentsTop Five Data Compliances for Financial ServicesFive Most Important United States Drivers for Financial Data Compliance1. Commodities Future Trading Commission (CFTC)The CFTC oversees the U.S. derivatives markets, including futures contracts, options, and over-the-counter (OTC) markets. While the rules that the CFTC has oversight of were part of the Commodities Exchange Act (CEA), which was passed in 1936, the Commission itself was established in 1974 by the passage of the Commodities Futures Trading Commission Act. The mission of the CFTC is to enforce laws, rules, and regulations within the derivatives markets it oversees to help ensure that firms and individuals operating in such markets provide legally required privacy rights to their consumers and promote the integrity, resilience, and vibrancy of the U.S. derivatives markets through sound regulation.2. Federal Deposit Insurance Corporation (FDIC)The FDIC provides deposit insurance of at least $250,000 for accounts with banks and thrift institutions. The FDIC aims to preserve and promote the public’s confidence in the U.S. financial system by providing deposit insurance. The FDIC is also responsible for enforcing financial data compliance as part of its mandate to ensure that banks comply with consumer data protection laws, including the Gramm-Leach-Bliley Act, which includes data security directives.3. Federal ReserveThe Federal Reserve Banking System provides guidance on managing outsourcing risk, including data for all financial institutions supervised by the Federal Reserve. It is also responsible for supervising, monitoring, inspecting, and examining institutions’ financial data compliance to ensure that they adhere to the necessary rules and regulations, including those published by the Federal Financial Institutions Examination Council (FFIEC) and Section 501 of the Gramm-Leach-Bliley Act (GLBA), regarding safeguards for customer data.4. Federal Financial Institutions Examination Council (FFIEC)The Federal Financial Institutions Examination\n",
+ "\n",
+ "Council is a five-member U.S. government interagency body. It consists of five banking regulators and is responsible for uniformity in the guidelines and supervision of financial institutions, including financial data compliance. It includes financial data management mandates, such as directing how organizations should govern the secure storage of all types of sensitive information, whether on computer systems, physical media, or hard-copy documents.5. Financial Industry Regulatory Authority (FINRA) SEC Rules 17a-4The Securities and Exchange Commission (SEC) requires broker-dealers to adhere to numerous financial data compliance regulations, including SEC Rule 17a-4(f)(3)(vii), which includes strict mandates on how electronic data must be stored, how it can be accessed, and how and when the archiving process should be reviewed. Several additional U.S. drivers for financial data compliance include the following:Dodd-Frank Wall Street Reform and Consumer Protection Act (Dodd-Frank)The Dodd-Frank Act was enacted in 2010 to facilitate financial stability by augmenting transparency and accountability. Financial data compliance requirements include specific directives related to email. To adhere to Dodd-Frank financial data compliance requirements related to communication, financial organizations must take steps to preserve email communication within proscribed periods of time with redundancy and other procedures to ensure that it is protected. Office for the Comptroller of the Currency (OCC) The Office for the Comptroller of the Currency (OCC) is an independent bureau of the U.S. Department of the Treasury that charters, regulates, and supervises all national banks and federal savings associations, as well as federal branches and agencies of foreign banks. As part of its charter to protect consumers, the OCC enforces federal data compliance regulations, such as FFIEC, GLBA, and the Privacy Act of 1974.Securities Exchange Commission (SEC)The Securities Exchange Commission (SEC) protects investors by ensuring the maintenance of fair, orderly, and efficient markets and facilitating capital formation. The SEC Cybersecurity Rule 2023 requires that market entities (e.g., broker-dealers, national securities exchanges) establish, maintain, and enforce written policies and procedures that are reasonably designed to address their cybersecurity risks. With regard to financial data compliance, the SEC requires any data breach that compromises the confidentiality, integrity, or availability of an information asset.Sarbanes-Oxley Act (SOX)SOX focuses on how organizations disclose and record their financial information. Financial data compliance provisions dictate that organizations must install cybersecurity\n",
+ "\n",
+ "PartnersMSP ProgramTECHNOLOGY PARTNERSGoogle WorkspaceMicrosoft 365SalesforceAll Apps and IntegrationsDeveloper's Toolkit Login Start Free Trial Get DemoResourcesPRODUCT HELPUniversityHelp DeskADDITIONAL RESOURCESBlogResource CenterGuidesEventsCase Studies Login Start Free Trial Get DemoCompanyCOMPANYAbout UsCareersContact Us Login Start Free Trial Get DemoSolutionsUse CasesPartnersResourcesCompanyPricing Login Start Free Trial Get Demo Submitted by on Tue, 11/14/2023 - 06:23 Home> Guides> Financial Services> Data Compliance for Financial ServicesHome > Data Compliance for Financial ServicesData Compliance for Financial ServicesShare this PageLet’s jump in and learn:What Is Financial Compliance?Top Five Data Compliances for Financial ServicesExamples of the United Kingdom (U.K.) and European Union (E.U.) Drivers for Financial Data ComplianceWhy are Compliance Laws Important?Limitations of Compliance LawsTips for Complying with Data Security RegulationsFinancial Data Compliance Improves Overall SecurityWhat Is Financial Compliance?Financial compliance is financial institutions’ (e.g., financial services organizations, capital markets) adherence to internal and external rules set forth by governments, industry groups, and internal governance policies. Financial data compliance is a governance structure focused on data protection that ensures that an organization complies with laws, regulations, and standards around its data. Financial data compliance includes processes and security tools to govern and secure the possession, organization, storage, and management of digital assets or data to prevent loss, theft, misuse, or compromise. Different financial data compliance regulations specify the types of data that require protection and, in some cases, direct what should be done to ensure its security.The growing stringency of financial data compliance requirements also pushes organizations to rethink updating legacy security systems.Origin of Financial ComplianceThe term was first used by the Basel Committee on Banking Supervision (BCBS, the primary global standard setter for the prudential regulation of banks) in 2005. In its document “Compliance and the compliance function in banks,” the body defines compliance, “The expression compliance risk is defined in this paper as the risk of legal or regulatory sanctions, material financial loss, or loss to reputation a bank may suffer as a result of its failure to comply with laws, regulations, rules, related self-regulatory organization standards, and codes of conduct applicable to its banking activities.”Financial data compliance helps organizations prevent legal issues and financial\n",
+ "\n",
+ "to protect financial data.Sections 302 and 304 of SOX set standards for data protection. While SOX does not specify any specific controls to protect financial data, encryption is widely considered to be the best practice.SWIFT CSPSociety for Worldwide Interbank Financial Telecommunication (SWIFT) is a Belgian cooperative society providing services related to the execution of financial transactions and payments between banks worldwide. Financial organizations that use SWIFT services must comply with SWIFT Customer Security Program (CSP) requirements. The framework outlines requirements for data protection, access management, and incident response. The SWIFT Customer Security Controls Framework has three objectives with eight principles.1. Secure your environmentRestrict internet accessSegregate critical systems from general IT environmentsReduce attack surface and vulnerabilitiesPhysically secure the environment2. Know and limit access Prevent compromise of credentials Manage identities and segregate privilege3. Detect and respondDetect anomalous activity in systems or transactional recordsPlan for incident response and information sharingExamples of the United Kingdom (U.K.) and European Union (E.U.) Drivers for Financial Data ComplianceGeneral Data Protection Regulation (GDPR) GDPR is the primary law regulating how organizations protect E.U. citizens’ personal data, setting forth stringent financial data compliance requirements. Several of the financial data requirements for GDPR that impact financial institutions are:Individuals must give explicit consent for their personal data to be collected and used.Individuals must understand how their information is going to be used.Organizations must clearly stipulate the legal channels available should data processing not comply with its agreed-upon use.All personal data must be destroyed after a prescribed period of time.In the event of a serious cyberattack, organizations must inform all those affected by the security breach and the Information Commissioner’s Office within 72 hours.Markets in Financial Instruments Directive II (MiFID II)The primary objective of MiFID II is to enhance investor protection with increased transparency and reporting, enhanced governance rules, and heightened regulation of markets.Related to financial data compliance, MiFID II, banks, and financial firms must implement common data processes and data quality metrics, which requires the adoption of data standards to ensure consistency of reporting across all regulated activities. Financial institutions must also manage multiple identifiers, including industry standard Market Identifier Codes (MIC) and the Global Legal Entity Identifier (LEI) to meet financial data compliance requirements.Why are Compliance Laws Important?Financial\u001b[0m\u001b[32;1m\u001b[1;3mRelevant Documents: 0\n",
+ "Cited Documents: 0\n",
+ "Answer: The Federal Financial Institutions Examination Council (FFIEC) is a five-member U.S. government interagency body, consisting of five banking regulators. It is responsible for uniformity in the guidelines and supervision of financial institutions, including financial data compliance. It includes financial data management mandates, such as directing how organisations should govern the secure storage of all types of sensitive information, whether on computer systems, physical media, or hard-copy documents.\n",
+ "Grounded answer: The Federal Financial Institutions Examination Council (FFIEC) is a five-member U.S. government interagency body, consisting of five banking regulators. It is responsible for uniformity in the guidelines and supervision of financial institutions, including financial data compliance. It includes financial data management mandates, such as directing how organisations should govern the secure storage of all types of sensitive information, whether on computer systems, physical media, or hard-copy documents.\u001b[0m\n",
+ "\n",
+ "\u001b[1m> Finished chain.\u001b[0m\n"
+ ]
+ }
+ ],
+ "source": [
+ "result = agent_executor.invoke(\n",
+ " {\n",
+ " \"input\": \"What are the primary functions of the Federal Financial Institutions Examination Council (FFIEC)?\",\n",
+ " \"preamble\": preamble,\n",
+ " }\n",
+ ")"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 15,
+ "metadata": {},
+ "outputs": [
+ {
+ "data": {
+ "text/plain": [
+ "'The Federal Financial Institutions Examination Council (FFIEC) is a five-member U.S. government interagency body, consisting of five banking regulators. It is responsible for uniformity in the guidelines and supervision of financial institutions, including financial data compliance. It includes financial data management mandates, such as directing how organisations should govern the secure storage of all types of sensitive information, whether on computer systems, physical media, or hard-copy documents.'"
+ ]
+ },
+ "execution_count": 15,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "result[\"output\"]"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "**Let's ask a question that requires the custom Market Cap tool.**"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 16,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "\n",
+ "\n",
+ "\u001b[1m> Entering new AgentExecutor chain...\u001b[0m\n",
+ "\u001b[32;1m\u001b[1;3m\n",
+ "I will use the market_cap tool to find the market capitalization of Apple and Microsoft. Then, I will use Python to calculate the difference between the two.\n",
+ "{'tool_name': 'market_cap', 'parameters': {'ticker': 'AAPL'}}\n",
+ "\u001b[0m\u001b[36;1m\u001b[1;3m[{'id': 0, 'text': 'Market cap for AAPL: $2927279690000'}]\u001b[0m\u001b[32;1m\u001b[1;3m\n",
+ "{'tool_name': 'market_cap', 'parameters': {'ticker': 'MSFT'}}\n",
+ "\u001b[0m\u001b[36;1m\u001b[1;3m[{'id': 0, 'text': 'Market cap for MSFT: $3023872485050'}]\u001b[0m"
+ ]
+ },
+ {
+ "name": "stderr",
+ "output_type": "stream",
+ "text": [
+ "Python REPL can execute arbitrary code. Use with caution.\n"
+ ]
+ },
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "\u001b[32;1m\u001b[1;3m\n",
+ "The market capitalization of Apple is $2,927,279,690,000 and the market capitalization of Microsoft is $3,023,872,485,050. I will now use Python to calculate the difference between the two.\n",
+ "{'tool_name': 'python_interpreter', 'parameters': {'code': 'apple_market_cap = 2927279690000\\nmicrosoft_market_cap = 3023872485050\\n\\ndifference = abs(microsoft_market_cap - apple_market_cap)\\n\\nprint(f\"The difference in market capitalization between Apple and Microsoft is {difference} dollars.\")'}}\n",
+ "\u001b[0m\u001b[38;5;200m\u001b[1;3mThe difference in market capitalization between Apple and Microsoft is 96592795050 dollars.\n",
+ "\u001b[0m\u001b[32;1m\u001b[1;3mRelevant Documents: 0,1,2\n",
+ "Cited Documents: 0,1,2\n",
+ "Answer: The market capitalization of Apple is $2,927,279,690,000 and the market capitalization of Microsoft is $3,023,872,485,050. The difference in market capitalization between the two companies is $96,592,795,050.\n",
+ "Grounded answer: The market capitalization of Apple is $2,927,279,690,000 and the market capitalization of Microsoft is $3,023,872,485,050. The difference in market capitalization between the two companies is $96,592,795,050.\u001b[0m\n",
+ "\n",
+ "\u001b[1m> Finished chain.\u001b[0m\n"
+ ]
+ }
+ ],
+ "source": [
+ "result = agent_executor.invoke(\n",
+ " {\n",
+ " \"input\": \"What is Apple capitalization compared to Microsoft? Calculate the difference\",\n",
+ " \"preamble\": preamble,\n",
+ " }\n",
+ ")"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 17,
+ "metadata": {},
+ "outputs": [
+ {
+ "data": {
+ "text/plain": [
+ "'The market capitalization of Apple is $2,927,279,690,000 and the market capitalization of Microsoft is $3,023,872,485,050. The difference in market capitalization between the two companies is $96,592,795,050.'"
+ ]
+ },
+ "execution_count": 17,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "result[\"output\"]"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "**Let's ask a question that does not require any tools.**"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 18,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "\n",
+ "\n",
+ "\u001b[1m> Entering new AgentExecutor chain...\u001b[0m\n",
+ "\u001b[32;1m\u001b[1;3m\n",
+ "I will respond to the user's greeting.\n",
+ "{'tool_name': 'directly_answer', 'parameters': {}}\n",
+ "\u001b[0mdirectly_answer is not a valid tool, try one of [vectorstore_search, internet_search, python_interpreter, market_cap].\u001b[32;1m\u001b[1;3mRelevant Documents: None\n",
+ "Cited Documents: None\n",
+ "Answer: Hi! How can I help?\n",
+ "Grounded answer: Hi! How can I help?\u001b[0m\n",
+ "\n",
+ "\u001b[1m> Finished chain.\u001b[0m\n"
+ ]
+ }
+ ],
+ "source": [
+ "result = agent_executor.invoke(\n",
+ " {\n",
+ " \"input\": \"Hi there!\",\n",
+ " \"preamble\": preamble,\n",
+ " }\n",
+ ")"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 19,
+ "metadata": {},
+ "outputs": [
+ {
+ "data": {
+ "text/plain": [
+ "'Hi! How can I help?'"
+ ]
+ },
+ "execution_count": 19,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "result[\"output\"]"
+ ]
+ }
+ ],
+ "metadata": {
+ "colab": {
+ "provenance": []
+ },
+ "kernelspec": {
+ "display_name": "Python 3 (ipykernel)",
+ "language": "python",
+ "name": "python3"
+ },
+ "language_info": {
+ "codemirror_mode": {
+ "name": "ipython",
+ "version": 3
+ },
+ "file_extension": ".py",
+ "mimetype": "text/x-python",
+ "name": "python",
+ "nbconvert_exporter": "python",
+ "pygments_lexer": "ipython3",
+ "version": "3.11.9"
+ }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 4
+}
diff --git a/notebooks/agents/Multi_Step_Tool_Use.ipynb b/notebooks/agents/Multi_Step_Tool_Use.ipynb
deleted file mode 100644
index 01454c4c..00000000
--- a/notebooks/agents/Multi_Step_Tool_Use.ipynb
+++ /dev/null
@@ -1,515 +0,0 @@
-{
- "cells": [
- {
- "cell_type": "markdown",
- "metadata": {
- "id": "EUYMLmXAF3W1"
- },
- "source": [
- "# A ReAct agent with Command R+, achieving the same goals as Adaptive RAG | Add the custom tools\n",
- "\n",
- "Adaptive RAG is a strategy for RAG that unites (1) [query analysis](https://blog.langchain.dev/query-construction/) with (2) [active / self-corrective RAG](https://blog.langchain.dev/agentic-rag-with-langgraph/).\n",
- "\n",
- "In the paper, they report query analysis to route across:\n",
- "- No Retrieval (LLM answers)\n",
- "- Single-shot RAG\n",
- "- Iterative RAG\n",
- "\n",
- "\n",
- "We'll use Command R+, a recent release from Cohere that:\n",
- "- Has strong accuracy on RAG, Tool Use and Agents\n",
- "- Has 128k context\n"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {
- "id": "W8yvOB2MQp07"
- },
- "source": [
- "# Environment"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "colab": {
- "base_uri": "https://localhost:8080/"
- },
- "id": "ujQVUvA9QlD4",
- "outputId": "230c6a56-3ee3-4e0d-95af-2a261b8541d6"
- },
- "outputs": [],
- "source": [
- "%pip install --quiet langchain langchain_cohere tiktoken faiss-cpu beautifulsoup4 langchain_experimental matplotlib tabulate python-dotenv"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "Save your credentials in a .env file in your user root directory, so that you can retrieve securely"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {},
- "outputs": [],
- "source": [
- "import os\n",
- "from dotenv import load_dotenv\n",
- "load_dotenv()\n",
- "\n",
- "COHERE_API_KEY = os.environ.get(\"COHERE_API_KEY\")\n",
- "TAVILY_API_KEY = os.environ.get(\"TAVILY_API_KEY\") # Get your Free API key at https://app.tavily.com once you sign up\n",
- "FMP_API_KEY = os.environ.get(\"FMP_API_KEY\") # Get your Free API key https://site.financialmodelingprep.com/ once you sign up"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {
- "id": "k3mrKxSsTfXO"
- },
- "source": [
- "# Create tools\n",
- "The ReAct agent will be equipped with these tools. The model can pick between\n",
- "- web search\n",
- "- RAG: retrieval from a vector store\n",
- "- custom tool (call an external API)\n",
- "- directly answering\n",
- "\n",
- "The model can use any of these tools, in any sequence of steps, and self-reflect."
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {
- "id": "4kGUB4IRF3W5"
- },
- "source": [
- "### Web search tool"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "id": "hYsJgVY0F3W5"
- },
- "outputs": [],
- "source": [
- "from langchain_community.tools.tavily_search import TavilySearchResults\n",
- "from langchain_core.pydantic_v1 import BaseModel, Field\n",
- "\n",
- "internet_search = TavilySearchResults()\n",
- "internet_search.name = \"internet_search\"\n",
- "internet_search.description = \"Returns a list of relevant document snippets for a textual query retrieved from the internet.\"\n",
- "\n",
- "\n",
- "class TavilySearchInput(BaseModel):\n",
- " query: str = Field(description=\"Query to search the internet with\")\n",
- "\n",
- "\n",
- "internet_search.args_schema = TavilySearchInput"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "### Python interpreter tool"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {},
- "outputs": [],
- "source": [
- "from langchain.agents import Tool\n",
- "from langchain_experimental.utilities import PythonREPL\n",
- "\n",
- "python_repl = PythonREPL()\n",
- "repl_tool = Tool(\n",
- " name=\"python_repl\",\n",
- " description=\"Executes python code and returns the result. The code runs in a static sandbox without interactive mode, so print output or save output to a file.\",\n",
- " func=python_repl.run,\n",
- ")\n",
- "repl_tool.name = \"python_interpreter\"\n",
- "\n",
- "# from langchain_core.pydantic_v1 import BaseModel, Field\n",
- "class ToolInput(BaseModel):\n",
- " code: str = Field(description=\"Python code to execute.\")\n",
- "repl_tool.args_schema = ToolInput"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {
- "id": "NxDa_Em0tuas"
- },
- "source": [
- "### RAG tool"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "id": "0QyahrNPtwEi"
- },
- "outputs": [],
- "source": [
- "from langchain.text_splitter import RecursiveCharacterTextSplitter\n",
- "from langchain_cohere import CohereEmbeddings\n",
- "from langchain_community.document_loaders import WebBaseLoader\n",
- "from langchain_community.vectorstores import FAISS\n",
- "\n",
- "# Set embeddings\n",
- "embd = CohereEmbeddings()\n",
- "\n",
- "# Docs to index\n",
- "urls = [\n",
- " \"https://www.mayerbrown.com/en/insights/publications/2023/03/new-data-standards-pending-for-federally-regulated-financial-entities\",\n",
- " \"https://plaid.com/resources/open-finance/what-is-fdx/\",\n",
- " \"https://www.egnyte.com/guides/financial-services/financial-data-compliance\",\n",
- "]\n",
- "\n",
- "# Load\n",
- "docs = [WebBaseLoader(url).load() for url in urls]\n",
- "docs_list = [item for sublist in docs for item in sublist]\n",
- "\n",
- "# Split\n",
- "text_splitter = RecursiveCharacterTextSplitter.from_tiktoken_encoder(\n",
- " chunk_size=512, chunk_overlap=0\n",
- ")\n",
- "doc_splits = text_splitter.split_documents(docs_list)\n",
- "\n",
- "# Add to vectorstore\n",
- "vectorstore = FAISS.from_documents(\n",
- " documents=doc_splits,\n",
- " embedding=embd,\n",
- ")\n",
- "\n",
- "vectorstore_retriever = vectorstore.as_retriever()"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "id": "eoa3toXfdGkI"
- },
- "outputs": [],
- "source": [
- "from langchain.tools.retriever import create_retriever_tool\n",
- "\n",
- "vectorstore_search = create_retriever_tool(\n",
- " retriever=vectorstore_retriever,\n",
- " name=\"vectorstore_search\",\n",
- " description=\"Retrieve relevant info from a vectorstore that contains documents related to Data Compliance for Financial Services and its regulation.\",\n",
- ")"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "### Custom Tool For Market Capitalization\n",
- "\n"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "1. Define the function"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {},
- "outputs": [],
- "source": [
- "import requests\n",
- "\n",
- "# Function to get the market capitalization of a ticker symbol\n",
- "def get_market_cap(ticker):\n",
- " url = f\"https://financialmodelingprep.com/api/v3/market-capitalization/{ticker}?apikey={FMP_API_KEY}\"\n",
- " response = requests.get(url)\n",
- " if response.status_code == 200:\n",
- " data = response.json()\n",
- " if data and isinstance(data, list) and len(data) > 0:\n",
- " market_cap = data[0].get('marketCap', 'No market cap data available')\n",
- " return [{'id': 0, 'text': f'Market cap for {ticker}: ${market_cap}'}]\n",
- " else:\n",
- " return \"No data available for the specified ticker.\"\n",
- " else:\n",
- " return \"Failed to retrieve data from the API.\""
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "2. Define the Custom Tool"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {},
- "outputs": [],
- "source": [
- "from langchain.tools import tool\n",
- "\n",
- "@tool\n",
- "def market_cap(ticker: str) -> list:\n",
- " \"\"\"This tool is only used to find market capitalization. Do not use it for anything else. Find the ticker when asked about capitalization. The ticker symbol of the company (e.g., AAPL which is the ticker for Apple Inc, MSFT for Microsoft )\"\"\"\n",
- " return get_market_cap(ticker)\n",
- " "
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {
- "id": "2ENQAfYXRG9Q"
- },
- "source": [
- "# Create the ReAct Agent\n",
- "The model can smartly pick the right tool(s) for the user query, call them in any sequence of steps, analyze the results and self-reflect."
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "id": "hx-Ew1i5F3W4"
- },
- "outputs": [],
- "source": [
- "from langchain.agents import AgentExecutor\n",
- "from langchain_cohere.react_multi_hop.agent import create_cohere_react_agent\n",
- "from langchain_core.prompts import ChatPromptTemplate"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "id": "BPo3ZIHkF3W4"
- },
- "outputs": [],
- "source": [
- "# LLM\n",
- "from langchain_cohere.chat_models import ChatCohere\n",
- "\n",
- "chat = ChatCohere(model=\"command-r-plus\", temperature=0.3)\n",
- "\n",
- "# Preamble\n",
- "preamble = \"\"\"\n",
- "Use all tools that are available to answear the question. \n",
- "If the query covers the topics of Federal Financial Institutions, use the vectorstore search first.\n",
- "If the query covers market capitalization, use the market cap tool first.\n",
- "You are equipped with an internet search tool, and python interpreter, a market cap api, and a special vectorstore of information about Data Compliance for Financial Services.\n",
- "\n",
- "\"\"\"\n",
- "\n",
- "# Prompt\n",
- "prompt = ChatPromptTemplate.from_template(\"{input}\")\n",
- "\n",
- "# Create the ReAct agent\n",
- "agent = create_cohere_react_agent(\n",
- " llm=chat,\n",
- " tools=[vectorstore_search, internet_search, repl_tool, market_cap],\n",
- " prompt=prompt,\n",
- ")"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "id": "dks-7TGGtdLE"
- },
- "outputs": [],
- "source": [
- "agent_executor = AgentExecutor(\n",
- " agent=agent, tools=[vectorstore_search, internet_search, repl_tool, market_cap], verbose=True\n",
- ")"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {
- "id": "v7L1aBHds1pj"
- },
- "source": [
- "# Testing the ReAct agent"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {
- "id": "ibo31dsHEjPv"
- },
- "source": [
- "**Let's ask a question that requires web search.**"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "colab": {
- "base_uri": "https://localhost:8080/"
- },
- "id": "AW_8C7mszsVE",
- "outputId": "9be7067f-a776-4688-8864-db873d1fb65a"
- },
- "outputs": [],
- "source": [
- "result = agent_executor.invoke(\n",
- " {\n",
- " \"input\": \"How does the current interest rate environment affect the bond market?\",\n",
- " \"preamble\": preamble,\n",
- " }\n",
- ")"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {},
- "outputs": [],
- "source": [
- "result[\"output\"]"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {
- "id": "lKkxSvJmEn8Z"
- },
- "source": [
- "**Let's ask a question that requires RAG retrieval from the vector db.**"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "colab": {
- "base_uri": "https://localhost:8080/"
- },
- "id": "Jl0JNX2TF3W5",
- "outputId": "2ccb747c-e702-4473-f587-47271e0c71b2"
- },
- "outputs": [],
- "source": [
- "result = agent_executor.invoke(\n",
- " {\n",
- " \"input\": \"What are the primary functions of the Federal Financial Institutions Examination Council (FFIEC)?\",\n",
- " \"preamble\": preamble,\n",
- " }\n",
- ")"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {},
- "outputs": [],
- "source": [
- "result[\"output\"]"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "**Let's ask a question that requires Market Cap.**"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {},
- "outputs": [],
- "source": [
- "result = agent_executor.invoke(\n",
- " {\n",
- " \"input\": \"What is Apple capitalization compared to Microsoft? Calculate the difference\",\n",
- " \"preamble\": preamble,\n",
- " }\n",
- ")"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {},
- "outputs": [],
- "source": [
- "result[\"output\"]"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "**Let's ask a question that requires Internet Search.**"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {},
- "outputs": [],
- "source": [
- "result = agent_executor.invoke(\n",
- " {\n",
- " \"input\": \"Hi there!\",\n",
- " \"preamble\": preamble,\n",
- " }\n",
- ")"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {},
- "outputs": [],
- "source": [
- "result[\"output\"]"
- ]
- }
- ],
- "metadata": {
- "colab": {
- "provenance": []
- },
- "kernelspec": {
- "display_name": "Python 3 (ipykernel)",
- "language": "python",
- "name": "python3"
- },
- "language_info": {
- "codemirror_mode": {
- "name": "ipython",
- "version": 3
- },
- "file_extension": ".py",
- "mimetype": "text/x-python",
- "name": "python",
- "nbconvert_exporter": "python",
- "pygments_lexer": "ipython3",
- "version": "3.11.9"
- }
- },
- "nbformat": 4,
- "nbformat_minor": 4
-}
diff --git a/notebooks/agents/Vanilla_Multi_Step_Tool_Use.ipynb b/notebooks/agents/Vanilla_Multi_Step_Tool_Use.ipynb
index 550e7d87..bfded608 100644
--- a/notebooks/agents/Vanilla_Multi_Step_Tool_Use.ipynb
+++ b/notebooks/agents/Vanilla_Multi_Step_Tool_Use.ipynb
@@ -1,979 +1,917 @@
{
- "nbformat": 4,
- "nbformat_minor": 0,
- "metadata": {
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "aH1iAdZiURXh"
+ },
+ "source": [
+ "# Multi-Step Tool Use\n",
+ "\n",
+ "Tool use allows developers to connect Cohere's models to external tools like search engines, APIs, functions, databases, etc.\n",
+ "\n",
+ "Multi-step tool use is an extension of this basic idea, and allows the model to call more than one tool in a sequence of steps, using the results from one tool call in a subsequent step. This process allows the language model to reason, perform dynamic actions, and quickly adapt on the basis of information coming from external sources.\n",
+ "\n",
+ "The recommended way to achieve [multi-step tool use with Cohere](https://docs.cohere.com/docs/multi-step-tool-use) is by leveraging the [Langchain framework](https://python.langchain.com/docs/integrations/providers/cohere#react-agent) in Python."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "PVT6Sl3msjNe"
+ },
+ "source": [
+ "## Install Dependencies"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 1,
+ "metadata": {
"colab": {
- "provenance": []
- },
- "kernelspec": {
- "name": "python3",
- "display_name": "Python 3"
- },
- "language_info": {
- "name": "python"
+ "base_uri": "https://localhost:8080/"
+ },
+ "id": "0m31LaFDjCIY",
+ "outputId": "7177e241-a864-47ed-cf0a-5fc9fa70a7ec"
+ },
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m812.8/812.8 kB\u001b[0m \u001b[31m6.5 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
+ "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m194.1/194.1 kB\u001b[0m \u001b[31m4.7 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
+ "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m1.9/1.9 MB\u001b[0m \u001b[31m9.6 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
+ "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m276.8/276.8 kB\u001b[0m \u001b[31m7.7 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
+ "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m87.5/87.5 kB\u001b[0m \u001b[31m3.7 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
+ "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m145.9/145.9 kB\u001b[0m \u001b[31m2.8 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
+ "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m3.1/3.1 MB\u001b[0m \u001b[31m21.7 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
+ "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m75.6/75.6 kB\u001b[0m \u001b[31m7.1 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
+ "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m49.4/49.4 kB\u001b[0m \u001b[31m4.1 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
+ "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m53.0/53.0 kB\u001b[0m \u001b[31m4.8 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
+ "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m144.8/144.8 kB\u001b[0m \u001b[31m4.9 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
+ "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m77.9/77.9 kB\u001b[0m \u001b[31m1.9 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
+ "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m58.3/58.3 kB\u001b[0m \u001b[31m5.5 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
+ "\u001b[?25h"
+ ]
}
+ ],
+ "source": [
+ "! pip install --quiet langchain langchain_cohere langchain_experimental"
+ ]
},
- "cells": [
- {
- "cell_type": "markdown",
- "source": [
- "# Multi-Step Tool Use\n",
- "\n",
- "Tool use allows developers to connect Cohere's models to external tools like search engines, APIs, functions, databases, etc.\n",
- "\n",
- "Multi-step tool use is an extension of this basic idea, and allows the model to call more than one tool in a sequence of steps, using the results from one tool call in a subsequent step. This process allows the language model to reason, perform dynamic actions, and quickly adapt on the basis of information coming from external sources.\n",
- "\n",
- "The recommended way to achieve [multi-step tool use with Cohere](https://docs.cohere.com/docs/multi-step-tool-use) is by leveraging the [Langchain framework](https://python.langchain.com/docs/integrations/providers/cohere#react-agent) in Python."
- ],
- "metadata": {
- "id": "aH1iAdZiURXh"
- }
- },
- {
- "cell_type": "markdown",
- "source": [
- "## Install Dependencies"
- ],
- "metadata": {
- "id": "PVT6Sl3msjNe"
- }
- },
- {
- "cell_type": "code",
- "execution_count": 1,
- "metadata": {
- "colab": {
- "base_uri": "https://localhost:8080/"
- },
- "id": "0m31LaFDjCIY",
- "outputId": "7177e241-a864-47ed-cf0a-5fc9fa70a7ec"
- },
- "outputs": [
- {
- "output_type": "stream",
- "name": "stdout",
- "text": [
- "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m812.8/812.8 kB\u001b[0m \u001b[31m6.5 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
- "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m194.1/194.1 kB\u001b[0m \u001b[31m4.7 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
- "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m1.9/1.9 MB\u001b[0m \u001b[31m9.6 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
- "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m276.8/276.8 kB\u001b[0m \u001b[31m7.7 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
- "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m87.5/87.5 kB\u001b[0m \u001b[31m3.7 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
- "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m145.9/145.9 kB\u001b[0m \u001b[31m2.8 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
- "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m3.1/3.1 MB\u001b[0m \u001b[31m21.7 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
- "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m75.6/75.6 kB\u001b[0m \u001b[31m7.1 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
- "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m49.4/49.4 kB\u001b[0m \u001b[31m4.1 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
- "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m53.0/53.0 kB\u001b[0m \u001b[31m4.8 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
- "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m144.8/144.8 kB\u001b[0m \u001b[31m4.9 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
- "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m77.9/77.9 kB\u001b[0m \u001b[31m1.9 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
- "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m58.3/58.3 kB\u001b[0m \u001b[31m5.5 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
- "\u001b[?25h"
- ]
- }
- ],
- "source": [
- "! pip install --quiet langchain langchain_cohere langchain_experimental"
- ]
- },
- {
- "cell_type": "code",
- "source": [
- "# LLM\n",
- "import os\n",
- "os.environ['COHERE_API_KEY'] = "
- ],
- "metadata": {
- "id": "K0GELKJVnadW"
- },
- "execution_count": 2,
- "outputs": []
- },
- {
- "cell_type": "markdown",
- "source": [
- "## Define tools\n",
- "\n",
- "Your agent will be equipped with the following tools. The model can pick between\n",
- "\n",
- "- doing a search on the web\n",
- "- and/or retrieving data from a vector store\n",
- "- and/or using a python interpreter\n",
- "- and/or directly answering [ this tool comes out of the box! ]\n",
- "\n",
- "Plus the model can self-reflect."
- ],
- "metadata": {
- "id": "6l1pDbAmsptW"
- }
- },
- {
- "cell_type": "markdown",
- "source": [
- "#### Web search\n",
- "You can easily equip your agent with web search!"
- ],
- "metadata": {
- "id": "_9riBTvIVnaN"
- }
- },
- {
- "cell_type": "code",
- "source": [
- "from langchain_community.tools.tavily_search import TavilySearchResults\n",
- "\n",
- "os.environ[\"TAVILY_API_KEY\"] = # you can create an API key for free on Tavily's website\n",
- "\n",
- "internet_search = TavilySearchResults()\n",
- "internet_search.name = \"internet_search\"\n",
- "internet_search.description = \"Returns a list of relevant document snippets for a textual query retrieved from the internet.\"\n",
- "\n",
- "\n",
- "from langchain_core.pydantic_v1 import BaseModel, Field\n",
- "class TavilySearchInput(BaseModel):\n",
- " query: str = Field(description=\"Query to search the internet with\")\n",
- "internet_search.args_schema = TavilySearchInput"
- ],
- "metadata": {
- "id": "6wMNTOGNVvwA"
- },
- "execution_count": 3,
- "outputs": []
- },
- {
- "cell_type": "markdown",
- "source": [
- "#### Vector store\n",
- "You can easily equip your agent with a vector store!"
- ],
- "metadata": {
- "id": "pax1SemzX6Kg"
- }
- },
- {
- "cell_type": "code",
- "source": [
- "!pip --quiet install faiss-cpu tiktoken"
- ],
- "metadata": {
- "id": "cc4d4pRFndYu",
- "colab": {
- "base_uri": "https://localhost:8080/"
- },
- "outputId": "6478fd6b-5618-4d0b-8928-44f78e3ca7e6"
- },
- "execution_count": 4,
- "outputs": [
- {
- "output_type": "stream",
- "name": "stdout",
- "text": [
- "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m27.0/27.0 MB\u001b[0m \u001b[31m41.4 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
- "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m1.8/1.8 MB\u001b[0m \u001b[31m58.3 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
- "\u001b[?25h"
- ]
- }
- ]
- },
- {
- "cell_type": "code",
- "source": [
- "from langchain.text_splitter import RecursiveCharacterTextSplitter\n",
- "from langchain_community.document_loaders import WebBaseLoader\n",
- "from langchain_community.vectorstores import FAISS\n",
- "from langchain_cohere import CohereEmbeddings\n",
- "\n",
- "# Set embeddings\n",
- "embd = CohereEmbeddings()\n",
- "\n",
- "# Docs to index\n",
- "urls = [\n",
- " \"https://paulgraham.com/best.html\",\n",
- "]\n",
- "\n",
- "# Load\n",
- "docs = [WebBaseLoader(url).load() for url in urls]\n",
- "docs_list = [item for sublist in docs for item in sublist]\n",
- "\n",
- "# Split\n",
- "text_splitter = RecursiveCharacterTextSplitter.from_tiktoken_encoder(\n",
- " chunk_size=512, chunk_overlap=0\n",
- ")\n",
- "doc_splits = text_splitter.split_documents(docs_list)\n",
- "\n",
- "# Add to vectorstore\n",
- "vectorstore = FAISS.from_documents(\n",
- " documents=doc_splits,\n",
- " embedding=embd,\n",
- ")\n",
- "\n",
- "vectorstore_retriever = vectorstore.as_retriever()\n"
- ],
- "metadata": {
- "id": "eqcorKP4YEH6"
- },
- "execution_count": 5,
- "outputs": []
- },
- {
- "cell_type": "code",
- "source": [
- "from langchain.tools.retriever import create_retriever_tool\n",
- "\n",
- "vectorstore_search = create_retriever_tool(\n",
- " retriever=vectorstore_retriever,\n",
- " name=\"vectorstore_search\",\n",
- " description=\"Retrieve relevant info from a vectorstore that contains information from Paul Graham about how to write good essays.\"\n",
- ")"
- ],
- "metadata": {
- "id": "eQFNJ-38abRd"
- },
- "execution_count": 6,
- "outputs": []
- },
- {
- "cell_type": "markdown",
- "source": [
- "#### Python interpreter tool\n",
- "You can easily equip your agent with a python interpreter!"
- ],
- "metadata": {
- "id": "aB90i9W5YqWi"
- }
- },
- {
- "cell_type": "code",
- "source": [
- "from langchain.agents import Tool\n",
- "from langchain_experimental.utilities import PythonREPL\n",
- "\n",
- "python_repl = PythonREPL()\n",
- "python_tool = Tool(\n",
- " name=\"python_repl\",\n",
- " description=\"Executes python code and returns the result. The code runs in a static sandbox without interactive mode, so print output or save output to a file.\",\n",
- " func=python_repl.run,\n",
- ")\n",
- "python_tool.name = \"python_interpreter\"\n",
- "\n",
- "# from langchain_core.pydantic_v1 import BaseModel, Field\n",
- "class ToolInput(BaseModel):\n",
- " code: str = Field(description=\"Python code to execute.\")\n",
- "python_tool.args_schema = ToolInput"
- ],
- "metadata": {
- "id": "AN-4FqXKYEFw"
- },
- "execution_count": 7,
- "outputs": []
- },
- {
- "cell_type": "code",
- "source": [],
- "metadata": {
- "id": "8vj868w_YED_"
- },
- "execution_count": 7,
- "outputs": []
- },
- {
- "cell_type": "markdown",
- "source": [
- "#### Transform any Python function in a Tool\n",
- "You can easily equip your agent with any Python function!"
- ],
- "metadata": {
- "id": "2SDCaoypexGL"
- }
- },
- {
- "cell_type": "code",
- "source": [
- "from langchain_core.tools import tool\n",
- "import random\n",
- "\n",
- "@tool\n",
- "def random_operation_tool(a: int, b: int):\n",
- " \"\"\"Calculates a random operation between the inputs.\"\"\"\n",
- " coin_toss = random.uniform(0, 1)\n",
- " if coin_toss > 0.5:\n",
- " return {'output': a*b}\n",
- " else:\n",
- " return {'output': a+b}\n",
- "\n",
- "random_operation_tool.name = \"random_operation\" # use python case\n",
- "random_operation_tool.description = \"Calculates a random operation between the inputs.\"\n",
- "\n",
- "from langchain_core.pydantic_v1 import BaseModel, Field\n",
- "class random_operation_inputs(BaseModel):\n",
- " a: int = Field(description=\"First input\")\n",
- " b: int = Field(description=\"Second input\")\n",
- "random_operation_tool.args_schema = random_operation_inputs"
- ],
- "metadata": {
- "id": "DYUJT2xKewny"
- },
- "execution_count": 8,
- "outputs": []
- },
- {
- "cell_type": "markdown",
- "source": [
- "## Create ReAct Agent\n",
- "\n",
- "The model can smartly pick the right tool(s) for the user query, call them in any sequence, analyze the results and self-reflect. \n",
- "Once the model considers it has enough information to answer the user question, it generates the final answer."
- ],
- "metadata": {
- "id": "hcspRBsRY2ED"
- }
- },
- {
- "cell_type": "code",
- "source": [
- "from langchain.agents import AgentExecutor\n",
- "from langchain_cohere.react_multi_hop.agent import create_cohere_react_agent\n",
- "from langchain_core.prompts import ChatPromptTemplate\n",
- "from langchain_cohere.chat_models import ChatCohere\n",
- "\n",
- "# LLM\n",
- "llm = ChatCohere(model=\"command-r-plus\", temperature=0.3)\n",
- "\n",
- "# Preamble\n",
- "preamble = \"\"\"\n",
- "You are an expert who answers the user's question with the most relevant datasource. You are equipped with an internet search tool and a special vectorstore of information about how to write good essays.\n",
- "You also have a 'random_operation_tool' tool, you must use it to compute the random operation between two numbers.\n",
- "\"\"\"\n",
- "\n",
- "\n",
- "# Prompt template\n",
- "prompt = ChatPromptTemplate.from_template(\"{input}\")\n",
- "\n",
- "# Create the ReAct agent\n",
- "agent = create_cohere_react_agent(\n",
- " llm=llm,\n",
- " tools=[internet_search, vectorstore_search, python_tool, random_operation_tool],\n",
- " prompt=prompt,\n",
- ")\n",
- "\n",
- "agent_executor = AgentExecutor(agent=agent, tools=[internet_search, vectorstore_search, python_tool, random_operation_tool], verbose=True)"
- ],
- "metadata": {
- "id": "CY83aAprYECO"
- },
- "execution_count": 13,
- "outputs": []
- },
- {
- "cell_type": "code",
- "source": [],
- "metadata": {
- "id": "RK0NaqngYD_J"
- },
- "execution_count": 13,
- "outputs": []
- },
- {
- "cell_type": "markdown",
- "source": [
- "## Ask a standalone question to the ReAct agent\n",
- "A question that requires using a predefined tool from Langchain"
- ],
- "metadata": {
- "id": "mzTQcVXSaBe7"
- }
- },
- {
- "cell_type": "code",
- "source": [
- "response = agent_executor.invoke({\n",
- " \"input\": \"I want to write an essay about the Roman Empire. Any tips for writing an essay? Any fun facts?\",\n",
- " \"preamble\": preamble,\n",
- "})\n",
- "\n",
- "response['output']\n",
- "\n",
- "# note that the model smartly looks in the vector db, and then online"
- ],
- "metadata": {
- "colab": {
- "base_uri": "https://localhost:8080/",
- "height": 1000
- },
- "id": "rR2hWVp-aEsU",
- "outputId": "fcf9b8b3-7027-4679-c06d-ef166f016892"
+ {
+ "cell_type": "code",
+ "execution_count": 2,
+ "metadata": {
+ "id": "K0GELKJVnadW"
+ },
+ "outputs": [],
+ "source": [
+ "# LLM\n",
+ "import os\n",
+ "os.environ['COHERE_API_KEY'] = "
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "6l1pDbAmsptW"
+ },
+ "source": [
+ "## Define tools\n",
+ "\n",
+ "Your agent will be equipped with the following tools. The model can pick between\n",
+ "\n",
+ "- doing a search on the web\n",
+ "- and/or retrieving data from a vector store\n",
+ "- and/or using a python interpreter\n",
+ "- and/or directly answering [ this tool comes out of the box! ]\n",
+ "\n",
+ "Plus the model can self-reflect."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "_9riBTvIVnaN"
+ },
+ "source": [
+ "#### Web search\n",
+ "You can easily equip your agent with web search!"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 3,
+ "metadata": {
+ "id": "6wMNTOGNVvwA"
+ },
+ "outputs": [],
+ "source": [
+ "from langchain_community.tools.tavily_search import TavilySearchResults\n",
+ "\n",
+ "os.environ[\"TAVILY_API_KEY\"] = # you can create an API key for free on Tavily's website\n",
+ "\n",
+ "internet_search = TavilySearchResults()\n",
+ "internet_search.name = \"internet_search\"\n",
+ "internet_search.description = \"Returns a list of relevant document snippets for a textual query retrieved from the internet.\"\n",
+ "\n",
+ "\n",
+ "from langchain_core.pydantic_v1 import BaseModel, Field\n",
+ "class TavilySearchInput(BaseModel):\n",
+ " query: str = Field(description=\"Query to search the internet with\")\n",
+ "internet_search.args_schema = TavilySearchInput"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "pax1SemzX6Kg"
+ },
+ "source": [
+ "#### Vector store\n",
+ "You can easily equip your agent with a vector store!"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 4,
+ "metadata": {
+ "colab": {
+ "base_uri": "https://localhost:8080/"
+ },
+ "id": "cc4d4pRFndYu",
+ "outputId": "6478fd6b-5618-4d0b-8928-44f78e3ca7e6"
+ },
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m27.0/27.0 MB\u001b[0m \u001b[31m41.4 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
+ "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m1.8/1.8 MB\u001b[0m \u001b[31m58.3 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
+ "\u001b[?25h"
+ ]
+ }
+ ],
+ "source": [
+ "!pip --quiet install faiss-cpu tiktoken"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 5,
+ "metadata": {
+ "id": "eqcorKP4YEH6"
+ },
+ "outputs": [],
+ "source": [
+ "from langchain.text_splitter import RecursiveCharacterTextSplitter\n",
+ "from langchain_community.document_loaders import WebBaseLoader\n",
+ "from langchain_community.vectorstores import FAISS\n",
+ "from langchain_cohere import CohereEmbeddings\n",
+ "\n",
+ "# Set embeddings\n",
+ "embd = CohereEmbeddings()\n",
+ "\n",
+ "# Docs to index\n",
+ "urls = [\n",
+ " \"https://paulgraham.com/best.html\",\n",
+ "]\n",
+ "\n",
+ "# Load\n",
+ "docs = [WebBaseLoader(url).load() for url in urls]\n",
+ "docs_list = [item for sublist in docs for item in sublist]\n",
+ "\n",
+ "# Split\n",
+ "text_splitter = RecursiveCharacterTextSplitter.from_tiktoken_encoder(\n",
+ " chunk_size=512, chunk_overlap=0\n",
+ ")\n",
+ "doc_splits = text_splitter.split_documents(docs_list)\n",
+ "\n",
+ "# Add to vectorstore\n",
+ "vectorstore = FAISS.from_documents(\n",
+ " documents=doc_splits,\n",
+ " embedding=embd,\n",
+ ")\n",
+ "\n",
+ "vectorstore_retriever = vectorstore.as_retriever()\n"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 6,
+ "metadata": {
+ "id": "eQFNJ-38abRd"
+ },
+ "outputs": [],
+ "source": [
+ "from langchain.tools.retriever import create_retriever_tool\n",
+ "\n",
+ "vectorstore_search = create_retriever_tool(\n",
+ " retriever=vectorstore_retriever,\n",
+ " name=\"vectorstore_search\",\n",
+ " description=\"Retrieve relevant info from a vectorstore that contains information from Paul Graham about how to write good essays.\"\n",
+ ")"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "aB90i9W5YqWi"
+ },
+ "source": [
+ "#### Python interpreter tool\n",
+ "You can easily equip your agent with a python interpreter!"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 7,
+ "metadata": {
+ "id": "AN-4FqXKYEFw"
+ },
+ "outputs": [],
+ "source": [
+ "from langchain.agents import Tool\n",
+ "from langchain_experimental.utilities import PythonREPL\n",
+ "\n",
+ "python_repl = PythonREPL()\n",
+ "python_tool = Tool(\n",
+ " name=\"python_repl\",\n",
+ " description=\"Executes python code and returns the result. The code runs in a static sandbox without interactive mode, so print output or save output to a file.\",\n",
+ " func=python_repl.run,\n",
+ ")\n",
+ "python_tool.name = \"python_interpreter\"\n",
+ "\n",
+ "# from langchain_core.pydantic_v1 import BaseModel, Field\n",
+ "class ToolInput(BaseModel):\n",
+ " code: str = Field(description=\"Python code to execute.\")\n",
+ "python_tool.args_schema = ToolInput"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "2SDCaoypexGL"
+ },
+ "source": [
+ "#### Transform any Python function in a Tool\n",
+ "You can easily equip your agent with any Python function!"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 8,
+ "metadata": {
+ "id": "DYUJT2xKewny"
+ },
+ "outputs": [],
+ "source": [
+ "from langchain_core.tools import tool\n",
+ "import random\n",
+ "\n",
+ "@tool\n",
+ "def random_operation_tool(a: int, b: int):\n",
+ " \"\"\"Calculates a random operation between the inputs.\"\"\"\n",
+ " coin_toss = random.uniform(0, 1)\n",
+ " if coin_toss > 0.5:\n",
+ " return {'output': a*b}\n",
+ " else:\n",
+ " return {'output': a+b}\n",
+ "\n",
+ "random_operation_tool.name = \"random_operation\" # use python case\n",
+ "random_operation_tool.description = \"Calculates a random operation between the inputs.\"\n",
+ "\n",
+ "from langchain_core.pydantic_v1 import BaseModel, Field\n",
+ "class random_operation_inputs(BaseModel):\n",
+ " a: int = Field(description=\"First input\")\n",
+ " b: int = Field(description=\"Second input\")\n",
+ "random_operation_tool.args_schema = random_operation_inputs"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "hcspRBsRY2ED"
+ },
+ "source": [
+ "## Create ReAct Agent\n",
+ "\n",
+ "The model can smartly pick the right tool(s) for the user query, call them in any sequence, analyze the results and self-reflect. \n",
+ "Once the model considers it has enough information to answer the user question, it generates the final answer."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 13,
+ "metadata": {
+ "id": "CY83aAprYECO"
+ },
+ "outputs": [],
+ "source": [
+ "from langchain.agents import AgentExecutor\n",
+ "from langchain_cohere.react_multi_hop.agent import create_cohere_react_agent\n",
+ "from langchain_core.prompts import ChatPromptTemplate\n",
+ "from langchain_cohere.chat_models import ChatCohere\n",
+ "\n",
+ "# LLM\n",
+ "llm = ChatCohere(model=\"command-r-plus\", temperature=0.3)\n",
+ "\n",
+ "# Preamble\n",
+ "preamble = \"\"\"\n",
+ "You are an expert who answers the user's question with the most relevant datasource. You are equipped with an internet search tool and a special vectorstore of information about how to write good essays.\n",
+ "You also have a 'random_operation_tool' tool, you must use it to compute the random operation between two numbers.\n",
+ "\"\"\"\n",
+ "\n",
+ "\n",
+ "# Prompt template\n",
+ "prompt = ChatPromptTemplate.from_template(\"{input}\")\n",
+ "\n",
+ "# Create the ReAct agent\n",
+ "agent = create_cohere_react_agent(\n",
+ " llm=llm,\n",
+ " tools=[internet_search, vectorstore_search, python_tool, random_operation_tool],\n",
+ " prompt=prompt,\n",
+ ")\n",
+ "\n",
+ "agent_executor = AgentExecutor(agent=agent, tools=[internet_search, vectorstore_search, python_tool, random_operation_tool], verbose=True)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "mzTQcVXSaBe7"
+ },
+ "source": [
+ "## Ask a standalone question to the ReAct agent\n",
+ "A question that requires using a predefined tool from Langchain"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 14,
+ "metadata": {
+ "colab": {
+ "base_uri": "https://localhost:8080/",
+ "height": 1000
+ },
+ "id": "rR2hWVp-aEsU",
+ "outputId": "fcf9b8b3-7027-4679-c06d-ef166f016892"
+ },
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "\n",
+ "\n",
+ "\u001b[1m> Entering new AgentExecutor chain...\u001b[0m\n",
+ "\u001b[32;1m\u001b[1;3m\n",
+ "I will search for tips on writing an essay and fun facts about the Roman Empire.\n",
+ "{'tool_name': 'vectorstore_search', 'parameters': {'query': 'tips for writing an essay'}}\n",
+ "\u001b[0m\u001b[33;1m\u001b[1;3mbe? I should have asked how do you write essays well? Though\n",
+ "these seem only phrasing apart, their answers diverge. The answer\n",
+ "to the first question, as we've seen, isn't really about essay\n",
+ "writing. The second question forces it to be.Writing essays, at its best, is a way of discovering ideas. How do\n",
+ "you do that well? How do you discover by writing?An essay should ordinarily start with what I'm going to call a\n",
+ "question, though I mean this in a very general sense: it doesn't\n",
+ "have to be a question grammatically, just something that acts like\n",
+ "one in the sense that it spurs some response.How do you get this initial question? It probably won't work to\n",
+ "choose some important-sounding topic at random and go at it.\n",
+ "Professional traders won't even trade unless they have what they\n",
+ "call an edge — a convincing story about why in some class of\n",
+ "trades they'll win more than they lose. Similarly, you shouldn't\n",
+ "attack a topic unless you have a way in — some new insight about\n",
+ "it or way of approaching it.You don't need to have a complete thesis; you just need some kind\n",
+ "of gap you can explore. In fact, merely having questions about\n",
+ "something other people take for granted can be edge enough.If you come across a question that's sufficiently puzzling, it could\n",
+ "be worth exploring even if it doesn't seem very momentous. Many an\n",
+ "important discovery has been made by pulling on a thread that seemed\n",
+ "insignificant at first. How can they all be finches? \n",
+ "[2]Once you've got a question, then what? You start thinking out loud\n",
+ "about it. Not literally out loud, but you commit to a specific\n",
+ "string of words in response, as you would if you were talking. This\n",
+ "initial response is usually mistaken or incomplete. Writing converts\n",
+ "your ideas from vague to bad. But that's a step forward, because\n",
+ "once you can see the brokenness, you can fix it.Perhaps beginning writers are alarmed at the thought of starting\n",
+ "with something mistaken or incomplete, but you shouldn't be, because\n",
+ "this is why essay writing works. Forcing yourself to commit to some\n",
+ "specific string of words gives you a starting point, and if it's\n",
+ "wrong, you'll see that when you reread it. At least half of essay\n",
+ "writing is rereading what you've written and asking is this correct\n",
+ "\n",
+ "didn't have edge with any of them. To start writing an essay, you\n",
+ "need a topic plus some initial insight about it, and you can't\n",
+ "generate those systematically. If only. \n",
+ "[9]You can probably cause yourself to have more of them, though. The\n",
+ "quality of the ideas that come out of your head depends on what goes\n",
+ "in, and you can improve that in two dimensions, breadth and depth.You can't learn everything, so getting breadth implies learning\n",
+ "about topics that are very different from one another. When I tell\n",
+ "people about my book-buying trips to Hay and they ask what I buy\n",
+ "books about, I usually feel a bit sheepish answering, because the\n",
+ "topics seem like a laundry list of unrelated subjects. But perhaps\n",
+ "that's actually optimal in this business.You can also get ideas by talking to people, by doing and building\n",
+ "things, and by going places and seeing things. I don't think it's\n",
+ "important to talk to new people so much as the sort of people who\n",
+ "make you have new ideas. I get more new ideas after talking for an\n",
+ "afternoon with Robert Morris than from talking to 20 new smart\n",
+ "people. I know because that's what a block of office hours at Y\n",
+ "Combinator consists of.While breadth comes from reading and talking and seeing, depth comes\n",
+ "from doing. The way to really learn about some domain is to have\n",
+ "to solve problems in it. Though this could take the form of writing,\n",
+ "I suspect that to be a good essayist you also have to do, or have\n",
+ "done, some other kind of work. That may not be true for most other\n",
+ "fields, but essay writing is different. You could spend half your\n",
+ "time working on something else and be net ahead, so long as it was\n",
+ "hard.I'm not proposing that as a recipe so much as an encouragement to\n",
+ "those already doing it. If you've spent all your life so far working\n",
+ "on other things, you're already halfway there. Though of course to\n",
+ "be good at writing you have to like it, and if you like writing\n",
+ "you'd probably have spent at least some time doing it.Everything I've said about initial questions applies also to the\n",
+ "questions you encounter in writing the essay. They're the same\n",
+ "thing; every subtree of an essay is usually a shorter essay, just\n",
+ "as every subtree of a Calder mobile is a smaller mobile. So any\n",
+ "\n",
+ "You don't have to get an answer right the first time, but there's\n",
+ "no excuse for not getting it right eventually, because you can keep\n",
+ "rewriting till you do. And this is not just a theoretical possibility.\n",
+ "It's a pretty accurate description of the way I work. I'm rewriting\n",
+ "as we speak.But although I wish I could say that writing great essays depends mostly\n",
+ "on effort, in the limit case it's inspiration that makes the\n",
+ "difference. In the limit case, the questions are the harder thing\n",
+ "to get. That pool has no bottom.How to get more questions? That is the most important question of\n",
+ "all.Notes[1]\n",
+ "There might be some resistance to this conclusion on the\n",
+ "grounds that some of these discoveries could only be understood by\n",
+ "a small number of readers. But you get into all sorts of difficulties\n",
+ "if you want to disqualify essays on this account. How do you decide\n",
+ "where the cutoff should be? If a virus kills off everyone except a \n",
+ "handful of people sequestered at Los Alamos,\n",
+ "could an essay that had been disqualified now be eligible? Etc.Darwin's 1844 essay was derived from an earlier version written in 1839.\n",
+ "Extracts from it were published in 1858.[2]\n",
+ "When you find yourself very curious about an apparently minor\n",
+ "question, that's an exciting sign. Evolution has designed you to\n",
+ "pay attention to things that matter. So when you're very curious\n",
+ "about something random, that could mean you've unconsciously noticed\n",
+ "it's less random than it seems.[3]\n",
+ "Corollary: If you're not intellectually honest, your writing\n",
+ "won't just be biased, but also boring, because you'll miss all the\n",
+ "ideas you'd have discovered if you pushed for the truth.[4]\n",
+ "Sometimes this process begins before you start writing.\n",
+ "Sometimes you've already figured out the first few things you want\n",
+ "to say. Schoolchildren are often taught they should decide everything\n",
+ "they want to say, and write this down as an outline before they\n",
+ "start writing the essay itself. Maybe that's a good way to get them\n",
+ "started — or not, I don't know — but it's antithetical to the\n",
+ "spirit of essay writing. The more detailed your outline, the less\n",
+ "your ideas can benefit from the sort of discovery that essays are for.[5]\n",
+ "The problem with this type of \"greedy\" algorithm is that you\n",
+ "\n",
+ "technique that gets you good initial questions also gets you good\n",
+ "whole essays.At some point the cycle of question and response reaches what feels\n",
+ "like a natural end. Which is a little suspicious; shouldn't every\n",
+ "answer suggest more questions? I think what happens is that you\n",
+ "start to feel sated. Once you've covered enough interesting ground,\n",
+ "you start to lose your appetite for new questions. Which is just\n",
+ "as well, because the reader is probably feeling sated too. And it's\n",
+ "not lazy to stop asking questions, because you could instead be\n",
+ "asking the initial question of a new essay.That's the ultimate source of drag on the connectedness of ideas:\n",
+ "the discoveries you make along the way. If you discover enough\n",
+ "starting from question A, you'll never make it to question B. Though\n",
+ "if you keep writing essays you'll gradually fix this problem by\n",
+ "burning off such discoveries. So bizarrely enough, writing lots of\n",
+ "essays makes it as if the space of ideas were more highly connected.When a subtree comes to an end, you can do one of two things. You\n",
+ "can either stop, or pull the Cubist trick of laying separate subtrees\n",
+ "end to end by returning to a question you skipped earlier. Usually\n",
+ "it requires some sleight of hand to make the essay flow continuously\n",
+ "at this point, but not this time. This time I actually need an\n",
+ "example of the phenomenon. For example, we discovered earlier that\n",
+ "the best possible essay wouldn't usually be timeless in the way the\n",
+ "best painting would. This seems surprising enough to be\n",
+ "worth investigating further.There are two senses in which an essay can be timeless: to be about\n",
+ "a matter of permanent importance, and always to have the same effect\n",
+ "on readers. With art these two senses blend together. Art that\n",
+ "looked beautiful to the ancient Greeks still looks beautiful to us.\n",
+ "But with essays the two senses diverge, because essays\n",
+ "teach, and you can't teach people something they already know.\n",
+ "Natural selection is certainly a matter of permanent importance,\n",
+ "but an essay explaining it couldn't have the same effect on us that\n",
+ "it would have had on Darwin's contemporaries, precisely because his\n",
+ "ideas were so successful that everyone already knows about them.\n",
+ "[10]I imagined when I started writing this that the best possible essay\n",
+ "would be timeless in the stricter, evergreen sense: that it would\u001b[0m\u001b[32;1m\u001b[1;3m\n",
+ "{'tool_name': 'internet_search', 'parameters': {'query': 'fun facts about the roman empire'}}\n",
+ "\u001b[0m\u001b[36;1m\u001b[1;3m[{'url': 'https://www.natgeokids.com/uk/discover/history/romans/10-facts-about-the-ancient-romans/', 'content': 'i love this website\\nBIG BOBBY\\nbooby\\nI love shell my bae;)\\ni like bobby fishes ;0\\nI like turtles\\nOmg soy cool\\ngreeeeeeeeeeeeaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaatttttttttttttttttttttttt\\nbest fact ever\\nthis artical is cool\\nHANDY\\nrubbish did not help what so ever\\nha\\nRocking\\nTHIS IS THE BEST\\nproper rad in it cool\\nthis is cool\\nawesomeness\\nawsome\\nawsome\\nthank you captain\\nit is a lot of help\\ni like this\\nwebsite it helps me on my projects and isabel likes munier\\nmark uses this for research\\nlot of help\\nthis is awsome\\nTHE BEST BOOBOO\\nCool webpage helped me get 4 housepoints\\n This helped me A LOT on a school project\\ncool wow awesomoe\\nCOOL WEBSITE LOL\\nthis helped me with a school project :)\\nthat was awesome\\ncool\\nthat helped me out for my research test\\nReally its very cool really COOL\\nLIKE COOL best website so far its nice\\nI love it\\nnice facts\\nIt help with my history\\n i mean u made animaljam a awesome nice safe place for kids and this site to have kids a safe website to get facts for reports and stuff\\nLots of Love ,\\nRose\\npretty good website if u ask me\\nbut definently not gonna use it on a daily basis\\nIll try it again another time\\ngood\\nCool webcite\\nterrible\\nquite impressive\\nAwesome website it real helps\\nits good\\nthis is a great website! You really a lot with my project!:)\\nthis has helleped\\nme get\\nmy progect\\ndone\\nthank you\\nsoooooooooooooooooo\\nmuchchchchchch\\nthis helleped me\\nsooooooooo much with my progect thank you\\nvery good website\\nthank us very much your nice one today!!\\n'}, {'url': 'https://ohfact.com/roman-empire-facts/', 'content': 'Learn about the ancient Roman Civilization, its history, culture, army, architecture, food and more from this list of 27 facts. Discover how the Romans started, conquered, lived, died and influenced the world with their legends, myths and facts.'}, {'url': 'https://factnight.com/fun-facts-about-the-roman-empire/', 'content': 'The Roman Empire was one of the most influential and significant civilizations in world history. At its peak, the empire stretched from North Africa to Britain, reigning over 60 million people. From its legendary beginnings and remarkable achievements to its eventual decline and fall, the Roman Empire is a fascinating topic full of little-known facts and intriguing trivia.'}, {'url': 'https://www.historyhit.com/facts-about-ancient-rome-and-the-romans/', 'content': 'The Enduring Legacy of C.S. Lewis\\nMargaret J. Winkler: A Forgotten Pioneer in Disney’s Success\\n10 Facts About Harper Lee\\nAntarctica Expedition Cruise\\nUncover Pompeii\\nSophie Hay and Tristan Hughes\\nRediscovering Richard III with Matt Lewis\\nOrder the History Hit Miscellany\\nHistory Hit Holidays\\nGift Subscriptions\\n100 Facts About Ancient Rome and the Romans\\nRome wasn’t built in a day, as the cliché reminds us. The Crossing of the Rhine in 405/6 AD brought around 100,000 barbarians into the Empire\\nBarbarian factions, tribes and war leaders were now a factor in the power struggles at the top of Roman politics and one of the once-strong boundaries of the Empire had proved to be permeable.\\n Related Articles\\n10 Facts About Saint Andrew\\nThe Rise of Pompey the Great, the ‘Roman Alexander’\\nWatch and Listen\\nCleopatra\\nSex in Ancient Rome\\nRelated Locations\\nBaelo Claudia\\nMausoleum of Cecilia Metella\\nColin Ricketts\\n30 July 2021\\n By the fourth century BC, the story was accepted by Romans who were proud of their warrior founder\\nThe story was included in the first history of the city, by the Greek writer Diocles of Peparethus, and the twins and their wolf step-mother were depicted on Rome’s first coins.\\n The History Hit Miscellany of Facts, Figures and Fascinating Finds\\nA History of England: Part One\\nDragons: Myth & Reality\\nA Tudor Wonder - Hardwick Hall\\nThe Battle of Shrewsbury\\nEurope’s 1848 Revolutions\\nThe Boston Tea Party\\nHow Did 3 People Seemingly Escape From Alcatraz?\\n'}, {'url': 'https://www.countryfaq.com/facts-about-the-roman-empire/', 'content': 'Facts about the Roman Empire. Explore some of the interesting, fun, cool facts bout the Roman Empire: 1. The Magnificent Roman Empire. The Roman Empire, a colossal entity of unparalleled grandeur, occupies an indomitable position within the annals of human history, a name that resonates resoundingly across the eons.'}]\u001b[0m\u001b[32;1m\u001b[1;3mRelevant Documents: 0,3,4,5\n",
+ "Cited Documents: 0,3,4,5\n",
+ "Answer: Here are some tips for writing an essay:\n",
+ "- Start with a question that spurs some response.\n",
+ "- Don't choose a topic at random, make sure you have a way in, a new insight or approach.\n",
+ "- You don't need a complete thesis, just a gap to explore.\n",
+ "- You can get ideas by talking to people, reading, doing and building things, and going places and seeing things.\n",
+ "- You can improve the quality of your ideas by increasing the breadth and depth of what goes in.\n",
+ "- You can get breadth by reading and talking about a wide range of topics.\n",
+ "- You can get depth by doing and having to solve problems.\n",
+ "- You can also get ideas by talking to people who make you have new ideas.\n",
+ "\n",
+ "Here are some fun facts about the Roman Empire:\n",
+ "- At its peak, the empire stretched from North Africa to Britain, reigning over 60 million people.\n",
+ "- The story of Rome's warrior founder and the twins and their wolf step-mother was depicted on Rome's first coins.\n",
+ "- The Crossing of the Rhine in 405/6 AD brought around 100,000 barbarians into the Empire.\n",
+ "Grounded answer: Here are some tips for writing an essay:\n",
+ "- Start with a question that spurs some response.\n",
+ "- Don't choose a topic at random, make sure you have a way in, a new insight or approach.\n",
+ "- You don't need a complete thesis, just a gap to explore.\n",
+ "- You can get ideas by talking to people, reading, doing and building things, and going places and seeing things.\n",
+ "- You can improve the quality of your ideas by increasing the breadth and depth of what goes in.\n",
+ "- You can get breadth by reading and talking about a wide range of topics.\n",
+ "- You can get depth by doing and having to solve problems.\n",
+ "- You can also get ideas by talking to people who make you have new ideas.\n",
+ "\n",
+ "Here are some fun facts about the Roman Empire:\n",
+ "- At its peak, the empire stretched from North Africa to Britain, reigning over 60 million people.\n",
+ "- The story of Rome's warrior founder and the twins and their wolf step-mother was depicted on Rome's first coins.\n",
+ "- The Crossing of the Rhine in 405/6 AD brought around 100,000 barbarians into the Empire.\u001b[0m\n",
+ "\n",
+ "\u001b[1m> Finished chain.\u001b[0m\n"
+ ]
+ },
+ {
+ "data": {
+ "application/vnd.google.colaboratory.intrinsic+json": {
+ "type": "string"
},
- "execution_count": 14,
- "outputs": [
- {
- "output_type": "stream",
- "name": "stdout",
- "text": [
- "\n",
- "\n",
- "\u001b[1m> Entering new AgentExecutor chain...\u001b[0m\n",
- "\u001b[32;1m\u001b[1;3m\n",
- "I will search for tips on writing an essay and fun facts about the Roman Empire.\n",
- "{'tool_name': 'vectorstore_search', 'parameters': {'query': 'tips for writing an essay'}}\n",
- "\u001b[0m\u001b[33;1m\u001b[1;3mbe? I should have asked how do you write essays well? Though\n",
- "these seem only phrasing apart, their answers diverge. The answer\n",
- "to the first question, as we've seen, isn't really about essay\n",
- "writing. The second question forces it to be.Writing essays, at its best, is a way of discovering ideas. How do\n",
- "you do that well? How do you discover by writing?An essay should ordinarily start with what I'm going to call a\n",
- "question, though I mean this in a very general sense: it doesn't\n",
- "have to be a question grammatically, just something that acts like\n",
- "one in the sense that it spurs some response.How do you get this initial question? It probably won't work to\n",
- "choose some important-sounding topic at random and go at it.\n",
- "Professional traders won't even trade unless they have what they\n",
- "call an edge — a convincing story about why in some class of\n",
- "trades they'll win more than they lose. Similarly, you shouldn't\n",
- "attack a topic unless you have a way in — some new insight about\n",
- "it or way of approaching it.You don't need to have a complete thesis; you just need some kind\n",
- "of gap you can explore. In fact, merely having questions about\n",
- "something other people take for granted can be edge enough.If you come across a question that's sufficiently puzzling, it could\n",
- "be worth exploring even if it doesn't seem very momentous. Many an\n",
- "important discovery has been made by pulling on a thread that seemed\n",
- "insignificant at first. How can they all be finches? \n",
- "[2]Once you've got a question, then what? You start thinking out loud\n",
- "about it. Not literally out loud, but you commit to a specific\n",
- "string of words in response, as you would if you were talking. This\n",
- "initial response is usually mistaken or incomplete. Writing converts\n",
- "your ideas from vague to bad. But that's a step forward, because\n",
- "once you can see the brokenness, you can fix it.Perhaps beginning writers are alarmed at the thought of starting\n",
- "with something mistaken or incomplete, but you shouldn't be, because\n",
- "this is why essay writing works. Forcing yourself to commit to some\n",
- "specific string of words gives you a starting point, and if it's\n",
- "wrong, you'll see that when you reread it. At least half of essay\n",
- "writing is rereading what you've written and asking is this correct\n",
- "\n",
- "didn't have edge with any of them. To start writing an essay, you\n",
- "need a topic plus some initial insight about it, and you can't\n",
- "generate those systematically. If only. \n",
- "[9]You can probably cause yourself to have more of them, though. The\n",
- "quality of the ideas that come out of your head depends on what goes\n",
- "in, and you can improve that in two dimensions, breadth and depth.You can't learn everything, so getting breadth implies learning\n",
- "about topics that are very different from one another. When I tell\n",
- "people about my book-buying trips to Hay and they ask what I buy\n",
- "books about, I usually feel a bit sheepish answering, because the\n",
- "topics seem like a laundry list of unrelated subjects. But perhaps\n",
- "that's actually optimal in this business.You can also get ideas by talking to people, by doing and building\n",
- "things, and by going places and seeing things. I don't think it's\n",
- "important to talk to new people so much as the sort of people who\n",
- "make you have new ideas. I get more new ideas after talking for an\n",
- "afternoon with Robert Morris than from talking to 20 new smart\n",
- "people. I know because that's what a block of office hours at Y\n",
- "Combinator consists of.While breadth comes from reading and talking and seeing, depth comes\n",
- "from doing. The way to really learn about some domain is to have\n",
- "to solve problems in it. Though this could take the form of writing,\n",
- "I suspect that to be a good essayist you also have to do, or have\n",
- "done, some other kind of work. That may not be true for most other\n",
- "fields, but essay writing is different. You could spend half your\n",
- "time working on something else and be net ahead, so long as it was\n",
- "hard.I'm not proposing that as a recipe so much as an encouragement to\n",
- "those already doing it. If you've spent all your life so far working\n",
- "on other things, you're already halfway there. Though of course to\n",
- "be good at writing you have to like it, and if you like writing\n",
- "you'd probably have spent at least some time doing it.Everything I've said about initial questions applies also to the\n",
- "questions you encounter in writing the essay. They're the same\n",
- "thing; every subtree of an essay is usually a shorter essay, just\n",
- "as every subtree of a Calder mobile is a smaller mobile. So any\n",
- "\n",
- "You don't have to get an answer right the first time, but there's\n",
- "no excuse for not getting it right eventually, because you can keep\n",
- "rewriting till you do. And this is not just a theoretical possibility.\n",
- "It's a pretty accurate description of the way I work. I'm rewriting\n",
- "as we speak.But although I wish I could say that writing great essays depends mostly\n",
- "on effort, in the limit case it's inspiration that makes the\n",
- "difference. In the limit case, the questions are the harder thing\n",
- "to get. That pool has no bottom.How to get more questions? That is the most important question of\n",
- "all.Notes[1]\n",
- "There might be some resistance to this conclusion on the\n",
- "grounds that some of these discoveries could only be understood by\n",
- "a small number of readers. But you get into all sorts of difficulties\n",
- "if you want to disqualify essays on this account. How do you decide\n",
- "where the cutoff should be? If a virus kills off everyone except a \n",
- "handful of people sequestered at Los Alamos,\n",
- "could an essay that had been disqualified now be eligible? Etc.Darwin's 1844 essay was derived from an earlier version written in 1839.\n",
- "Extracts from it were published in 1858.[2]\n",
- "When you find yourself very curious about an apparently minor\n",
- "question, that's an exciting sign. Evolution has designed you to\n",
- "pay attention to things that matter. So when you're very curious\n",
- "about something random, that could mean you've unconsciously noticed\n",
- "it's less random than it seems.[3]\n",
- "Corollary: If you're not intellectually honest, your writing\n",
- "won't just be biased, but also boring, because you'll miss all the\n",
- "ideas you'd have discovered if you pushed for the truth.[4]\n",
- "Sometimes this process begins before you start writing.\n",
- "Sometimes you've already figured out the first few things you want\n",
- "to say. Schoolchildren are often taught they should decide everything\n",
- "they want to say, and write this down as an outline before they\n",
- "start writing the essay itself. Maybe that's a good way to get them\n",
- "started — or not, I don't know — but it's antithetical to the\n",
- "spirit of essay writing. The more detailed your outline, the less\n",
- "your ideas can benefit from the sort of discovery that essays are for.[5]\n",
- "The problem with this type of \"greedy\" algorithm is that you\n",
- "\n",
- "technique that gets you good initial questions also gets you good\n",
- "whole essays.At some point the cycle of question and response reaches what feels\n",
- "like a natural end. Which is a little suspicious; shouldn't every\n",
- "answer suggest more questions? I think what happens is that you\n",
- "start to feel sated. Once you've covered enough interesting ground,\n",
- "you start to lose your appetite for new questions. Which is just\n",
- "as well, because the reader is probably feeling sated too. And it's\n",
- "not lazy to stop asking questions, because you could instead be\n",
- "asking the initial question of a new essay.That's the ultimate source of drag on the connectedness of ideas:\n",
- "the discoveries you make along the way. If you discover enough\n",
- "starting from question A, you'll never make it to question B. Though\n",
- "if you keep writing essays you'll gradually fix this problem by\n",
- "burning off such discoveries. So bizarrely enough, writing lots of\n",
- "essays makes it as if the space of ideas were more highly connected.When a subtree comes to an end, you can do one of two things. You\n",
- "can either stop, or pull the Cubist trick of laying separate subtrees\n",
- "end to end by returning to a question you skipped earlier. Usually\n",
- "it requires some sleight of hand to make the essay flow continuously\n",
- "at this point, but not this time. This time I actually need an\n",
- "example of the phenomenon. For example, we discovered earlier that\n",
- "the best possible essay wouldn't usually be timeless in the way the\n",
- "best painting would. This seems surprising enough to be\n",
- "worth investigating further.There are two senses in which an essay can be timeless: to be about\n",
- "a matter of permanent importance, and always to have the same effect\n",
- "on readers. With art these two senses blend together. Art that\n",
- "looked beautiful to the ancient Greeks still looks beautiful to us.\n",
- "But with essays the two senses diverge, because essays\n",
- "teach, and you can't teach people something they already know.\n",
- "Natural selection is certainly a matter of permanent importance,\n",
- "but an essay explaining it couldn't have the same effect on us that\n",
- "it would have had on Darwin's contemporaries, precisely because his\n",
- "ideas were so successful that everyone already knows about them.\n",
- "[10]I imagined when I started writing this that the best possible essay\n",
- "would be timeless in the stricter, evergreen sense: that it would\u001b[0m\u001b[32;1m\u001b[1;3m\n",
- "{'tool_name': 'internet_search', 'parameters': {'query': 'fun facts about the roman empire'}}\n",
- "\u001b[0m\u001b[36;1m\u001b[1;3m[{'url': 'https://www.natgeokids.com/uk/discover/history/romans/10-facts-about-the-ancient-romans/', 'content': 'i love this website\\nBIG BOBBY\\nbooby\\nI love shell my bae;)\\ni like bobby fishes ;0\\nI like turtles\\nOmg soy cool\\ngreeeeeeeeeeeeaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaatttttttttttttttttttttttt\\nbest fact ever\\nthis artical is cool\\nHANDY\\nrubbish did not help what so ever\\nha\\nRocking\\nTHIS IS THE BEST\\nproper rad in it cool\\nthis is cool\\nawesomeness\\nawsome\\nawsome\\nthank you captain\\nit is a lot of help\\ni like this\\nwebsite it helps me on my projects and isabel likes munier\\nmark uses this for research\\nlot of help\\nthis is awsome\\nTHE BEST BOOBOO\\nCool webpage helped me get 4 housepoints\\n This helped me A LOT on a school project\\ncool wow awesomoe\\nCOOL WEBSITE LOL\\nthis helped me with a school project :)\\nthat was awesome\\ncool\\nthat helped me out for my research test\\nReally its very cool really COOL\\nLIKE COOL best website so far its nice\\nI love it\\nnice facts\\nIt help with my history\\n i mean u made animaljam a awesome nice safe place for kids and this site to have kids a safe website to get facts for reports and stuff\\nLots of Love ,\\nRose\\npretty good website if u ask me\\nbut definently not gonna use it on a daily basis\\nIll try it again another time\\ngood\\nCool webcite\\nterrible\\nquite impressive\\nAwesome website it real helps\\nits good\\nthis is a great website! You really a lot with my project!:)\\nthis has helleped\\nme get\\nmy progect\\ndone\\nthank you\\nsoooooooooooooooooo\\nmuchchchchchch\\nthis helleped me\\nsooooooooo much with my progect thank you\\nvery good website\\nthank us very much your nice one today!!\\n'}, {'url': 'https://ohfact.com/roman-empire-facts/', 'content': 'Learn about the ancient Roman Civilization, its history, culture, army, architecture, food and more from this list of 27 facts. Discover how the Romans started, conquered, lived, died and influenced the world with their legends, myths and facts.'}, {'url': 'https://factnight.com/fun-facts-about-the-roman-empire/', 'content': 'The Roman Empire was one of the most influential and significant civilizations in world history. At its peak, the empire stretched from North Africa to Britain, reigning over 60 million people. From its legendary beginnings and remarkable achievements to its eventual decline and fall, the Roman Empire is a fascinating topic full of little-known facts and intriguing trivia.'}, {'url': 'https://www.historyhit.com/facts-about-ancient-rome-and-the-romans/', 'content': 'The Enduring Legacy of C.S. Lewis\\nMargaret J. Winkler: A Forgotten Pioneer in Disney’s Success\\n10 Facts About Harper Lee\\nAntarctica Expedition Cruise\\nUncover Pompeii\\nSophie Hay and Tristan Hughes\\nRediscovering Richard III with Matt Lewis\\nOrder the History Hit Miscellany\\nHistory Hit Holidays\\nGift Subscriptions\\n100 Facts About Ancient Rome and the Romans\\nRome wasn’t built in a day, as the cliché reminds us. The Crossing of the Rhine in 405/6 AD brought around 100,000 barbarians into the Empire\\nBarbarian factions, tribes and war leaders were now a factor in the power struggles at the top of Roman politics and one of the once-strong boundaries of the Empire had proved to be permeable.\\n Related Articles\\n10 Facts About Saint Andrew\\nThe Rise of Pompey the Great, the ‘Roman Alexander’\\nWatch and Listen\\nCleopatra\\nSex in Ancient Rome\\nRelated Locations\\nBaelo Claudia\\nMausoleum of Cecilia Metella\\nColin Ricketts\\n30 July 2021\\n By the fourth century BC, the story was accepted by Romans who were proud of their warrior founder\\nThe story was included in the first history of the city, by the Greek writer Diocles of Peparethus, and the twins and their wolf step-mother were depicted on Rome’s first coins.\\n The History Hit Miscellany of Facts, Figures and Fascinating Finds\\nA History of England: Part One\\nDragons: Myth & Reality\\nA Tudor Wonder - Hardwick Hall\\nThe Battle of Shrewsbury\\nEurope’s 1848 Revolutions\\nThe Boston Tea Party\\nHow Did 3 People Seemingly Escape From Alcatraz?\\n'}, {'url': 'https://www.countryfaq.com/facts-about-the-roman-empire/', 'content': 'Facts about the Roman Empire. Explore some of the interesting, fun, cool facts bout the Roman Empire: 1. The Magnificent Roman Empire. The Roman Empire, a colossal entity of unparalleled grandeur, occupies an indomitable position within the annals of human history, a name that resonates resoundingly across the eons.'}]\u001b[0m\u001b[32;1m\u001b[1;3mRelevant Documents: 0,3,4,5\n",
- "Cited Documents: 0,3,4,5\n",
- "Answer: Here are some tips for writing an essay:\n",
- "- Start with a question that spurs some response.\n",
- "- Don't choose a topic at random, make sure you have a way in, a new insight or approach.\n",
- "- You don't need a complete thesis, just a gap to explore.\n",
- "- You can get ideas by talking to people, reading, doing and building things, and going places and seeing things.\n",
- "- You can improve the quality of your ideas by increasing the breadth and depth of what goes in.\n",
- "- You can get breadth by reading and talking about a wide range of topics.\n",
- "- You can get depth by doing and having to solve problems.\n",
- "- You can also get ideas by talking to people who make you have new ideas.\n",
- "\n",
- "Here are some fun facts about the Roman Empire:\n",
- "- At its peak, the empire stretched from North Africa to Britain, reigning over 60 million people.\n",
- "- The story of Rome's warrior founder and the twins and their wolf step-mother was depicted on Rome's first coins.\n",
- "- The Crossing of the Rhine in 405/6 AD brought around 100,000 barbarians into the Empire.\n",
- "Grounded answer: Here are some tips for writing an essay:\n",
- "- Start with a question that spurs some response.\n",
- "- Don't choose a topic at random, make sure you have a way in, a new insight or approach.\n",
- "- You don't need a complete thesis, just a gap to explore.\n",
- "- You can get ideas by talking to people, reading, doing and building things, and going places and seeing things.\n",
- "- You can improve the quality of your ideas by increasing the breadth and depth of what goes in.\n",
- "- You can get breadth by reading and talking about a wide range of topics.\n",
- "- You can get depth by doing and having to solve problems.\n",
- "- You can also get ideas by talking to people who make you have new ideas.\n",
- "\n",
- "Here are some fun facts about the Roman Empire:\n",
- "- At its peak, the empire stretched from North Africa to Britain, reigning over 60 million people.\n",
- "- The story of Rome's warrior founder and the twins and their wolf step-mother was depicted on Rome's first coins.\n",
- "- The Crossing of the Rhine in 405/6 AD brought around 100,000 barbarians into the Empire.\u001b[0m\n",
- "\n",
- "\u001b[1m> Finished chain.\u001b[0m\n"
- ]
- },
- {
- "output_type": "execute_result",
- "data": {
- "text/plain": [
- "\"Here are some tips for writing an essay:\\n- Start with a question that spurs some response.\\n- Don't choose a topic at random, make sure you have a way in, a new insight or approach.\\n- You don't need a complete thesis, just a gap to explore.\\n- You can get ideas by talking to people, reading, doing and building things, and going places and seeing things.\\n- You can improve the quality of your ideas by increasing the breadth and depth of what goes in.\\n- You can get breadth by reading and talking about a wide range of topics.\\n- You can get depth by doing and having to solve problems.\\n- You can also get ideas by talking to people who make you have new ideas.\\n\\nHere are some fun facts about the Roman Empire:\\n- At its peak, the empire stretched from North Africa to Britain, reigning over 60 million people.\\n- The story of Rome's warrior founder and the twins and their wolf step-mother was depicted on Rome's first coins.\\n- The Crossing of the Rhine in 405/6 AD brought around 100,000 barbarians into the Empire.\""
- ],
- "application/vnd.google.colaboratory.intrinsic+json": {
- "type": "string"
- }
- },
- "metadata": {},
- "execution_count": 14
- }
+ "text/plain": [
+ "\"Here are some tips for writing an essay:\\n- Start with a question that spurs some response.\\n- Don't choose a topic at random, make sure you have a way in, a new insight or approach.\\n- You don't need a complete thesis, just a gap to explore.\\n- You can get ideas by talking to people, reading, doing and building things, and going places and seeing things.\\n- You can improve the quality of your ideas by increasing the breadth and depth of what goes in.\\n- You can get breadth by reading and talking about a wide range of topics.\\n- You can get depth by doing and having to solve problems.\\n- You can also get ideas by talking to people who make you have new ideas.\\n\\nHere are some fun facts about the Roman Empire:\\n- At its peak, the empire stretched from North Africa to Britain, reigning over 60 million people.\\n- The story of Rome's warrior founder and the twins and their wolf step-mother was depicted on Rome's first coins.\\n- The Crossing of the Rhine in 405/6 AD brought around 100,000 barbarians into the Empire.\""
]
- },
- {
- "cell_type": "code",
- "source": [],
- "metadata": {
- "id": "DIP1YkXCg7rQ"
- },
- "execution_count": 14,
- "outputs": []
- },
- {
- "cell_type": "markdown",
- "source": [
- "A question that requires the large language model to use a custom tool."
- ],
- "metadata": {
- "id": "FhwS3VHvg3_l"
- }
- },
- {
- "cell_type": "code",
- "source": [
- "response = agent_executor.invoke({\n",
- " \"input\": \"Calculate the result of the random operation of 10 and 20. Then find a few fun facts about that number, as well as its prime factors.\",\n",
- " \"preamble\": preamble,\n",
- "})\n",
- "\n",
- "response['output']\n",
- "\n",
- "# note that the model uses a sequence of tools"
- ],
- "metadata": {
- "colab": {
- "base_uri": "https://localhost:8080/",
- "height": 645
- },
- "id": "79Dw6Zhrg3xI",
- "outputId": "a41c6de9-cb64-4cff-a58b-e9481d438f6c"
+ },
+ "execution_count": 14,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "response = agent_executor.invoke({\n",
+ " \"input\": \"I want to write an essay about the Roman Empire. Any tips for writing an essay? Any fun facts?\",\n",
+ " \"preamble\": preamble,\n",
+ "})\n",
+ "\n",
+ "response['output']\n",
+ "\n",
+ "# note that the model smartly looks in the vector db, and then online"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "FhwS3VHvg3_l"
+ },
+ "source": [
+ "A question that requires the large language model to use a custom tool."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 22,
+ "metadata": {
+ "colab": {
+ "base_uri": "https://localhost:8080/",
+ "height": 645
+ },
+ "id": "79Dw6Zhrg3xI",
+ "outputId": "a41c6de9-cb64-4cff-a58b-e9481d438f6c"
+ },
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "\n",
+ "\n",
+ "\u001b[1m> Entering new AgentExecutor chain...\u001b[0m\n",
+ "\u001b[32;1m\u001b[1;3m\n",
+ "First, I will calculate the result of the random operation between 10 and 20. Then, I will search for fun facts about that number and its prime factors.\n",
+ "{'tool_name': 'random_operation_tool', 'parameters': {'a': 10, 'b': 20}}\n",
+ "\u001b[0mrandom_operation_tool is not a valid tool, try one of [internet_search, vectorstore_search, python_interpreter, random_operation].\u001b[32;1m\u001b[1;3m\n",
+ "I received an error message when trying to use the random_operation_tool. I will now try using the python_interpreter tool to calculate the random operation between 10 and 20.\n",
+ "{'tool_name': 'python_interpreter', 'parameters': {'code': \"import random\\n\\n# Define the two numbers\\na = 10\\nb = 20\\n\\n# Calculate the random operation\\nresult = random.choice(['+', '-', '*', '/'])\\n\\n# Perform the operation\\nif result == '+':\\n answer = a + b\\nelif result == '-':\\n answer = a - b\\nelif result == '*':\\n answer = a * b\\nelif result == '/':\\n answer = a / b\\n\\nprint(f'The result of the random operation is {answer:.0f}')\"}}\n",
+ "\u001b[0m\u001b[38;5;200m\u001b[1;3mThe result of the random operation is 200\n",
+ "\u001b[0m\u001b[32;1m\u001b[1;3m\n",
+ "The result of the random operation is 200. Now I will search for fun facts about the number 200 and its prime factors.\n",
+ "{'tool_name': 'internet_search', 'parameters': {'query': 'fun facts about the number 200'}}\n",
+ "\u001b[0m\u001b[36;1m\u001b[1;3m[{'url': 'https://www.express.co.uk/life-style/top10facts/690340/Top-10-facts-number-200', 'content': \"Top 10 facts about the number 200 TODAY is the 200th day of 2016, so to celebrate let's have some facts about the number 200. By WILLIAM HARTSTON. 00:01, Mon, Jul 18, 2016.\"}, {'url': 'https://en.wikipedia.org/wiki/200_(number)', 'content': \"The number appears in the Padovan sequence, preceded by 86, 114, 151 (it is the sum of the first two of these). The sum of Euler's totient function φ(x) over the first twenty-five integers is 200. 200 is the smallest base 10 unprimeable number - it cannot be turned into a prime number by changing just one of its digits to any other digit.\"}, {'url': 'https://www.archimedes-lab.org/numbers/Num70_200.html', 'content': \"With 189 pages filled with an incredible variety of fun facts on numbers (and their peculiar properties), both mathematical and cultural, as well as tantalizing problems and anecdotes, there is much to learn for everyone. ... The number 200, according to Bullinger's study of biblical literature, signifies 'insufficiency'. The word 200 (ducenti) ...\"}, {'url': 'https://owlcation.com/misc/Over-200-Odd-Facts-Did-You-Know-Them', 'content': \"Over 200 odd facts about science, sports, history and more that you and your friends probably don't already know! ... Strange and Interesting Facts. ... Average number of people airborne over the U.S. at any given hour: 61,000. Portion of land in the U.S. owned by the government: 1/3. Ninety percent of New York City cabbies are recently arrived ...\"}, {'url': 'https://numbermatics.com/n/200/', 'content': 'Mathematical info, prime factorization, fun facts and numerical data for STEM, education and fun. Your guide to the number 200, an even composite number composed of two distinct primes. Mathematical info, prime factorization, fun facts and numerical data for STEM, education and fun. ... Number 200 - Facts about the integer. Retrieved 2 April ...'}]\u001b[0m\u001b[32;1m\u001b[1;3m\n",
+ "{'tool_name': 'internet_search', 'parameters': {'query': 'prime factors of 200'}}\n",
+ "\u001b[0m\u001b[36;1m\u001b[1;3m[{'url': 'https://www.cuemath.com/numbers/factors-of-200/', 'content': 'Therefore, the factors of 200 are 1, 2, 4, 5, 8, 10, 20, 25, 40, 50, 100, and 200. Which means the number 200 is an even composite number.'}, {'url': 'https://byjus.com/maths/factors-of-200/', 'content': \"The factors of 200 are 1, 2, 4, 5, 8, 10, 20, 25, 40, 50, 100 and 200. Visit BYJU'S to learn the pair factors and the prime factors of 200 with complete\\xa0...\"}, {'url': 'https://byjus.com/us/math/factors-of-200/', 'content': 'The factors of 200 are 1, 2, 4, 5, 8, 10, 20, 25, 40, 50, 100 and 200 because all these numbers divide the number 200 evenly.'}, {'url': 'https://homework.study.com/explanation/what-is-the-prime-factorization-of-200-using-exponents.html', 'content': 'The prime factorization of 200 using exponents is 2 3 ∗ 5 2 . First, we need to find the prime factorization of 200. 200 = 2 ∗ 100. 200 = 2 ∗ 2 ∗ 50.'}]\u001b[0m\u001b[32;1m\u001b[1;3mRelevant Documents: 1,3,4,6,7,8,9,10\n",
+ "Cited Documents: 1,3,4,6,7,8,9,10\n",
+ "Answer: The result of the random operation is **200**. Here are some fun facts about the number 200:\n",
+ "- It is the smallest base 10 unprimeable number, meaning it cannot be turned into a prime number by changing just one of its digits to any other digit.\n",
+ "- According to Bullinger's study of biblical literature, the number 200 signifies 'insufficiency'.\n",
+ "- The number 200 is an even composite number composed of two distinct primes.\n",
+ "\n",
+ "The prime factors of 200 are 2 and 5.\n",
+ "Grounded answer: The result of the random operation is **200**. Here are some fun facts about the number 200:\n",
+ "- It is the smallest base 10 unprimeable number, meaning it cannot be turned into a prime number by changing just one of its digits to any other digit.\n",
+ "- According to Bullinger's study of biblical literature, the number 200 signifies 'insufficiency'.\n",
+ "- The number 200 is an evencomposite number composed of two distinct primes.\n",
+ "\n",
+ "The prime factors of 200 are 2 and 5.\u001b[0m\n",
+ "\n",
+ "\u001b[1m> Finished chain.\u001b[0m\n"
+ ]
+ },
+ {
+ "data": {
+ "application/vnd.google.colaboratory.intrinsic+json": {
+ "type": "string"
},
- "execution_count": 22,
- "outputs": [
- {
- "output_type": "stream",
- "name": "stdout",
- "text": [
- "\n",
- "\n",
- "\u001b[1m> Entering new AgentExecutor chain...\u001b[0m\n",
- "\u001b[32;1m\u001b[1;3m\n",
- "First, I will calculate the result of the random operation between 10 and 20. Then, I will search for fun facts about that number and its prime factors.\n",
- "{'tool_name': 'random_operation_tool', 'parameters': {'a': 10, 'b': 20}}\n",
- "\u001b[0mrandom_operation_tool is not a valid tool, try one of [internet_search, vectorstore_search, python_interpreter, random_operation].\u001b[32;1m\u001b[1;3m\n",
- "I received an error message when trying to use the random_operation_tool. I will now try using the python_interpreter tool to calculate the random operation between 10 and 20.\n",
- "{'tool_name': 'python_interpreter', 'parameters': {'code': \"import random\\n\\n# Define the two numbers\\na = 10\\nb = 20\\n\\n# Calculate the random operation\\nresult = random.choice(['+', '-', '*', '/'])\\n\\n# Perform the operation\\nif result == '+':\\n answer = a + b\\nelif result == '-':\\n answer = a - b\\nelif result == '*':\\n answer = a * b\\nelif result == '/':\\n answer = a / b\\n\\nprint(f'The result of the random operation is {answer:.0f}')\"}}\n",
- "\u001b[0m\u001b[38;5;200m\u001b[1;3mThe result of the random operation is 200\n",
- "\u001b[0m\u001b[32;1m\u001b[1;3m\n",
- "The result of the random operation is 200. Now I will search for fun facts about the number 200 and its prime factors.\n",
- "{'tool_name': 'internet_search', 'parameters': {'query': 'fun facts about the number 200'}}\n",
- "\u001b[0m\u001b[36;1m\u001b[1;3m[{'url': 'https://www.express.co.uk/life-style/top10facts/690340/Top-10-facts-number-200', 'content': \"Top 10 facts about the number 200 TODAY is the 200th day of 2016, so to celebrate let's have some facts about the number 200. By WILLIAM HARTSTON. 00:01, Mon, Jul 18, 2016.\"}, {'url': 'https://en.wikipedia.org/wiki/200_(number)', 'content': \"The number appears in the Padovan sequence, preceded by 86, 114, 151 (it is the sum of the first two of these). The sum of Euler's totient function φ(x) over the first twenty-five integers is 200. 200 is the smallest base 10 unprimeable number - it cannot be turned into a prime number by changing just one of its digits to any other digit.\"}, {'url': 'https://www.archimedes-lab.org/numbers/Num70_200.html', 'content': \"With 189 pages filled with an incredible variety of fun facts on numbers (and their peculiar properties), both mathematical and cultural, as well as tantalizing problems and anecdotes, there is much to learn for everyone. ... The number 200, according to Bullinger's study of biblical literature, signifies 'insufficiency'. The word 200 (ducenti) ...\"}, {'url': 'https://owlcation.com/misc/Over-200-Odd-Facts-Did-You-Know-Them', 'content': \"Over 200 odd facts about science, sports, history and more that you and your friends probably don't already know! ... Strange and Interesting Facts. ... Average number of people airborne over the U.S. at any given hour: 61,000. Portion of land in the U.S. owned by the government: 1/3. Ninety percent of New York City cabbies are recently arrived ...\"}, {'url': 'https://numbermatics.com/n/200/', 'content': 'Mathematical info, prime factorization, fun facts and numerical data for STEM, education and fun. Your guide to the number 200, an even composite number composed of two distinct primes. Mathematical info, prime factorization, fun facts and numerical data for STEM, education and fun. ... Number 200 - Facts about the integer. Retrieved 2 April ...'}]\u001b[0m\u001b[32;1m\u001b[1;3m\n",
- "{'tool_name': 'internet_search', 'parameters': {'query': 'prime factors of 200'}}\n",
- "\u001b[0m\u001b[36;1m\u001b[1;3m[{'url': 'https://www.cuemath.com/numbers/factors-of-200/', 'content': 'Therefore, the factors of 200 are 1, 2, 4, 5, 8, 10, 20, 25, 40, 50, 100, and 200. Which means the number 200 is an even composite number.'}, {'url': 'https://byjus.com/maths/factors-of-200/', 'content': \"The factors of 200 are 1, 2, 4, 5, 8, 10, 20, 25, 40, 50, 100 and 200. Visit BYJU'S to learn the pair factors and the prime factors of 200 with complete\\xa0...\"}, {'url': 'https://byjus.com/us/math/factors-of-200/', 'content': 'The factors of 200 are 1, 2, 4, 5, 8, 10, 20, 25, 40, 50, 100 and 200 because all these numbers divide the number 200 evenly.'}, {'url': 'https://homework.study.com/explanation/what-is-the-prime-factorization-of-200-using-exponents.html', 'content': 'The prime factorization of 200 using exponents is 2 3 ∗ 5 2 . First, we need to find the prime factorization of 200. 200 = 2 ∗ 100. 200 = 2 ∗ 2 ∗ 50.'}]\u001b[0m\u001b[32;1m\u001b[1;3mRelevant Documents: 1,3,4,6,7,8,9,10\n",
- "Cited Documents: 1,3,4,6,7,8,9,10\n",
- "Answer: The result of the random operation is **200**. Here are some fun facts about the number 200:\n",
- "- It is the smallest base 10 unprimeable number, meaning it cannot be turned into a prime number by changing just one of its digits to any other digit.\n",
- "- According to Bullinger's study of biblical literature, the number 200 signifies 'insufficiency'.\n",
- "- The number 200 is an even composite number composed of two distinct primes.\n",
- "\n",
- "The prime factors of 200 are 2 and 5.\n",
- "Grounded answer: The result of the random operation is **200**. Here are some fun facts about the number 200:\n",
- "- It is the smallest base 10 unprimeable number, meaning it cannot be turned into a prime number by changing just one of its digits to any other digit.\n",
- "- According to Bullinger's study of biblical literature, the number 200 signifies 'insufficiency'.\n",
- "- The number 200 is an evencomposite number composed of two distinct primes.\n",
- "\n",
- "The prime factors of 200 are 2 and 5.\u001b[0m\n",
- "\n",
- "\u001b[1m> Finished chain.\u001b[0m\n"
- ]
- },
- {
- "output_type": "execute_result",
- "data": {
- "text/plain": [
- "\"The result of the random operation is **200**. Here are some fun facts about the number 200:\\n- It is the smallest base 10 unprimeable number, meaning it cannot be turned into a prime number by changing just one of its digits to any other digit.\\n- According to Bullinger's study of biblical literature, the number 200 signifies 'insufficiency'.\\n- The number 200 is an even composite number composed of two distinct primes.\\n\\nThe prime factors of 200 are 2 and 5.\""
- ],
- "application/vnd.google.colaboratory.intrinsic+json": {
- "type": "string"
- }
- },
- "metadata": {},
- "execution_count": 22
- }
+ "text/plain": [
+ "\"The result of the random operation is **200**. Here are some fun facts about the number 200:\\n- It is the smallest base 10 unprimeable number, meaning it cannot be turned into a prime number by changing just one of its digits to any other digit.\\n- According to Bullinger's study of biblical literature, the number 200 signifies 'insufficiency'.\\n- The number 200 is an even composite number composed of two distinct primes.\\n\\nThe prime factors of 200 are 2 and 5.\""
]
- },
- {
- "cell_type": "code",
- "source": [],
- "metadata": {
- "id": "V8LcAh8vaEqR"
- },
- "execution_count": 15,
- "outputs": []
- },
- {
- "cell_type": "markdown",
- "source": [
- "A question that requires the large language model to directly answer."
- ],
- "metadata": {
- "id": "n9nV_jiaAaD1"
- }
- },
- {
- "cell_type": "code",
- "source": [
- "response = agent_executor.invoke({\n",
- " \"input\": \"Hey how are you?\",\n",
- " \"preamble\": preamble,\n",
- "})\n",
- "\n",
- "response['output']\n",
- "\n",
- "# note that the modle can directly answer!"
- ],
- "metadata": {
- "colab": {
- "base_uri": "https://localhost:8080/",
- "height": 250
- },
- "id": "tRf6V3gJAZkO",
- "outputId": "aec1acdb-876c-48e9-a1f5-e3ba8a0f19ed"
+ },
+ "execution_count": 22,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "response = agent_executor.invoke({\n",
+ " \"input\": \"Calculate the result of the random operation of 10 and 20. Then find a few fun facts about that number, as well as its prime factors.\",\n",
+ " \"preamble\": preamble,\n",
+ "})\n",
+ "\n",
+ "response['output']\n",
+ "\n",
+ "# note that the model uses a sequence of tools"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "n9nV_jiaAaD1"
+ },
+ "source": [
+ "A question that requires the large language model to directly answer."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 23,
+ "metadata": {
+ "colab": {
+ "base_uri": "https://localhost:8080/",
+ "height": 250
+ },
+ "id": "tRf6V3gJAZkO",
+ "outputId": "aec1acdb-876c-48e9-a1f5-e3ba8a0f19ed"
+ },
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "\n",
+ "\n",
+ "\u001b[1m> Entering new AgentExecutor chain...\u001b[0m\n",
+ "\u001b[32;1m\u001b[1;3m\n",
+ "I will respond to the user's greeting.\n",
+ "{'tool_name': 'directly_answer', 'parameters': {}}\n",
+ "\u001b[0mdirectly_answer is not a valid tool, try one of [internet_search, vectorstore_search, python_interpreter, random_operation].\u001b[32;1m\u001b[1;3mRelevant Documents: None\n",
+ "Cited Documents: None\n",
+ "Answer: I'm an AI chatbot, so I don't have feelings as such, but I'm here to help you with your queries. How can I help?\n",
+ "Grounded answer: I'm an AI chatbot, so I don't have feelings as such, but I'm here to help you with your queries. How can I help?\u001b[0m\n",
+ "\n",
+ "\u001b[1m> Finished chain.\u001b[0m\n"
+ ]
+ },
+ {
+ "data": {
+ "application/vnd.google.colaboratory.intrinsic+json": {
+ "type": "string"
},
- "execution_count": 23,
- "outputs": [
- {
- "output_type": "stream",
- "name": "stdout",
- "text": [
- "\n",
- "\n",
- "\u001b[1m> Entering new AgentExecutor chain...\u001b[0m\n",
- "\u001b[32;1m\u001b[1;3m\n",
- "I will respond to the user's greeting.\n",
- "{'tool_name': 'directly_answer', 'parameters': {}}\n",
- "\u001b[0mdirectly_answer is not a valid tool, try one of [internet_search, vectorstore_search, python_interpreter, random_operation].\u001b[32;1m\u001b[1;3mRelevant Documents: None\n",
- "Cited Documents: None\n",
- "Answer: I'm an AI chatbot, so I don't have feelings as such, but I'm here to help you with your queries. How can I help?\n",
- "Grounded answer: I'm an AI chatbot, so I don't have feelings as such, but I'm here to help you with your queries. How can I help?\u001b[0m\n",
- "\n",
- "\u001b[1m> Finished chain.\u001b[0m\n"
- ]
- },
- {
- "output_type": "execute_result",
- "data": {
- "text/plain": [
- "\"I'm an AI chatbot, so I don't have feelings as such, but I'm here to help you with your queries. How can I help?\""
- ],
- "application/vnd.google.colaboratory.intrinsic+json": {
- "type": "string"
- }
- },
- "metadata": {},
- "execution_count": 23
- }
+ "text/plain": [
+ "\"I'm an AI chatbot, so I don't have feelings as such, but I'm here to help you with your queries. How can I help?\""
]
- },
- {
- "cell_type": "code",
- "source": [],
- "metadata": {
- "id": "hnLkln_ckYXJ"
- },
- "execution_count": null,
- "outputs": []
- },
- {
- "cell_type": "markdown",
- "source": [
- "## Ask a more complex question to the ReAct agent\n",
- "A question that requires using multipe tools, in sequence"
- ],
- "metadata": {
- "id": "3h-icRL2_5iu"
- }
- },
- {
- "cell_type": "code",
- "source": [
- "response = agent_executor.invoke({\n",
- " \"input\": \"In what year was the company that was founded as Sound of Music went public? What was its stock price in 2000 and 2010.\",\n",
- " \"preamble\": preamble,\n",
- "})\n",
- "\n",
- "response['output']"
- ],
- "metadata": {
- "colab": {
- "base_uri": "https://localhost:8080/",
- "height": 377
- },
- "id": "Q4IyoXXRaEoL",
- "outputId": "0e935f68-8b7c-4d9a-b8f1-1be57b7c9a1f"
+ },
+ "execution_count": 23,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "response = agent_executor.invoke({\n",
+ " \"input\": \"Hey how are you?\",\n",
+ " \"preamble\": preamble,\n",
+ "})\n",
+ "\n",
+ "response['output']\n",
+ "\n",
+ "# note that the modle can directly answer!"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "3h-icRL2_5iu"
+ },
+ "source": [
+ "## Ask a more complex question to the ReAct agent\n",
+ "A question that requires using multipe tools, in sequence"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 27,
+ "metadata": {
+ "colab": {
+ "base_uri": "https://localhost:8080/",
+ "height": 377
+ },
+ "id": "Q4IyoXXRaEoL",
+ "outputId": "0e935f68-8b7c-4d9a-b8f1-1be57b7c9a1f"
+ },
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "\n",
+ "\n",
+ "\u001b[1m> Entering new AgentExecutor chain...\u001b[0m\n",
+ "\u001b[32;1m\u001b[1;3m\n",
+ "I will search for the company that was founded as Sound of Music. Then, I will search for the year it went public. Finally, I will search for its stock price in 2000 and 2010.\n",
+ "{'tool_name': 'internet_search', 'parameters': {'query': 'company founded as Sound of Music'}}\n",
+ "\u001b[0m\u001b[36;1m\u001b[1;3m[{'url': 'https://www.mprnews.org/story/2012/05/15/best-buy-richard-schulze-legacy', 'content': 'Amazon is taking a piece of everyone\\'s business,\"\\nSome analysts have questioned the extent to which Schulze\\'s strong hand over the company has held it back in the face of digital competition. \"\\nSchulze hit on the \"Best Buy\" strategy when a twister ripped open a store and the company held a huge sale with low prices and lots of promotion. And Richard Schulz was able to outthink and outmaneuver the competition to put up Best Buy as the definitive place in the brick-and-mortar world to buy electronic equipment,\" Spector said.\\n Best Buy\\'s Richard Schulze: Stereo seller to retail giant\\nShare\\nIn 1966, Best Buy was a stereo specialty retailer called Sound of Music, founded by Richard Schulze in St. Paul.\\n Former CEO Anderson said it is possible that Schulze\\'s goals for the company were out of touch with the digital age, and that Schulze may have stuck around too long.\\n'}, {'url': 'https://corporate.bestbuy.com/tag/sound-of-music/', 'content': 'As we celebrate Best Buy’s 50th anniversary, we sat down with founder and Chairman Emeritus Dick Schulze and current Chairman and CEO Hubert Joly to talk about how we got to where we are today and get a glimpse into where we’re going next.\\n Sound of Music\\n22 Aug: Best Buy at 50: A Q&A with Founder Dick Schulze and CEO Hubert Joly\\nTechnology has changed a lot over the past half century, and so has Best Buy.\\n 14 Jun: 35 Years Ago Today, A Tornado Transformed Best Buy\\nOn the afternoon of June 14, 1981, a tornado slammed into the Minneapolis suburbs, mere miles from where the Best Buy corporate headquarters now stands.\\n Little did he know that it would become Best Buy, a nearly $40 billion company that now sells everything from TVs and laptops to drones and virtual-reality headsets.\\n It was the most significant tornado to hit the Minneapolis-St. Paul area in 20 years — and its wake would change the path of our company.\\n'}, {'url': 'https://historydraft.com/story/best-buy/timeline/841', 'content': \"Best Buy announced the shutdown of the Future Shop chain in Canada\\nOn March 28, 2015, Best Buy announced the shutdown of the Future Shop chain in Canada; 65 of its 131 former locations were converted into Best Buy locations, while the rest (primarily those in close proximity to an existing Best Buy) were closed permanently.\\n Best Buy Company, Inc\\nIn 1983, with seven stores and $10 million in annual sales, Sound of Music was renamed Best Buy Company, Inc.\\nBest Buy debuted on the New York Stock Exchange\\nBest Buy was taken public in 1985, and two years later it debuted on the New York Stock Exchange.\\n The company closed all of its Best Buy-branded stores in China\\nThe company closed all of its Best Buy-branded stores in China by February 2011, when it merged Best Buy China's operations with Jiangsu Five Star, which had become a wholly-owned subsidiary of Best Buy in 2009.\\n Best Buy announced that it would start selling musical instruments and related gear in over 80 of its retail stores\\nIn July 2008, Best Buy announced that it would start selling musical instruments and related gear in over 80 of its retail stores, making the company the second-largest musical-instrument distributor in the US.\\n Best Buy hired Virtucom Group to revamp Best Buy's website and handle all of the company's online content\\nIn January 2004, Best Buy hired Virtucom Group to revamp Best Buy's website and handle all of the company's online content.\\n\"}, {'url': 'https://en.wikipedia.org/wiki/Best_Buy', 'content': 'Under the Geek Squad brand, Best Buy offers computer repair, warranty service, and accidental service plans.[2] Best Buy provides an online community forum for members, where consumers can discuss product experiences, ask questions, and get answers from other members or retail product experts.[82]\\nThe building exteriors of Best Buy-branded stores are typically light brown, with the entrance designed to look like a blue box emerging from the structure.[83] Corporate employees operated under a results only work environment from 2005 until March 2013, when the management style was abandoned by Best Buy CEO Hubert Joly.[84][85]\\nAs of October 29, 2016, Best Buy operated 1,026 Best Buy, 331 Best Buy Mobile stand-alone stores, and 28 stand-alone Pacific Sales stores in the US.[2] Best Buy also operated: 135 Best Buy and 53 Best Buy Mobile stand-alone stores in Canada; and 18 Best Buy stores and 5 Best Buy Express stores in Mexico.[2] Best Buy exited the European market in April 2013, selling its stake in the business back to its partner Carphone Warehouse.[71][72]\\nHouse brands[edit]\\nBest Buy also produces products under eight house brands:[2]\\nControversies[edit]\\nWarranty[edit]\\n The company, in announcing the result, said it was focusing more on digital media in its marketing, moving away from newspaper, magazine, and television advertising.[73]\\nOn March 28, 2015, Best Buy announced the shutdown of the Future Shop chain in Canada; 65 of its 131 former locations were converted into Best Buy locations, while the rest (primarily those in close proximity to an existing Best Buy) were closed permanently.[74]\\nOn March 1, 2018, the company announced that it would shut down its 250 standalone Best Buy Mobile stores in the United States by the end of May, due to low revenue and high costs. In August 2022, Best Buy said it would be laying off employees across the country after warnings of weaker sales, and the company cut its forecast for the remainder of 2022.[79]\\nOn October 13, 2023, Best Buy announced that it would phase out the sale of physical home media in early 2024, citing changes in the market due to the prevalence of streaming video on demand services.[80][81]\\nCorporate affairs[edit]\\nBusiness operations[edit]\\nBest Buy sells consumer electronics and a variety of related merchandise, including software, video games, music, mobile phones, digital cameras, car stereos, and video cameras, in addition to home appliances (washing machines, dryers, and refrigerators), in a noncommissioned sales environment.[2] The Best Buy Mobile stores were reported to account for 1% of the company\\'s revenue.[75]\\nOn May 9, 2018, the company unveiled a new logo for the first time in nearly three decades.[76]\\nOn July 2, 2018, Best Buy announced it was cutting the amount of store space devoted to selling physical music, citing the popularity of streaming services as having reduced sales.[77]\\nOn April 15, 2019, Best Buy announced that in June 2019, its current CFO, Corie Barry, would replace Hubert Joly[5] who held the position of CEO since August 2012. The customer was indicted for possession of child pornography, although the judge in the case later threw out nearly all the evidence against the defendant due to \"false and misleading statements\" made by an FBI agent while trying to secure a search warrant for the customer\\'s house, and the government ultimately dropped the case.[97]\\nPrivacy[edit]\\nOn October 20, 2023, CBC News released the results of a Marketplace investigation which found that that Best Buy technicians had viewed private files, such as intimate photos, on customer devices.'}, {'url': 'https://www.company-histories.com/Best-Buy-Co-Inc-Company-History.html', 'content': 'As the Best Buy chain pushed past the 500-store mark in 2003 with the opening of 67 new stores in the United States, including the first stores in Alaska, Idaho, Utah, and West Virginia, the situation at the Musicland chains was deteriorating. Returning to the Core: 2003 and Beyond\\nDespite the completion of this acquisition, Best Buy pushed ahead with a previously planned expansion of the Best Buy chain into Canada, opening eight stores in the Toronto area in the fall of 2002. Significant changes were made to the product mix, particularly by eliminating slower selling product lines and models; a greater emphasis was placed on selling service plans to customers; and \"high touch\" areas were added to the stores to help sell the burgeoning array of digital consumer products, such as cameras, cellular phones, satellite systems, and the fast-selling DVD player (first introduced in 1996) for which customers often needed more assistance. This rapid growth in digital product sales also inspired a shift in the overall product mix: sales of consumer electronics products (33 percent of the total) surpassed the sales of home office products (31 percent) for the first time (in 1999 these figures were 27 percent and 36 percent, respectively). Magnolia was founded in 1954 by Len Tweten, who built the firm into one of the most respected audio-video retailers in the nation based on the high quality of its merchandise, its dedication to exceptional customer service, and its renowned in-house repair/installation department.'}]\u001b[0m\u001b[32;1m\u001b[1;3m\n",
+ "I have found that Best Buy was founded as Sound of Music. Now, I will search for the year it went public and its stock price in 2000 and 2010.\n",
+ "{'tool_name': 'internet_search', 'parameters': {'query': 'Best Buy went public'}}\n",
+ "\u001b[0m\u001b[36;1m\u001b[1;3m[{'url': 'https://www.zippia.com/best-buy-careers-1455/history/', 'content': 'Meantime, Best Buy was taken public in 1985, raising $8 million through an IPO, and two years later gained a listing on the New York Stock Exchange (NYSE). ... Best Buy may also be known as or be related to Best Buy, Best Buy Co Inc, Best Buy Co., Inc., Sound of Music (1966-1983) Best Buy Co. Superstores (1983-1984) Best Buy Superstores ...'}, {'url': 'https://atouchofbusiness.com/companies/best-buy/', 'content': 'Public Listing and IPO: Best Buy went public in 1985 and listed on the NYSE in 1987. Conceptual Changes: The late 1980s and 1990s brought new store formats and growth, with revenues surpassing $1 billion. Overcoming Market Challenges. Supplier Relations: Mid-1990s challenges led to a merchandising revamp.'}, {'url': 'https://www.encyclopedia.com/social-sciences-and-law/economics-business-and-labor/businesses-and-occupations/best-buy-co-inc', 'content': 'Best Buy grew rapidly with $28.5 million in sales in 1984—compared to $9.9 million in 1983. In 1985, with 9 stores, Best Buy became a public company. By 1987, Best Buy had 24 stores and sales of $240 million, but it was beginning to feel the crunch as other rapidly expanding consumer electronics retailers pushed their way into the market.'}, {'url': 'https://www.company-histories.com/Best-Buy-Co-Inc-Company-History.html', 'content': 'As the Best Buy chain pushed past the 500-store mark in 2003 with the opening of 67 new stores in the United States, including the first stores in Alaska, Idaho, Utah, and West Virginia, the situation at the Musicland chains was deteriorating. Returning to the Core: 2003 and Beyond\\nDespite the completion of this acquisition, Best Buy pushed ahead with a previously planned expansion of the Best Buy chain into Canada, opening eight stores in the Toronto area in the fall of 2002. Significant changes were made to the product mix, particularly by eliminating slower selling product lines and models; a greater emphasis was placed on selling service plans to customers; and \"high touch\" areas were added to the stores to help sell the burgeoning array of digital consumer products, such as cameras, cellular phones, satellite systems, and the fast-selling DVD player (first introduced in 1996) for which customers often needed more assistance. This rapid growth in digital product sales also inspired a shift in the overall product mix: sales of consumer electronics products (33 percent of the total) surpassed the sales of home office products (31 percent) for the first time (in 1999 these figures were 27 percent and 36 percent, respectively). Magnolia was founded in 1954 by Len Tweten, who built the firm into one of the most respected audio-video retailers in the nation based on the high quality of its merchandise, its dedication to exceptional customer service, and its renowned in-house repair/installation department.'}, {'url': 'https://en.wikipedia.org/wiki/Best_Buy', 'content': 'Under the Geek Squad brand, Best Buy offers computer repair, warranty service, and accidental service plans.[2] Best Buy provides an online community forum for members, where consumers can discuss product experiences, ask questions, and get answers from other members or retail product experts.[82]\\nThe building exteriors of Best Buy-branded stores are typically light brown, with the entrance designed to look like a blue box emerging from the structure.[83] Corporate employees operated under a results only work environment from 2005 until March 2013, when the management style was abandoned by Best Buy CEO Hubert Joly.[84][85]\\nAs of October 29, 2016, Best Buy operated 1,026 Best Buy, 331 Best Buy Mobile stand-alone stores, and 28 stand-alone Pacific Sales stores in the US.[2] Best Buy also operated: 135 Best Buy and 53 Best Buy Mobile stand-alone stores in Canada; and 18 Best Buy stores and 5 Best Buy Express stores in Mexico.[2] Best Buy exited the European market in April 2013, selling its stake in the business back to its partner Carphone Warehouse.[71][72]\\nHouse brands[edit]\\nBest Buy also produces products under eight house brands:[2]\\nControversies[edit]\\nWarranty[edit]\\n The company, in announcing the result, said it was focusing more on digital media in its marketing, moving away from newspaper, magazine, and television advertising.[73]\\nOn March 28, 2015, Best Buy announced the shutdown of the Future Shop chain in Canada; 65 of its 131 former locations were converted into Best Buy locations, while the rest (primarily those in close proximity to an existing Best Buy) were closed permanently.[74]\\nOn March 1, 2018, the company announced that it would shut down its 250 standalone Best Buy Mobile stores in the United States by the end of May, due to low revenue and high costs. In August 2022, Best Buy said it would be laying off employees across the country after warnings of weaker sales, and the company cut its forecast for the remainder of 2022.[79]\\nOn October 13, 2023, Best Buy announced that it would phase out the sale of physical home media in early 2024, citing changes in the market due to the prevalence of streaming video on demand services.[80][81]\\nCorporate affairs[edit]\\nBusiness operations[edit]\\nBest Buy sells consumer electronics and a variety of related merchandise, including software, video games, music, mobile phones, digital cameras, car stereos, and video cameras, in addition to home appliances (washing machines, dryers, and refrigerators), in a noncommissioned sales environment.[2] The Best Buy Mobile stores were reported to account for 1% of the company\\'s revenue.[75]\\nOn May 9, 2018, the company unveiled a new logo for the first time in nearly three decades.[76]\\nOn July 2, 2018, Best Buy announced it was cutting the amount of store space devoted to selling physical music, citing the popularity of streaming services as having reduced sales.[77]\\nOn April 15, 2019, Best Buy announced that in June 2019, its current CFO, Corie Barry, would replace Hubert Joly[5] who held the position of CEO since August 2012. The customer was indicted for possession of child pornography, although the judge in the case later threw out nearly all the evidence against the defendant due to \"false and misleading statements\" made by an FBI agent while trying to secure a search warrant for the customer\\'s house, and the government ultimately dropped the case.[97]\\nPrivacy[edit]\\nOn October 20, 2023, CBC News released the results of a Marketplace investigation which found that that Best Buy technicians had viewed private files, such as intimate photos, on customer devices.'}]\u001b[0m\u001b[32;1m\u001b[1;3m\n",
+ "{'tool_name': 'internet_search', 'parameters': {'query': 'Best Buy stock price in 2000 and 2010'}}\n",
+ "\u001b[0m\u001b[36;1m\u001b[1;3m[{'url': 'https://finance.yahoo.com/quote/BBY/history/', 'content': 'Discover historical prices for BBY stock on Yahoo Finance. View daily, weekly or monthly format back to when Best Buy Co., Inc. stock was issued.'}, {'url': 'https://www.macrotrends.net/stocks/charts/BBY/best-buy/stock-price-history', 'content': 'Best Buy - 39 Year Stock Price History | BBY. Prices ... 2010, 25.1877, 25.7674, 31.2441, 20.2713, 22.3228 ... 2000, 16.0083, 15.2882, 22.8658, 5.9318, 7.8595, -\\xa0...'}, {'url': 'https://investors.bestbuy.com/investor-relations/stock-info/quote-and-chart/', 'content': 'Price 79.30. Change -0.88. Volume 666,622.'}, {'url': 'https://companiesmarketcap.com/best-buy/stock-price-history/', 'content': 'Stock price history for Best Buy (BBY). Highest end of day price: $138.00 USD on 2021-11-22. Lowest end of day price: $0.14 USD on 1985-05-02\\xa0...'}, {'url': 'https://www.netcials.com/stock-price-chart-history-nyse/BBY-Best-Buy-Co-Inc/', 'content': '1 Best Buy Co Inc (BBY) 20 Years Stock Chart History ; 2000, 16.24 (17.6%), 23.199 (29.07%) ; 2001, 14.80 (-8.87%), 20.1319 (-13.22%) ; 2002, 14.59 (-1.42%)\\xa0...'}]\u001b[0m\u001b[32;1m\u001b[1;3mRelevant Documents: 0,1,2,3,4,5,6,7,11,13,14\n",
+ "Cited Documents: 0,1,2,3,4,5,6,7,11,14\n",
+ "Answer: Best Buy, which was founded as Sound of Music, went public in 1985. Its stock price in 2000 was between $5.93 and $16.24 and in 2010, it was between $20.27 and $31.24.\n",
+ "Grounded answer: Best Buy, which was founded as Sound of Music, went public in 1985. Its stock price in 2000 was between $5.93 and $16.24 and in 2010, it was between $20.27 and $31.24.\u001b[0m\n",
+ "\n",
+ "\u001b[1m> Finished chain.\u001b[0m\n"
+ ]
+ },
+ {
+ "data": {
+ "application/vnd.google.colaboratory.intrinsic+json": {
+ "type": "string"
},
- "execution_count": 27,
- "outputs": [
- {
- "output_type": "stream",
- "name": "stdout",
- "text": [
- "\n",
- "\n",
- "\u001b[1m> Entering new AgentExecutor chain...\u001b[0m\n",
- "\u001b[32;1m\u001b[1;3m\n",
- "I will search for the company that was founded as Sound of Music. Then, I will search for the year it went public. Finally, I will search for its stock price in 2000 and 2010.\n",
- "{'tool_name': 'internet_search', 'parameters': {'query': 'company founded as Sound of Music'}}\n",
- "\u001b[0m\u001b[36;1m\u001b[1;3m[{'url': 'https://www.mprnews.org/story/2012/05/15/best-buy-richard-schulze-legacy', 'content': 'Amazon is taking a piece of everyone\\'s business,\"\\nSome analysts have questioned the extent to which Schulze\\'s strong hand over the company has held it back in the face of digital competition. \"\\nSchulze hit on the \"Best Buy\" strategy when a twister ripped open a store and the company held a huge sale with low prices and lots of promotion. And Richard Schulz was able to outthink and outmaneuver the competition to put up Best Buy as the definitive place in the brick-and-mortar world to buy electronic equipment,\" Spector said.\\n Best Buy\\'s Richard Schulze: Stereo seller to retail giant\\nShare\\nIn 1966, Best Buy was a stereo specialty retailer called Sound of Music, founded by Richard Schulze in St. Paul.\\n Former CEO Anderson said it is possible that Schulze\\'s goals for the company were out of touch with the digital age, and that Schulze may have stuck around too long.\\n'}, {'url': 'https://corporate.bestbuy.com/tag/sound-of-music/', 'content': 'As we celebrate Best Buy’s 50th anniversary, we sat down with founder and Chairman Emeritus Dick Schulze and current Chairman and CEO Hubert Joly to talk about how we got to where we are today and get a glimpse into where we’re going next.\\n Sound of Music\\n22 Aug: Best Buy at 50: A Q&A with Founder Dick Schulze and CEO Hubert Joly\\nTechnology has changed a lot over the past half century, and so has Best Buy.\\n 14 Jun: 35 Years Ago Today, A Tornado Transformed Best Buy\\nOn the afternoon of June 14, 1981, a tornado slammed into the Minneapolis suburbs, mere miles from where the Best Buy corporate headquarters now stands.\\n Little did he know that it would become Best Buy, a nearly $40 billion company that now sells everything from TVs and laptops to drones and virtual-reality headsets.\\n It was the most significant tornado to hit the Minneapolis-St. Paul area in 20 years — and its wake would change the path of our company.\\n'}, {'url': 'https://historydraft.com/story/best-buy/timeline/841', 'content': \"Best Buy announced the shutdown of the Future Shop chain in Canada\\nOn March 28, 2015, Best Buy announced the shutdown of the Future Shop chain in Canada; 65 of its 131 former locations were converted into Best Buy locations, while the rest (primarily those in close proximity to an existing Best Buy) were closed permanently.\\n Best Buy Company, Inc\\nIn 1983, with seven stores and $10 million in annual sales, Sound of Music was renamed Best Buy Company, Inc.\\nBest Buy debuted on the New York Stock Exchange\\nBest Buy was taken public in 1985, and two years later it debuted on the New York Stock Exchange.\\n The company closed all of its Best Buy-branded stores in China\\nThe company closed all of its Best Buy-branded stores in China by February 2011, when it merged Best Buy China's operations with Jiangsu Five Star, which had become a wholly-owned subsidiary of Best Buy in 2009.\\n Best Buy announced that it would start selling musical instruments and related gear in over 80 of its retail stores\\nIn July 2008, Best Buy announced that it would start selling musical instruments and related gear in over 80 of its retail stores, making the company the second-largest musical-instrument distributor in the US.\\n Best Buy hired Virtucom Group to revamp Best Buy's website and handle all of the company's online content\\nIn January 2004, Best Buy hired Virtucom Group to revamp Best Buy's website and handle all of the company's online content.\\n\"}, {'url': 'https://en.wikipedia.org/wiki/Best_Buy', 'content': 'Under the Geek Squad brand, Best Buy offers computer repair, warranty service, and accidental service plans.[2] Best Buy provides an online community forum for members, where consumers can discuss product experiences, ask questions, and get answers from other members or retail product experts.[82]\\nThe building exteriors of Best Buy-branded stores are typically light brown, with the entrance designed to look like a blue box emerging from the structure.[83] Corporate employees operated under a results only work environment from 2005 until March 2013, when the management style was abandoned by Best Buy CEO Hubert Joly.[84][85]\\nAs of October 29, 2016, Best Buy operated 1,026 Best Buy, 331 Best Buy Mobile stand-alone stores, and 28 stand-alone Pacific Sales stores in the US.[2] Best Buy also operated: 135 Best Buy and 53 Best Buy Mobile stand-alone stores in Canada; and 18 Best Buy stores and 5 Best Buy Express stores in Mexico.[2] Best Buy exited the European market in April 2013, selling its stake in the business back to its partner Carphone Warehouse.[71][72]\\nHouse brands[edit]\\nBest Buy also produces products under eight house brands:[2]\\nControversies[edit]\\nWarranty[edit]\\n The company, in announcing the result, said it was focusing more on digital media in its marketing, moving away from newspaper, magazine, and television advertising.[73]\\nOn March 28, 2015, Best Buy announced the shutdown of the Future Shop chain in Canada; 65 of its 131 former locations were converted into Best Buy locations, while the rest (primarily those in close proximity to an existing Best Buy) were closed permanently.[74]\\nOn March 1, 2018, the company announced that it would shut down its 250 standalone Best Buy Mobile stores in the United States by the end of May, due to low revenue and high costs. In August 2022, Best Buy said it would be laying off employees across the country after warnings of weaker sales, and the company cut its forecast for the remainder of 2022.[79]\\nOn October 13, 2023, Best Buy announced that it would phase out the sale of physical home media in early 2024, citing changes in the market due to the prevalence of streaming video on demand services.[80][81]\\nCorporate affairs[edit]\\nBusiness operations[edit]\\nBest Buy sells consumer electronics and a variety of related merchandise, including software, video games, music, mobile phones, digital cameras, car stereos, and video cameras, in addition to home appliances (washing machines, dryers, and refrigerators), in a noncommissioned sales environment.[2] The Best Buy Mobile stores were reported to account for 1% of the company\\'s revenue.[75]\\nOn May 9, 2018, the company unveiled a new logo for the first time in nearly three decades.[76]\\nOn July 2, 2018, Best Buy announced it was cutting the amount of store space devoted to selling physical music, citing the popularity of streaming services as having reduced sales.[77]\\nOn April 15, 2019, Best Buy announced that in June 2019, its current CFO, Corie Barry, would replace Hubert Joly[5] who held the position of CEO since August 2012. The customer was indicted for possession of child pornography, although the judge in the case later threw out nearly all the evidence against the defendant due to \"false and misleading statements\" made by an FBI agent while trying to secure a search warrant for the customer\\'s house, and the government ultimately dropped the case.[97]\\nPrivacy[edit]\\nOn October 20, 2023, CBC News released the results of a Marketplace investigation which found that that Best Buy technicians had viewed private files, such as intimate photos, on customer devices.'}, {'url': 'https://www.company-histories.com/Best-Buy-Co-Inc-Company-History.html', 'content': 'As the Best Buy chain pushed past the 500-store mark in 2003 with the opening of 67 new stores in the United States, including the first stores in Alaska, Idaho, Utah, and West Virginia, the situation at the Musicland chains was deteriorating. Returning to the Core: 2003 and Beyond\\nDespite the completion of this acquisition, Best Buy pushed ahead with a previously planned expansion of the Best Buy chain into Canada, opening eight stores in the Toronto area in the fall of 2002. Significant changes were made to the product mix, particularly by eliminating slower selling product lines and models; a greater emphasis was placed on selling service plans to customers; and \"high touch\" areas were added to the stores to help sell the burgeoning array of digital consumer products, such as cameras, cellular phones, satellite systems, and the fast-selling DVD player (first introduced in 1996) for which customers often needed more assistance. This rapid growth in digital product sales also inspired a shift in the overall product mix: sales of consumer electronics products (33 percent of the total) surpassed the sales of home office products (31 percent) for the first time (in 1999 these figures were 27 percent and 36 percent, respectively). Magnolia was founded in 1954 by Len Tweten, who built the firm into one of the most respected audio-video retailers in the nation based on the high quality of its merchandise, its dedication to exceptional customer service, and its renowned in-house repair/installation department.'}]\u001b[0m\u001b[32;1m\u001b[1;3m\n",
- "I have found that Best Buy was founded as Sound of Music. Now, I will search for the year it went public and its stock price in 2000 and 2010.\n",
- "{'tool_name': 'internet_search', 'parameters': {'query': 'Best Buy went public'}}\n",
- "\u001b[0m\u001b[36;1m\u001b[1;3m[{'url': 'https://www.zippia.com/best-buy-careers-1455/history/', 'content': 'Meantime, Best Buy was taken public in 1985, raising $8 million through an IPO, and two years later gained a listing on the New York Stock Exchange (NYSE). ... Best Buy may also be known as or be related to Best Buy, Best Buy Co Inc, Best Buy Co., Inc., Sound of Music (1966-1983) Best Buy Co. Superstores (1983-1984) Best Buy Superstores ...'}, {'url': 'https://atouchofbusiness.com/companies/best-buy/', 'content': 'Public Listing and IPO: Best Buy went public in 1985 and listed on the NYSE in 1987. Conceptual Changes: The late 1980s and 1990s brought new store formats and growth, with revenues surpassing $1 billion. Overcoming Market Challenges. Supplier Relations: Mid-1990s challenges led to a merchandising revamp.'}, {'url': 'https://www.encyclopedia.com/social-sciences-and-law/economics-business-and-labor/businesses-and-occupations/best-buy-co-inc', 'content': 'Best Buy grew rapidly with $28.5 million in sales in 1984—compared to $9.9 million in 1983. In 1985, with 9 stores, Best Buy became a public company. By 1987, Best Buy had 24 stores and sales of $240 million, but it was beginning to feel the crunch as other rapidly expanding consumer electronics retailers pushed their way into the market.'}, {'url': 'https://www.company-histories.com/Best-Buy-Co-Inc-Company-History.html', 'content': 'As the Best Buy chain pushed past the 500-store mark in 2003 with the opening of 67 new stores in the United States, including the first stores in Alaska, Idaho, Utah, and West Virginia, the situation at the Musicland chains was deteriorating. Returning to the Core: 2003 and Beyond\\nDespite the completion of this acquisition, Best Buy pushed ahead with a previously planned expansion of the Best Buy chain into Canada, opening eight stores in the Toronto area in the fall of 2002. Significant changes were made to the product mix, particularly by eliminating slower selling product lines and models; a greater emphasis was placed on selling service plans to customers; and \"high touch\" areas were added to the stores to help sell the burgeoning array of digital consumer products, such as cameras, cellular phones, satellite systems, and the fast-selling DVD player (first introduced in 1996) for which customers often needed more assistance. This rapid growth in digital product sales also inspired a shift in the overall product mix: sales of consumer electronics products (33 percent of the total) surpassed the sales of home office products (31 percent) for the first time (in 1999 these figures were 27 percent and 36 percent, respectively). Magnolia was founded in 1954 by Len Tweten, who built the firm into one of the most respected audio-video retailers in the nation based on the high quality of its merchandise, its dedication to exceptional customer service, and its renowned in-house repair/installation department.'}, {'url': 'https://en.wikipedia.org/wiki/Best_Buy', 'content': 'Under the Geek Squad brand, Best Buy offers computer repair, warranty service, and accidental service plans.[2] Best Buy provides an online community forum for members, where consumers can discuss product experiences, ask questions, and get answers from other members or retail product experts.[82]\\nThe building exteriors of Best Buy-branded stores are typically light brown, with the entrance designed to look like a blue box emerging from the structure.[83] Corporate employees operated under a results only work environment from 2005 until March 2013, when the management style was abandoned by Best Buy CEO Hubert Joly.[84][85]\\nAs of October 29, 2016, Best Buy operated 1,026 Best Buy, 331 Best Buy Mobile stand-alone stores, and 28 stand-alone Pacific Sales stores in the US.[2] Best Buy also operated: 135 Best Buy and 53 Best Buy Mobile stand-alone stores in Canada; and 18 Best Buy stores and 5 Best Buy Express stores in Mexico.[2] Best Buy exited the European market in April 2013, selling its stake in the business back to its partner Carphone Warehouse.[71][72]\\nHouse brands[edit]\\nBest Buy also produces products under eight house brands:[2]\\nControversies[edit]\\nWarranty[edit]\\n The company, in announcing the result, said it was focusing more on digital media in its marketing, moving away from newspaper, magazine, and television advertising.[73]\\nOn March 28, 2015, Best Buy announced the shutdown of the Future Shop chain in Canada; 65 of its 131 former locations were converted into Best Buy locations, while the rest (primarily those in close proximity to an existing Best Buy) were closed permanently.[74]\\nOn March 1, 2018, the company announced that it would shut down its 250 standalone Best Buy Mobile stores in the United States by the end of May, due to low revenue and high costs. In August 2022, Best Buy said it would be laying off employees across the country after warnings of weaker sales, and the company cut its forecast for the remainder of 2022.[79]\\nOn October 13, 2023, Best Buy announced that it would phase out the sale of physical home media in early 2024, citing changes in the market due to the prevalence of streaming video on demand services.[80][81]\\nCorporate affairs[edit]\\nBusiness operations[edit]\\nBest Buy sells consumer electronics and a variety of related merchandise, including software, video games, music, mobile phones, digital cameras, car stereos, and video cameras, in addition to home appliances (washing machines, dryers, and refrigerators), in a noncommissioned sales environment.[2] The Best Buy Mobile stores were reported to account for 1% of the company\\'s revenue.[75]\\nOn May 9, 2018, the company unveiled a new logo for the first time in nearly three decades.[76]\\nOn July 2, 2018, Best Buy announced it was cutting the amount of store space devoted to selling physical music, citing the popularity of streaming services as having reduced sales.[77]\\nOn April 15, 2019, Best Buy announced that in June 2019, its current CFO, Corie Barry, would replace Hubert Joly[5] who held the position of CEO since August 2012. The customer was indicted for possession of child pornography, although the judge in the case later threw out nearly all the evidence against the defendant due to \"false and misleading statements\" made by an FBI agent while trying to secure a search warrant for the customer\\'s house, and the government ultimately dropped the case.[97]\\nPrivacy[edit]\\nOn October 20, 2023, CBC News released the results of a Marketplace investigation which found that that Best Buy technicians had viewed private files, such as intimate photos, on customer devices.'}]\u001b[0m\u001b[32;1m\u001b[1;3m\n",
- "{'tool_name': 'internet_search', 'parameters': {'query': 'Best Buy stock price in 2000 and 2010'}}\n",
- "\u001b[0m\u001b[36;1m\u001b[1;3m[{'url': 'https://finance.yahoo.com/quote/BBY/history/', 'content': 'Discover historical prices for BBY stock on Yahoo Finance. View daily, weekly or monthly format back to when Best Buy Co., Inc. stock was issued.'}, {'url': 'https://www.macrotrends.net/stocks/charts/BBY/best-buy/stock-price-history', 'content': 'Best Buy - 39 Year Stock Price History | BBY. Prices ... 2010, 25.1877, 25.7674, 31.2441, 20.2713, 22.3228 ... 2000, 16.0083, 15.2882, 22.8658, 5.9318, 7.8595, -\\xa0...'}, {'url': 'https://investors.bestbuy.com/investor-relations/stock-info/quote-and-chart/', 'content': 'Price 79.30. Change -0.88. Volume 666,622.'}, {'url': 'https://companiesmarketcap.com/best-buy/stock-price-history/', 'content': 'Stock price history for Best Buy (BBY). Highest end of day price: $138.00 USD on 2021-11-22. Lowest end of day price: $0.14 USD on 1985-05-02\\xa0...'}, {'url': 'https://www.netcials.com/stock-price-chart-history-nyse/BBY-Best-Buy-Co-Inc/', 'content': '1 Best Buy Co Inc (BBY) 20 Years Stock Chart History ; 2000, 16.24 (17.6%), 23.199 (29.07%) ; 2001, 14.80 (-8.87%), 20.1319 (-13.22%) ; 2002, 14.59 (-1.42%)\\xa0...'}]\u001b[0m\u001b[32;1m\u001b[1;3mRelevant Documents: 0,1,2,3,4,5,6,7,11,13,14\n",
- "Cited Documents: 0,1,2,3,4,5,6,7,11,14\n",
- "Answer: Best Buy, which was founded as Sound of Music, went public in 1985. Its stock price in 2000 was between $5.93 and $16.24 and in 2010, it was between $20.27 and $31.24.\n",
- "Grounded answer: Best Buy, which was founded as Sound of Music, went public in 1985. Its stock price in 2000 was between $5.93 and $16.24 and in 2010, it was between $20.27 and $31.24.\u001b[0m\n",
- "\n",
- "\u001b[1m> Finished chain.\u001b[0m\n"
- ]
- },
- {
- "output_type": "execute_result",
- "data": {
- "text/plain": [
- "'Best Buy, which was founded as Sound of Music, went public in 1985. Its stock price in 2000 was between $5.93 and $16.24 and in 2010, it was between $20.27 and $31.24.'"
- ],
- "application/vnd.google.colaboratory.intrinsic+json": {
- "type": "string"
- }
- },
- "metadata": {},
- "execution_count": 27
- }
+ "text/plain": [
+ "'Best Buy, which was founded as Sound of Music, went public in 1985. Its stock price in 2000 was between $5.93 and $16.24 and in 2010, it was between $20.27 and $31.24.'"
]
- },
- {
- "cell_type": "code",
- "source": [],
- "metadata": {
- "id": "L1BsKueTaEmg"
- },
- "execution_count": null,
- "outputs": []
- },
- {
- "cell_type": "markdown",
- "source": [
- "## Have a multi-turn conversation with the ReAct agent\n",
- "The chat history enables you to have multi-turn conversations with the ReAct agent."
- ],
- "metadata": {
- "id": "53FDSFzQBFyf"
- }
- },
- {
- "cell_type": "code",
- "source": [
- "# Step 1: Construct the chat history as a list of LangChain Messages, ending with the last user message\n",
- "from langchain_core.messages import HumanMessage, AIMessage\n",
- "\n",
- "chat_history = [\n",
- " HumanMessage(content=\"I'm considering switching to Oracle for my CRM.\"),\n",
- " AIMessage(content=\"That sounds like a good idea! How can I help you?\"),\n",
- " HumanMessage(content=\"Recap me all the info you can find about their offering.\"),\n",
- "]\n",
- "\n",
- "prompt = ChatPromptTemplate.from_messages(chat_history)"
- ],
- "metadata": {
- "id": "1iYyAExsaEk9"
- },
- "execution_count": 28,
- "outputs": []
- },
- {
- "cell_type": "code",
- "source": [
- "# Step 2: When you make the agent, specify the chat_history as the prompt, e.g.\n",
- "# Create the ReAct agent\n",
- "agent = create_cohere_react_agent(\n",
- " llm=llm,\n",
- " tools=[internet_search, vectorstore_search, python_tool],\n",
- " prompt=prompt,\n",
- ")\n",
- "\n",
- "agent_executor = AgentExecutor(agent=agent, tools=[internet_search, vectorstore_search, python_tool], verbose=True)"
- ],
- "metadata": {
- "id": "fyv_1LedaEi7"
- },
- "execution_count": 29,
- "outputs": []
- },
- {
- "cell_type": "code",
- "source": [
- "# Step 3: When you invoke the agent_executor there's no need to pass anything else into invoke\n",
- "response = agent_executor.invoke({\n",
- " \"preamble\": preamble,\n",
- "})\n",
- "\n",
- "response['output']"
- ],
- "metadata": {
- "id": "-ZCFj-m5nqFw",
- "colab": {
- "base_uri": "https://localhost:8080/",
- "height": 538
- },
- "outputId": "3d71931e-1644-49b9-903c-4ff117787868"
+ },
+ "execution_count": 27,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "response = agent_executor.invoke({\n",
+ " \"input\": \"In what year was the company that was founded as Sound of Music went public? What was its stock price in 2000 and 2010.\",\n",
+ " \"preamble\": preamble,\n",
+ "})\n",
+ "\n",
+ "response['output']"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "53FDSFzQBFyf"
+ },
+ "source": [
+ "## Have a multi-turn conversation with the ReAct agent\n",
+ "The chat history enables you to have multi-turn conversations with the ReAct agent."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 28,
+ "metadata": {
+ "id": "1iYyAExsaEk9"
+ },
+ "outputs": [],
+ "source": [
+ "# Step 1: Construct the chat history as a list of LangChain Messages, ending with the last user message\n",
+ "from langchain_core.messages import HumanMessage, AIMessage\n",
+ "\n",
+ "chat_history = [\n",
+ " HumanMessage(content=\"I'm considering switching to Oracle for my CRM.\"),\n",
+ " AIMessage(content=\"That sounds like a good idea! How can I help you?\"),\n",
+ " HumanMessage(content=\"Recap me all the info you can find about their offering.\"),\n",
+ "]\n",
+ "\n",
+ "prompt = ChatPromptTemplate.from_messages(chat_history)"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 29,
+ "metadata": {
+ "id": "fyv_1LedaEi7"
+ },
+ "outputs": [],
+ "source": [
+ "# Step 2: When you make the agent, specify the chat_history as the prompt, e.g.\n",
+ "# Create the ReAct agent\n",
+ "agent = create_cohere_react_agent(\n",
+ " llm=llm,\n",
+ " tools=[internet_search, vectorstore_search, python_tool],\n",
+ " prompt=prompt,\n",
+ ")\n",
+ "\n",
+ "agent_executor = AgentExecutor(agent=agent, tools=[internet_search, vectorstore_search, python_tool], verbose=True)"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 30,
+ "metadata": {
+ "colab": {
+ "base_uri": "https://localhost:8080/",
+ "height": 538
+ },
+ "id": "-ZCFj-m5nqFw",
+ "outputId": "3d71931e-1644-49b9-903c-4ff117787868"
+ },
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "\n",
+ "\n",
+ "\u001b[1m> Entering new AgentExecutor chain...\u001b[0m\n",
+ "\u001b[32;1m\u001b[1;3m\n",
+ "I will search for 'Oracle CRM offering' and relay the information I find to the user.\n",
+ "{'tool_name': 'internet_search', 'parameters': {'query': 'Oracle CRM offering'}}\n",
+ "\u001b[0m\u001b[36;1m\u001b[1;3m[{'url': 'https://docs.oracle.com/en/applications/siebel/', 'content': 'Siebel CRM delivers a combination of transactional, analytical, and engagement features to manage all customer-facing operations. With solutions tailored to more than 20 industries, Siebel CRM delivers comprehensive on-premise CRM solutions that are tailored industry solutions with role-based customer intelligence and pre-built integration.'}, {'url': 'https://www.softwareadvice.com/resources/breaking-down-oracle-crm/', 'content': \"Oracle's Marketing Cloud provides a comprehensive set of tools that cover a range of digital marketing needs. This includes tools for cross-channel marketing, i.e., marketing automation for both B2B and B2C marketers, as well as data management, content marketing and social media marketing. Cross-Channel Marketing.\"}, {'url': 'https://www.trustradius.com/products/oracle-crm-on-demand/reviews?qs=pros-and-cons', 'content': \"What is Oracle CRM On Demand?The basis of this offering is the Market2Lead product that Oracle acquired in 2010. It has now been fully integrated with Oracle's On Demand CRM product and is a full-featured marketing automation product with features from lead management and nurturing, to measuring marketing ROI.\"}, {'url': 'https://www.oracle.com/cx/siebel/', 'content': 'In addition to standard CRM functionality, this industry solution includes asset, premises, and contracts management; work orders; oil well management and oil field service; B2C and B2B web portals; and credit and fraud management capabilities.\\nDesigned for pharmaceutical sales, Siebel CRM Life Sciences provides personalized content delivery tools to help sales and marketing teams deliver the right messages during each customer interaction.\\n Marketing, sales, and customer service applications are fully integrated and designed to manage the complex interactions and relationships between brand owners, their partners (including brokers and distributors), their customers, and the end consumer.\\n It leverages data and AI to transform CX–launching offers, acquiring and retaining customers, omnichannel ecommerce and customer care, and fulfilling and monetizing services.\\n It leverages data and AI to transform CX–launching offers, acquiring and retaining customers, omnichannel ecommerce and customer care, and fulfilling and monetizing services.\\n Provide world-class citizen services while delivering comprehensive, cost-efficient case management and policy management, including social services, justice and public safety, constituent services/311, self-service citizen portals, tax and revenue, and licensing and permitting.\\n'}, {'url': 'https://www.suretysystems.com/insights/oracle-customer-relationship-management-a-complete-overview/', 'content': 'Effective CRM systems offer the following features for enterprise users: Simple, easy-to-use interface; ... Oracle CRM simplifies customer relationship management by enhancing customer interactions and improving customer satisfaction and sales growth. Oracle Client Experience (CX), a connected suite of tools that transcends standard CRM ...'}]\u001b[0m\u001b[32;1m\u001b[1;3mRelevant Documents: 0,1,2,3,4\n",
+ "Cited Documents: 0,1,2,3,4\n",
+ "Answer: Oracle's CRM offering includes the following:\n",
+ "- Marketing Cloud, which provides a comprehensive set of tools that cover a range of digital marketing needs, including cross-channel marketing, marketing automation, data management, content marketing and social media marketing\n",
+ "- CRM On Demand, which is a full-featured marketing automation product with features from lead management and nurturing, to measuring marketing ROI\n",
+ "- Siebel CRM, which delivers a combination of transactional, analytical, and engagement features to manage all customer-facing operations, with solutions tailored to more than 20 industries\n",
+ "- Siebel CRM Life Sciences, which provides personalised content delivery tools to help sales and marketing teams deliver the right messages during each customer interaction\n",
+ "- Oracle Client Experience (CX), a connected suite of tools that transcends standard CRM\n",
+ "Grounded answer: Oracle's CRM offering includes the following:\n",
+ "- Marketing Cloud, which provides a comprehensive set of tools that cover a range of digital marketing needs, including cross-channel marketing, marketing automation, data management, content marketing and social media marketing\n",
+ "- CRM On Demand, which is a full-featured marketing automation product with features from lead management and nurturing, to measuring marketing ROI\n",
+ "- Siebel CRM, which delivers a combination of transactional, analytical, and engagement features to manage all customer-facing operations, with solutions tailored to more than 20 industries\n",
+ "- Siebel CRM Life Sciences, which provides personalised content delivery tools to help sales and marketing teams deliver the right messages during each customer interaction\n",
+ "- Oracle Client Experience (CX), a connected suite of tools that transcends standard CRM\u001b[0m\n",
+ "\n",
+ "\u001b[1m> Finished chain.\u001b[0m\n"
+ ]
+ },
+ {
+ "data": {
+ "application/vnd.google.colaboratory.intrinsic+json": {
+ "type": "string"
},
- "execution_count": 30,
- "outputs": [
- {
- "output_type": "stream",
- "name": "stdout",
- "text": [
- "\n",
- "\n",
- "\u001b[1m> Entering new AgentExecutor chain...\u001b[0m\n",
- "\u001b[32;1m\u001b[1;3m\n",
- "I will search for 'Oracle CRM offering' and relay the information I find to the user.\n",
- "{'tool_name': 'internet_search', 'parameters': {'query': 'Oracle CRM offering'}}\n",
- "\u001b[0m\u001b[36;1m\u001b[1;3m[{'url': 'https://docs.oracle.com/en/applications/siebel/', 'content': 'Siebel CRM delivers a combination of transactional, analytical, and engagement features to manage all customer-facing operations. With solutions tailored to more than 20 industries, Siebel CRM delivers comprehensive on-premise CRM solutions that are tailored industry solutions with role-based customer intelligence and pre-built integration.'}, {'url': 'https://www.softwareadvice.com/resources/breaking-down-oracle-crm/', 'content': \"Oracle's Marketing Cloud provides a comprehensive set of tools that cover a range of digital marketing needs. This includes tools for cross-channel marketing, i.e., marketing automation for both B2B and B2C marketers, as well as data management, content marketing and social media marketing. Cross-Channel Marketing.\"}, {'url': 'https://www.trustradius.com/products/oracle-crm-on-demand/reviews?qs=pros-and-cons', 'content': \"What is Oracle CRM On Demand?The basis of this offering is the Market2Lead product that Oracle acquired in 2010. It has now been fully integrated with Oracle's On Demand CRM product and is a full-featured marketing automation product with features from lead management and nurturing, to measuring marketing ROI.\"}, {'url': 'https://www.oracle.com/cx/siebel/', 'content': 'In addition to standard CRM functionality, this industry solution includes asset, premises, and contracts management; work orders; oil well management and oil field service; B2C and B2B web portals; and credit and fraud management capabilities.\\nDesigned for pharmaceutical sales, Siebel CRM Life Sciences provides personalized content delivery tools to help sales and marketing teams deliver the right messages during each customer interaction.\\n Marketing, sales, and customer service applications are fully integrated and designed to manage the complex interactions and relationships between brand owners, their partners (including brokers and distributors), their customers, and the end consumer.\\n It leverages data and AI to transform CX–launching offers, acquiring and retaining customers, omnichannel ecommerce and customer care, and fulfilling and monetizing services.\\n It leverages data and AI to transform CX–launching offers, acquiring and retaining customers, omnichannel ecommerce and customer care, and fulfilling and monetizing services.\\n Provide world-class citizen services while delivering comprehensive, cost-efficient case management and policy management, including social services, justice and public safety, constituent services/311, self-service citizen portals, tax and revenue, and licensing and permitting.\\n'}, {'url': 'https://www.suretysystems.com/insights/oracle-customer-relationship-management-a-complete-overview/', 'content': 'Effective CRM systems offer the following features for enterprise users: Simple, easy-to-use interface; ... Oracle CRM simplifies customer relationship management by enhancing customer interactions and improving customer satisfaction and sales growth. Oracle Client Experience (CX), a connected suite of tools that transcends standard CRM ...'}]\u001b[0m\u001b[32;1m\u001b[1;3mRelevant Documents: 0,1,2,3,4\n",
- "Cited Documents: 0,1,2,3,4\n",
- "Answer: Oracle's CRM offering includes the following:\n",
- "- Marketing Cloud, which provides a comprehensive set of tools that cover a range of digital marketing needs, including cross-channel marketing, marketing automation, data management, content marketing and social media marketing\n",
- "- CRM On Demand, which is a full-featured marketing automation product with features from lead management and nurturing, to measuring marketing ROI\n",
- "- Siebel CRM, which delivers a combination of transactional, analytical, and engagement features to manage all customer-facing operations, with solutions tailored to more than 20 industries\n",
- "- Siebel CRM Life Sciences, which provides personalised content delivery tools to help sales and marketing teams deliver the right messages during each customer interaction\n",
- "- Oracle Client Experience (CX), a connected suite of tools that transcends standard CRM\n",
- "Grounded answer: Oracle's CRM offering includes the following:\n",
- "- Marketing Cloud, which provides a comprehensive set of tools that cover a range of digital marketing needs, including cross-channel marketing, marketing automation, data management, content marketing and social media marketing\n",
- "- CRM On Demand, which is a full-featured marketing automation product with features from lead management and nurturing, to measuring marketing ROI\n",
- "- Siebel CRM, which delivers a combination of transactional, analytical, and engagement features to manage all customer-facing operations, with solutions tailored to more than 20 industries\n",
- "- Siebel CRM Life Sciences, which provides personalised content delivery tools to help sales and marketing teams deliver the right messages during each customer interaction\n",
- "- Oracle Client Experience (CX), a connected suite of tools that transcends standard CRM\u001b[0m\n",
- "\n",
- "\u001b[1m> Finished chain.\u001b[0m\n"
- ]
- },
- {
- "output_type": "execute_result",
- "data": {
- "text/plain": [
- "\"Oracle's CRM offering includes the following:\\n- Marketing Cloud, which provides a comprehensive set of tools that cover a range of digital marketing needs, including cross-channel marketing, marketing automation, data management, content marketing and social media marketing\\n- CRM On Demand, which is a full-featured marketing automation product with features from lead management and nurturing, to measuring marketing ROI\\n- Siebel CRM, which delivers a combination of transactional, analytical, and engagement features to manage all customer-facing operations, with solutions tailored to more than 20 industries\\n- Siebel CRM Life Sciences, which provides personalised content delivery tools to help sales and marketing teams deliver the right messages during each customer interaction\\n- Oracle Client Experience (CX), a connected suite of tools that transcends standard CRM\""
- ],
- "application/vnd.google.colaboratory.intrinsic+json": {
- "type": "string"
- }
- },
- "metadata": {},
- "execution_count": 30
- }
+ "text/plain": [
+ "\"Oracle's CRM offering includes the following:\\n- Marketing Cloud, which provides a comprehensive set of tools that cover a range of digital marketing needs, including cross-channel marketing, marketing automation, data management, content marketing and social media marketing\\n- CRM On Demand, which is a full-featured marketing automation product with features from lead management and nurturing, to measuring marketing ROI\\n- Siebel CRM, which delivers a combination of transactional, analytical, and engagement features to manage all customer-facing operations, with solutions tailored to more than 20 industries\\n- Siebel CRM Life Sciences, which provides personalised content delivery tools to help sales and marketing teams deliver the right messages during each customer interaction\\n- Oracle Client Experience (CX), a connected suite of tools that transcends standard CRM\""
]
- },
- {
- "cell_type": "code",
- "source": [],
- "metadata": {
- "id": "8Zh01M4hWw3L"
- },
- "execution_count": 30,
- "outputs": []
- },
- {
- "cell_type": "code",
- "source": [],
- "metadata": {
- "id": "0qEPY6xj_qwr"
- },
- "execution_count": null,
- "outputs": []
+ },
+ "execution_count": 30,
+ "metadata": {},
+ "output_type": "execute_result"
}
- ]
-}
\ No newline at end of file
+ ],
+ "source": [
+ "# Step 3: When you invoke the agent_executor there's no need to pass anything else into invoke\n",
+ "response = agent_executor.invoke({\n",
+ " \"preamble\": preamble,\n",
+ "})\n",
+ "\n",
+ "response['output']"
+ ]
+ }
+ ],
+ "metadata": {
+ "colab": {
+ "provenance": []
+ },
+ "kernelspec": {
+ "display_name": "Python 3 (ipykernel)",
+ "language": "python",
+ "name": "python3"
+ },
+ "language_info": {
+ "codemirror_mode": {
+ "name": "ipython",
+ "version": 3
+ },
+ "file_extension": ".py",
+ "mimetype": "text/x-python",
+ "name": "python",
+ "nbconvert_exporter": "python",
+ "pygments_lexer": "ipython3",
+ "version": "3.8.16"
+ }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 4
+}
diff --git a/notebooks/agents/Vanilla_Tool_Use.ipynb b/notebooks/agents/Vanilla_Tool_Use.ipynb
index 26aa6fdc..69d21c5c 100644
--- a/notebooks/agents/Vanilla_Tool_Use.ipynb
+++ b/notebooks/agents/Vanilla_Tool_Use.ipynb
@@ -1,674 +1,631 @@
{
- "cells": [
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "\n",
- " \n",
- ""
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {
- "id": "YN-eakfxtLGd"
- },
- "source": [
- "# Tool Use\n",
- "\n",
- "Tool use allows customers to **connect their large language models to external tools like search engines, APIs, functions, databases**, etc.\n",
- "\n",
- "This allows the customer's model to unlock a richer set of behaviors by leveraging data stored in tools, taking actions through APIs, interacting with a vector database, querying a search engine, etc.\n",
- "\n",
- "This is particularly valuable for enterprise customers, since a lot of enterprise data lives in external sources.\n",
- "\n",
- "Tool Use consists of 4 steps:\n",
- "- Step 1: the user configures the request to the model\n",
- "- Step 2: the **model smartly decides which tool(s) to use and how**\n",
- "- Step 3: the tool calls are executed to mock database\n",
- "- Step 4: the **model generates a final answer with precise citations based on the tool results**"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "colab": {
- "base_uri": "https://localhost:8080/"
- },
- "id": "us5dkKrLCbXW",
- "outputId": "94c97f62-77fb-4492-a4e4-d9eeee4e438c"
- },
- "outputs": [
- {
- "name": "stdout",
- "output_type": "stream",
- "text": [
- "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m52.8/52.8 kB\u001b[0m \u001b[31m1.6 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
- "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m3.1/3.1 MB\u001b[0m \u001b[31m28.1 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
- "\u001b[?25h"
- ]
- }
- ],
- "source": [
- "# we'll use Cohere to do Tool Use\n",
- "# TODO: upgrade to \"cohere>5\"\n",
- "%pip install \"cohere<5\" --quiet"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "id": "j0DC3iPftLGo"
- },
- "outputs": [],
- "source": [
- "import cohere, json\n",
- "API_KEY = \"...\" # fill in your Cohere API key here\n",
- "co = cohere.Client(API_KEY)"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {
- "id": "0T7yc1PltLGp"
- },
- "source": [
- "## Step 0: Create a mock database\n",
- "First, we'll define the mock data that our tools will query. This data represents sales reports and a product catalog."
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "id": "yZffY8xItLGp"
- },
- "outputs": [],
- "source": [
- "# Mock database containing daily sales reports\n",
- "sales_database = {\n",
- " '2023-09-28': {\n",
- " 'total_sales_amount': 5000,\n",
- " 'total_units_sold': 100,\n",
- " },\n",
- " '2023-09-29': {\n",
- " 'total_sales_amount': 10000,\n",
- " 'total_units_sold': 250,\n",
- " },\n",
- " '2023-09-30': {\n",
- " 'total_sales_amount': 8000,\n",
- " 'total_units_sold': 200,\n",
- " }\n",
- "}\n",
- "\n",
- "# Mock product catalog\n",
- "product_catalog = {\n",
- " 'Electronics': [\n",
- " {'product_id': 'E1001', 'name': 'Smartphone', 'price': 500, 'stock_level': 20},\n",
- " {'product_id': 'E1002', 'name': 'Laptop', 'price': 1000, 'stock_level': 15},\n",
- " {'product_id': 'E1003', 'name': 'Tablet', 'price': 300, 'stock_level': 25},\n",
- " ],\n",
- " 'Clothing': [\n",
- " {'product_id': 'C1001', 'name': 'T-Shirt', 'price': 20, 'stock_level': 100},\n",
- " {'product_id': 'C1002', 'name': 'Jeans', 'price': 50, 'stock_level': 80},\n",
- " {'product_id': 'C1003', 'name': 'Jacket', 'price': 100, 'stock_level': 40},\n",
- " ]\n",
- "}"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {
- "id": "6TGWYiOdtLGp"
- },
- "source": [
- "Now, we'll define the tools that simulate querying this database. \n",
- "You could for example use the API of an enterprise sales platform.\n",
- "\n"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "id": "YuIH4us8tLGp"
- },
- "outputs": [],
- "source": [
- "def query_daily_sales_report(day: str) -> dict:\n",
- " \"\"\"\n",
- " Function to retrieve the sales report for the given day\n",
- " \"\"\"\n",
- " report = sales_database.get(day, {})\n",
- " if report:\n",
- " return {\n",
- " 'date': day,\n",
- " 'summary': f\"Total Sales Amount: {report['total_sales_amount']}, Total Units Sold: {report['total_units_sold']}\"\n",
- " }\n",
- " else:\n",
- " return {'date': day, 'summary': 'No sales data available for this day.'}\n",
- "\n",
- "\n",
- "def query_product_catalog(category: str) -> dict:\n",
- " \"\"\"\n",
- " Function to retrieve products for the given category\n",
- " \"\"\"\n",
- " products = product_catalog.get(category, [])\n",
- " return {\n",
- " 'category': category,\n",
- " 'products': products\n",
- " }\n",
- "\n",
- "\n",
- "functions_map = {\n",
- " \"query_daily_sales_report\": query_daily_sales_report,\n",
- " \"query_product_catalog\": query_product_catalog\n",
- "}"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {
- "id": "HZRhTu4ftLGp"
- },
- "source": [
- "## Step 1 - the user configures the request to the model\n",
- "\n",
- "The developer provides a few things to the model:\n",
- "- A preamble containing instructions about the task and the desired style for the output.\n",
- "- The user request.\n",
- "- A list of tools to the model.\n",
- "- (Optionally) a chat history for the model to work with.\n",
- "\n",
- "\n",
- "You can specify one or many tools to the model. Every tool needs to be described with a JSON schema, indicating the tool name, description, and parameters (code snippets below).\n",
- "\n",
- "In our example, we provide two tools to the model: `daily_sales_report` and `product_catalog`.\n"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "id": "aIk-of_OtLGp"
- },
- "outputs": [],
- "source": [
- "# tool descriptions that the model has access to\n",
- "# note: Cohere always adds a \"directly_answer\" tool under the hood, so that the model can decide to not leverage any tool, if they're not needed.\n",
- "tools = [\n",
- " {\n",
- " \"name\": \"query_daily_sales_report\",\n",
- " \"description\": \"Connects to a database to retrieve overall sales volumes and sales information for a given day.\",\n",
- " \"parameter_definitions\": {\n",
- " \"day\": {\n",
- " \"description\": \"Retrieves sales data for this day, formatted as YYYY-MM-DD.\",\n",
- " \"type\": \"str\",\n",
- " \"required\": True\n",
- " }\n",
- " }\n",
- " },\n",
- " {\n",
- " \"name\": \"query_product_catalog\",\n",
- " \"description\": \"Connects to a a product catalog with information about all the products being sold, including categories, prices, and stock levels.\",\n",
- " \"parameter_definitions\": {\n",
- " \"category\": {\n",
- " \"description\": \"Retrieves product information data for all products in this category.\",\n",
- " \"type\": \"str\",\n",
- " \"required\": True\n",
- " }\n",
- " }\n",
- " }\n",
- "]"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {
- "id": "k2AHJRnztLGp"
- },
- "source": [
- "Now let's define the user request. \n",
- "\n",
- "In our example we'll use: \"Can you provide a sales summary for 29th September 2023, and also give me the details of all products in the 'Electronics' category that were sold that day, including their prices and stock levels?\"\n",
- "\n",
- "Only a langage model with Tool Use can answer this request: it requires looking up information in the right external tools (step 2), and then providing a final answer based on the tool results (step 4)."
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "id": "JuDgJ7fjtLGq"
- },
- "outputs": [],
- "source": [
- "# preamble containing instructions about the task and the desired style for the output.\n",
- "preamble = \"\"\"\n",
- "## Task & Context\n",
- "You help people answer their questions and other requests interactively. You will be asked a very wide array of requests on all kinds of topics. You will be equipped with a wide range of search engines or similar tools to help you, which you use to research your answer. You should focus on serving the user's needs as best you can, which will be wide-ranging.\n",
- "\n",
- "## Style Guide\n",
- "Unless the user asks for a different style of answer, you should answer in full sentences, using proper grammar and spelling.\n",
- "\"\"\"\n",
- "\n",
- "# user request\n",
- "message = \"Can you provide a sales summary for 29th September 2023, and also give me some details about the products in the 'Electronics' category, for example their prices and stock levels?\""
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {
- "id": "1NhW-G_JtLGq"
- },
- "source": [
- "## Step 2 – the model smartly decides which tool(s) to use and how\n",
- "The model intelligently selects the right tool(s) to call -- and the right parameters for each tool call -- based on the content of the user message."
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "colab": {
- "base_uri": "https://localhost:8080/"
- },
- "id": "o79_n99GtLGq",
- "outputId": "81789d00-01b9-4c17-d1b0-1668d75a2b86"
- },
- "outputs": [
- {
- "name": "stdout",
- "output_type": "stream",
- "text": [
- "The model recommends doing the following tool calls:\n",
- "cohere.ToolCall {\n",
- "\tname: query_daily_sales_report\n",
- "\tparameters: {'day': '2023-09-29'}\n",
- "\tgeneration_id: eaf955e3-623d-4796-bf51-23b07c66ed2c\n",
- "}\n",
- "cohere.ToolCall {\n",
- "\tname: query_product_catalog\n",
- "\tparameters: {'category': 'Electronics'}\n",
- "\tgeneration_id: eaf955e3-623d-4796-bf51-23b07c66ed2c\n",
- "}\n"
- ]
- }
- ],
- "source": [
- "response = co.chat(\n",
- " message=message,\n",
- " tools=tools,\n",
- " preamble=preamble,\n",
- " model=\"command-r\"\n",
- ")\n",
- "\n",
- "# Note that the Cohere Chat API also exposes:\n",
- "# - stream (for streaming mode)\n",
- "# - chat_history\n",
- "# - among other parameters\n",
- "# See https://docs.cohere.com/reference/chat for details.\n",
- "\n",
- "print(\"The model recommends doing the following tool calls:\")\n",
- "print(\"\\n\".join(str(tool_call) for tool_call in response.tool_calls))"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {
- "id": "md_9QPcxtLGq"
- },
- "source": [
- "## Step 3 – the tool calls are executed\n",
- "\n",
- "You can then execute the appropriate calls, using the tool calls and tool parameters generated by the model. \n",
- "These tool calls return tool results that will be fed to the model in Step 4."
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "colab": {
- "base_uri": "https://localhost:8080/"
- },
- "id": "1LuDCRpFtLGr",
- "outputId": "42ead35e-225a-4b9a-c954-b526f2865350"
- },
- "outputs": [
- {
- "name": "stdout",
- "output_type": "stream",
- "text": [
- "= running tool query_daily_sales_report, with parameters: {'day': '2023-09-29'}\n",
- "== tool results: [{'date': '2023-09-29', 'summary': 'Total Sales Amount: 10000, Total Units Sold: 250'}]\n",
- "= running tool query_product_catalog, with parameters: {'category': 'Electronics'}\n",
- "== tool results: [{'category': 'Electronics', 'products': [{'product_id': 'E1001', 'name': 'Smartphone', 'price': 500, 'stock_level': 20}, {'product_id': 'E1002', 'name': 'Laptop', 'price': 1000, 'stock_level': 15}, {'product_id': 'E1003', 'name': 'Tablet', 'price': 300, 'stock_level': 25}]}]\n",
- "Tool results that will be fed back to the model in step 4:\n",
- "[\n",
- " {\n",
- " \"call\": {\n",
- " \"name\": \"query_daily_sales_report\",\n",
- " \"parameters\": {\n",
- " \"day\": \"2023-09-29\"\n",
- " },\n",
- " \"generation_id\": \"eaf955e3-623d-4796-bf51-23b07c66ed2c\"\n",
- " },\n",
- " \"outputs\": [\n",
- " {\n",
- " \"date\": \"2023-09-29\",\n",
- " \"summary\": \"Total Sales Amount: 10000, Total Units Sold: 250\"\n",
- " }\n",
- " ]\n",
- " },\n",
- " {\n",
- " \"call\": {\n",
- " \"name\": \"query_product_catalog\",\n",
- " \"parameters\": {\n",
- " \"category\": \"Electronics\"\n",
- " },\n",
- " \"generation_id\": \"eaf955e3-623d-4796-bf51-23b07c66ed2c\"\n",
- " },\n",
- " \"outputs\": [\n",
- " {\n",
- " \"category\": \"Electronics\",\n",
- " \"products\": [\n",
- " {\n",
- " \"product_id\": \"E1001\",\n",
- " \"name\": \"Smartphone\",\n",
- " \"price\": 500,\n",
- " \"stock_level\": 20\n",
- " },\n",
- " {\n",
- " \"product_id\": \"E1002\",\n",
- " \"name\": \"Laptop\",\n",
- " \"price\": 1000,\n",
- " \"stock_level\": 15\n",
- " },\n",
- " {\n",
- " \"product_id\": \"E1003\",\n",
- " \"name\": \"Tablet\",\n",
- " \"price\": 300,\n",
- " \"stock_level\": 25\n",
- " }\n",
- " ]\n",
- " }\n",
- " ]\n",
- " }\n",
- "]\n"
- ]
- }
- ],
- "source": [
- "tool_results = []\n",
- "# Iterate over the tool calls generated by the model\n",
- "for tool_call in response.tool_calls:\n",
- " # here is where you would call the tool recommended by the model, using the parameters recommended by the model\n",
- " print(f\"= running tool {tool_call.name}, with parameters: {tool_call.parameters}\")\n",
- " output = functions_map[tool_call.name](**tool_call.parameters)\n",
- " # store the output in a list\n",
- " outputs = [output]\n",
- " print(f\"== tool results: {outputs}\")\n",
- " # store your tool results in this format\n",
- " tool_results.append({\n",
- " \"call\": tool_call,\n",
- " \"outputs\": outputs\n",
- " })\n",
- "\n",
- "print(\"Tool results that will be fed back to the model in step 4:\")\n",
- "print(json.dumps(tool_results, indent=4))"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {
- "id": "8cKlLk18tLGr"
- },
- "source": [
- "## Step 4 - the model generates a final answer based on the tool results\n",
- "Finally, the developer calls the Cohere model, providing the tools results, in order to generate the model's final answer. \n",
- "\n",
- "Bonus: At Cohere, all Tool Use calls come with... **precise citations**! 🎉\n",
- "The model cites which tool results were used to generate the final answer. \n",
- "These citations make it easy to check where the model’s generated response claims are coming from. \n",
- "They help users gain visibility into the model reasoning, and sanity check the final model generation. \n",
- "These citations are optional — you can decide to ignore them.\n",
- "\n"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "id": "MKnjXVfXtLGr"
- },
- "outputs": [],
- "source": [
- "response = co.chat(\n",
- " message=message,\n",
- " tools=tools,\n",
- " tool_results=tool_results,\n",
- " preamble=preamble,\n",
- " model=\"command-r\",\n",
- " temperature=0.3\n",
- ")"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "colab": {
- "base_uri": "https://localhost:8080/"
- },
- "id": "jlxKTsaztLGr",
- "outputId": "b2cd8667-bca9-4928-c423-61930b4b49fa"
- },
- "outputs": [
- {
- "name": "stdout",
- "output_type": "stream",
- "text": [
- "Final answer:\n",
- "On the 29th of September 2023, there were 10,000 sales with 250 units sold. \n",
- "\n",
- "The Electronics category contains three products. There are details below:\n",
- "\n",
- "| Product Name | Price | Stock Level |\n",
- "| ------------ | ----- | ----------- |\n",
- "| Smartphone | 500 | 20 |\n",
- "| Laptop | 1000 | 15 |\n",
- "| Tablet | 300 | 25 | \n",
- "\n",
- "The total stock level for Electronics items is 50.\n"
- ]
- }
- ],
- "source": [
- "print(\"Final answer:\")\n",
- "print(response.text)"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {
- "id": "niMkAr2PN9j4"
- },
- "source": [
- "## Bonus: Citations come for free with Cohere! 🎉\n",
- "\n",
- "At Cohere, model generations come with... precise citations! 🎉\n",
- "The model cites which groups of words, in the tool results, were used to generate the final answer. \n",
- "These citations make it easy to check where the model’s generated response claims are coming from. \n",
- "They help users gain visibility into the model reasoning, and sanity check the final model generation. \n",
- "These citations are optional — you can decide to ignore them."
- ]
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "\n",
+ " \n",
+ ""
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "YN-eakfxtLGd"
+ },
+ "source": [
+ "# Basic Tool Use\n",
+ "\n",
+ "Tool use allows customers to **connect their large language models to external tools, such as search engines, APIs, functions, and databases**.\n",
+ "\n",
+ "This allows the customer to unlock a richer set of model behaviors by, for instance, leveraging data stored in tools, taking actions through APIs, interacting with a vector database, or querying a search engine.\n",
+ "\n",
+ "Below, we illustrate tool use in four steps:\n",
+ "- Step 1: the user configures the request to the model\n",
+ "- Step 2: the model uses context to **determine which tool(s) to use and how**\n",
+ "- Step 3: the tool calls are executed\n",
+ "- Step 4: the model **generates a final answer with precise citations** based on the tool results"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "j0DC3iPftLGo"
+ },
+ "outputs": [],
+ "source": [
+ "import cohere, json\n",
+ "API_KEY = \"...\" # fill in your Cohere API key here\n",
+ "co = cohere.Client(API_KEY)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "0T7yc1PltLGp"
+ },
+ "source": [
+ "## Step 0: Create a mock database\n",
+ "\n",
+ "Before we can illustrate tool use, we first need to do some setup. Here, we'll define the mock data that our tools will query. This data represents sales reports and a product catalog."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "yZffY8xItLGp"
+ },
+ "outputs": [],
+ "source": [
+ "# Mock database containing daily sales reports\n",
+ "sales_database = {\n",
+ " '2023-09-28': {\n",
+ " 'total_sales_amount': 5000,\n",
+ " 'total_units_sold': 100,\n",
+ " },\n",
+ " '2023-09-29': {\n",
+ " 'total_sales_amount': 10000,\n",
+ " 'total_units_sold': 250,\n",
+ " },\n",
+ " '2023-09-30': {\n",
+ " 'total_sales_amount': 8000,\n",
+ " 'total_units_sold': 200,\n",
+ " }\n",
+ "}\n",
+ "\n",
+ "# Mock product catalog\n",
+ "product_catalog = {\n",
+ " 'Electronics': [\n",
+ " {'product_id': 'E1001', 'name': 'Smartphone', 'price': 500, 'stock_level': 20},\n",
+ " {'product_id': 'E1002', 'name': 'Laptop', 'price': 1000, 'stock_level': 15},\n",
+ " {'product_id': 'E1003', 'name': 'Tablet', 'price': 300, 'stock_level': 25},\n",
+ " ],\n",
+ " 'Clothing': [\n",
+ " {'product_id': 'C1001', 'name': 'T-Shirt', 'price': 20, 'stock_level': 100},\n",
+ " {'product_id': 'C1002', 'name': 'Jeans', 'price': 50, 'stock_level': 80},\n",
+ " {'product_id': 'C1003', 'name': 'Jacket', 'price': 100, 'stock_level': 40},\n",
+ " ]\n",
+ "}"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "6TGWYiOdtLGp"
+ },
+ "source": [
+ "Now, we'll define the tools that simulate querying this database. \n",
+ "For example, you could use the API of an enterprise sales platform.\n",
+ "\n"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "YuIH4us8tLGp",
+ "tags": []
+ },
+ "outputs": [],
+ "source": [
+ "def query_daily_sales_report(day: str) -> dict:\n",
+ " \"\"\"\n",
+ " Function to retrieve the sales report for the given day\n",
+ " \"\"\"\n",
+ " report = sales_database.get(day, {})\n",
+ " if report:\n",
+ " return {\n",
+ " 'date': day,\n",
+ " 'summary': f\"Total Sales Amount: {report['total_sales_amount']}, Total Units Sold: {report['total_units_sold']}\"\n",
+ " }\n",
+ " else:\n",
+ " return {'date': day, 'summary': 'No sales data available for this day.'}\n",
+ "\n",
+ "\n",
+ "def query_product_catalog(category: str) -> dict:\n",
+ " \"\"\"\n",
+ " Function to retrieve products for the given category\n",
+ " \"\"\"\n",
+ " products = product_catalog.get(category, [])\n",
+ " return {\n",
+ " 'category': category,\n",
+ " 'products': products\n",
+ " }\n",
+ "\n",
+ "\n",
+ "functions_map = {\n",
+ " \"query_daily_sales_report\": query_daily_sales_report,\n",
+ " \"query_product_catalog\": query_product_catalog\n",
+ "}"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "HZRhTu4ftLGp"
+ },
+ "source": [
+ "## Step 1 - User configures the request to the model\n",
+ "\n",
+ "The developer provides a few things to the model:\n",
+ "- A preamble containing instructions about the task and the desired style for the output.\n",
+ "- The user request.\n",
+ "- A list of tools to the model.\n",
+ "- (Optionally) a chat history for the model to work with.\n",
+ "\n",
+ "\n",
+ "You can specify one or many tools to the model. Every tool needs to be described with a JSON schema, indicating the tool name, description, and parameters (code snippets below).\n",
+ "\n",
+ "In our example, we provide two tools to the model: `daily_sales_report` and `product_catalog`.\n"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "aIk-of_OtLGp"
+ },
+ "outputs": [],
+ "source": [
+ "# tool descriptions that the model has access to\n",
+ "# note: Cohere always adds a \"directly_answer\" tool under the hood, so that the model can decide to not leverage any tool, if they're not needed.\n",
+ "tools = [\n",
+ " {\n",
+ " \"name\": \"query_daily_sales_report\",\n",
+ " \"description\": \"Connects to a database to retrieve overall sales volumes and sales information for a given day.\",\n",
+ " \"parameter_definitions\": {\n",
+ " \"day\": {\n",
+ " \"description\": \"Retrieves sales data for this day, formatted as YYYY-MM-DD.\",\n",
+ " \"type\": \"str\",\n",
+ " \"required\": True\n",
+ " }\n",
+ " }\n",
+ " },\n",
+ " {\n",
+ " \"name\": \"query_product_catalog\",\n",
+ " \"description\": \"Connects to a a product catalog with information about all the products being sold, including categories, prices, and stock levels.\",\n",
+ " \"parameter_definitions\": {\n",
+ " \"category\": {\n",
+ " \"description\": \"Retrieves product information data for all products in this category.\",\n",
+ " \"type\": \"str\",\n",
+ " \"required\": True\n",
+ " }\n",
+ " }\n",
+ " }\n",
+ "]"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "k2AHJRnztLGp"
+ },
+ "source": [
+ "Now let's define the user request. \n",
+ "\n",
+ "In our example we'll use: \"Can you provide a sales summary for 29th September 2023, and also give me the details of all products in the 'Electronics' category that were sold that day, including their prices and stock levels?\"\n",
+ "\n",
+ "Only a langage model with Tool Use can answer this request: it requires looking up information in the right external tools (step 2), and then providing a final answer based on the tool results (step 4)."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "JuDgJ7fjtLGq"
+ },
+ "outputs": [],
+ "source": [
+ "# preamble containing instructions about the task and the desired style for the output.\n",
+ "preamble = \"\"\"\n",
+ "## Task & Context\n",
+ "You help people answer their questions and other requests interactively. You will be asked a very wide array of requests on all kinds of topics. You will be equipped with a wide range of search engines or similar tools to help you, which you use to research your answer. You should focus on serving the user's needs as best you can, which will be wide-ranging.\n",
+ "\n",
+ "## Style Guide\n",
+ "Unless the user asks for a different style of answer, you should answer in full sentences, using proper grammar and spelling.\n",
+ "\"\"\"\n",
+ "\n",
+ "# user request\n",
+ "message = \"Can you provide a sales summary for 29th September 2023, and also give me some details about the products in the 'Electronics' category, for example their prices and stock levels?\""
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "1NhW-G_JtLGq"
+ },
+ "source": [
+ "## Step 2 – The model smartly decides which tool(s) to use and how\n",
+ "The model intelligently selects the right tool(s) to call -- and the right parameters for each tool call -- based on the content of the user message."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "colab": {
+ "base_uri": "https://localhost:8080/"
},
+ "id": "o79_n99GtLGq",
+ "outputId": "81789d00-01b9-4c17-d1b0-1668d75a2b86"
+ },
+ "outputs": [
{
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "colab": {
- "base_uri": "https://localhost:8080/"
- },
- "id": "9wuoCUBwtLGr",
- "outputId": "da3c0dc5-6b87-42ea-d64b-e7e85c40273e"
- },
- "outputs": [
- {
- "name": "stdout",
- "output_type": "stream",
- "text": [
- "Citations that support the final answer:\n",
- "{'start': 7, 'end': 29, 'text': '29th of September 2023', 'document_ids': ['query_daily_sales_report:0:0']}\n",
- "{'start': 42, 'end': 75, 'text': '10,000 sales with 250 units sold.', 'document_ids': ['query_daily_sales_report:0:0']}\n",
- "{'start': 112, 'end': 127, 'text': 'three products.', 'document_ids': ['query_product_catalog:1:0']}\n",
- "{'start': 234, 'end': 244, 'text': 'Smartphone', 'document_ids': ['query_product_catalog:1:0']}\n",
- "{'start': 247, 'end': 250, 'text': '500', 'document_ids': ['query_product_catalog:1:0']}\n",
- "{'start': 253, 'end': 255, 'text': '20', 'document_ids': ['query_product_catalog:1:0']}\n",
- "{'start': 260, 'end': 266, 'text': 'Laptop', 'document_ids': ['query_product_catalog:1:0']}\n",
- "{'start': 269, 'end': 273, 'text': '1000', 'document_ids': ['query_product_catalog:1:0']}\n",
- "{'start': 276, 'end': 278, 'text': '15', 'document_ids': ['query_product_catalog:1:0']}\n",
- "{'start': 283, 'end': 289, 'text': 'Tablet', 'document_ids': ['query_product_catalog:1:0']}\n",
- "{'start': 292, 'end': 295, 'text': '300', 'document_ids': ['query_product_catalog:1:0']}\n",
- "{'start': 298, 'end': 300, 'text': '25', 'document_ids': ['query_product_catalog:1:0']}\n"
- ]
- }
- ],
- "source": [
- "print(\"Citations that support the final answer:\")\n",
- "for cite in response.citations:\n",
- " print(cite)"
- ]
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "The model recommends doing the following tool calls:\n",
+ "cohere.ToolCall {\n",
+ "\tname: query_daily_sales_report\n",
+ "\tparameters: {'day': '2023-09-29'}\n",
+ "\tgeneration_id: eaf955e3-623d-4796-bf51-23b07c66ed2c\n",
+ "}\n",
+ "cohere.ToolCall {\n",
+ "\tname: query_product_catalog\n",
+ "\tparameters: {'category': 'Electronics'}\n",
+ "\tgeneration_id: eaf955e3-623d-4796-bf51-23b07c66ed2c\n",
+ "}\n"
+ ]
+ }
+ ],
+ "source": [
+ "response = co.chat(\n",
+ " message=message,\n",
+ " tools=tools,\n",
+ " preamble=preamble,\n",
+ " model=\"command-r\"\n",
+ ")\n",
+ "\n",
+ "# Note that the Cohere Chat API also exposes:\n",
+ "# - stream (for streaming mode)\n",
+ "# - chat_history\n",
+ "# - among other parameters\n",
+ "# See https://docs.cohere.com/reference/chat for details.\n",
+ "\n",
+ "print(\"The model recommends doing the following tool calls:\")\n",
+ "print(\"\\n\".join(str(tool_call) for tool_call in response.tool_calls))"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "md_9QPcxtLGq"
+ },
+ "source": [
+ "## Step 3 – The tool calls are executed\n",
+ "\n",
+ "You can now execute the appropriate calls, using the tool calls and tool parameters generated by the model. \n",
+ "These tool calls return tool results that will be fed to the model in Step 4."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "colab": {
+ "base_uri": "https://localhost:8080/"
},
+ "id": "1LuDCRpFtLGr",
+ "outputId": "42ead35e-225a-4b9a-c954-b526f2865350"
+ },
+ "outputs": [
{
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "colab": {
- "base_uri": "https://localhost:8080/"
- },
- "id": "7td1J80DOKTM",
- "outputId": "e6e4fa7d-67eb-42ca-e736-9a45a47a6c1d"
- },
- "outputs": [
- {
- "name": "stdout",
- "output_type": "stream",
- "text": [
- "On the **29th of September 2023**[1], there were **10,000 sales with 250 units sold.**[1] \n",
- "\n",
- "The Electronics category contains **three products.**[2] There are details below:\n",
- "\n",
- "| Product Name | Price | Stock Level |\n",
- "| ------------ | ----- | ----------- |\n",
- "| **Smartphone**[2] | **500**[2] | **20**[2] |\n",
- "| **Laptop**[2] | **1000**[2] | **15**[2] |\n",
- "| **Tablet**[2] | **300**[2] | **25**[2] | \n",
- "\n",
- "The total stock level for Electronics items is 50.\n",
- "\n",
- "[1] source: [{'date': '2023-09-29', 'summary': 'Total Sales Amount: 10000, Total Units Sold: 250'}] \n",
- " based on tool call: {'name': 'query_daily_sales_report', 'parameters': {'day': '2023-09-29'}, 'generation_id': 'eaf955e3-623d-4796-bf51-23b07c66ed2c'}\n",
- "[2] source: [{'category': 'Electronics', 'products': [{'product_id': 'E1001', 'name': 'Smartphone', 'price': 500, 'stock_level': 20}, {'product_id': 'E1002', 'name': 'Laptop', 'price': 1000, 'stock_level': 15}, {'product_id': 'E1003', 'name': 'Tablet', 'price': 300, 'stock_level': 25}]}] \n",
- " based on tool call: {'name': 'query_product_catalog', 'parameters': {'category': 'Electronics'}, 'generation_id': 'eaf955e3-623d-4796-bf51-23b07c66ed2c'}\n"
- ]
- }
- ],
- "source": [
- "def insert_citations_in_order(text, citations):\n",
- " \"\"\"\n",
- " A helper function to pretty print citations.\n",
- " \"\"\"\n",
- " offset = 0\n",
- " document_id_to_number = {}\n",
- " citation_number = 0\n",
- " modified_citations = []\n",
- "\n",
- " # Process citations, assigning numbers based on unique document_ids\n",
- " for citation in citations:\n",
- " citation_numbers = []\n",
- " for document_id in sorted(citation[\"document_ids\"]):\n",
- " if document_id not in document_id_to_number:\n",
- " citation_number += 1 # Increment for a new document_id\n",
- " document_id_to_number[document_id] = citation_number\n",
- " citation_numbers.append(document_id_to_number[document_id])\n",
- "\n",
- " # Adjust start/end with offset\n",
- " start, end = citation['start'] + offset, citation['end'] + offset\n",
- " placeholder = ''.join([f'[{number}]' for number in citation_numbers])\n",
- " # Bold the cited text and append the placeholder\n",
- " modification = f'**{text[start:end]}**{placeholder}'\n",
- " # Replace the cited text with its bolded version + placeholder\n",
- " text = text[:start] + modification + text[end:]\n",
- " # Update the offset for subsequent replacements\n",
- " offset += len(modification) - (end - start)\n",
- "\n",
- " # Prepare citations for listing at the bottom, ensuring unique document_ids are listed once\n",
- " unique_citations = {number: doc_id for doc_id, number in document_id_to_number.items()}\n",
- " citation_list = '\\n'.join([f'[{doc_id}] source: {tool_results[doc_id - 1][\"outputs\"]} \\n based on tool call: {dict(tool_results[doc_id - 1][\"call\"])}' for doc_id, number in sorted(unique_citations.items(), key=lambda item: item[1])])\n",
- " text_with_citations = f'{text}\\n\\n{citation_list}'\n",
- "\n",
- " return text_with_citations\n",
- "\n",
- "\n",
- "print(insert_citations_in_order(response.text, response.citations))\n"
- ]
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "= running tool query_daily_sales_report, with parameters: {'day': '2023-09-29'}\n",
+ "== tool results: [{'date': '2023-09-29', 'summary': 'Total Sales Amount: 10000, Total Units Sold: 250'}]\n",
+ "= running tool query_product_catalog, with parameters: {'category': 'Electronics'}\n",
+ "== tool results: [{'category': 'Electronics', 'products': [{'product_id': 'E1001', 'name': 'Smartphone', 'price': 500, 'stock_level': 20}, {'product_id': 'E1002', 'name': 'Laptop', 'price': 1000, 'stock_level': 15}, {'product_id': 'E1003', 'name': 'Tablet', 'price': 300, 'stock_level': 25}]}]\n",
+ "Tool results that will be fed back to the model in step 4:\n",
+ "[\n",
+ " {\n",
+ " \"call\": {\n",
+ " \"name\": \"query_daily_sales_report\",\n",
+ " \"parameters\": {\n",
+ " \"day\": \"2023-09-29\"\n",
+ " },\n",
+ " \"generation_id\": \"eaf955e3-623d-4796-bf51-23b07c66ed2c\"\n",
+ " },\n",
+ " \"outputs\": [\n",
+ " {\n",
+ " \"date\": \"2023-09-29\",\n",
+ " \"summary\": \"Total Sales Amount: 10000, Total Units Sold: 250\"\n",
+ " }\n",
+ " ]\n",
+ " },\n",
+ " {\n",
+ " \"call\": {\n",
+ " \"name\": \"query_product_catalog\",\n",
+ " \"parameters\": {\n",
+ " \"category\": \"Electronics\"\n",
+ " },\n",
+ " \"generation_id\": \"eaf955e3-623d-4796-bf51-23b07c66ed2c\"\n",
+ " },\n",
+ " \"outputs\": [\n",
+ " {\n",
+ " \"category\": \"Electronics\",\n",
+ " \"products\": [\n",
+ " {\n",
+ " \"product_id\": \"E1001\",\n",
+ " \"name\": \"Smartphone\",\n",
+ " \"price\": 500,\n",
+ " \"stock_level\": 20\n",
+ " },\n",
+ " {\n",
+ " \"product_id\": \"E1002\",\n",
+ " \"name\": \"Laptop\",\n",
+ " \"price\": 1000,\n",
+ " \"stock_level\": 15\n",
+ " },\n",
+ " {\n",
+ " \"product_id\": \"E1003\",\n",
+ " \"name\": \"Tablet\",\n",
+ " \"price\": 300,\n",
+ " \"stock_level\": 25\n",
+ " }\n",
+ " ]\n",
+ " }\n",
+ " ]\n",
+ " }\n",
+ "]\n"
+ ]
+ }
+ ],
+ "source": [
+ "tool_results = []\n",
+ "# Iterate over the tool calls generated by the model\n",
+ "for tool_call in response.tool_calls:\n",
+ " # here is where you would call the tool recommended by the model, using the parameters recommended by the model\n",
+ " print(f\"= running tool {tool_call.name}, with parameters: {tool_call.parameters}\")\n",
+ " output = functions_map[tool_call.name](**tool_call.parameters)\n",
+ " # store the output in a list\n",
+ " outputs = [output]\n",
+ " print(f\"== tool results: {outputs}\")\n",
+ " # store your tool results in this format\n",
+ " tool_results.append({\n",
+ " \"call\": tool_call,\n",
+ " \"outputs\": outputs\n",
+ " })\n",
+ "\n",
+ "print(\"Tool results that will be fed back to the model in step 4:\")\n",
+ "print(json.dumps(tool_results, indent=4))"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "8cKlLk18tLGr"
+ },
+ "source": [
+ "## Step 4 - The model generates a final answer based on the tool results\n",
+ "Finally, the developer calls the Cohere model, providing the tools results, in order to generate the model's final answer. "
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "MKnjXVfXtLGr"
+ },
+ "outputs": [],
+ "source": [
+ "response = co.chat(\n",
+ " message=message,\n",
+ " tools=tools,\n",
+ " tool_results=tool_results,\n",
+ " preamble=preamble,\n",
+ " model=\"command-r\",\n",
+ " temperature=0.3\n",
+ ")"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "colab": {
+ "base_uri": "https://localhost:8080/"
},
+ "id": "jlxKTsaztLGr",
+ "outputId": "b2cd8667-bca9-4928-c423-61930b4b49fa"
+ },
+ "outputs": [
{
- "cell_type": "markdown",
- "metadata": {
- "id": "RwH2O3FptLGr"
- },
- "source": [
- "Yiha. You've used Cohere for Tool Use. Tool use opens up a wide range of new use cases. Here are a few examples:\n",
- "\n",
- "- **Function calling**: It's now possible to ask the model to output a JSON object with specific function parameters.\n",
- "For instance, this allows your chatbot to interact with your CRM to change the status of a deal, or to engage with a Python interpreter to conduct data science analyses.\n",
- "\n",
- "- **Query transformation**: You can transform a user message into a search query for a vector database or any search engine.\n",
- "For instance, this enables your work assistant to automatically retrieve the appropriate data from your company's documentation by creating the right query for your vector database.\n",
- "\n",
- "- **Advanced searches**: You can transform a user message into one-or-many queries, to do multiple subtasks based on the content of the message.\n",
- "For instance, this allows your chatbot to search across different databases and platforms to retrieve relevant information or to conduct comparative analysis.\n",
- "\n"
- ]
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "Final answer:\n",
+ "On the 29th of September 2023, there were 10,000 sales with 250 units sold. \n",
+ "\n",
+ "The Electronics category contains three products. There are details below:\n",
+ "\n",
+ "| Product Name | Price | Stock Level |\n",
+ "| ------------ | ----- | ----------- |\n",
+ "| Smartphone | 500 | 20 |\n",
+ "| Laptop | 1000 | 15 |\n",
+ "| Tablet | 300 | 25 | \n",
+ "\n",
+ "The total stock level for Electronics items is 50.\n"
+ ]
+ }
+ ],
+ "source": [
+ "print(\"Final answer:\")\n",
+ "print(response.text)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "niMkAr2PN9j4"
+ },
+ "source": [
+ "## Bonus: Citations come for free with Cohere! 🎉\n",
+ "\n",
+ "At Cohere, model generations come with... precise citations! 🎉\n",
+ "The model cites which groups of words, in the tool results, were used to generate the final answer. \n",
+ "These citations make it easy to check where the model’s generated response claims are coming from. \n",
+ "They help users gain visibility into the model reasoning, and sanity check the final model generation. \n",
+ "These citations are optional — you can decide to ignore them."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "colab": {
+ "base_uri": "https://localhost:8080/"
},
+ "id": "9wuoCUBwtLGr",
+ "outputId": "da3c0dc5-6b87-42ea-d64b-e7e85c40273e"
+ },
+ "outputs": [
{
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "id": "FXOSzfqRCLBH"
- },
- "outputs": [],
- "source": []
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "Citations that support the final answer:\n",
+ "{'start': 7, 'end': 29, 'text': '29th of September 2023', 'document_ids': ['query_daily_sales_report:0:0']}\n",
+ "{'start': 42, 'end': 75, 'text': '10,000 sales with 250 units sold.', 'document_ids': ['query_daily_sales_report:0:0']}\n",
+ "{'start': 112, 'end': 127, 'text': 'three products.', 'document_ids': ['query_product_catalog:1:0']}\n",
+ "{'start': 234, 'end': 244, 'text': 'Smartphone', 'document_ids': ['query_product_catalog:1:0']}\n",
+ "{'start': 247, 'end': 250, 'text': '500', 'document_ids': ['query_product_catalog:1:0']}\n",
+ "{'start': 253, 'end': 255, 'text': '20', 'document_ids': ['query_product_catalog:1:0']}\n",
+ "{'start': 260, 'end': 266, 'text': 'Laptop', 'document_ids': ['query_product_catalog:1:0']}\n",
+ "{'start': 269, 'end': 273, 'text': '1000', 'document_ids': ['query_product_catalog:1:0']}\n",
+ "{'start': 276, 'end': 278, 'text': '15', 'document_ids': ['query_product_catalog:1:0']}\n",
+ "{'start': 283, 'end': 289, 'text': 'Tablet', 'document_ids': ['query_product_catalog:1:0']}\n",
+ "{'start': 292, 'end': 295, 'text': '300', 'document_ids': ['query_product_catalog:1:0']}\n",
+ "{'start': 298, 'end': 300, 'text': '25', 'document_ids': ['query_product_catalog:1:0']}\n"
+ ]
}
- ],
- "metadata": {
+ ],
+ "source": [
+ "print(\"Citations that support the final answer:\")\n",
+ "for cite in response.citations:\n",
+ " print(cite)"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
"colab": {
- "provenance": []
+ "base_uri": "https://localhost:8080/"
},
- "kernelspec": {
- "display_name": "hackathon_demo",
- "language": "python",
- "name": "python3"
- },
- "language_info": {
- "codemirror_mode": {
- "name": "ipython",
- "version": 3
- },
- "file_extension": ".py",
- "mimetype": "text/x-python",
- "name": "python",
- "nbconvert_exporter": "python",
- "pygments_lexer": "ipython3",
- "version": "3.11.5"
+ "id": "7td1J80DOKTM",
+ "outputId": "e6e4fa7d-67eb-42ca-e736-9a45a47a6c1d"
+ },
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "On the **29th of September 2023**[1], there were **10,000 sales with 250 units sold.**[1] \n",
+ "\n",
+ "The Electronics category contains **three products.**[2] There are details below:\n",
+ "\n",
+ "| Product Name | Price | Stock Level |\n",
+ "| ------------ | ----- | ----------- |\n",
+ "| **Smartphone**[2] | **500**[2] | **20**[2] |\n",
+ "| **Laptop**[2] | **1000**[2] | **15**[2] |\n",
+ "| **Tablet**[2] | **300**[2] | **25**[2] | \n",
+ "\n",
+ "The total stock level for Electronics items is 50.\n",
+ "\n",
+ "[1] source: [{'date': '2023-09-29', 'summary': 'Total Sales Amount: 10000, Total Units Sold: 250'}] \n",
+ " based on tool call: {'name': 'query_daily_sales_report', 'parameters': {'day': '2023-09-29'}, 'generation_id': 'eaf955e3-623d-4796-bf51-23b07c66ed2c'}\n",
+ "[2] source: [{'category': 'Electronics', 'products': [{'product_id': 'E1001', 'name': 'Smartphone', 'price': 500, 'stock_level': 20}, {'product_id': 'E1002', 'name': 'Laptop', 'price': 1000, 'stock_level': 15}, {'product_id': 'E1003', 'name': 'Tablet', 'price': 300, 'stock_level': 25}]}] \n",
+ " based on tool call: {'name': 'query_product_catalog', 'parameters': {'category': 'Electronics'}, 'generation_id': 'eaf955e3-623d-4796-bf51-23b07c66ed2c'}\n"
+ ]
}
+ ],
+ "source": [
+ "def insert_citations_in_order(text, citations):\n",
+ " \"\"\"\n",
+ " A helper function to pretty print citations.\n",
+ " \"\"\"\n",
+ " offset = 0\n",
+ " document_id_to_number = {}\n",
+ " citation_number = 0\n",
+ " modified_citations = []\n",
+ "\n",
+ " # Process citations, assigning numbers based on unique document_ids\n",
+ " for citation in citations:\n",
+ " citation_numbers = []\n",
+ " for document_id in sorted(citation[\"document_ids\"]):\n",
+ " if document_id not in document_id_to_number:\n",
+ " citation_number += 1 # Increment for a new document_id\n",
+ " document_id_to_number[document_id] = citation_number\n",
+ " citation_numbers.append(document_id_to_number[document_id])\n",
+ "\n",
+ " # Adjust start/end with offset\n",
+ " start, end = citation['start'] + offset, citation['end'] + offset\n",
+ " placeholder = ''.join([f'[{number}]' for number in citation_numbers])\n",
+ " # Bold the cited text and append the placeholder\n",
+ " modification = f'**{text[start:end]}**{placeholder}'\n",
+ " # Replace the cited text with its bolded version + placeholder\n",
+ " text = text[:start] + modification + text[end:]\n",
+ " # Update the offset for subsequent replacements\n",
+ " offset += len(modification) - (end - start)\n",
+ "\n",
+ " # Prepare citations for listing at the bottom, ensuring unique document_ids are listed once\n",
+ " unique_citations = {number: doc_id for doc_id, number in document_id_to_number.items()}\n",
+ " citation_list = '\\n'.join([f'[{doc_id}] source: {tool_results[doc_id - 1][\"outputs\"]} \\n based on tool call: {dict(tool_results[doc_id - 1][\"call\"])}' for doc_id, number in sorted(unique_citations.items(), key=lambda item: item[1])])\n",
+ " text_with_citations = f'{text}\\n\\n{citation_list}'\n",
+ "\n",
+ " return text_with_citations\n",
+ "\n",
+ "\n",
+ "print(insert_citations_in_order(response.text, response.citations))\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "RwH2O3FptLGr"
+ },
+ "source": [
+ "Now, you've used Cohere for Tool Use. Tool use opens up a wide range of new use cases. Here are a few examples:\n",
+ "\n",
+ "- **Function calling**: It's now possible to ask the model to output a JSON object with specific function parameters.\n",
+ "For instance, this allows your chatbot to interact with your CRM to change the status of a deal, or to engage with a Python interpreter to conduct data science analyses.\n",
+ "\n",
+ "- **Query transformation**: You can transform a user message into a search query for a vector database or any search engine.\n",
+ "For instance, this enables your work assistant to automatically retrieve the appropriate data from your company's documentation by creating the right query for your vector database.\n",
+ "\n",
+ "- **Advanced searches**: You can transform a user message into one-or-many queries, to do multiple subtasks based on the content of the message.\n",
+ "For instance, this allows your chatbot to search across different databases and platforms to retrieve relevant information or to conduct comparative analysis.\n",
+ "\n"
+ ]
+ }
+ ],
+ "metadata": {
+ "colab": {
+ "provenance": []
+ },
+ "kernelspec": {
+ "display_name": "Python 3 (ipykernel)",
+ "language": "python",
+ "name": "python3"
},
- "nbformat": 4,
- "nbformat_minor": 0
+ "language_info": {
+ "codemirror_mode": {
+ "name": "ipython",
+ "version": 3
+ },
+ "file_extension": ".py",
+ "mimetype": "text/x-python",
+ "name": "python",
+ "nbconvert_exporter": "python",
+ "pygments_lexer": "ipython3",
+ "version": "3.8.16"
+ }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 4
}
diff --git a/notebooks/agents/agent_memory_walkthrough.ipynb b/notebooks/agents/agent_memory_walkthrough.ipynb
index cbb0a524..b0717cf0 100644
--- a/notebooks/agents/agent_memory_walkthrough.ipynb
+++ b/notebooks/agents/agent_memory_walkthrough.ipynb
@@ -1,19 +1,9 @@
{
"cells": [
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- ""
- ]
- },
{
"cell_type": "markdown",
"metadata": {
- "collapsed": false,
- "jupyter": {
- "outputs_hidden": false
- }
+ "collapsed": false
},
"source": [
"# Notebook Overview\n",
@@ -23,9 +13,9 @@
"The way we make LLMs aware of previous information is by simply providing them the full conversation history, i.e., the concatenation of previous input queries and generations.\n",
"\n",
"As Agents become more and more used, we face the issue to make Agents fully aware of the info from previous turns.\n",
- "The current approach is to pass the Agent the generations of the previous turns (see https://python.langchain.com/docs/modules/agents/how_to/custom_agent/), i.e., to do the same thing we do with LLMs. However, as we show below, while this is a good approach for LLMs, it is not for Agents. The reason is that, given the same input, LLMs *only* produce the final generation; conversely, Agents *first* produce a reasoning chain (intermediate steps), *then* produce the final outcome. Hence, if we only retain the final generation, we are loosing some crucial info: the reasoning chain.\n",
+ "The current approach is to pass the Agent the generations of the previous turns (see https://python.langchain.com/docs/modules/agents/how_to/custom_agent/). However, as we show below, while this is a good approach for LLMs it is not for Agents. Given the same input, LLMs *only* produce the final generation; conversely, Agents *first* produce a reasoning chain (intermediate steps), *then* produce the final outcome. Hence, if we only retain the final generation we lose some crucial info: the reasoning chain.\n",
"\n",
- "A straightforward solution to this issue would be to append to the conversation history from both the reasoning chain and the generations. This is problematic due to the fact that reasoning chains can be very long, especially when the model makes mistakes, and corrects itself. Using the full reasoning chains would (i) introduce a lot of noise; (ii) quickly fill the whole input window of the model.\n",
+ "A straightforward solution to this issue would be to append to the conversation history from both the reasoning chain and the generations. This is problematic due to the fact that reasoning chains can be very long, especially when the model makes mistakes and corrects itself. Using the full reasoning chains would (i) introduce a lot of noise; (ii) quickly fill the whole input window of the model.\n",
"\n",
"## Objective\n",
"\n",
@@ -45,33 +35,23 @@
{
"cell_type": "markdown",
"metadata": {
- "collapsed": false,
- "jupyter": {
- "outputs_hidden": false
- }
+ "collapsed": false
},
"source": [
- "\n",
+ "\n",
"# Step 1: Setup the Prompt and the Agent"
]
},
{
"cell_type": "code",
- "execution_count": 16,
+ "execution_count": null,
"metadata": {
"collapsed": false,
- "jupyter": {
- "outputs_hidden": false
- }
+ "is_executing": true
},
"outputs": [],
"source": [
- "####################################################################################################\n",
- "#\n",
"# Uncomment if you need to install the following packages\n",
- "#\n",
- "####################################################################################################\n",
- "\n",
"# !pip install cohere\n",
"# !pip install python-dotenv\n",
"# !pip install pandas"
@@ -81,10 +61,7 @@
"cell_type": "code",
"execution_count": 17,
"metadata": {
- "collapsed": false,
- "jupyter": {
- "outputs_hidden": false
- }
+ "collapsed": false
},
"outputs": [],
"source": [
@@ -107,10 +84,7 @@
"cell_type": "code",
"execution_count": 2,
"metadata": {
- "collapsed": false,
- "jupyter": {
- "outputs_hidden": false
- }
+ "collapsed": false
},
"outputs": [],
"source": [
@@ -122,10 +96,7 @@
"cell_type": "code",
"execution_count": 3,
"metadata": {
- "collapsed": false,
- "jupyter": {
- "outputs_hidden": false
- }
+ "collapsed": false
},
"outputs": [],
"source": [
@@ -137,10 +108,7 @@
"cell_type": "code",
"execution_count": 4,
"metadata": {
- "collapsed": false,
- "jupyter": {
- "outputs_hidden": false
- }
+ "collapsed": false
},
"outputs": [],
"source": [
@@ -152,10 +120,7 @@
"cell_type": "code",
"execution_count": 5,
"metadata": {
- "collapsed": false,
- "jupyter": {
- "outputs_hidden": false
- }
+ "collapsed": false
},
"outputs": [],
"source": [
@@ -179,10 +144,7 @@
"cell_type": "code",
"execution_count": 18,
"metadata": {
- "collapsed": false,
- "jupyter": {
- "outputs_hidden": false
- }
+ "collapsed": false
},
"outputs": [],
"source": [
@@ -205,13 +167,10 @@
{
"cell_type": "markdown",
"metadata": {
- "collapsed": false,
- "jupyter": {
- "outputs_hidden": false
- }
+ "collapsed": false
},
"source": [
- "\n",
+ "\n",
"# Step 2: Conversation without memory\n"
]
},
@@ -219,10 +178,7 @@
"cell_type": "code",
"execution_count": 10,
"metadata": {
- "collapsed": false,
- "jupyter": {
- "outputs_hidden": false
- }
+ "collapsed": false
},
"outputs": [
{
@@ -231,12 +187,12 @@
"text": [
"\n",
"\n",
- "\u001b[1m> Entering new AgentExecutor chain...\u001b[0m\n",
- "\u001b[32;1m\u001b[1;3m\n",
+ "\u001B[1m> Entering new AgentExecutor chain...\u001B[0m\n",
+ "\u001B[32;1m\u001B[1;3m\n",
"I will use python to read the CSV file and extract the column names.\n",
"{'tool_name': 'python_interpreter', 'parameters': {'code': \"import pandas as pd\\n\\ndf = pd.read_csv('revenue_table.csv')\\n\\nprint(df.columns)\"}}\n",
- "\u001b[0m\u001b[36;1m\u001b[1;3mIndex(['Unnamed: 0', 'time', 'revenue', 'loss'], dtype='object')\n",
- "\u001b[0m\u001b[32;1m\u001b[1;3mRelevant Documents: 0\n",
+ "\u001B[0m\u001B[36;1m\u001B[1;3mIndex(['Unnamed: 0', 'time', 'revenue', 'loss'], dtype='object')\n",
+ "\u001B[0m\u001B[32;1m\u001B[1;3mRelevant Documents: 0\n",
"Cited Documents: 0\n",
"Answer: The column names in the CSV file are:\n",
"- Unnamed: 0\n",
@@ -247,9 +203,9 @@
"- Unnamed: 0\n",
"- time\n",
"- revenue\n",
- "- loss\u001b[0m\n",
+ "- loss\u001B[0m\n",
"\n",
- "\u001b[1m> Finished chain.\u001b[0m\n"
+ "\u001B[1m> Finished chain.\u001B[0m\n"
]
}
],
@@ -265,10 +221,7 @@
"cell_type": "code",
"execution_count": 12,
"metadata": {
- "collapsed": false,
- "jupyter": {
- "outputs_hidden": false
- }
+ "collapsed": false
},
"outputs": [
{
@@ -277,8 +230,8 @@
"text": [
"\n",
"\n",
- "\u001b[1m> Entering new AgentExecutor chain...\u001b[0m\n",
- "\u001b[32;1m\u001b[1;3mPlan: I will ask the user for clarification on what data they would like to visualise.\n",
+ "\u001B[1m> Entering new AgentExecutor chain...\u001B[0m\n",
+ "\u001B[32;1m\u001B[1;3mPlan: I will ask the user for clarification on what data they would like to visualise.\n",
"Action: ```json\n",
"[\n",
" {\n",
@@ -290,9 +243,9 @@
"Relevant Documents: None\n",
"Cited Documents: None\n",
"Answer: Hello, could you please clarify what data you would like to see plotted?\n",
- "Grounded answer: Hello, could you please clarify what data you would like to see plotted?\u001b[0m\n",
+ "Grounded answer: Hello, could you please clarify what data you would like to see plotted?\u001B[0m\n",
"\n",
- "\u001b[1m> Finished chain.\u001b[0m\n"
+ "\u001B[1m> Finished chain.\u001B[0m\n"
]
}
],
@@ -307,10 +260,7 @@
{
"cell_type": "markdown",
"metadata": {
- "collapsed": false,
- "jupyter": {
- "outputs_hidden": false
- }
+ "collapsed": false
},
"source": [
"Without memory, the model cannot answer follow up questions because it misses the necessary previous context"
@@ -319,13 +269,10 @@
{
"cell_type": "markdown",
"metadata": {
- "collapsed": false,
- "jupyter": {
- "outputs_hidden": false
- }
+ "collapsed": false
},
"source": [
- "\n",
+ "\n",
"# Step 3. Conversation with Memory using AI Messages\n",
"\n",
"Here we will populate the chat history only with the generations from the model. This is the current approach used, e.g., here: https://python.langchain.com/docs/modules/agents/how_to/custom_agent/"
@@ -335,10 +282,7 @@
"cell_type": "code",
"execution_count": 88,
"metadata": {
- "collapsed": false,
- "jupyter": {
- "outputs_hidden": false
- }
+ "collapsed": false
},
"outputs": [
{
@@ -347,23 +291,23 @@
"text": [
"\n",
"\n",
- "\u001b[1m> Entering new AgentExecutor chain...\u001b[0m\n",
- "\u001b[32;1m\u001b[1;3m\n",
+ "\u001B[1m> Entering new AgentExecutor chain...\u001B[0m\n",
+ "\u001B[32;1m\u001B[1;3m\n",
"I will copy and paste the code from the previous conversation and add code to plot the revenue numbers.\n",
"{'tool_name': 'python_interpreter', 'parameters': {'code': \"import pandas as pd\\nimport matplotlib.pyplot as plt\\n\\ndata = pd.read_csv('data/sample.csv')\\n\\n# Plot revenue numbers\\nplt.plot(data['revenue'])\\n\\nplt.savefig('revenue_plot.png')\"}}\n",
- "\u001b[0m\u001b[36;1m\u001b[1;3mFileNotFoundError(2, 'No such file or directory')\u001b[0m\u001b[32;1m\u001b[1;3m\n",
+ "\u001B[0m\u001B[36;1m\u001B[1;3mFileNotFoundError(2, 'No such file or directory')\u001B[0m\u001B[32;1m\u001B[1;3m\n",
"The code is throwing an error because the file doesn't exist in the specified location. I will amend the file path and re-run the code.\n",
"{'tool_name': 'python_interpreter', 'parameters': {'code': \"import pandas as pd\\nimport matplotlib.pyplot as plt\\n\\ndata = pd.read_csv('sample.csv')\\n\\n# Plot revenue numbers\\nplt.plot(data['revenue'])\\n\\nplt.savefig('revenue_plot.png')\"}}\n",
- "\u001b[0m\u001b[36;1m\u001b[1;3mFileNotFoundError(2, 'No such file or directory')\u001b[0m\u001b[32;1m\u001b[1;3m\n",
+ "\u001B[0m\u001B[36;1m\u001B[1;3mFileNotFoundError(2, 'No such file or directory')\u001B[0m\u001B[32;1m\u001B[1;3m\n",
"The file path is still incorrect. I will amend it and re-run the code.\n",
"{'tool_name': 'python_interpreter', 'parameters': {'code': \"import pandas as pd\\nimport matplotlib.pyplot as plt\\n\\ndata = pd.read_csv('../data/sample.csv')\\n\\n# Plot revenue numbers\\nplt.plot(data['revenue'])\\n\\nplt.savefig('revenue_plot.png')\"}}\n",
- "\u001b[0m\u001b[36;1m\u001b[1;3mFileNotFoundError(2, 'No such file or directory')\u001b[0m\u001b[32;1m\u001b[1;3mReflection: The file path is still incorrect. I will ask the user to provide the correct file path.\n",
+ "\u001B[0m\u001B[36;1m\u001B[1;3mFileNotFoundError(2, 'No such file or directory')\u001B[0m\u001B[32;1m\u001B[1;3mReflection: The file path is still incorrect. I will ask the user to provide the correct file path.\n",
"Relevant Documents: 0,1,2\n",
"Cited Documents: 0,1,2\n",
"Answer: The file path provided is incorrect. Please provide the correct file path so that I can plot the revenue numbers. I have tried '../data/sample.csv' but this also doesn't work.\n",
- "Grounded answer: The file path provided is incorrect. Please provide the correct file path so that I can plot the revenue numbers. I have tried '../data/sample.csv' but this also doesn't work.\u001b[0m\n",
+ "Grounded answer: The file path provided is incorrect. Please provide the correct file path so that I can plot the revenue numbers. I have tried '../data/sample.csv' but this also doesn't work.\u001B[0m\n",
"\n",
- "\u001b[1m> Finished chain.\u001b[0m\n"
+ "\u001B[1m> Finished chain.\u001B[0m\n"
]
}
],
@@ -378,25 +322,19 @@
{
"cell_type": "markdown",
"metadata": {
- "collapsed": false,
- "jupyter": {
- "outputs_hidden": false
- }
+ "collapsed": false
},
"source": [
- "Also in this case, the model cannot manage the follow up question. The reason is that the AI message tells is only part of the necessary context: we need more info from previous turns.\n"
+ "Also in this case, the model cannot manage the follow up question. The reason is that the AI message only tells part of the necessary context: we need more info from previous turns.\n"
]
},
{
"cell_type": "markdown",
"metadata": {
- "collapsed": false,
- "jupyter": {
- "outputs_hidden": false
- }
+ "collapsed": false
},
"source": [
- "\n",
+ "\n",
"# Step 4: Conversation with Memory using AI Messages and Human Messages"
]
},
@@ -404,10 +342,7 @@
"cell_type": "code",
"execution_count": 14,
"metadata": {
- "collapsed": false,
- "jupyter": {
- "outputs_hidden": false
- }
+ "collapsed": false
},
"outputs": [
{
@@ -416,18 +351,18 @@
"text": [
"\n",
"\n",
- "\u001b[1m> Entering new AgentExecutor chain...\u001b[0m\n",
- "\u001b[32;1m\u001b[1;3m\n",
+ "\u001B[1m> Entering new AgentExecutor chain...\u001B[0m\n",
+ "\u001B[32;1m\u001B[1;3m\n",
"I will copy and paste the code from the previous conversation into this one, and then use it to plot the revenue numbers.\n",
"{'tool_name': 'python_interpreter', 'parameters': {'code': \"import pandas as pd\\nimport matplotlib.pyplot as plt\\n\\ndata = pd.read_csv('modified_revenue_table.csv')\\n\\n# Plot the revenue numbers\\nplt.plot(data['revenue'])\\n\\nplt.savefig('revenue_plot.png')\"}}\n",
- "\u001b[0m\u001b[36;1m\u001b[1;3m\u001b[0m\u001b[32;1m\u001b[1;3mRelevant Documents: 0\n",
+ "\u001B[0m\u001B[36;1m\u001B[1;3m\u001B[0m\u001B[32;1m\u001B[1;3mRelevant Documents: 0\n",
"Cited Documents: 0\n",
"Answer: Here's a plot of the revenue numbers:\n",
"![Revenue plot](\"revenue_plot.png\")\n",
"Grounded answer: Here's a plot of the revenue numbers:\n",
- "! [Revenue plot](\"revenue_plot.png\")\u001b[0m\n",
+ "! [Revenue plot](\"revenue_plot.png\")\u001B[0m\n",
"\n",
- "\u001b[1m> Finished chain.\u001b[0m\n"
+ "\u001B[1m> Finished chain.\u001B[0m\n"
]
},
{
@@ -454,23 +389,17 @@
{
"cell_type": "markdown",
"metadata": {
- "collapsed": false,
- "jupyter": {
- "outputs_hidden": false
- }
+ "collapsed": false
},
"source": [
- "It works! let's go on with the conversation."
+ "It works! Let's go on with the conversation."
]
},
{
"cell_type": "code",
"execution_count": 15,
"metadata": {
- "collapsed": false,
- "jupyter": {
- "outputs_hidden": false
- }
+ "collapsed": false
},
"outputs": [
{
@@ -479,18 +408,18 @@
"text": [
"\n",
"\n",
- "\u001b[1m> Entering new AgentExecutor chain...\u001b[0m\n",
- "\u001b[32;1m\u001b[1;3m\n",
+ "\u001B[1m> Entering new AgentExecutor chain...\u001B[0m\n",
+ "\u001B[32;1m\u001B[1;3m\n",
"I will copy and paste the previous code and make the changes requested by the user. Then I will execute the code to plot the graph with the changes applied.\n",
"{'tool_name': 'python_interpreter', 'parameters': {'code': 'import pandas as pd\\nimport matplotlib.pyplot as plt\\n\\n# Read the CSV file into a DataFrame\\ndf = pd.read_csv(\"modified_revenue_table.csv\")\\n\\n# Plot revenue against time\\nplt.plot(df[\"time\"], df[\"revenue\"], marker=\"o\")\\n\\n# Set minimum and maximum values for y-axis\\nplt.ylim(0, 1000)\\n\\nplt.savefig(\"revenue_plot.png\")'}}\n",
- "\u001b[0m\u001b[36;1m\u001b[1;3m\u001b[0m\u001b[32;1m\u001b[1;3mRelevant Documents: 0\n",
+ "\u001B[0m\u001B[36;1m\u001B[1;3m\u001B[0m\u001B[32;1m\u001B[1;3mRelevant Documents: 0\n",
"Cited Documents: 0\n",
"Answer: Here's the plot with the requested changes:\n",
"![Revenue plot](\"revenue_plot.png\")\n",
"Grounded answer: Here's the plot with the requested changes:\n",
- "! [Revenue plot](\"revenue_plot.png\")\u001b[0m\n",
+ "! [Revenue plot](\"revenue_plot.png\")\u001B[0m\n",
"\n",
- "\u001b[1m> Finished chain.\u001b[0m\n"
+ "\u001B[1m> Finished chain.\u001B[0m\n"
]
},
{
@@ -520,30 +449,24 @@
{
"cell_type": "markdown",
"metadata": {
- "collapsed": false,
- "jupyter": {
- "outputs_hidden": false
- }
+ "collapsed": false
},
"source": [
"The model does what we asked, but it decides to introduce the marker=\"o\" in the plotting function. While in this case the modification of the code does not affect the quality of the output, this is still an undesidered behaviour, since the model is introducing a modification that was not required.\n",
"\n",
- "To address this problem, we can further enrich the chat history, by adding information from the reasoning chain\n"
+ "To address this problem, we can further enrich the chat history by adding information from the reasoning chain.\n"
]
},
{
"cell_type": "markdown",
"metadata": {
- "collapsed": false,
- "jupyter": {
- "outputs_hidden": false
- }
+ "collapsed": false
},
"source": [
- "\n",
+ "\n",
"# Step 5: Conversation with Memory using AI Messages, Human Messages and the Reasoning Chain\n",
"\n",
- "To enrich the chat history, we will include in it also the resoning chains produced by the model. Such chains can be very long, especially in those cases in which errors are made, and the agent needs several attempts to get to the final output. Hence, by concatenating all the reasoning chains, we might have two issues: (i) noisy information; (ii) we would quickly hit max input length.\n",
+ "Reasoning chains can be very long, especially in the cases that contain errors and the agent needs several attempts to get to the final output. Hence, by concatenating all the reasoning chains, we might have two issues: (i) noisy information; (ii) we would quickly hit max input length.\n",
"\n",
"To avoid this issue, we need a way to extract the relevant info from the previous turns. Below, we propose a simple approach to info extraction. We format the extracted info in such a way to enhance human interpretability. We call the objects passed in the chat history *augmented memory objects*."
]
@@ -552,10 +475,7 @@
"cell_type": "code",
"execution_count": 14,
"metadata": {
- "collapsed": false,
- "jupyter": {
- "outputs_hidden": false
- }
+ "collapsed": false
},
"outputs": [],
"source": [
@@ -586,10 +506,7 @@
"cell_type": "code",
"execution_count": 91,
"metadata": {
- "collapsed": false,
- "jupyter": {
- "outputs_hidden": false
- }
+ "collapsed": false
},
"outputs": [],
"source": [
@@ -600,10 +517,7 @@
{
"cell_type": "markdown",
"metadata": {
- "collapsed": false,
- "jupyter": {
- "outputs_hidden": false
- }
+ "collapsed": false
},
"source": [
"Below, an example of the augmented memory object generated by the model. You can see that the agent now has full visibility on what it did in the previous step."
@@ -613,10 +527,7 @@
"cell_type": "code",
"execution_count": 92,
"metadata": {
- "collapsed": false,
- "jupyter": {
- "outputs_hidden": false
- }
+ "collapsed": false
},
"outputs": [
{
@@ -650,10 +561,7 @@
"cell_type": "code",
"execution_count": 87,
"metadata": {
- "collapsed": false,
- "jupyter": {
- "outputs_hidden": false
- }
+ "collapsed": false
},
"outputs": [
{
@@ -662,20 +570,20 @@
"text": [
"\n",
"\n",
- "\u001b[1m> Entering new AgentExecutor chain...\u001b[0m\n",
- "\u001b[32;1m\u001b[1;3m\n",
+ "\u001B[1m> Entering new AgentExecutor chain...\u001B[0m\n",
+ "\u001B[32;1m\u001B[1;3m\n",
"I will copy and paste the previous code, and modify the y axis limits as requested.\n",
"{'tool_name': 'python_interpreter', 'parameters': {'code': \"import pandas as pd\\nimport matplotlib.pyplot as plt\\n\\ndata = pd.read_csv('modified_revenue_table.csv')\\n\\n# Plot the revenue numbers\\nplt.plot(data['revenue'])\\n\\n# Set y axis limits\\nplt.ylim(0, 1000)\\n\\nplt.savefig('revenue_plot.png')\"}}\n",
- "\u001b[0m\u001b[36;1m\u001b[1;3m\u001b[0m\u001b[32;1m\u001b[1;3mRelevant Documents: 0\n",
+ "\u001B[0m\u001B[36;1m\u001B[1;3m\u001B[0m\u001B[32;1m\u001B[1;3mRelevant Documents: 0\n",
"Cited Documents: 0\n",
"Answer: Here's the plot with the requested y axis limits:\n",
"\n",
"![Revenue plot](\"revenue_plot.png\")\n",
"Grounded answer: Here's the plot with the requested y axis limits:\n",
"\n",
- "! [Revenue plot](\"revenue_plot.png\")\u001b[0m\n",
+ "! [Revenue plot](\"revenue_plot.png\")\u001B[0m\n",
"\n",
- "\u001b[1m> Finished chain.\u001b[0m\n"
+ "\u001B[1m> Finished chain.\u001B[0m\n"
]
},
{
@@ -702,10 +610,7 @@
{
"cell_type": "markdown",
"metadata": {
- "collapsed": false,
- "jupyter": {
- "outputs_hidden": false
- }
+ "collapsed": false
},
"source": [
"We can see that, now, the plot only includes the modification we asked for, and nothing else. This is possible because we are now providing the Agent with the code it previously generated, and the Agent re-uses that code, making only the necessary modifications. This is fundamentally different from what we observed before, when the Agent had to re-create from scratch the code.\n",
@@ -714,18 +619,6 @@
"\n",
"In a future post, we will explore how to handle really long historical context using vector databases."
]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "collapsed": false,
- "jupyter": {
- "outputs_hidden": false
- }
- },
- "outputs": [],
- "source": []
}
],
"metadata": {
diff --git a/notebooks/agents/agentic-RAG/agentic_rag_langchain.ipynb b/notebooks/agents/agentic-RAG/agentic_rag_langchain.ipynb
index cdefbbaa..de23b8a7 100644
--- a/notebooks/agents/agentic-RAG/agentic_rag_langchain.ipynb
+++ b/notebooks/agents/agentic-RAG/agentic_rag_langchain.ipynb
@@ -1,12 +1,5 @@
{
"cells": [
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- ""
- ]
- },
{
"cell_type": "markdown",
"metadata": {},
@@ -14,21 +7,23 @@
"# Notebook Overview\n",
"\n",
"## Motivation\n",
- "Asking questions over documents continues to be an important retrieval augmented generation (RAG) task. However, the document complexity can significantly influence overall RAG performance, particularly when the documents are PDFs that contain a mix of text and tables. Finding an optimal strategy to parse this information, chunk, embed and retrieve it is thus quite critical to obtaining accurate results. Furthermore, if the questions being asked over the retrieved documents require mathematical reasoning, then having a model that can validate those operations is quite useful.\n",
+ "Retrieval-augmented generation (RAG) allows language models to generate grounded answers to questions about documents. However, the complexity of the documents can significantly influence overall RAG performance. For instance, the documents may be PDFs that contain a mix of text and tables. \n",
+ "\n",
+ "More broadly, the implementation of a RAG pipeline - including parsing and chunking of documents, along with the embedding and retrieval of the chunks - is critical to the accuracy of grounded answers. Additionally, it is sometimes not sufficient to merely retrieve the answers; a user may want further postprocessing performed on the output. This use case would benefit from giving the model access to tools.\n",
"\n",
"## Objective\n",
- "In this notebook we will guide you through the best practices of setting up a RAG pipeline to process documents that contain both tables and text. In addition, we will show you how to create a Cohere ReAct agent with access to a RAG pipeline tool to improve accuracy. The general structure of the nb is as follows:\n",
+ "In this notebook, we will guide you through best practices for setting up a RAG pipeline to process documents that contain both tables and text. We will also demonstrate how to create a [ReAct](https://python.langchain.com/v0.1/docs/modules/agents/agent_types/react/) agent with a Cohere model, and then give the agent access to a RAG pipeline tool to improve accuracy. The general structure of the notebook is as follows:\n",
"\n",
- "1. individual components around parsing, retrieval and generation are covered for documents with mixed tabular and textual data\n",
- "2. a class object is created that can be used to instantiate the pipeline with parametric input\n",
- "3. the RAG pipeline is then used as a tool for a cohere react agent\n",
+ "- individual components around parsing, retrieval and generation are covered for documents with mixed tabular and textual data\n",
+ "- a class object is created that can be used to instantiate the pipeline with parametric input\n",
+ "- the RAG pipeline is then used as a tool for a Cohere ReACT agent\n",
"\n",
"# Reference Documents\n",
- "we recommend the following as a guide on doing [semi-structured RAG](https://github.com/langchain-ai/langchain/blob/master/cookbook/Semi_Structured_RAG.ipynb)\n",
+ "We recommend the following notebook as a guide to [semi-structured RAG](https://github.com/langchain-ai/langchain/blob/master/cookbook/Semi_Structured_RAG.ipynb).\n",
"\n",
- "we recommend this notebook to explore various parsing techniques for [PDFs](https://github.com/cohere-ai/notebooks/blob/main/notebooks/guides/Document_Parsing_For_Enterprises.ipynb)\n",
+ "We also recommend the following notebook to explore various parsing techniques for [PDFs](https://github.com/cohere-ai/notebooks/blob/main/notebooks/guides/Document_Parsing_For_Enterprises.ipynb).\n",
"\n",
- "various langchain supported [parsers](https://python.langchain.com/docs/modules/data_connection/document_loaders/pdf/)\n",
+ "Various LangChain-supported parsers can be found [here](https://python.langchain.com/docs/modules/data_connection/document_loaders/pdf/).\n",
"\n",
"## Table of Contents\n",
"- Section 1\n",
@@ -101,7 +96,6 @@
"from datasets import load_dataset\n",
"from joblib import Parallel, delayed\n",
"\n",
- "\n",
"os.environ['COHERE_API_KEY'] = \"\""
]
},
@@ -109,13 +103,13 @@
"cell_type": "markdown",
"metadata": {},
"source": [
- "\n",
+ "\n",
"# Parsing\n",
"\n",
"To improve RAG performance on PDFs with mixed types (text and tables), we investigated a number of parsing and chunking strategies from various libraries:\n",
- "- PyPDFLoader (LC)\n",
- "- LlamaParse (Llama-Index)\n",
- "- Unstructured IO\n",
+ "- [PyPDFLoader (LC)](https://api.python.langchain.com/en/latest/document_loaders/langchain_community.document_loaders.pdf.PyPDFLoader.html)\n",
+ "- [LlamaParse](https://docs.llamaindex.ai/en/stable/llama_cloud/llama_parse/) (Llama-Index)\n",
+ "- [Unstructured](https://unstructured.io/)\n",
"\n",
"\n",
"We have found that the best option for parsing is unstructured.io since the parser can:\n",
@@ -125,22 +119,11 @@
},
{
"cell_type": "code",
- "execution_count": 3,
+ "execution_count": null,
"metadata": {
"id": "eqcorKP4YEH6"
},
- "outputs": [
- {
- "name": "stderr",
- "output_type": "stream",
- "text": [
- "This function will be deprecated in a future release and `unstructured` will simply use the DEFAULT_MODEL from `unstructured_inference.model.base` to set default model name\n",
- "Some weights of the model checkpoint at microsoft/table-transformer-structure-recognition were not used when initializing TableTransformerForObjectDetection: ['model.backbone.conv_encoder.model.layer4.0.downsample.1.num_batches_tracked', 'model.backbone.conv_encoder.model.layer3.0.downsample.1.num_batches_tracked', 'model.backbone.conv_encoder.model.layer2.0.downsample.1.num_batches_tracked']\n",
- "- This IS expected if you are initializing TableTransformerForObjectDetection from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n",
- "- This IS NOT expected if you are initializing TableTransformerForObjectDetection from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n"
- ]
- }
- ],
+ "outputs": [],
"source": [
"# UNSTRUCTURED pdf loader\n",
"# Get elements\n",
@@ -204,15 +187,15 @@
"cell_type": "markdown",
"metadata": {},
"source": [
- "\n",
+ "\n",
"# Vector Store Setup\n",
"\n",
- "There are many options to setup a vector store. Here we show how to set one up using Chroma and Langchains Multi-vector retrieval.\n",
- "We use multi-vector retrieval because oftentimes a summary may be able to distill more accurately what a chunk is about, leading to better retrieval.\n",
+ "There are many options for setting up a vector store. Here, we show how to do so using [Chroma](https://www.trychroma.com/) and Langchain's Multi-vector retrieval.\n",
+ "As the name implies, multi-vector retrieval allows us to store multiple vectors per document; for instance, for a single document chunk, one could keep embeddings for both the chunk itself, and a summary of that document. A summary may be able to distill more accurately what a chunk is about, leading to better retrieval.\n",
"\n",
"You can read more about this here: https://python.langchain.com/docs/modules/data_connection/retrievers/multi_vector/\n",
"\n",
- "The process is as follows:\n",
+ "Below, we demonstrate the following process:\n",
"- summaries of each chunk are embedded\n",
"- during inference, the multi-vector retrieval returns the full context document related to the summary"
]
@@ -223,9 +206,7 @@
"metadata": {},
"outputs": [],
"source": [
- "\n",
"co = cohere.Client()\n",
- "\n",
"def get_chat_output(message, preamble, chat_history, model, temp, documents=None):\n",
" return co.chat(\n",
" message=message,\n",
@@ -260,7 +241,6 @@
"outputs": [],
"source": [
"# generate table and text summaries\n",
- "\n",
"prompt_text = \"\"\"You are an assistant tasked with summarizing tables and text. \\ \n",
"Give a concise summary of the table or text. Table or text chunk: {element}. Only provide the summary and no other text.\"\"\"\n",
"\n",
@@ -311,7 +291,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
- "\n",
+ "\n",
"# RAG Pipeline"
]
},
@@ -319,11 +299,11 @@
"cell_type": "markdown",
"metadata": {},
"source": [
- "the query process can be broken down into the following steps:\n",
+ "With our database in place, we can run queries against it. The query process can be broken down into the following steps:\n",
"\n",
- "1. augment the query, this really helps retrieve all the relevant information\n",
- "2. use each augmented query to retrieve the top k docs and then rerank them\n",
- "3. concatenate all the shortlisted/reranked docs and pass them to the generation model"
+ "- augment the query, this really helps retrieve all the relevant information\n",
+ "- use each augmented query to retrieve the top k docs and then rerank them\n",
+ "- concatenate all the shortlisted/reranked docs and pass them to the generation model"
]
},
{
@@ -382,6 +362,13 @@
"## Example "
]
},
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "We can now test out a query. In this example, the final answer can be found on page 12 of the PDF, which aligns with the response provided by the model:"
+ ]
+ },
{
"cell_type": "code",
"execution_count": 10,
@@ -413,25 +400,18 @@
"cell_type": "markdown",
"metadata": {},
"source": [
- "the final answer is correct based on page 12 in the PDF and we can see that the information retrieved is linked to that table :) "
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "### chat history management"
+ "### Chat History Management"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
- "an example of asking a follow up question that relies on the chat history but does not require a re-run of RAG.\n",
+ "In the example below, we ask a follow up question that relies on the chat history, but does not require a rerun of the RAG pipeline.\n",
"\n",
- "The search_queries_only flag can be used to determine whether RAG needs to be rerun or not i.e. it can help easily identify if the query passed needs retrieval.\n",
+ "We detect questions that do not require RAG by examining the `search_queries` object returned by calling `co.chat` to generate candidate queries to answer our question. If this object is empty, then the model has determined that a document query is not needed to answer the question.\n",
"\n",
- "In the example below, the else statement is invoked based on query2. In the else we pass in history without documents as the new query does not need to call the RAG pipeline "
+ "In the example below, the `else` statement is invoked based on `query2`. We still pass in the chat history, allowing the question to be answered with only the prior context."
]
},
{
@@ -480,10 +460,10 @@
"cell_type": "markdown",
"metadata": {},
"source": [
- "\n",
+ "\n",
"# RAG Pipeline Class\n",
"\n",
- "Here we connect all the pieces discussed above into one class object that is then used as a tool for a cohere react agent. This class is to help consolidate and clarify the key parameters used to define the RAG pipeline."
+ "Here, we connect all of the pieces discussed above into one class object, which is then used as a tool for a Cohere ReAct agent. This class definition consolidates and clarify the key parameters used to define the RAG pipeline."
]
},
{
@@ -737,15 +717,20 @@
"cell_type": "markdown",
"metadata": {},
"source": [
- "\n",
- "# Cohere REACT Agent with RAG Tool"
+ "\n",
+ "# Cohere ReAct Agent with RAG Tool"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
- "we build a simple agent using the RAG pipeline previously defined. The e2e rag pipeline is provided as a tool in addition to a python tool. The premise in coupling these tools is so that the mathematical steps can be done using a python tool to improve accuracy."
+ "Finally, we build a simple agent that utilizes the RAG pipeline defined above. We do this by granting the agent access to two tools:\n",
+ "\n",
+ "- the end-to-end RAG pipeline \n",
+ "- a Python interpreter\n",
+ "\n",
+ "The intention behind coupling these tools is to enable the model to perform mathematical and other postprocessing operations on RAG outputs using Python."
]
},
{
@@ -856,18 +841,18 @@
"text": [
"\n",
"\n",
- "\u001b[1m> Entering new AgentExecutor chain...\u001b[0m\n",
- "\u001b[32;1m\u001b[1;3m\n",
+ "\u001B[1m> Entering new AgentExecutor chain...\u001B[0m\n",
+ "\u001B[32;1m\u001B[1;3m\n",
"I will search for the charges for services in 2022 and 2023.\n",
"{'tool_name': 'vectorsearch', 'parameters': {'query': 'charges for services in 2022 and 2023'}}\n",
- "\u001b[0m\u001b[36;1m\u001b[1;3mThe charges for services in 2022 were $5,266 million and in 2023 were $5,769 million.The final answer is from the documents below:\n",
+ "\u001B[0m\u001B[36;1m\u001B[1;3mThe charges for services in 2022 were $5,266 million and in 2023 were $5,769 million.The final answer is from the documents below:\n",
" \n",
- " [{'id': 'doc_0', 'snippet': 'Program and General Revenues FY 2023 FY 2022 FY 2021 Category (in millions) Charges for Services (CS) $5,769 $5,266 $5,669 Operating Grants and Contributions (OGC) 27,935 31,757 28,109 Capital Grants and Contributions (CGC) 657 656 675 Real Estate Taxes (RET) 31,502 29,507 31,421 Sales and Use Taxes (SUT) 10,577 10,106 7,614 Personal Income Taxes (PIT) 15,313 15,520 15,795 Income Taxes, Other (ITO) 13,181 9,521 9,499 Other Taxes* (OT) 3,680 3,777 2,755 Investment Income* (II) 694 151 226 Unrestricted Federal and State Aid (UFSA) 234 549 108 Other* (O) Total Program and General Revenues - Primary Government 2,305 $110,250 $107,535 $104,176 708 725', 'title': 'chunk 0'}]\u001b[0m\u001b[32;1m\u001b[1;3mRelevant Documents: 0\n",
+ " [{'id': 'doc_0', 'snippet': 'Program and General Revenues FY 2023 FY 2022 FY 2021 Category (in millions) Charges for Services (CS) $5,769 $5,266 $5,669 Operating Grants and Contributions (OGC) 27,935 31,757 28,109 Capital Grants and Contributions (CGC) 657 656 675 Real Estate Taxes (RET) 31,502 29,507 31,421 Sales and Use Taxes (SUT) 10,577 10,106 7,614 Personal Income Taxes (PIT) 15,313 15,520 15,795 Income Taxes, Other (ITO) 13,181 9,521 9,499 Other Taxes* (OT) 3,680 3,777 2,755 Investment Income* (II) 694 151 226 Unrestricted Federal and State Aid (UFSA) 234 549 108 Other* (O) Total Program and General Revenues - Primary Government 2,305 $110,250 $107,535 $104,176 708 725', 'title': 'chunk 0'}]\u001B[0m\u001B[32;1m\u001B[1;3mRelevant Documents: 0\n",
"Cited Documents: 0\n",
"Answer: The charges for services in 2022 were $5,266 million and in 2023 were $5,769 million.\n",
- "Grounded answer: The charges for services in 2022 were $5,266 million and in 2023 were $5,769 million.\u001b[0m\n",
+ "Grounded answer: The charges for services in 2022 were $5,266 million and in 2023 were $5,769 million.\u001B[0m\n",
"\n",
- "\u001b[1m> Finished chain.\u001b[0m\n"
+ "\u001B[1m> Finished chain.\u001B[0m\n"
]
}
],
@@ -879,7 +864,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
- "Example of how to handle history with the langchain agent"
+ "Just like earlier, we can also pass chat history to the LangChain agent to refer to for any other queries."
]
},
{
@@ -914,7 +899,7 @@
"text": [
"\n",
"\n",
- "\u001b[1m> Entering new AgentExecutor chain...\u001b[0m\n"
+ "\u001B[1m> Entering new AgentExecutor chain...\u001B[0m\n"
]
},
{
@@ -928,16 +913,16 @@
"name": "stdout",
"output_type": "stream",
"text": [
- "\u001b[32;1m\u001b[1;3m\n",
+ "\u001B[32;1m\u001B[1;3m\n",
"I will use the Python Interpreter tool to calculate the mean of the two values.\n",
"{'tool_name': 'python_interpreter', 'parameters': {'code': 'import numpy as np\\n\\n# Data\\nvalues = [5266, 5769]\\n\\n# Calculate the mean\\nmean_value = np.mean(values)\\n\\nprint(f\"The mean of the two values is: {mean_value:.0f} million\")'}}\n",
- "\u001b[0m\u001b[33;1m\u001b[1;3mThe mean of the two values is: 5518 million\n",
- "\u001b[0m\u001b[32;1m\u001b[1;3mRelevant Documents: 0\n",
+ "\u001B[0m\u001B[33;1m\u001B[1;3mThe mean of the two values is: 5518 million\n",
+ "\u001B[0m\u001B[32;1m\u001B[1;3mRelevant Documents: 0\n",
"Cited Documents: 0\n",
"Answer: The mean of the two values is 5518 million.\n",
- "Grounded answer: The mean of the two values is 5518 million.\u001b[0m\n",
+ "Grounded answer: The mean of the two values is 5518 million.\u001B[0m\n",
"\n",
- "\u001b[1m> Finished chain.\u001b[0m\n"
+ "\u001B[1m> Finished chain.\u001B[0m\n"
]
},
{
@@ -963,11 +948,14 @@
]
},
{
- "cell_type": "code",
- "execution_count": null,
- "metadata": {},
- "outputs": [],
- "source": []
+ "cell_type": "markdown",
+ "source": [
+ "# Conclusion\n",
+ "As you can see, the RAG pipeline can be used as a tool for a Cohere ReAct agent. This allows the agent to access the RAG pipeline for document retrieval and generation, as well as a Python interpreter for postprocessing mathematical operations to improve accuracy. This setup can be used to improve the accuracy of grounded answers to questions about documents that contain both tables and text."
+ ],
+ "metadata": {
+ "collapsed": false
+ }
}
],
"metadata": {
diff --git a/notebooks/agents/agents_with_deterministic_functions.ipynb b/notebooks/agents/agents_with_deterministic_functions.ipynb
index 6fa7b94d..8564db87 100644
--- a/notebooks/agents/agents_with_deterministic_functions.ipynb
+++ b/notebooks/agents/agents_with_deterministic_functions.ipynb
@@ -1,19 +1,9 @@
{
"cells": [
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- ""
- ]
- },
{
"cell_type": "markdown",
"metadata": {
- "collapsed": false,
- "jupyter": {
- "outputs_hidden": false
- }
+ "collapsed": false
},
"source": [
"# Notebook Overview\n",
@@ -33,9 +23,9 @@
"## Solution\n",
"In this notebook, we propose a simple but effective solution to the challenge defined above: We create a [Langchain ReAct Agent](https://github.com/langchain-ai/langchain-cohere/blob/main/libs/cohere/langchain_cohere/cohere_agent.py) that has access to a deterministic tool that extracts the alphanumeric patterns in the query, and returns a dictionary in which the keys are the parameters, and the values the extracted patterns. The output of the tool is then used to generate the final request.\n",
"\n",
- "Using such a tool is just one of the possibilities that the Agent has to generate the query: As we will see below, when a more semantic understanding of the query is required, we can ignore the tool and leverage the linguistic capabilities of the LLM powering the Agent to generate the final output.\n",
+ "Using such a tool is just one of the possibilities that the Agent has to generate the query. As we will see below, when a more semantic understanding of the query is required, we can ignore the tool and leverage the linguistic capabilities of the LLM powering the Agent to generate the final output.\n",
"\n",
- "With this approach, we bring together the best of two worlds: on the one hand, the ability of LLMs to use tool and generate outputs, on the other one, the reliability and efficiency of deterministic functions.\n",
+ "With this approach, we bring together the best of two worlds: the ability of LLMs to use tools and generate outputs and the reliability and efficiency of deterministic functions.\n",
"\n",
"## Table of Contents\n",
"\n",
@@ -49,13 +39,10 @@
{
"cell_type": "markdown",
"metadata": {
- "collapsed": false,
- "jupyter": {
- "outputs_hidden": false
- }
+ "collapsed": false
},
"source": [
- "\n",
+ "\n",
"# Step 1: Setup"
]
},
@@ -63,32 +50,21 @@
"cell_type": "code",
"execution_count": 1,
"metadata": {
- "collapsed": false,
- "jupyter": {
- "outputs_hidden": false
- }
+ "collapsed": false
},
"outputs": [],
"source": [
- "####################################################################################################\n",
- "#\n",
"# Uncomment if you need to install the following packages\n",
- "#\n",
- "####################################################################################################\n",
- "\n",
"# !pip install cohere\n",
"# !pip install python-dotenv\n",
- "# !pip install pandas\n"
+ "# !pip install pandas"
]
},
{
"cell_type": "code",
"execution_count": 2,
"metadata": {
- "collapsed": false,
- "jupyter": {
- "outputs_hidden": false
- }
+ "collapsed": false
},
"outputs": [],
"source": [
@@ -108,10 +84,7 @@
"cell_type": "code",
"execution_count": 3,
"metadata": {
- "collapsed": false,
- "jupyter": {
- "outputs_hidden": false
- }
+ "collapsed": false
},
"outputs": [],
"source": [
@@ -122,13 +95,10 @@
{
"cell_type": "markdown",
"metadata": {
- "collapsed": false,
- "jupyter": {
- "outputs_hidden": false
- }
+ "collapsed": false
},
"source": [
- "\n",
+ "\n",
"# Step 2: Define the Tool and the Agent\n",
"Here we create a tool which implements the deterministic function to extract alphanumeric strings from the user's query and match them to the right parameter."
]
@@ -137,17 +107,13 @@
"cell_type": "code",
"execution_count": 4,
"metadata": {
- "collapsed": false,
- "jupyter": {
- "outputs_hidden": false
- }
+ "collapsed": false
},
"outputs": [],
"source": [
"@tool\n",
"def regex_extractor(user_query: str) -> dict:\n",
" \"\"\"Function which, given the query from the user, returns a dictionary parameter:value.\"\"\"\n",
- "\n",
" uuid = re.findall(\"\\s([a-f0-9]{8}-[a-f0-9]{4}-[a-f0-9]{4}-[a-f0-9]{4}-[a-f0-9]{12})\", user_query)\n",
" nmgs = re.findall(\"(0000[A-Z0-9]{21})\", user_query)\n",
" objref = re.findall(\"\\s([A-Z]{5,9}\\d{3,4}[A-Z]{3,8})\", user_query)\n",
@@ -159,12 +125,10 @@
" d_filtered = {k: v for k, v in d.items() if v != []}\n",
" return d_filtered\n",
"\n",
- "\n",
"class extract_code_v1productssearch(BaseModel):\n",
" user_query: str = Field(\n",
" description=\"This is the full input query as received from the user. Do not truncate or modify the query in any way.\"\n",
" )\n",
- "\n",
"regex_extractor.name = \"regex_extractor\"\n",
"regex_extractor.args_schema = extract_code_v1productssearch\n",
"tools=[regex_extractor]"
@@ -174,10 +138,7 @@
"cell_type": "code",
"execution_count": 5,
"metadata": {
- "collapsed": false,
- "jupyter": {
- "outputs_hidden": false
- }
+ "collapsed": false
},
"outputs": [],
"source": [
@@ -187,44 +148,39 @@
"# - the cases in which the Agent has to produce an output without using the tool\n",
"# - some examples to clarify the task\n",
"preamble = \"\"\"You are an assistant that given a user's query about, generates a request an API.\n",
- " You can use the tool named \"regex_extractor\".\n",
- " Pass to the \"regex_extractor\" tool the entire text of the the input query.\n",
- " The tool returns a dictionary, in which keys are the name of the codes, and values a list of extracted codes.\n",
- " Create a JSON for each of the key-value pairs in the dictionary.\n",
+ "You can use the tool named \"regex_extractor\".\n",
+ "Pass to the \"regex_extractor\" tool the entire text of the the input query.\n",
+ "The tool returns a dictionary, in which keys are the name of the codes, and values a list of extracted codes.\n",
+ "Create a JSON for each of the key-value pairs in the dictionary.\n",
"\n",
- " Return a list of JSONs. Make sure each JSON is properly defined. Do not return any other explanation or comment.\n",
- " You MUST generate a perfect JSON object: make sure that each string in the lists is included between quotes.\n",
+ "Return a list of JSONs. Make sure each JSON is properly defined. Do not return any other explanation or comment.\n",
+ "You MUST generate a perfect JSON object: make sure that each string in the lists is included between quotes.\n",
"\n",
- " If the request mentions one of the tags listed below, or a related word, create a dictionary in which the key is \"taxonomies\" and the value the list of capitalized tags.\n",
- " Tags list: cars, trucks, clothes, sport\n",
+ "If the request mentions one of the tags listed below, or a related word, create a dictionary in which the key is \"taxonomies\" and the value the list of capitalized tags.\n",
+ "Tags list: cars, trucks, clothes, sport\n",
"\n",
"\n",
- " Find a list of examples here:\n",
- " User question | parameter for the tool | what you should understand\n",
- " Look products GLCMS004AGTCAMIS; 0000234GLCMS0100ANORAKCAA, GLCHL000CGUCHALE | Look products GLCMS004AGTCAMIS; 0000234GLCMS0100ANORAKCAA, GLCHL000CGUCHALE | [{\"objref\":[\"GLCMS004AGTCAMIS\",\"GLCHL000CGUCHALE\"]},{\"nmgs\":[\"0000234GLCMS0100ANORAKCAA\"]}]\n",
- " Retrieve id 7410e652-639d-402e-984e-8fd7025f0aac 8bb21b93-2ddf-4a63-af63-ddb6b1be49a1, ref GLPNT0005GUJOGGE GLSBR000BGASOBRE, nmg 0000234GLCHL0200ARTINDIUS | Retrieve id 7410e652-639d-402e-984e-8fd7025f0aac 8bb21b93-2ddf-4a63-af63-ddb6b1be49a1, ref GLPNT0005GUJOGGE GLSBR000BGASOBRE, nmg 0000234GLCHL0200ARTINDIUS | [{\"uuid\":[\"7410e652-639d-402e-984e-8fd7025f0aac\",\"8bb21b93-2ddf-4a63-af63-ddb6b1be49a1\"]}, {\"objref\":[\"GLPNT0005GUJOGGE\",\"GLSBR000BGASOBRE\"]},{\"nmgs\":[\"0000234GLCHL0200ARTINDIUS\"]}]\n",
- " Look for items of cars and trucks | Look for items of pants and t-shirts | [{'taxonomies': ['CARS', 'TRUCKS']}]\n",
- " Search products sport | Search products dress and jumpsuit | [{'taxonomies': ['SPORT']}]\n",
- " \"\"\""
+ "Find a list of examples here:\n",
+ "User question | parameter for the tool | what you should understand\n",
+ "Look products GLCMS004AGTCAMIS; 0000234GLCMS0100ANORAKCAA, GLCHL000CGUCHALE | Look products GLCMS004AGTCAMIS; 0000234GLCMS0100ANORAKCAA, GLCHL000CGUCHALE | [{\"objref\":[\"GLCMS004AGTCAMIS\",\"GLCHL000CGUCHALE\"]},{\"nmgs\":[\"0000234GLCMS0100ANORAKCAA\"]}]\n",
+ "Retrieve id 7410e652-639d-402e-984e-8fd7025f0aac 8bb21b93-2ddf-4a63-af63-ddb6b1be49a1, ref GLPNT0005GUJOGGE GLSBR000BGASOBRE, nmg 0000234GLCHL0200ARTINDIUS | Retrieve id 7410e652-639d-402e-984e-8fd7025f0aac 8bb21b93-2ddf-4a63-af63-ddb6b1be49a1, ref GLPNT0005GUJOGGE GLSBR000BGASOBRE, nmg 0000234GLCHL0200ARTINDIUS | [{\"uuid\":[\"7410e652-639d-402e-984e-8fd7025f0aac\",\"8bb21b93-2ddf-4a63-af63-ddb6b1be49a1\"]}, {\"objref\":[\"GLPNT0005GUJOGGE\",\"GLSBR000BGASOBRE\"]},{\"nmgs\":[\"0000234GLCHL0200ARTINDIUS\"]}]\n",
+ "Look for items of cars and trucks | Look for items of pants and t-shirts | [{'taxonomies': ['CARS', 'TRUCKS']}]\n",
+ "Search products sport | Search products dress and jumpsuit | [{'taxonomies': ['SPORT']}]\n",
+ "\"\"\""
]
},
{
"cell_type": "code",
"execution_count": 6,
"metadata": {
- "collapsed": false,
- "jupyter": {
- "outputs_hidden": false
- }
+ "collapsed": false
},
"outputs": [],
"source": [
"# Define the prompt\n",
"prompt = ChatPromptTemplate.from_template(\"{input}\")\n",
- "\n",
"# Define the agent\n",
"llm = ChatCohere(model=\"command-r-plus\", temperature=0)\n",
- "\n",
"# instantiate agent and agent executor\n",
"agent = create_cohere_react_agent(\n",
" llm=llm,\n",
@@ -242,10 +198,7 @@
"cell_type": "code",
"execution_count": 7,
"metadata": {
- "collapsed": false,
- "jupyter": {
- "outputs_hidden": false
- }
+ "collapsed": false
},
"outputs": [],
"source": [
@@ -256,19 +209,16 @@
" .replace(\"json\", \"\")\n",
" .replace(\"`\", \"\")\n",
" .replace(\"`\", \"\")\n",
- " )\n"
+ " )"
]
},
{
"cell_type": "markdown",
"metadata": {
- "collapsed": false,
- "jupyter": {
- "outputs_hidden": false
- }
+ "collapsed": false
},
"source": [
- "\n",
+ "\n",
"# Step 3: Run the Agent\n",
"Let's now test the Agent we just defined!"
]
@@ -277,10 +227,7 @@
"cell_type": "code",
"execution_count": 8,
"metadata": {
- "collapsed": false,
- "jupyter": {
- "outputs_hidden": false
- }
+ "collapsed": false
},
"outputs": [
{
@@ -289,11 +236,11 @@
"text": [
"\n",
"\n",
- "\u001b[1m> Entering new AgentExecutor chain...\u001b[0m\n",
- "\u001b[32;1m\u001b[1;3m\n",
+ "\u001B[1m> Entering new AgentExecutor chain...\u001B[0m\n",
+ "\u001B[32;1m\u001B[1;3m\n",
"I will use the regex_extractor tool to extract the codes from the user query.\n",
"{'tool_name': 'regex_extractor', 'parameters': {'user_query': 'Look for urn:75f2b737-06dd-4399-9206-a6c11b65138e, GLCMS004AGTCAMIS; 0000234GLCMS0100ANORAKCAA, GLCHL000CGUCHALE'}}\n",
- "\u001b[0m\u001b[36;1m\u001b[1;3m{'nmgs': ['0000234GLCMS0100ANORAKCAA'], 'objref': ['GLCMS004AGTCAMIS', 'GLCHL000CGUCHALE'], 'urn': ['urn:75f2b737-06dd-4399-9206-a6c11b65138e']}\u001b[0m\u001b[32;1m\u001b[1;3mRelevant Documents: 0\n",
+ "\u001B[0m\u001B[36;1m\u001B[1;3m{'nmgs': ['0000234GLCMS0100ANORAKCAA'], 'objref': ['GLCMS004AGTCAMIS', 'GLCHL000CGUCHALE'], 'urn': ['urn:75f2b737-06dd-4399-9206-a6c11b65138e']}\u001B[0m\u001B[32;1m\u001B[1;3mRelevant Documents: 0\n",
"Cited Documents: 0\n",
"Answer: ```json\n",
"[\n",
@@ -312,9 +259,9 @@
" \"nmgs\": [\"0000234GLCMS0100ANORAKCAA\"]\n",
" }\n",
"]\n",
- "```\u001b[0m\n",
+ "```\u001B[0m\n",
"\n",
- "\u001b[1m> Finished chain.\u001b[0m\n"
+ "\u001B[1m> Finished chain.\u001B[0m\n"
]
}
],
@@ -331,10 +278,7 @@
{
"cell_type": "markdown",
"metadata": {
- "collapsed": false,
- "jupyter": {
- "outputs_hidden": false
- }
+ "collapsed": false
},
"source": [
"In the reasoning chain above, we can see that the Agent uses the tool we provided it to extract the strings in the query.\n",
@@ -345,10 +289,7 @@
"cell_type": "code",
"execution_count": 9,
"metadata": {
- "collapsed": false,
- "jupyter": {
- "outputs_hidden": false
- }
+ "collapsed": false
},
"outputs": [
{
@@ -372,10 +313,7 @@
{
"cell_type": "markdown",
"metadata": {
- "collapsed": false,
- "jupyter": {
- "outputs_hidden": false
- }
+ "collapsed": false
},
"source": [
"As mentioned above, the Agent can use the tool when specific alphanumeric patterns have to be extracted from the query; however, it can also generate the output based on its semantic understanding of the query. For example:"
@@ -385,10 +323,7 @@
"cell_type": "code",
"execution_count": 10,
"metadata": {
- "collapsed": false,
- "jupyter": {
- "outputs_hidden": false
- }
+ "collapsed": false
},
"outputs": [
{
@@ -397,11 +332,11 @@
"text": [
"\n",
"\n",
- "\u001b[1m> Entering new AgentExecutor chain...\u001b[0m\n",
- "\u001b[32;1m\u001b[1;3m\n",
+ "\u001B[1m> Entering new AgentExecutor chain...\u001B[0m\n",
+ "\u001B[32;1m\u001B[1;3m\n",
"I will use the regex_extractor tool to extract the relevant information from the user request.\n",
"{'tool_name': 'regex_extractor', 'parameters': {'user_query': 'I need tennis products'}}\n",
- "\u001b[0m\u001b[36;1m\u001b[1;3m{}\u001b[0m\u001b[32;1m\u001b[1;3mRelevant Documents: None\n",
+ "\u001B[0m\u001B[36;1m\u001B[1;3m{}\u001B[0m\u001B[32;1m\u001B[1;3mRelevant Documents: None\n",
"Cited Documents: None\n",
"Answer: ```json\n",
"[\n",
@@ -420,9 +355,9 @@
" ]\n",
" }\n",
"]\n",
- "```\u001b[0m\n",
+ "```\u001B[0m\n",
"\n",
- "\u001b[1m> Finished chain.\u001b[0m\n"
+ "\u001B[1m> Finished chain.\u001B[0m\n"
]
}
],
@@ -440,23 +375,17 @@
{
"cell_type": "markdown",
"metadata": {
- "collapsed": false,
- "jupyter": {
- "outputs_hidden": false
- }
+ "collapsed": false
},
"source": [
- "The Agent run the tool to check if any target string was in the query, then it generated the request body based on its understanding."
+ "The Agent runs the tool to check if any target string was in the query, then it generated the request body based on its understanding."
]
},
{
"cell_type": "code",
"execution_count": 11,
"metadata": {
- "collapsed": false,
- "jupyter": {
- "outputs_hidden": false
- }
+ "collapsed": false
},
"outputs": [
{
@@ -477,10 +406,7 @@
{
"cell_type": "markdown",
"metadata": {
- "collapsed": false,
- "jupyter": {
- "outputs_hidden": false
- }
+ "collapsed": false
},
"source": [
"Finally, the two paths to generation - deterministic and semantic - can be applied in parallel by the Agent, as shown below:"
@@ -490,10 +416,7 @@
"cell_type": "code",
"execution_count": 12,
"metadata": {
- "collapsed": false,
- "jupyter": {
- "outputs_hidden": false
- }
+ "collapsed": false
},
"outputs": [
{
@@ -502,11 +425,11 @@
"text": [
"\n",
"\n",
- "\u001b[1m> Entering new AgentExecutor chain...\u001b[0m\n",
- "\u001b[32;1m\u001b[1;3m\n",
+ "\u001B[1m> Entering new AgentExecutor chain...\u001B[0m\n",
+ "\u001B[32;1m\u001B[1;3m\n",
"I will use the regex_extractor tool to extract the codes from the user query. Then, I will create a JSON for each of the key-value pairs in the dictionary.\n",
"{'tool_name': 'regex_extractor', 'parameters': {'user_query': 'Look for GLBRL0000GACHALE, nmg 0000234GLCZD0000GUREDTOAA and car products'}}\n",
- "\u001b[0m\u001b[36;1m\u001b[1;3m{'nmgs': ['0000234GLCZD0000GUREDTOAA'], 'objref': ['GLBRL0000GACHALE']}\u001b[0m\u001b[32;1m\u001b[1;3mRelevant Documents: 0\n",
+ "\u001B[0m\u001B[36;1m\u001B[1;3m{'nmgs': ['0000234GLCZD0000GUREDTOAA'], 'objref': ['GLBRL0000GACHALE']}\u001B[0m\u001B[32;1m\u001B[1;3mRelevant Documents: 0\n",
"Cited Documents: 0\n",
"Answer: ```json\n",
"[\n",
@@ -529,9 +452,9 @@
" \"taxonomies\": [\"CARS\"]\n",
" }\n",
"]\n",
- "```\u001b[0m\n",
+ "```\u001B[0m\n",
"\n",
- "\u001b[1m> Finished chain.\u001b[0m\n"
+ "\u001B[1m> Finished chain.\u001B[0m\n"
]
}
],
@@ -550,10 +473,7 @@
"cell_type": "code",
"execution_count": 13,
"metadata": {
- "collapsed": false,
- "jupyter": {
- "outputs_hidden": false
- }
+ "collapsed": false
},
"outputs": [
{
@@ -575,29 +495,14 @@
{
"cell_type": "markdown",
"metadata": {
- "collapsed": false,
- "jupyter": {
- "outputs_hidden": false
- }
+ "collapsed": false
},
"source": [
- "\n",
+ "\n",
"# Conclusions\n",
"\n",
"In this notebook we showed how Agents can be used to solve a real-world use case, in which the goal is to create API requests based on the user's query. We did it by providing the Agent with a deterministic tool to extract relevant alphanumeric strings in the query, and matching them to the right parameter name. In parallel, the Agent can leverage the semantic understanding of the query provided by the LLM powering it."
]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "collapsed": false,
- "jupyter": {
- "outputs_hidden": false
- }
- },
- "outputs": [],
- "source": []
}
],
"metadata": {
diff --git a/notebooks/agents/financial-csv-agent/financial_csv_publication.ipynb b/notebooks/agents/financial-csv-agent/financial_csv_publication.ipynb
index c7ee3fbf..d36ad183 100644
--- a/notebooks/agents/financial-csv-agent/financial_csv_publication.ipynb
+++ b/notebooks/agents/financial-csv-agent/financial_csv_publication.ipynb
@@ -1,18 +1,12 @@
{
"cells": [
- {
- "cell_type": "markdown",
- "id": "5c0d5271-8d35-4471-b667-265695cbf11a",
- "metadata": {},
- "source": [
- ""
- ]
- },
{
"cell_type": "markdown",
"id": "1d71f595-01ef-4c24-99eb-81b01eec048b",
"metadata": {},
"source": [
+ "# Financial CSV Agent with Langchain\n",
+ "\n",
"# Notebook Overview\n",
"\n",
"## Motivation\n",
@@ -27,7 +21,7 @@
"Having an Agent which is able to correctly compute these and other ratios would be a great help for any analyst in the field of Finance.\n",
"\n",
"## Objective\n",
- "In this notebook we explore how to setup a [Cohere ReAct Agent](https://github.com/langchain-ai/langchain-cohere/blob/main/libs/cohere/langchain_cohere/cohere_agent.py) to answer questions over tables in Apple's SEC10K 2020 form. We show how this can be done with two variants of a langchain python tool, one that requires you to pass the full path of the dataframe and another that requires the dataframe objects to be loaded in memory. While there is no major difference between the two approaches, we present both to show how to manage files that are loaded in memory vs files in storage. \n",
+ "In this notebook we explore how to setup a [Cohere ReAct Agent](https://github.com/langchain-ai/langchain-cohere/blob/main/libs/cohere/langchain_cohere/cohere_agent.py) to answer questions over tables in Apple's SEC10K 2020 form. We show how this can be done with two variants of a Langchain Python tool, one that requires you to pass the full path of the dataframe and another that requires the dataframe objects to be loaded in memory. While there is no major difference between the two approaches, we present both to show how to manage files that are loaded in memory vs files in storage. \n",
"\n",
"## Table of Contents\n",
"\n",
@@ -84,12 +78,7 @@
"metadata": {},
"outputs": [],
"source": [
- "####################################################################################################\n",
- "#\n",
"# Uncomment if you need to install the following packages\n",
- "#\n",
- "####################################################################################################\n",
- "\n",
"#!pip install --quiet langchain langchain_cohere langchain_experimental --upgrade\n",
"#!pip install sec-api"
]
@@ -108,7 +97,7 @@
"# Introduction\n",
"\n",
"The aim of this notebook is to showcase how Cohere Langchain Agents can be used to answer questions over tabular data.\n",
- "This notebook assumes the input is a set of csv files extracted from Apples SEC 10K filings."
+ "This notebook assumes the input is a set of csv files extracted from Apple's SEC 10K filings."
]
},
{
@@ -118,7 +107,7 @@
"source": [
"### Data Loading\n",
"\n",
- "We use the sec-api to download the income statement and balance sheet from the SEC 10K .htm file. Please note that the tables need to be parsed such that the index is numerical as the python code generation struggles to filter based on index. We have processed the tables and provided them for you."
+ "We use the sec-api to download the income statement and balance sheet from the SEC 10K .htm file. Please note that the tables need to be parsed such that the index is numerical as the python code generation struggles to filter based on index. We have processed the tables and provided them for you. They can be found [here](https://github.com/cohere-ai/notebooks/tree/main/notebooks/agents/financial-csv-agent). "
]
},
{
@@ -151,10 +140,10 @@
"\n",
"In the example below, we show how the python tool can be used to load a dataframe and extract information from it. To do this successfully we need to:\n",
"\n",
- "1) pass the file name to the preamble so the model knows how to load the dataframe\n",
- "2) pass a preview of the dataframe in the preamble so the model knows which columns/rows to query\n",
+ "* Pass the file name to the preamble so the model knows how to load the dataframe.\n",
+ "* Pass a preview of the dataframe in the preamble so the model knows which columns/rows to query.\n",
"\n",
- "First, let's implement the ReAct agent"
+ "First, let's implement the ReAct agent."
]
},
{
@@ -179,7 +168,7 @@
"class ToolInput(BaseModel):\n",
" code: str = Field(description=\"Python code to execute.\")\n",
"python_tool.args_schema = ToolInput\n",
- "tools=[python_tool] \n",
+ "tools=[python_tool]\n",
"\n",
"# define the prompt template\n",
"prompt = ChatPromptTemplate.from_template(\"{input}\")\n",
@@ -245,7 +234,7 @@
}
},
"source": [
- "Let's now see how the Agent answers to each of the questions"
+ "Let's now see how the Agent answers each of the questions."
]
},
{
@@ -383,7 +372,7 @@
}
},
"source": [
- "Let's loop again over the same dictionary of questions"
+ "Let's loop again over the same dictionary of questions."
]
},
{
@@ -501,7 +490,7 @@
"class ToolInput(BaseModel):\n",
" code: str = Field(description=\"Python code to execute.\")\n",
"python_tool.args_schema = ToolInput\n",
- "tools=[python_tool] \n",
+ "tools=[python_tool]\n",
"\n",
"prompt = ChatPromptTemplate.from_template(\"{input}\")\n",
"agent = create_cohere_react_agent(\n",
@@ -536,7 +525,7 @@
}
},
"source": [
- "We now define a new question"
+ "We now define a new question."
]
},
{
@@ -566,7 +555,7 @@
}
},
"source": [
- "The answer to this question can be obtained only by accessing both the balance sheet and the income statement, as show below:"
+ "The answer to this question can be obtained only by accessing both the balance sheet and the income statement, as shown below:"
]
},
{
@@ -609,7 +598,7 @@
}
},
"source": [
- "Let's now get the answer from the Agent"
+ "Let's now get the answer from the Agent."
]
},
{
@@ -666,19 +655,6 @@
"source": [
"The Agent defined a plan (\"write and execute Python code to find the largest and smallest values in the relevant columns of the dataframes, then calculate the ratio\") and executed it. It made some mistakes when coding, that resulted in NameError errors, but it fixed them and finally got to the correct answer"
]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "id": "77a3f5ac",
- "metadata": {
- "collapsed": false,
- "jupyter": {
- "outputs_hidden": false
- }
- },
- "outputs": [],
- "source": []
}
],
"metadata": {
diff --git a/notebooks/agents/financial-csv-agent/financial_csv_publication_native.ipynb b/notebooks/agents/financial-csv-agent/financial_csv_publication_native.ipynb
index 6532a8d0..0c58e05a 100644
--- a/notebooks/agents/financial-csv-agent/financial_csv_publication_native.ipynb
+++ b/notebooks/agents/financial-csv-agent/financial_csv_publication_native.ipynb
@@ -2,10 +2,10 @@
"cells": [
{
"cell_type": "markdown",
- "id": "549c3504",
+ "id": "df17583b",
"metadata": {},
"source": [
- ""
+ "# Financial CSV Agent with Native Multi-step Cohere API \n"
]
},
{
@@ -29,7 +29,7 @@
"\n",
"## Objective\n",
"\n",
- "In this notebook we explore how to setup a [Cohere Agent](https://docs.cohere.com/docs/multi-step-tool-use) to answer questions over tables in Apple's SEC10K 2020 form. [financial_csv_publication.ipynb](#TODO) already showed how to use langchain to ask questions about your data. This notebook will demonstrate how you can build the same agent using Cohere's native API with langchain python tool. We will also explore how to make your agent more resilient to errors. \n",
+ "In this notebook we explore how to setup a [Cohere Agent](https://docs.cohere.com/docs/multi-step-tool-use) to answer questions over tables in Apple's SEC10K 2020 form. [Financial CSV Agent](https://docs.cohere.com/page/csv-agent) already showed how to use Langchain to ask questions about your data. This notebook will demonstrate how you can build the same agent using Cohere's native API with Langchain Python tool. We will also explore how to make your agent more resilient to errors. \n",
"\n",
"## Table of Contents\n",
"\n",
@@ -47,7 +47,7 @@
"id": "a9f349f1",
"metadata": {},
"source": [
- "\n",
+ "\n",
"\n",
"# Setup"
]
@@ -85,12 +85,7 @@
"metadata": {},
"outputs": [],
"source": [
- "####################################################################################################\n",
- "#\n",
"# Uncomment if you need to install the following packages\n",
- "#\n",
- "####################################################################################################\n",
- "\n",
"# !pip install --quiet langchain langchain_experimental cohere --upgrade"
]
},
@@ -455,7 +450,7 @@
"id": "58a1e1ce",
"metadata": {},
"source": [
- "\n",
+ "\n",
"\n",
"# Define Python Tool \n",
"\n",
@@ -511,11 +506,11 @@
"id": "e58c2745",
"metadata": {},
"source": [
- "\n",
+ "\n",
"\n",
"# Create Cohere Agent \n",
"\n",
- "As [Vanilla_Multi_Step_Tool_Use.ipynb](https://github.com/cohere-ai/notebooks/blob/fbf6c8dad47d7557314e9248a267175c7a6908d8/notebooks/Vanilla_Multi_Step_Tool_Use.ipynb) shows, you have a lot of flexiblity on how you can customize and interact with the cohere agent. Here I am creating a wrapper so that it automatically determines when to stop calling the tools and output final answer. It will run maximum of 15 steps. "
+ "As [Multi-Step Tool Use](https://docs.cohere.com/page/basic-multi-step) shows, you have a lot of flexiblity on how you can customize and interact with the cohere agent. Here I am creating a wrapper so that it automatically determines when to stop calling the tools and output final answer. It will run maximum of 15 steps. "
]
},
{
@@ -618,19 +613,19 @@
"id": "d3e42dd8-20d0-4b97-90fd-4a4ce5836c64",
"metadata": {},
"source": [
- "\n",
+ "\n",
"\n",
"# QnA over Single Table \n",
"\n",
- "In the example below, we show how the python tool can be used to load a dataframe and extract information from it. To do this successfully we need to:\n",
+ "In the example below, we show how the Python tool can be used to load a dataframe and extract information from it. To do this successfully we need to:\n",
"\n",
- "1) pass the file name to the preamble so the model knows how to load the dataframe\n",
- "2) pass a preview of the dataframe in the preamble so the model knows which columns/rows to query\n",
+ "* Pass the file name to the preamble so the model knows how to load the dataframe\n",
+ "* Pass a preview of the dataframe in the preamble so the model knows which columns/rows to query\n",
"\n",
"We will ask the following questions given income statement data. \n",
- "1. what is the highest value of cost of goods and service?\n",
- "2. what is the largest gross profit margin?\n",
- "3. what is the minimum ratio of operating income loss divided by non operating income expense? "
+ "* What is the highest value of cost of goods and service?\n",
+ "* What is the largest gross profit margin?\n",
+ "* What is the minimum ratio of operating income loss divided by non operating income expense? "
]
},
{
@@ -664,8 +659,7 @@
"|---:|-------------:|:----------------------|------------------------------------------------------:|-----------------------------:|--------------:|--------------------------------:|-----------------------------------------:|--------------------:|----------------------:|----------------------------:|----------------------------------------------------------------------------------------------:|--------------------------:|----------------:|------------------------:|--------------------------:|------------------------------------------------:|--------------------------------------------------:|\n",
"| 0 | 0 | 2017-10-01-2018-09-29 | 265595000000 | 1.63756e+11 | 101839000000 | 1.4236e+10 | 1.6705e+10 | 3.0941e+10 | 7.0898e+10 | 2.005e+09 | 7.2903e+10 | 1.3372e+10 | 59531000000 | 3 | 2.98 | 1.98215e+10 | 2.00004e+10 |\n",
"| 1 | 1 | 2018-09-30-2018-12-29 | 84310000000 | nan | 32031000000 | nan | nan | nan | nan | nan | nan | nan | 19965000000 | 1.05 | 1.05 | nan | nan |\n",
- "| 2 | 2 | 2018-09-30-2019-09-28 | 260174000000 | 1.61782e+11 | 98392000000 | 1.6217e+10 | 1.8245e+10 | 3.4462e+10 | 6.393e+10 | 1.807e+09 | 6.5737e+10 | 1.0481e+10 | 55256000000 | 2.99 | 2.97 | 1.84713e+10 | 1.85957e+10 |\n",
- "\n"
+ "| 2 | 2 | 2018-09-30-2019-09-28 | 260174000000 | 1.61782e+11 | 98392000000 | 1.6217e+10 | 1.8245e+10 | 3.4462e+10 | 6.393e+10 | 1.807e+09 | 6.5737e+10 | 1.0481e+10 | 55256000000 | 2.99 | 2.97 | 1.84713e+10 | 1.85957e+10 |\n"
]
}
],
@@ -740,13 +734,13 @@
"id": "d0200ea4",
"metadata": {},
"source": [
- "\n",
+ "\n",
"\n",
"# QnA over Multiple Tables \n",
"\n",
- "We now make the task for the Agent more complicated, by asking it a question that answer can be computed only by retrieving relevant information from multiple tables: \n",
+ "We now make the task for the Agent more complicated by asking a question that can be only answered by retrieving relevant information from multiple tables: \n",
"\n",
- "- Q: What is the ratio of the largest stockholders equity to the smallest revenue?\n",
+ "* Q: What is the ratio of the largest stockholders equity to the smallest revenue?\n",
"\n",
"As you will see below, this question can be obtained only by accessing both the balance sheet and the income statement. \n",
"\n"
@@ -819,8 +813,7 @@
"|---:|-------------:|:-----------|----------------------------------------:|------------------------------:|-------------------------------:|---------------:|-----------------------------:|---------------------:|----------------:|---------------------------------:|-------------------------------:|------------------------:|-------------------:|--------------:|-------------------------:|--------------------------:|---------------------------------------:|------------------:|----------------------:|---------------------:|-------------------------:|-----------------------------:|------------------------:|--------------:|------------------------------:|-----------------------------------------------:|-------------------------------------:|--------------------------------------------------:|---------------------:|-----------------------------------:|\n",
"| 0 | 0 | 2017-09-30 | nan | nan | nan | nan | nan | nan | nan | nan | nan | nan | nan | nan | nan | nan | nan | nan | nan | nan | nan | nan | nan | nan | nan | nan | nan | nan | 134047000000 | nan |\n",
"| 1 | 1 | 2018-09-29 | nan | nan | nan | nan | nan | nan | nan | nan | nan | nan | nan | nan | nan | nan | nan | nan | nan | nan | nan | nan | nan | nan | nan | nan | nan | nan | 107147000000 | nan |\n",
- "| 2 | 2 | 2019-09-28 | 4.8844e+10 | 5.1713e+10 | 2.2926e+10 | 4.106e+09 | 2.2878e+10 | 1.2352e+10 | 1.62819e+11 | 1.05341e+11 | 3.7378e+10 | 3.2978e+10 | 1.75697e+11 | 3.38516e+11 | 4.6236e+10 | 3.772e+10 | 5.522e+09 | 5.98e+09 | 1.026e+10 | 1.05718e+11 | 9.1807e+10 | 5.0503e+10 | 1.4231e+11 | 2.48028e+11 | 0 | 4.5174e+10 | 4.5898e+10 | -5.84e+08 | 90488000000 | 3.38516e+11 |\n",
- "\n"
+ "| 2 | 2 | 2019-09-28 | 4.8844e+10 | 5.1713e+10 | 2.2926e+10 | 4.106e+09 | 2.2878e+10 | 1.2352e+10 | 1.62819e+11 | 1.05341e+11 | 3.7378e+10 | 3.2978e+10 | 1.75697e+11 | 3.38516e+11 | 4.6236e+10 | 3.772e+10 | 5.522e+09 | 5.98e+09 | 1.026e+10 | 1.05718e+11 | 9.1807e+10 | 5.0503e+10 | 1.4231e+11 | 2.48028e+11 | 0 | 4.5174e+10 | 4.5898e+10 | -5.84e+08 | 90488000000 | 3.38516e+11 |\n"
]
}
],
@@ -873,14 +866,14 @@
"id": "571c8c96",
"metadata": {},
"source": [
- "\n",
+ "\n",
"\n",
"# Error Resilience\n",
"\n",
- "In the previous example over single table, the model successfully answered your questions. However, the model may not always have access to the preview of the data. You will see that when we remove the preview from the preamble, the model is run into an error and not produce the answer. We will solve this problem with two different ways: \n",
+ "In the previous example over single table, the model successfully answered your questions. However, the model may not always have access to the preview of the data. You will see that when we remove the preview from the preamble, the model runs into an error and is not produce the answer. We will solve this problem with two different ways: \n",
"\n",
- "1. Asking the model to keep trying until it fixes the issue. \n",
- "2. Giving the model another tool to view the data and telling it to preview the data before writing code. \n",
+ "* Asking the model to keep trying until it fixes the issue. \n",
+ "* Giving the model another tool to view the data and telling it to preview the data before writing code. \n",
"\n",
"You will see that the second method is able to come to the answer with fewer steps. \n"
]
diff --git a/notebooks/agents/sql_agent/sql_agent.ipynb b/notebooks/agents/sql_agent/sql_agent.ipynb
index 62ec38d8..76b525f0 100644
--- a/notebooks/agents/sql_agent/sql_agent.ipynb
+++ b/notebooks/agents/sql_agent/sql_agent.ipynb
@@ -1,13 +1,5 @@
{
"cells": [
- {
- "cell_type": "markdown",
- "id": "5c0d5271-8d35-4471-b667-265695cbf11a",
- "metadata": {},
- "source": [
- ""
- ]
- },
{
"cell_type": "markdown",
"id": "1d71f595-01ef-4c24-99eb-81b01eec048b",
@@ -16,10 +8,10 @@
"# Notebook Overview\n",
"\n",
"## Motivation\n",
- "Enterprise customers often store and handle information in relational databases but querying such databases effectively requires bespoke knowledge of the underlying database structure as well as strong SQL coding skills. One way to address these challenges is to build an LLM agent capable of generating and executing SQL queries based on natural language. For example If a user asks: ``what are the top 4 rows in table X``, the agent should be able to generate ``SELECT * FROM X LIMIT 4``, execute this query and return the output to the user. \n",
+ "Enterprise customers often store and handle information in relational databases but querying such databases effectively requires bespoke knowledge of the underlying database structure as well as strong SQL coding skills. One way to address these challenges is to build an LLM agent capable of generating and executing SQL queries based on natural language. For example, if a user asks: ``what are the top 4 rows in table X``, the agent should be able to generate ``SELECT * FROM X LIMIT 4``, execute this query and return the output to the user. \n",
"\n",
"## Objective\n",
- "In this notebook we explore how to setup a [Cohere ReAct Agent](https://github.com/langchain-ai/langchain-cohere/blob/main/libs/cohere/langchain_cohere/cohere_agent.py) to answer questions over SQL Databases. We show how this can be done seamlessly with langchain's existing SQLDBToolkit.\n",
+ "In this notebook we explore how to setup a [Cohere ReAct Agent](https://github.com/langchain-ai/langchain-cohere/blob/main/libs/cohere/langchain_cohere/cohere_agent.py) to answer questions over SQL Databases. We show how this can be done seamlessly with langchain's existing [SQLDBToolkit](https://python.langchain.com/v0.1/docs/integrations/toolkits/sql_database/).\n",
"\n",
"## Table of Contents\n",
"\n",
@@ -32,17 +24,20 @@
"cell_type": "markdown",
"id": "68e373fd",
"metadata": {
- "collapsed": false
+ "collapsed": false,
+ "jupyter": {
+ "outputs_hidden": false
+ }
},
"source": [
- "\n",
+ "\n",
"# Toolkit Setup\n",
"\n"
]
},
{
"cell_type": "code",
- "execution_count": 1,
+ "execution_count": 3,
"id": "bce0ca82-fc00-40bb-9d0e-a274bd2eb8a7",
"metadata": {
"colab": {
@@ -59,22 +54,20 @@
"from langchain_cohere.chat_models import ChatCohere\n",
"from langchain_community.utilities.sql_database import SQLDatabase\n",
"from langchain_community.agent_toolkits import SQLDatabaseToolkit\n",
- "import os"
+ "import os\n",
+ "import json"
]
},
{
"cell_type": "code",
"execution_count": 2,
"id": "76287bf9-d98d-4a34-84c8-46134b85de53",
- "metadata": {},
+ "metadata": {
+ "is_executing": true
+ },
"outputs": [],
"source": [
- "####################################################################################################\n",
- "#\n",
"# Uncomment if you need to install the following packages\n",
- "#\n",
- "####################################################################################################\n",
- "\n",
"#!pip install --quiet langchain langchain_cohere langchain_experimental --upgrade"
]
},
@@ -96,7 +89,7 @@
},
{
"cell_type": "code",
- "execution_count": 4,
+ "execution_count": 5,
"id": "6195f401-f0be-4a31-ab9f-80e722bb0fbb",
"metadata": {},
"outputs": [],
@@ -107,7 +100,7 @@
},
{
"cell_type": "code",
- "execution_count": 11,
+ "execution_count": 23,
"id": "936212a7-fda6-4e12-a5e3-2fdd373660ae",
"metadata": {},
"outputs": [
@@ -116,31 +109,21 @@
"output_type": "stream",
"text": [
"**List of pre-defined Langchain Tools**\n",
- "['sql_db_query', 'sql_db_schema', 'sql_db_list_tables', 'sql_db_query_checker']\n",
- "\n",
- "**Context to pass to LLM on tables**\n",
- "{'table_info': '\\nCREATE TABLE \"Album\" (\\n\\t\"AlbumId\" INTEGER NOT NULL, \\n\\t\"Title\" NVARCHAR(160) NOT NULL, \\n\\t\"ArtistId\" INTEGER NOT NULL, \\n\\tPRIMARY KEY (\"AlbumId\"), \\n\\tFOREIGN KEY(\"ArtistId\") REFERENCES \"Artist\" (\"ArtistId\")\\n)\\n\\n/*\\n3 rows from Album table:\\nAlbumId\\tTitle\\tArtistId\\n1\\tFor Those About To Rock We Salute You\\t1\\n2\\tBalls to the Wall\\t2\\n3\\tRestless and Wild\\t2\\n*/\\n\\n\\nCREATE TABLE \"Artist\" (\\n\\t\"ArtistId\" INTEGER NOT NULL, \\n\\t\"Name\" NVARCHAR(120), \\n\\tPRIMARY KEY (\"ArtistId\")\\n)\\n\\n/*\\n3 rows from Artist table:\\nArtistId\\tName\\n1\\tAC/DC\\n2\\tAccept\\n3\\tAerosmith\\n*/\\n\\n\\nCREATE TABLE \"Customer\" (\\n\\t\"CustomerId\" INTEGER NOT NULL, \\n\\t\"FirstName\" NVARCHAR(40) NOT NULL, \\n\\t\"LastName\" NVARCHAR(20) NOT NULL, \\n\\t\"Company\" NVARCHAR(80), \\n\\t\"Address\" NVARCHAR(70), \\n\\t\"City\" NVARCHAR(40), \\n\\t\"State\" NVARCHAR(40), \\n\\t\"Country\" NVARCHAR(40), \\n\\t\"PostalCode\" NVARCHAR(10), \\n\\t\"Phone\" NVARCHAR(24), \\n\\t\"Fax\" NVARCHAR(24), \\n\\t\"Email\" NVARCHAR(60) NOT NULL, \\n\\t\"SupportRepId\" INTEGER, \\n\\tPRIMARY KEY (\"CustomerId\"), \\n\\tFOREIGN KEY(\"SupportRepId\") REFERENCES \"Employee\" (\"EmployeeId\")\\n)\\n\\n/*\\n3 rows from Customer table:\\nCustomerId\\tFirstName\\tLastName\\tCompany\\tAddress\\tCity\\tState\\tCountry\\tPostalCode\\tPhone\\tFax\\tEmail\\tSupportRepId\\n1\\tLuís\\tGonçalves\\tEmbraer - Empresa Brasileira de Aeronáutica S.A.\\tAv. Brigadeiro Faria Lima, 2170\\tSão José dos Campos\\tSP\\tBrazil\\t12227-000\\t+55 (12) 3923-5555\\t+55 (12) 3923-5566\\tluisg@embraer.com.br\\t3\\n2\\tLeonie\\tKöhler\\tNone\\tTheodor-Heuss-Straße 34\\tStuttgart\\tNone\\tGermany\\t70174\\t+49 0711 2842222\\tNone\\tleonekohler@surfeu.de\\t5\\n3\\tFrançois\\tTremblay\\tNone\\t1498 rue Bélanger\\tMontréal\\tQC\\tCanada\\tH2G 1A7\\t+1 (514) 721-4711\\tNone\\tftremblay@gmail.com\\t3\\n*/\\n\\n\\nCREATE TABLE \"Employee\" (\\n\\t\"EmployeeId\" INTEGER NOT NULL, \\n\\t\"LastName\" NVARCHAR(20) NOT NULL, \\n\\t\"FirstName\" NVARCHAR(20) NOT NULL, \\n\\t\"Title\" NVARCHAR(30), \\n\\t\"ReportsTo\" INTEGER, \\n\\t\"BirthDate\" DATETIME, \\n\\t\"HireDate\" DATETIME, \\n\\t\"Address\" NVARCHAR(70), \\n\\t\"City\" NVARCHAR(40), \\n\\t\"State\" NVARCHAR(40), \\n\\t\"Country\" NVARCHAR(40), \\n\\t\"PostalCode\" NVARCHAR(10), \\n\\t\"Phone\" NVARCHAR(24), \\n\\t\"Fax\" NVARCHAR(24), \\n\\t\"Email\" NVARCHAR(60), \\n\\tPRIMARY KEY (\"EmployeeId\"), \\n\\tFOREIGN KEY(\"ReportsTo\") REFERENCES \"Employee\" (\"EmployeeId\")\\n)\\n\\n/*\\n3 rows from Employee table:\\nEmployeeId\\tLastName\\tFirstName\\tTitle\\tReportsTo\\tBirthDate\\tHireDate\\tAddress\\tCity\\tState\\tCountry\\tPostalCode\\tPhone\\tFax\\tEmail\\n1\\tAdams\\tAndrew\\tGeneral Manager\\tNone\\t1962-02-18 00:00:00\\t2002-08-14 00:00:00\\t11120 Jasper Ave NW\\tEdmonton\\tAB\\tCanada\\tT5K 2N1\\t+1 (780) 428-9482\\t+1 (780) 428-3457\\tandrew@chinookcorp.com\\n2\\tEdwards\\tNancy\\tSales Manager\\t1\\t1958-12-08 00:00:00\\t2002-05-01 00:00:00\\t825 8 Ave SW\\tCalgary\\tAB\\tCanada\\tT2P 2T3\\t+1 (403) 262-3443\\t+1 (403) 262-3322\\tnancy@chinookcorp.com\\n3\\tPeacock\\tJane\\tSales Support Agent\\t2\\t1973-08-29 00:00:00\\t2002-04-01 00:00:00\\t1111 6 Ave SW\\tCalgary\\tAB\\tCanada\\tT2P 5M5\\t+1 (403) 262-3443\\t+1 (403) 262-6712\\tjane@chinookcorp.com\\n*/\\n\\n\\nCREATE TABLE \"Genre\" (\\n\\t\"GenreId\" INTEGER NOT NULL, \\n\\t\"Name\" NVARCHAR(120), \\n\\tPRIMARY KEY (\"GenreId\")\\n)\\n\\n/*\\n3 rows from Genre table:\\nGenreId\\tName\\n1\\tRock\\n2\\tJazz\\n3\\tMetal\\n*/\\n\\n\\nCREATE TABLE \"Invoice\" (\\n\\t\"InvoiceId\" INTEGER NOT NULL, \\n\\t\"CustomerId\" INTEGER NOT NULL, \\n\\t\"InvoiceDate\" DATETIME NOT NULL, \\n\\t\"BillingAddress\" NVARCHAR(70), \\n\\t\"BillingCity\" NVARCHAR(40), \\n\\t\"BillingState\" NVARCHAR(40), \\n\\t\"BillingCountry\" NVARCHAR(40), \\n\\t\"BillingPostalCode\" NVARCHAR(10), \\n\\t\"Total\" NUMERIC(10, 2) NOT NULL, \\n\\tPRIMARY KEY (\"InvoiceId\"), \\n\\tFOREIGN KEY(\"CustomerId\") REFERENCES \"Customer\" (\"CustomerId\")\\n)\\n\\n/*\\n3 rows from Invoice table:\\nInvoiceId\\tCustomerId\\tInvoiceDate\\tBillingAddress\\tBillingCity\\tBillingState\\tBillingCountry\\tBillingPostalCode\\tTotal\\n1\\t2\\t2021-01-01 00:00:00\\tTheodor-Heuss-Straße 34\\tStuttgart\\tNone\\tGermany\\t70174\\t1.98\\n2\\t4\\t2021-01-02 00:00:00\\tUllevålsveien 14\\tOslo\\tNone\\tNorway\\t0171\\t3.96\\n3\\t8\\t2021-01-03 00:00:00\\tGrétrystraat 63\\tBrussels\\tNone\\tBelgium\\t1000\\t5.94\\n*/\\n\\n\\nCREATE TABLE \"InvoiceLine\" (\\n\\t\"InvoiceLineId\" INTEGER NOT NULL, \\n\\t\"InvoiceId\" INTEGER NOT NULL, \\n\\t\"TrackId\" INTEGER NOT NULL, \\n\\t\"UnitPrice\" NUMERIC(10, 2) NOT NULL, \\n\\t\"Quantity\" INTEGER NOT NULL, \\n\\tPRIMARY KEY (\"InvoiceLineId\"), \\n\\tFOREIGN KEY(\"TrackId\") REFERENCES \"Track\" (\"TrackId\"), \\n\\tFOREIGN KEY(\"InvoiceId\") REFERENCES \"Invoice\" (\"InvoiceId\")\\n)\\n\\n/*\\n3 rows from InvoiceLine table:\\nInvoiceLineId\\tInvoiceId\\tTrackId\\tUnitPrice\\tQuantity\\n1\\t1\\t2\\t0.99\\t1\\n2\\t1\\t4\\t0.99\\t1\\n3\\t2\\t6\\t0.99\\t1\\n*/\\n\\n\\nCREATE TABLE \"MediaType\" (\\n\\t\"MediaTypeId\" INTEGER NOT NULL, \\n\\t\"Name\" NVARCHAR(120), \\n\\tPRIMARY KEY (\"MediaTypeId\")\\n)\\n\\n/*\\n3 rows from MediaType table:\\nMediaTypeId\\tName\\n1\\tMPEG audio file\\n2\\tProtected AAC audio file\\n3\\tProtected MPEG-4 video file\\n*/\\n\\n\\nCREATE TABLE \"Playlist\" (\\n\\t\"PlaylistId\" INTEGER NOT NULL, \\n\\t\"Name\" NVARCHAR(120), \\n\\tPRIMARY KEY (\"PlaylistId\")\\n)\\n\\n/*\\n3 rows from Playlist table:\\nPlaylistId\\tName\\n1\\tMusic\\n2\\tMovies\\n3\\tTV Shows\\n*/\\n\\n\\nCREATE TABLE \"PlaylistTrack\" (\\n\\t\"PlaylistId\" INTEGER NOT NULL, \\n\\t\"TrackId\" INTEGER NOT NULL, \\n\\tPRIMARY KEY (\"PlaylistId\", \"TrackId\"), \\n\\tFOREIGN KEY(\"TrackId\") REFERENCES \"Track\" (\"TrackId\"), \\n\\tFOREIGN KEY(\"PlaylistId\") REFERENCES \"Playlist\" (\"PlaylistId\")\\n)\\n\\n/*\\n3 rows from PlaylistTrack table:\\nPlaylistId\\tTrackId\\n1\\t3402\\n1\\t3389\\n1\\t3390\\n*/\\n\\n\\nCREATE TABLE \"Track\" (\\n\\t\"TrackId\" INTEGER NOT NULL, \\n\\t\"Name\" NVARCHAR(200) NOT NULL, \\n\\t\"AlbumId\" INTEGER, \\n\\t\"MediaTypeId\" INTEGER NOT NULL, \\n\\t\"GenreId\" INTEGER, \\n\\t\"Composer\" NVARCHAR(220), \\n\\t\"Milliseconds\" INTEGER NOT NULL, \\n\\t\"Bytes\" INTEGER, \\n\\t\"UnitPrice\" NUMERIC(10, 2) NOT NULL, \\n\\tPRIMARY KEY (\"TrackId\"), \\n\\tFOREIGN KEY(\"MediaTypeId\") REFERENCES \"MediaType\" (\"MediaTypeId\"), \\n\\tFOREIGN KEY(\"GenreId\") REFERENCES \"Genre\" (\"GenreId\"), \\n\\tFOREIGN KEY(\"AlbumId\") REFERENCES \"Album\" (\"AlbumId\")\\n)\\n\\n/*\\n3 rows from Track table:\\nTrackId\\tName\\tAlbumId\\tMediaTypeId\\tGenreId\\tComposer\\tMilliseconds\\tBytes\\tUnitPrice\\n1\\tFor Those About To Rock (We Salute You)\\t1\\t1\\t1\\tAngus Young, Malcolm Young, Brian Johnson\\t343719\\t11170334\\t0.99\\n2\\tBalls to the Wall\\t2\\t2\\t1\\tU. Dirkschneider, W. Hoffmann, H. Frank, P. Baltes, S. Kaufmann, G. Hoffmann\\t342562\\t5510424\\t0.99\\n3\\tFast As a Shark\\t3\\t2\\t1\\tF. Baltes, S. Kaufman, U. Dirkscneider & W. Hoffman\\t230619\\t3990994\\t0.99\\n*/', 'table_names': 'Album, Artist, Customer, Employee, Genre, Invoice, InvoiceLine, MediaType, Playlist, PlaylistTrack, Track'}\n"
+ "['sql_db_query', 'sql_db_schema', 'sql_db_list_tables', 'sql_db_query_checker']\n"
]
}
],
"source": [
- "\n",
"DB_NAME='Chinook.db'\n",
"MODEL=\"command-r-plus\"\n",
- "\n",
"llm = ChatCohere(model=MODEL, temperature=0.1,verbose=True)\n",
"db = SQLDatabase.from_uri(f\"sqlite:///{DB_NAME}\")\n",
- "\n",
- "\n",
"toolkit = SQLDatabaseToolkit(db=db, llm=llm)\n",
"context = toolkit.get_context()\n",
"tools = toolkit.get_tools()\n",
"\n",
"print('**List of pre-defined Langchain Tools**')\n",
- "print([tool.name for tool in tools])\n",
- "print('')\n",
- "print('**Context to pass to LLM on tables**')\n",
- "print(context)"
+ "print([tool.name for tool in tools])"
]
},
{
@@ -148,23 +131,21 @@
"id": "7554c397-76ae-46c8-9c76-b906f926febf",
"metadata": {},
"source": [
- "\n",
+ "\n",
"# SQL Agent\n",
"\n",
- "we follow the general cohere react agent setup in Langchain to build our SQL agent."
+ "We follow the general cohere react agent setup in Langchain to build our SQL agent."
]
},
{
"cell_type": "code",
- "execution_count": 13,
+ "execution_count": 28,
"id": "9be6f9eb-b1ba-4a4f-a5f3-5b775f0db69f",
"metadata": {},
"outputs": [],
"source": [
- "\n",
"# define the prompt template\n",
"prompt = ChatPromptTemplate.from_template(\"{input}\")\n",
- "\n",
"# instantiate the ReAct agent\n",
"agent = create_cohere_react_agent(\n",
" llm=llm,\n",
@@ -175,8 +156,7 @@
" tools=tools,\n",
" verbose=True,\n",
" return_intermediate_steps=True\n",
- " )\n",
- "\n"
+ " )"
]
},
{
@@ -191,65 +171,65 @@
"text": [
"\n",
"\n",
- "\u001B[1m> Entering new AgentExecutor chain...\u001B[0m\n",
- "\u001B[32;1m\u001B[1;3m\n",
- "I will use the sql_db_list_tables tool to find out what tables are available.\n",
+ "\u001b[1m> Entering new AgentExecutor chain...\u001b[0m\n",
+ "\u001b[32;1m\u001b[1;3m\n",
+ "I will use the sql_db_list_tables tool to find out which tables are available.\n",
"{'tool_name': 'sql_db_list_tables', 'parameters': {'tool_input': ''}}\n",
- "\u001B[0m\u001B[38;5;200m\u001B[1;3mAlbum, Artist, Customer, Employee, Genre, Invoice, InvoiceLine, MediaType, Playlist, PlaylistTrack, Track\u001B[0m\u001B[32;1m\u001B[1;3mRelevant Documents: 0\n",
+ "\u001b[0m\u001b[38;5;200m\u001b[1;3mAlbum, Artist, Customer, Employee, Genre, Invoice, InvoiceLine, MediaType, Playlist, PlaylistTrack, Track\u001b[0m\u001b[32;1m\u001b[1;3mRelevant Documents: 0\n",
"Cited Documents: 0\n",
"Answer: The following tables are available: Album, Artist, Customer, Employee, Genre, Invoice, InvoiceLine, MediaType, Playlist, PlaylistTrack, Track.\n",
- "Grounded answer: The following tables are available: Album, Artist, Customer, Employee, Genre, Invoice, InvoiceLine, MediaType, Playlist, PlaylistTrack, Track.\u001B[0m\n",
+ "Grounded answer: The following tables are available: Album, Artist, Customer, Employee, Genre, Invoice, InvoiceLine, MediaType, Playlist, PlaylistTrack, Track.\u001b[0m\n",
"\n",
- "\u001B[1m> Finished chain.\u001B[0m\n"
+ "\u001b[1m> Finished chain.\u001b[0m\n"
]
- },
- {
- "data": {
- "text/plain": [
- "{'input': 'what tables are available?',\n",
- " 'output': 'The following tables are available: Album, Artist, Customer, Employee, Genre, Invoice, InvoiceLine, MediaType, Playlist, PlaylistTrack, Track.',\n",
- " 'citations': [CohereCitation(start=36, end=41, text='Album', documents=[{'output': 'Album, Artist, Customer, Employee, Genre, Invoice, InvoiceLine, MediaType, Playlist, PlaylistTrack, Track'}]),\n",
- " CohereCitation(start=43, end=49, text='Artist', documents=[{'output': 'Album, Artist, Customer, Employee, Genre, Invoice, InvoiceLine, MediaType, Playlist, PlaylistTrack, Track'}]),\n",
- " CohereCitation(start=51, end=59, text='Customer', documents=[{'output': 'Album, Artist, Customer, Employee, Genre, Invoice, InvoiceLine, MediaType, Playlist, PlaylistTrack, Track'}]),\n",
- " CohereCitation(start=61, end=69, text='Employee', documents=[{'output': 'Album, Artist, Customer, Employee, Genre, Invoice, InvoiceLine, MediaType, Playlist, PlaylistTrack, Track'}]),\n",
- " CohereCitation(start=71, end=76, text='Genre', documents=[{'output': 'Album, Artist, Customer, Employee, Genre, Invoice, InvoiceLine, MediaType, Playlist, PlaylistTrack, Track'}]),\n",
- " CohereCitation(start=78, end=85, text='Invoice', documents=[{'output': 'Album, Artist, Customer, Employee, Genre, Invoice, InvoiceLine, MediaType, Playlist, PlaylistTrack, Track'}]),\n",
- " CohereCitation(start=87, end=98, text='InvoiceLine', documents=[{'output': 'Album, Artist, Customer, Employee, Genre, Invoice, InvoiceLine, MediaType, Playlist, PlaylistTrack, Track'}]),\n",
- " CohereCitation(start=100, end=109, text='MediaType', documents=[{'output': 'Album, Artist, Customer, Employee, Genre, Invoice, InvoiceLine, MediaType, Playlist, PlaylistTrack, Track'}]),\n",
- " CohereCitation(start=111, end=119, text='Playlist', documents=[{'output': 'Album, Artist, Customer, Employee, Genre, Invoice, InvoiceLine, MediaType, Playlist, PlaylistTrack, Track'}]),\n",
- " CohereCitation(start=121, end=134, text='PlaylistTrack', documents=[{'output': 'Album, Artist, Customer, Employee, Genre, Invoice, InvoiceLine, MediaType, Playlist, PlaylistTrack, Track'}]),\n",
- " CohereCitation(start=136, end=141, text='Track', documents=[{'output': 'Album, Artist, Customer, Employee, Genre, Invoice, InvoiceLine, MediaType, Playlist, PlaylistTrack, Track'}])],\n",
- " 'intermediate_steps': [(AgentActionMessageLog(tool='sql_db_list_tables', tool_input={'tool_input': ''}, log=\"\\nI will use the sql_db_list_tables tool to find out what tables are available.\\n{'tool_name': 'sql_db_list_tables', 'parameters': {'tool_input': ''}}\\n\", message_log=[AIMessage(content='\\nPlan: I will use the sql_db_list_tables tool to find out what tables are available.\\nAction: ```json\\n[\\n {\\n \"tool_name\": \"sql_db_list_tables\",\\n \"parameters\": {\\n \"tool_input\": \"\"\\n }\\n }\\n]\\n```')]),\n",
- " 'Album, Artist, Customer, Employee, Genre, Invoice, InvoiceLine, MediaType, Playlist, PlaylistTrack, Track')]}"
- ]
- },
- "execution_count": 14,
- "metadata": {},
- "output_type": "execute_result"
}
],
"source": [
- "agent_executor.invoke({\n",
+ "output=agent_executor.invoke({\n",
" \"input\": 'what tables are available?',\n",
"})"
]
},
+ {
+ "cell_type": "code",
+ "execution_count": 15,
+ "id": "e7118c7a-f92b-4cbc-81b5-7d005801027c",
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "The following tables are available: Album, Artist, Customer, Employee, Genre, Invoice, InvoiceLine, MediaType, Playlist, PlaylistTrack, Track.\n"
+ ]
+ }
+ ],
+ "source": [
+ "print(output['output'])"
+ ]
+ },
{
"cell_type": "markdown",
"id": "8a1fc000",
"metadata": {
- "collapsed": false
+ "collapsed": false,
+ "jupyter": {
+ "outputs_hidden": false
+ }
},
"source": [
- "The agent uses the list_tables tool to effectively highlight all the tables in the DB"
+ "The agent uses the list_tables tool to effectively highlight all the tables in the DB."
]
},
{
"cell_type": "code",
- "execution_count": 16,
+ "execution_count": 17,
"id": "77a3f5ac",
"metadata": {
- "collapsed": false
+ "collapsed": false,
+ "jupyter": {
+ "outputs_hidden": false
+ }
},
"outputs": [
{
@@ -258,11 +238,11 @@
"text": [
"\n",
"\n",
- "\u001B[1m> Entering new AgentExecutor chain...\u001B[0m\n",
- "\u001B[32;1m\u001B[1;3m\n",
+ "\u001b[1m> Entering new AgentExecutor chain...\u001b[0m\n",
+ "\u001b[32;1m\u001b[1;3m\n",
"I will use the sql_db_schema tool to find the first row of the Playlist and Genre tables.\n",
"{'tool_name': 'sql_db_schema', 'parameters': {'table_names': 'Playlist, Genre'}}\n",
- "\u001B[0m\u001B[33;1m\u001B[1;3m\n",
+ "\u001b[0m\u001b[33;1m\u001b[1;3m\n",
"CREATE TABLE \"Genre\" (\n",
"\t\"GenreId\" INTEGER NOT NULL, \n",
"\t\"Name\" NVARCHAR(120), \n",
@@ -290,73 +270,80 @@
"1\tMusic\n",
"2\tMovies\n",
"3\tTV Shows\n",
- "*/\u001B[0m\u001B[32;1m\u001B[1;3mRelevant Documents: 0\n",
+ "*/\u001b[0m\u001b[32;1m\u001b[1;3mRelevant Documents: 0\n",
"Cited Documents: 0\n",
"Answer: Here is the first row of the Genre table:\n",
"\n",
"| GenreId | Name |\n",
- "| --- | --- |\n",
+ "|---|---|\n",
"| 1 | Rock |\n",
"\n",
"Here is the first row of the Playlist table:\n",
"\n",
"| PlaylistId | Name |\n",
- "| --- | --- |\n",
+ "|---|---|\n",
"| 1 | Music |\n",
"Grounded answer: Here is the first row of the Genre table:\n",
"\n",
"| GenreId | Name |\n",
- "| --- | --- |\n",
+ "|---|---|\n",
"| 1 | Rock |\n",
"\n",
"Here is the first row of the Playlist table:\n",
"\n",
"| PlaylistId | Name |\n",
- "| --- | --- |\n",
- "| 1 | Music |\u001B[0m\n",
+ "|---|---|\n",
+ "| 1 | Music |\u001b[0m\n",
"\n",
- "\u001B[1m> Finished chain.\u001B[0m\n"
+ "\u001b[1m> Finished chain.\u001b[0m\n"
]
- },
- {
- "data": {
- "text/plain": [
- "{'input': 'show the first row of the Playlist and Genre tables?',\n",
- " 'output': 'Here is the first row of the Genre table:\\n\\n| GenreId | Name |\\n| --- | --- |\\n| 1 | Rock |\\n\\nHere is the first row of the Playlist table:\\n\\n| PlaylistId | Name |\\n| --- | --- |\\n| 1 | Music |',\n",
- " 'citations': [CohereCitation(start=45, end=52, text='GenreId', documents=[{'output': '\\nCREATE TABLE \"Genre\" (\\n\\t\"GenreId\" INTEGER NOT NULL, \\n\\t\"Name\" NVARCHAR(120), \\n\\tPRIMARY KEY (\"GenreId\")\\n)\\n\\n/*\\n3 rows from Genre table:\\nGenreId\\tName\\n1\\tRock\\n2\\tJazz\\n3\\tMetal\\n*/\\n\\n\\nCREATE TABLE \"Playlist\" (\\n\\t\"PlaylistId\" INTEGER NOT NULL, \\n\\t\"Name\" NVARCHAR(120), \\n\\tPRIMARY KEY (\"PlaylistId\")\\n)\\n\\n/*\\n3 rows from Playlist table:\\nPlaylistId\\tName\\n1\\tMusic\\n2\\tMovies\\n3\\tTV Shows\\n*/'}]),\n",
- " CohereCitation(start=55, end=59, text='Name', documents=[{'output': '\\nCREATE TABLE \"Genre\" (\\n\\t\"GenreId\" INTEGER NOT NULL, \\n\\t\"Name\" NVARCHAR(120), \\n\\tPRIMARY KEY (\"GenreId\")\\n)\\n\\n/*\\n3 rows from Genre table:\\nGenreId\\tName\\n1\\tRock\\n2\\tJazz\\n3\\tMetal\\n*/\\n\\n\\nCREATE TABLE \"Playlist\" (\\n\\t\"PlaylistId\" INTEGER NOT NULL, \\n\\t\"Name\" NVARCHAR(120), \\n\\tPRIMARY KEY (\"PlaylistId\")\\n)\\n\\n/*\\n3 rows from Playlist table:\\nPlaylistId\\tName\\n1\\tMusic\\n2\\tMovies\\n3\\tTV Shows\\n*/'}]),\n",
- " CohereCitation(start=78, end=79, text='1', documents=[{'output': '\\nCREATE TABLE \"Genre\" (\\n\\t\"GenreId\" INTEGER NOT NULL, \\n\\t\"Name\" NVARCHAR(120), \\n\\tPRIMARY KEY (\"GenreId\")\\n)\\n\\n/*\\n3 rows from Genre table:\\nGenreId\\tName\\n1\\tRock\\n2\\tJazz\\n3\\tMetal\\n*/\\n\\n\\nCREATE TABLE \"Playlist\" (\\n\\t\"PlaylistId\" INTEGER NOT NULL, \\n\\t\"Name\" NVARCHAR(120), \\n\\tPRIMARY KEY (\"PlaylistId\")\\n)\\n\\n/*\\n3 rows from Playlist table:\\nPlaylistId\\tName\\n1\\tMusic\\n2\\tMovies\\n3\\tTV Shows\\n*/'}]),\n",
- " CohereCitation(start=82, end=86, text='Rock', documents=[{'output': '\\nCREATE TABLE \"Genre\" (\\n\\t\"GenreId\" INTEGER NOT NULL, \\n\\t\"Name\" NVARCHAR(120), \\n\\tPRIMARY KEY (\"GenreId\")\\n)\\n\\n/*\\n3 rows from Genre table:\\nGenreId\\tName\\n1\\tRock\\n2\\tJazz\\n3\\tMetal\\n*/\\n\\n\\nCREATE TABLE \"Playlist\" (\\n\\t\"PlaylistId\" INTEGER NOT NULL, \\n\\t\"Name\" NVARCHAR(120), \\n\\tPRIMARY KEY (\"PlaylistId\")\\n)\\n\\n/*\\n3 rows from Playlist table:\\nPlaylistId\\tName\\n1\\tMusic\\n2\\tMovies\\n3\\tTV Shows\\n*/'}]),\n",
- " CohereCitation(start=138, end=148, text='PlaylistId', documents=[{'output': '\\nCREATE TABLE \"Genre\" (\\n\\t\"GenreId\" INTEGER NOT NULL, \\n\\t\"Name\" NVARCHAR(120), \\n\\tPRIMARY KEY (\"GenreId\")\\n)\\n\\n/*\\n3 rows from Genre table:\\nGenreId\\tName\\n1\\tRock\\n2\\tJazz\\n3\\tMetal\\n*/\\n\\n\\nCREATE TABLE \"Playlist\" (\\n\\t\"PlaylistId\" INTEGER NOT NULL, \\n\\t\"Name\" NVARCHAR(120), \\n\\tPRIMARY KEY (\"PlaylistId\")\\n)\\n\\n/*\\n3 rows from Playlist table:\\nPlaylistId\\tName\\n1\\tMusic\\n2\\tMovies\\n3\\tTV Shows\\n*/'}]),\n",
- " CohereCitation(start=151, end=155, text='Name', documents=[{'output': '\\nCREATE TABLE \"Genre\" (\\n\\t\"GenreId\" INTEGER NOT NULL, \\n\\t\"Name\" NVARCHAR(120), \\n\\tPRIMARY KEY (\"GenreId\")\\n)\\n\\n/*\\n3 rows from Genre table:\\nGenreId\\tName\\n1\\tRock\\n2\\tJazz\\n3\\tMetal\\n*/\\n\\n\\nCREATE TABLE \"Playlist\" (\\n\\t\"PlaylistId\" INTEGER NOT NULL, \\n\\t\"Name\" NVARCHAR(120), \\n\\tPRIMARY KEY (\"PlaylistId\")\\n)\\n\\n/*\\n3 rows from Playlist table:\\nPlaylistId\\tName\\n1\\tMusic\\n2\\tMovies\\n3\\tTV Shows\\n*/'}]),\n",
- " CohereCitation(start=174, end=175, text='1', documents=[{'output': '\\nCREATE TABLE \"Genre\" (\\n\\t\"GenreId\" INTEGER NOT NULL, \\n\\t\"Name\" NVARCHAR(120), \\n\\tPRIMARY KEY (\"GenreId\")\\n)\\n\\n/*\\n3 rows from Genre table:\\nGenreId\\tName\\n1\\tRock\\n2\\tJazz\\n3\\tMetal\\n*/\\n\\n\\nCREATE TABLE \"Playlist\" (\\n\\t\"PlaylistId\" INTEGER NOT NULL, \\n\\t\"Name\" NVARCHAR(120), \\n\\tPRIMARY KEY (\"PlaylistId\")\\n)\\n\\n/*\\n3 rows from Playlist table:\\nPlaylistId\\tName\\n1\\tMusic\\n2\\tMovies\\n3\\tTV Shows\\n*/'}]),\n",
- " CohereCitation(start=178, end=183, text='Music', documents=[{'output': '\\nCREATE TABLE \"Genre\" (\\n\\t\"GenreId\" INTEGER NOT NULL, \\n\\t\"Name\" NVARCHAR(120), \\n\\tPRIMARY KEY (\"GenreId\")\\n)\\n\\n/*\\n3 rows from Genre table:\\nGenreId\\tName\\n1\\tRock\\n2\\tJazz\\n3\\tMetal\\n*/\\n\\n\\nCREATE TABLE \"Playlist\" (\\n\\t\"PlaylistId\" INTEGER NOT NULL, \\n\\t\"Name\" NVARCHAR(120), \\n\\tPRIMARY KEY (\"PlaylistId\")\\n)\\n\\n/*\\n3 rows from Playlist table:\\nPlaylistId\\tName\\n1\\tMusic\\n2\\tMovies\\n3\\tTV Shows\\n*/'}])],\n",
- " 'intermediate_steps': [(AgentActionMessageLog(tool='sql_db_schema', tool_input={'table_names': 'Playlist, Genre'}, log=\"\\nI will use the sql_db_schema tool to find the first row of the Playlist and Genre tables.\\n{'tool_name': 'sql_db_schema', 'parameters': {'table_names': 'Playlist, Genre'}}\\n\", message_log=[AIMessage(content='\\nPlan: I will use the sql_db_schema tool to find the first row of the Playlist and Genre tables.\\nAction: ```json\\n[\\n {\\n \"tool_name\": \"sql_db_schema\",\\n \"parameters\": {\\n \"table_names\": \"Playlist, Genre\"\\n }\\n }\\n]\\n```')]),\n",
- " '\\nCREATE TABLE \"Genre\" (\\n\\t\"GenreId\" INTEGER NOT NULL, \\n\\t\"Name\" NVARCHAR(120), \\n\\tPRIMARY KEY (\"GenreId\")\\n)\\n\\n/*\\n3 rows from Genre table:\\nGenreId\\tName\\n1\\tRock\\n2\\tJazz\\n3\\tMetal\\n*/\\n\\n\\nCREATE TABLE \"Playlist\" (\\n\\t\"PlaylistId\" INTEGER NOT NULL, \\n\\t\"Name\" NVARCHAR(120), \\n\\tPRIMARY KEY (\"PlaylistId\")\\n)\\n\\n/*\\n3 rows from Playlist table:\\nPlaylistId\\tName\\n1\\tMusic\\n2\\tMovies\\n3\\tTV Shows\\n*/')]}"
- ]
- },
- "execution_count": 16,
- "metadata": {},
- "output_type": "execute_result"
}
],
"source": [
- "agent_executor.invoke({\n",
+ "output=agent_executor.invoke({\n",
" \"input\": 'show the first row of the Playlist and Genre tables?',\n",
"})"
]
},
+ {
+ "cell_type": "code",
+ "execution_count": 18,
+ "id": "55556951-3643-4494-b4bd-6e1c952b9e6b",
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "Here is the first row of the Genre table:\n",
+ "\n",
+ "| GenreId | Name |\n",
+ "|---|---|\n",
+ "| 1 | Rock |\n",
+ "\n",
+ "Here is the first row of the Playlist table:\n",
+ "\n",
+ "| PlaylistId | Name |\n",
+ "|---|---|\n",
+ "| 1 | Music |\n"
+ ]
+ }
+ ],
+ "source": [
+ "print(output['output'])"
+ ]
+ },
{
"cell_type": "markdown",
"id": "3e77d393-7fd8-42df-8ed2-59badb3b6d5b",
"metadata": {},
"source": [
- "Here we see that the tool takes a list of tables to query the sql_db_schema tool to retrieve the various schemas"
+ "Here we see that the tool takes a list of tables to query the sql_db_schema tool to retrieve the various schemas."
]
},
{
"cell_type": "code",
- "execution_count": 17,
+ "execution_count": 19,
"id": "4466e2b1-8cd7-4e50-aab4-8a6d9f809491",
"metadata": {},
"outputs": [
@@ -366,24 +353,24 @@
"text": [
"\n",
"\n",
- "\u001B[1m> Entering new AgentExecutor chain...\u001B[0m\n",
- "\u001B[32;1m\u001B[1;3m\n",
- "I will search for the number of invoices per country and then write an answer based on the results.\n",
+ "\u001b[1m> Entering new AgentExecutor chain...\u001b[0m\n",
+ "\u001b[32;1m\u001b[1;3m\n",
+ "I will search for the number of invoices per country and then write an answer.\n",
"{'tool_name': 'sql_db_query', 'parameters': {'query': 'SELECT country, COUNT(*) AS invoice_count FROM invoices GROUP BY country ORDER BY invoice_count DESC'}}\n",
- "\u001B[0m\u001B[36;1m\u001B[1;3mError: (sqlite3.OperationalError) no such table: invoices\n",
+ "\u001b[0m\u001b[36;1m\u001b[1;3mError: (sqlite3.OperationalError) no such table: invoices\n",
"[SQL: SELECT country, COUNT(*) AS invoice_count FROM invoices GROUP BY country ORDER BY invoice_count DESC]\n",
- "(Background on this error at: https://sqlalche.me/e/20/e3q8)\u001B[0m\u001B[32;1m\u001B[1;3m\n",
- "I received an error message. I will now use the sql_db_list_tables tool to find out the name of the table containing the invoices data.\n",
+ "(Background on this error at: https://sqlalche.me/e/20/e3q8)\u001b[0m\u001b[32;1m\u001b[1;3m\n",
+ "I received an error message. I will now use the sql_db_list_tables tool to find out the correct table name.\n",
"{'tool_name': 'sql_db_list_tables', 'parameters': {'tool_input': ''}}\n",
- "\u001B[0m\u001B[38;5;200m\u001B[1;3mAlbum, Artist, Customer, Employee, Genre, Invoice, InvoiceLine, MediaType, Playlist, PlaylistTrack, Track\u001B[0m\u001B[32;1m\u001B[1;3m\n",
- "I found out that the table name is 'Invoice'. I will now rewrite my query and run it again.\n",
+ "\u001b[0m\u001b[38;5;200m\u001b[1;3mAlbum, Artist, Customer, Employee, Genre, Invoice, InvoiceLine, MediaType, Playlist, PlaylistTrack, Track\u001b[0m\u001b[32;1m\u001b[1;3m\n",
+ "I found out that the correct table name is 'Invoice'. I will now rewrite my query and run it again.\n",
"{'tool_name': 'sql_db_query', 'parameters': {'query': 'SELECT country, COUNT(*) AS invoice_count FROM Invoice GROUP BY country ORDER BY invoice_count DESC'}}\n",
- "\u001B[0m\u001B[36;1m\u001B[1;3mError: (sqlite3.OperationalError) no such column: country\n",
+ "\u001b[0m\u001b[36;1m\u001b[1;3mError: (sqlite3.OperationalError) no such column: country\n",
"[SQL: SELECT country, COUNT(*) AS invoice_count FROM Invoice GROUP BY country ORDER BY invoice_count DESC]\n",
- "(Background on this error at: https://sqlalche.me/e/20/e3q8)\u001B[0m\u001B[32;1m\u001B[1;3m\n",
- "I received another error message. I will now use the sql_db_schema tool to find out the column names in the 'Invoice' table.\n",
+ "(Background on this error at: https://sqlalche.me/e/20/e3q8)\u001b[0m\u001b[32;1m\u001b[1;3m\n",
+ "I received another error message. I will now use the sql_db_schema tool to find out the correct column name.\n",
"{'tool_name': 'sql_db_schema', 'parameters': {'table_names': 'Invoice'}}\n",
- "\u001B[0m\u001B[33;1m\u001B[1;3m\n",
+ "\u001b[0m\u001b[33;1m\u001b[1;3m\n",
"CREATE TABLE \"Invoice\" (\n",
"\t\"InvoiceId\" INTEGER NOT NULL, \n",
"\t\"CustomerId\" INTEGER NOT NULL, \n",
@@ -404,59 +391,53 @@
"1\t2\t2021-01-01 00:00:00\tTheodor-Heuss-Straße 34\tStuttgart\tNone\tGermany\t70174\t1.98\n",
"2\t4\t2021-01-02 00:00:00\tUllevålsveien 14\tOslo\tNone\tNorway\t0171\t3.96\n",
"3\t8\t2021-01-03 00:00:00\tGrétrystraat 63\tBrussels\tNone\tBelgium\t1000\t5.94\n",
- "*/\u001B[0m\u001B[32;1m\u001B[1;3m\n",
- "I found out that the column name for country is 'BillingCountry'. I will now rewrite my query and run it again.\n",
- "{'tool_name': 'sql_db_query', 'parameters': {'query': 'SELECT BillingCountry AS country, COUNT(*) AS invoice_count FROM Invoice GROUP BY country ORDER BY invoice_count DESC'}}\n",
- "\u001B[0m\u001B[36;1m\u001B[1;3m[('USA', 91), ('Canada', 56), ('France', 35), ('Brazil', 35), ('Germany', 28), ('United Kingdom', 21), ('Portugal', 14), ('Czech Republic', 14), ('India', 13), ('Sweden', 7), ('Spain', 7), ('Poland', 7), ('Norway', 7), ('Netherlands', 7), ('Italy', 7), ('Ireland', 7), ('Hungary', 7), ('Finland', 7), ('Denmark', 7), ('Chile', 7), ('Belgium', 7), ('Austria', 7), ('Australia', 7), ('Argentina', 7)]\u001B[0m\u001B[32;1m\u001B[1;3mRelevant Documents: 1,3,4\n",
+ "*/\u001b[0m\u001b[32;1m\u001b[1;3m\n",
+ "I found out that the correct column name is 'BillingCountry'. I will now rewrite my query and run it again.\n",
+ "{'tool_name': 'sql_db_query', 'parameters': {'query': 'SELECT BillingCountry AS country, COUNT(*) AS invoice_count FROM Invoice GROUP BY BillingCountry ORDER BY invoice_count DESC'}}\n",
+ "\u001b[0m\u001b[36;1m\u001b[1;3m[('USA', 91), ('Canada', 56), ('France', 35), ('Brazil', 35), ('Germany', 28), ('United Kingdom', 21), ('Portugal', 14), ('Czech Republic', 14), ('India', 13), ('Sweden', 7), ('Spain', 7), ('Poland', 7), ('Norway', 7), ('Netherlands', 7), ('Italy', 7), ('Ireland', 7), ('Hungary', 7), ('Finland', 7), ('Denmark', 7), ('Chile', 7), ('Belgium', 7), ('Austria', 7), ('Australia', 7), ('Argentina', 7)]\u001b[0m\u001b[32;1m\u001b[1;3mRelevant Documents: 1,3,4\n",
"Cited Documents: 4\n",
"Answer: The countries with the most invoices are the USA (91), Canada (56), and France (35).\n",
- "Grounded answer: The countries with the most invoices are the USA (91), Canada (56), and France (35).\u001B[0m\n",
+ "Grounded answer: The countries with the most invoices are the USA (91), Canada (56), and France (35).\u001b[0m\n",
"\n",
- "\u001B[1m> Finished chain.\u001B[0m\n"
+ "\u001b[1m> Finished chain.\u001b[0m\n"
]
- },
- {
- "data": {
- "text/plain": [
- "{'input': 'which countries have the most invoices?',\n",
- " 'output': 'The countries with the most invoices are the USA (91), Canada (56), and France (35).',\n",
- " 'citations': [CohereCitation(start=45, end=52, text='USA (91', documents=[{'output': \"[('USA', 91), ('Canada', 56), ('France', 35), ('Brazil', 35), ('Germany', 28), ('United Kingdom', 21), ('Portugal', 14), ('Czech Republic', 14), ('India', 13), ('Sweden', 7), ('Spain', 7), ('Poland', 7), ('Norway', 7), ('Netherlands', 7), ('Italy', 7), ('Ireland', 7), ('Hungary', 7), ('Finland', 7), ('Denmark', 7), ('Chile', 7), ('Belgium', 7), ('Austria', 7), ('Australia', 7), ('Argentina', 7)]\"}]),\n",
- " CohereCitation(start=55, end=65, text='Canada (56', documents=[{'output': \"[('USA', 91), ('Canada', 56), ('France', 35), ('Brazil', 35), ('Germany', 28), ('United Kingdom', 21), ('Portugal', 14), ('Czech Republic', 14), ('India', 13), ('Sweden', 7), ('Spain', 7), ('Poland', 7), ('Norway', 7), ('Netherlands', 7), ('Italy', 7), ('Ireland', 7), ('Hungary', 7), ('Finland', 7), ('Denmark', 7), ('Chile', 7), ('Belgium', 7), ('Austria', 7), ('Australia', 7), ('Argentina', 7)]\"}]),\n",
- " CohereCitation(start=72, end=82, text='France (35', documents=[{'output': \"[('USA', 91), ('Canada', 56), ('France', 35), ('Brazil', 35), ('Germany', 28), ('United Kingdom', 21), ('Portugal', 14), ('Czech Republic', 14), ('India', 13), ('Sweden', 7), ('Spain', 7), ('Poland', 7), ('Norway', 7), ('Netherlands', 7), ('Italy', 7), ('Ireland', 7), ('Hungary', 7), ('Finland', 7), ('Denmark', 7), ('Chile', 7), ('Belgium', 7), ('Austria', 7), ('Australia', 7), ('Argentina', 7)]\"}])],\n",
- " 'intermediate_steps': [(AgentActionMessageLog(tool='sql_db_query', tool_input={'query': 'SELECT country, COUNT(*) AS invoice_count FROM invoices GROUP BY country ORDER BY invoice_count DESC'}, log=\"\\nI will search for the number of invoices per country and then write an answer based on the results.\\n{'tool_name': 'sql_db_query', 'parameters': {'query': 'SELECT country, COUNT(*) AS invoice_count FROM invoices GROUP BY country ORDER BY invoice_count DESC'}}\\n\", message_log=[AIMessage(content='\\nPlan: I will search for the number of invoices per country and then write an answer based on the results.\\nAction: ```json\\n[\\n {\\n \"tool_name\": \"sql_db_query\",\\n \"parameters\": {\\n \"query\": \"SELECT country, COUNT(*) AS invoice_count FROM invoices GROUP BY country ORDER BY invoice_count DESC\"\\n }\\n }\\n]\\n```')]),\n",
- " 'Error: (sqlite3.OperationalError) no such table: invoices\\n[SQL: SELECT country, COUNT(*) AS invoice_count FROM invoices GROUP BY country ORDER BY invoice_count DESC]\\n(Background on this error at: https://sqlalche.me/e/20/e3q8)'),\n",
- " (AgentActionMessageLog(tool='sql_db_list_tables', tool_input={'tool_input': ''}, log=\"\\nI received an error message. I will now use the sql_db_list_tables tool to find out the name of the table containing the invoices data.\\n{'tool_name': 'sql_db_list_tables', 'parameters': {'tool_input': ''}}\\n\", message_log=[AIMessage(content='\\nReflection: I received an error message. I will now use the sql_db_list_tables tool to find out the name of the table containing the invoices data.\\nAction: ```json\\n[\\n {\\n \"tool_name\": \"sql_db_list_tables\",\\n \"parameters\": {\\n \"tool_input\": \"\"\\n }\\n }\\n]\\n```')]),\n",
- " 'Album, Artist, Customer, Employee, Genre, Invoice, InvoiceLine, MediaType, Playlist, PlaylistTrack, Track'),\n",
- " (AgentActionMessageLog(tool='sql_db_query', tool_input={'query': 'SELECT country, COUNT(*) AS invoice_count FROM Invoice GROUP BY country ORDER BY invoice_count DESC'}, log=\"\\nI found out that the table name is 'Invoice'. I will now rewrite my query and run it again.\\n{'tool_name': 'sql_db_query', 'parameters': {'query': 'SELECT country, COUNT(*) AS invoice_count FROM Invoice GROUP BY country ORDER BY invoice_count DESC'}}\\n\", message_log=[AIMessage(content='\\nReflection: I found out that the table name is \\'Invoice\\'. I will now rewrite my query and run it again.\\nAction: ```json\\n[\\n {\\n \"tool_name\": \"sql_db_query\",\\n \"parameters\": {\\n \"query\": \"SELECT country, COUNT(*) AS invoice_count FROM Invoice GROUP BY country ORDER BY invoice_count DESC\"\\n }\\n }\\n]\\n```')]),\n",
- " 'Error: (sqlite3.OperationalError) no such column: country\\n[SQL: SELECT country, COUNT(*) AS invoice_count FROM Invoice GROUP BY country ORDER BY invoice_count DESC]\\n(Background on this error at: https://sqlalche.me/e/20/e3q8)'),\n",
- " (AgentActionMessageLog(tool='sql_db_schema', tool_input={'table_names': 'Invoice'}, log=\"\\nI received another error message. I will now use the sql_db_schema tool to find out the column names in the 'Invoice' table.\\n{'tool_name': 'sql_db_schema', 'parameters': {'table_names': 'Invoice'}}\\n\", message_log=[AIMessage(content='\\nReflection: I received another error message. I will now use the sql_db_schema tool to find out the column names in the \\'Invoice\\' table.\\nAction: ```json\\n[\\n {\\n \"tool_name\": \"sql_db_schema\",\\n \"parameters\": {\\n \"table_names\": \"Invoice\"\\n }\\n }\\n]\\n```')]),\n",
- " '\\nCREATE TABLE \"Invoice\" (\\n\\t\"InvoiceId\" INTEGER NOT NULL, \\n\\t\"CustomerId\" INTEGER NOT NULL, \\n\\t\"InvoiceDate\" DATETIME NOT NULL, \\n\\t\"BillingAddress\" NVARCHAR(70), \\n\\t\"BillingCity\" NVARCHAR(40), \\n\\t\"BillingState\" NVARCHAR(40), \\n\\t\"BillingCountry\" NVARCHAR(40), \\n\\t\"BillingPostalCode\" NVARCHAR(10), \\n\\t\"Total\" NUMERIC(10, 2) NOT NULL, \\n\\tPRIMARY KEY (\"InvoiceId\"), \\n\\tFOREIGN KEY(\"CustomerId\") REFERENCES \"Customer\" (\"CustomerId\")\\n)\\n\\n/*\\n3 rows from Invoice table:\\nInvoiceId\\tCustomerId\\tInvoiceDate\\tBillingAddress\\tBillingCity\\tBillingState\\tBillingCountry\\tBillingPostalCode\\tTotal\\n1\\t2\\t2021-01-01 00:00:00\\tTheodor-Heuss-Straße 34\\tStuttgart\\tNone\\tGermany\\t70174\\t1.98\\n2\\t4\\t2021-01-02 00:00:00\\tUllevålsveien 14\\tOslo\\tNone\\tNorway\\t0171\\t3.96\\n3\\t8\\t2021-01-03 00:00:00\\tGrétrystraat 63\\tBrussels\\tNone\\tBelgium\\t1000\\t5.94\\n*/'),\n",
- " (AgentActionMessageLog(tool='sql_db_query', tool_input={'query': 'SELECT BillingCountry AS country, COUNT(*) AS invoice_count FROM Invoice GROUP BY country ORDER BY invoice_count DESC'}, log=\"\\nI found out that the column name for country is 'BillingCountry'. I will now rewrite my query and run it again.\\n{'tool_name': 'sql_db_query', 'parameters': {'query': 'SELECT BillingCountry AS country, COUNT(*) AS invoice_count FROM Invoice GROUP BY country ORDER BY invoice_count DESC'}}\\n\", message_log=[AIMessage(content='\\nReflection: I found out that the column name for country is \\'BillingCountry\\'. I will now rewrite my query and run it again.\\nAction: ```json\\n[\\n {\\n \"tool_name\": \"sql_db_query\",\\n \"parameters\": {\\n \"query\": \"SELECT BillingCountry AS country, COUNT(*) AS invoice_count FROM Invoice GROUP BY country ORDER BY invoice_count DESC\"\\n }\\n }\\n]\\n```')]),\n",
- " \"[('USA', 91), ('Canada', 56), ('France', 35), ('Brazil', 35), ('Germany', 28), ('United Kingdom', 21), ('Portugal', 14), ('Czech Republic', 14), ('India', 13), ('Sweden', 7), ('Spain', 7), ('Poland', 7), ('Norway', 7), ('Netherlands', 7), ('Italy', 7), ('Ireland', 7), ('Hungary', 7), ('Finland', 7), ('Denmark', 7), ('Chile', 7), ('Belgium', 7), ('Austria', 7), ('Australia', 7), ('Argentina', 7)]\")]}"
- ]
- },
- "execution_count": 17,
- "metadata": {},
- "output_type": "execute_result"
}
],
"source": [
- "agent_executor.invoke({\n",
+ "output=agent_executor.invoke({\n",
" \"input\": 'which countries have the most invoices?',\n",
"})"
]
},
+ {
+ "cell_type": "code",
+ "execution_count": 20,
+ "id": "ca2de5e9-b75a-4eb2-8797-ebaf90edb3bf",
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "The countries with the most invoices are the USA (91), Canada (56), and France (35).\n"
+ ]
+ }
+ ],
+ "source": [
+ "print(output['output'])"
+ ]
+ },
{
"cell_type": "markdown",
"id": "634500b5-8980-4ee9-93a3-172130feeb5a",
"metadata": {},
"source": [
- "The agent initially makes some errors as it jumps to answer the question using the db_query tool, but it then realizes it needs to figure out what tables it has access to and what they look like. It then fixes the sql code and is able to generate the right answer"
+ "The agent initially makes some errors as it jumps to answer the question using the db_query tool, but it then realizes it needs to figure out what tables it has access to and what they look like. It then fixes the SQL code and is able to generate the right answer."
]
},
{
"cell_type": "code",
- "execution_count": 18,
+ "execution_count": 29,
"id": "95144085-9a57-4b7d-ac8e-c0996c0d3e81",
"metadata": {},
"outputs": [
@@ -466,19 +447,19 @@
"text": [
"\n",
"\n",
- "\u001B[1m> Entering new AgentExecutor chain...\u001B[0m\n",
- "\u001B[32;1m\u001B[1;3m\n",
- "I will search for the customer who has spent the most money and write an answer based on the results.\n",
- "{'tool_name': 'sql_db_query', 'parameters': {'query': 'SELECT customer_name, SUM(total_price) AS total_spent FROM orders GROUP BY customer_name ORDER BY total_spent DESC LIMIT 1;'}}\n",
- "\u001B[0m\u001B[36;1m\u001B[1;3mError: (sqlite3.OperationalError) no such table: orders\n",
- "[SQL: SELECT customer_name, SUM(total_price) AS total_spent FROM orders GROUP BY customer_name ORDER BY total_spent DESC LIMIT 1;]\n",
- "(Background on this error at: https://sqlalche.me/e/20/e3q8)\u001B[0m\u001B[32;1m\u001B[1;3m\n",
- "I received an error message. I will now use the sql_db_list_tables tool to find out which tables are available.\n",
+ "\u001b[1m> Entering new AgentExecutor chain...\u001b[0m\n",
+ "\u001b[32;1m\u001b[1;3m\n",
+ "I will run a SQL query to find the customer who has spent the most money.\n",
+ "{'tool_name': 'sql_db_query', 'parameters': {'query': 'SELECT customer_name, SUM(total_cost) AS total_spent FROM orders GROUP BY customer_name ORDER BY total_spent DESC LIMIT 1;'}}\n",
+ "\u001b[0m\u001b[36;1m\u001b[1;3mError: (sqlite3.OperationalError) no such table: orders\n",
+ "[SQL: SELECT customer_name, SUM(total_cost) AS total_spent FROM orders GROUP BY customer_name ORDER BY total_spent DESC LIMIT 1;]\n",
+ "(Background on this error at: https://sqlalche.me/e/20/e3q8)\u001b[0m\u001b[32;1m\u001b[1;3m\n",
+ "I received an error message saying that there is no table called 'orders'. I will now use the sql_db_list_tables tool to find out what tables are available.\n",
"{'tool_name': 'sql_db_list_tables', 'parameters': {'tool_input': ''}}\n",
- "\u001B[0m\u001B[38;5;200m\u001B[1;3mAlbum, Artist, Customer, Employee, Genre, Invoice, InvoiceLine, MediaType, Playlist, PlaylistTrack, Track\u001B[0m\u001B[32;1m\u001B[1;3m\n",
- "I found that there is a 'Customer' table and an 'Invoice' table. I will now use the sql_db_schema tool to find out more about these tables.\n",
- "{'tool_name': 'sql_db_schema', 'parameters': {'table_names': 'Customer,Invoice'}}\n",
- "\u001B[0m\u001B[33;1m\u001B[1;3m\n",
+ "\u001b[0m\u001b[38;5;200m\u001b[1;3mAlbum, Artist, Customer, Employee, Genre, Invoice, InvoiceLine, MediaType, Playlist, PlaylistTrack, Track\u001b[0m\u001b[32;1m\u001b[1;3m\n",
+ "I found that there is a table called 'Customer'. I will now use the sql_db_schema tool to find out what columns are in the 'Customer' table.\n",
+ "{'tool_name': 'sql_db_schema', 'parameters': {'table_names': 'Customer'}}\n",
+ "\u001b[0m\u001b[33;1m\u001b[1;3m\n",
"CREATE TABLE \"Customer\" (\n",
"\t\"CustomerId\" INTEGER NOT NULL, \n",
"\t\"FirstName\" NVARCHAR(40) NOT NULL, \n",
@@ -503,9 +484,10 @@
"1\tLuís\tGonçalves\tEmbraer - Empresa Brasileira de Aeronáutica S.A.\tAv. Brigadeiro Faria Lima, 2170\tSão José dos Campos\tSP\tBrazil\t12227-000\t+55 (12) 3923-5555\t+55 (12) 3923-5566\tluisg@embraer.com.br\t3\n",
"2\tLeonie\tKöhler\tNone\tTheodor-Heuss-Straße 34\tStuttgart\tNone\tGermany\t70174\t+49 0711 2842222\tNone\tleonekohler@surfeu.de\t5\n",
"3\tFrançois\tTremblay\tNone\t1498 rue Bélanger\tMontréal\tQC\tCanada\tH2G 1A7\t+1 (514) 721-4711\tNone\tftremblay@gmail.com\t3\n",
- "*/\n",
- "\n",
- "\n",
+ "*/\u001b[0m\u001b[32;1m\u001b[1;3m\n",
+ "I found that the 'Customer' table does not contain any information about how much money a customer has spent. I will now use the sql_db_schema tool to find out what columns are in the 'Invoice' table.\n",
+ "{'tool_name': 'sql_db_schema', 'parameters': {'table_names': 'Invoice'}}\n",
+ "\u001b[0m\u001b[33;1m\u001b[1;3m\n",
"CREATE TABLE \"Invoice\" (\n",
"\t\"InvoiceId\" INTEGER NOT NULL, \n",
"\t\"CustomerId\" INTEGER NOT NULL, \n",
@@ -526,51 +508,48 @@
"1\t2\t2021-01-01 00:00:00\tTheodor-Heuss-Straße 34\tStuttgart\tNone\tGermany\t70174\t1.98\n",
"2\t4\t2021-01-02 00:00:00\tUllevålsveien 14\tOslo\tNone\tNorway\t0171\t3.96\n",
"3\t8\t2021-01-03 00:00:00\tGrétrystraat 63\tBrussels\tNone\tBelgium\t1000\t5.94\n",
- "*/\u001B[0m\u001B[32;1m\u001B[1;3m\n",
- "I found that the 'Customer' table contains information about customers, including their names, addresses, and phone numbers. The 'Invoice' table contains information about invoices, including the customer ID, invoice date, and total amount. I will now write a new query to find the customer who has spent the most money.\n",
+ "*/\u001b[0m\u001b[32;1m\u001b[1;3m\n",
+ "I found that the 'Invoice' table contains a 'Total' column, which is likely to be the total amount spent by the customer on that invoice. I will now use the sql_db_query tool to find the customer who has spent the most money.\n",
"{'tool_name': 'sql_db_query', 'parameters': {'query': 'SELECT c.FirstName, c.LastName, SUM(i.Total) AS total_spent FROM Customer c JOIN Invoice i ON c.CustomerId = i.CustomerId GROUP BY c.CustomerId ORDER BY total_spent DESC LIMIT 1;'}}\n",
- "\u001B[0m\u001B[36;1m\u001B[1;3m[('Helena', 'Holý', 49.62)]\u001B[0m\u001B[32;1m\u001B[1;3mRelevant Documents: 1,2,3\n",
- "Cited Documents: 3\n",
+ "\u001b[0m\u001b[36;1m\u001b[1;3m[('Helena', 'Holý', 49.62)]\u001b[0m\u001b[32;1m\u001b[1;3mRelevant Documents: 1,2,3,4\n",
+ "Cited Documents: 4\n",
"Answer: The best customer is Helena Holý, who has spent a total of 49.62.\n",
- "Grounded answer: The best customer is Helena Holý, who has spent a total of 49.62.\u001B[0m\n",
+ "Grounded answer: The best customer is Helena Holý, who has spent a total of 49.62.\u001b[0m\n",
"\n",
- "\u001B[1m> Finished chain.\u001B[0m\n"
+ "\u001b[1m> Finished chain.\u001b[0m\n"
]
- },
- {
- "data": {
- "text/plain": [
- "{'input': 'who is the best customer? The customer who has spent the most money is the best.',\n",
- " 'output': 'The best customer is Helena Holý, who has spent a total of 49.62.',\n",
- " 'citations': [CohereCitation(start=21, end=32, text='Helena Holý', documents=[{'output': \"[('Helena', 'Holý', 49.62)]\"}]),\n",
- " CohereCitation(start=59, end=64, text='49.62', documents=[{'output': \"[('Helena', 'Holý', 49.62)]\"}])],\n",
- " 'intermediate_steps': [(AgentActionMessageLog(tool='sql_db_query', tool_input={'query': 'SELECT customer_name, SUM(total_price) AS total_spent FROM orders GROUP BY customer_name ORDER BY total_spent DESC LIMIT 1;'}, log=\"\\nI will search for the customer who has spent the most money and write an answer based on the results.\\n{'tool_name': 'sql_db_query', 'parameters': {'query': 'SELECT customer_name, SUM(total_price) AS total_spent FROM orders GROUP BY customer_name ORDER BY total_spent DESC LIMIT 1;'}}\\n\", message_log=[AIMessage(content='\\nPlan: I will search for the customer who has spent the most money and write an answer based on the results.\\nAction: ```json\\n[\\n {\\n \"tool_name\": \"sql_db_query\",\\n \"parameters\": {\\n \"query\": \"SELECT customer_name, SUM(total_price) AS total_spent FROM orders GROUP BY customer_name ORDER BY total_spent DESC LIMIT 1;\"\\n }\\n }\\n]\\n```')]),\n",
- " 'Error: (sqlite3.OperationalError) no such table: orders\\n[SQL: SELECT customer_name, SUM(total_price) AS total_spent FROM orders GROUP BY customer_name ORDER BY total_spent DESC LIMIT 1;]\\n(Background on this error at: https://sqlalche.me/e/20/e3q8)'),\n",
- " (AgentActionMessageLog(tool='sql_db_list_tables', tool_input={'tool_input': ''}, log=\"\\nI received an error message. I will now use the sql_db_list_tables tool to find out which tables are available.\\n{'tool_name': 'sql_db_list_tables', 'parameters': {'tool_input': ''}}\\n\", message_log=[AIMessage(content='\\nReflection: I received an error message. I will now use the sql_db_list_tables tool to find out which tables are available.\\nAction: ```json\\n[\\n {\\n \"tool_name\": \"sql_db_list_tables\",\\n \"parameters\": {\\n \"tool_input\": \"\"\\n }\\n }\\n]\\n```')]),\n",
- " 'Album, Artist, Customer, Employee, Genre, Invoice, InvoiceLine, MediaType, Playlist, PlaylistTrack, Track'),\n",
- " (AgentActionMessageLog(tool='sql_db_schema', tool_input={'table_names': 'Customer,Invoice'}, log=\"\\nI found that there is a 'Customer' table and an 'Invoice' table. I will now use the sql_db_schema tool to find out more about these tables.\\n{'tool_name': 'sql_db_schema', 'parameters': {'table_names': 'Customer,Invoice'}}\\n\", message_log=[AIMessage(content='\\nReflection: I found that there is a \\'Customer\\' table and an \\'Invoice\\' table. I will now use the sql_db_schema tool to find out more about these tables.\\nAction: ```json\\n[\\n {\\n \"tool_name\": \"sql_db_schema\",\\n \"parameters\": {\\n \"table_names\": \"Customer,Invoice\"\\n }\\n }\\n]\\n```')]),\n",
- " '\\nCREATE TABLE \"Customer\" (\\n\\t\"CustomerId\" INTEGER NOT NULL, \\n\\t\"FirstName\" NVARCHAR(40) NOT NULL, \\n\\t\"LastName\" NVARCHAR(20) NOT NULL, \\n\\t\"Company\" NVARCHAR(80), \\n\\t\"Address\" NVARCHAR(70), \\n\\t\"City\" NVARCHAR(40), \\n\\t\"State\" NVARCHAR(40), \\n\\t\"Country\" NVARCHAR(40), \\n\\t\"PostalCode\" NVARCHAR(10), \\n\\t\"Phone\" NVARCHAR(24), \\n\\t\"Fax\" NVARCHAR(24), \\n\\t\"Email\" NVARCHAR(60) NOT NULL, \\n\\t\"SupportRepId\" INTEGER, \\n\\tPRIMARY KEY (\"CustomerId\"), \\n\\tFOREIGN KEY(\"SupportRepId\") REFERENCES \"Employee\" (\"EmployeeId\")\\n)\\n\\n/*\\n3 rows from Customer table:\\nCustomerId\\tFirstName\\tLastName\\tCompany\\tAddress\\tCity\\tState\\tCountry\\tPostalCode\\tPhone\\tFax\\tEmail\\tSupportRepId\\n1\\tLuís\\tGonçalves\\tEmbraer - Empresa Brasileira de Aeronáutica S.A.\\tAv. Brigadeiro Faria Lima, 2170\\tSão José dos Campos\\tSP\\tBrazil\\t12227-000\\t+55 (12) 3923-5555\\t+55 (12) 3923-5566\\tluisg@embraer.com.br\\t3\\n2\\tLeonie\\tKöhler\\tNone\\tTheodor-Heuss-Straße 34\\tStuttgart\\tNone\\tGermany\\t70174\\t+49 0711 2842222\\tNone\\tleonekohler@surfeu.de\\t5\\n3\\tFrançois\\tTremblay\\tNone\\t1498 rue Bélanger\\tMontréal\\tQC\\tCanada\\tH2G 1A7\\t+1 (514) 721-4711\\tNone\\tftremblay@gmail.com\\t3\\n*/\\n\\n\\nCREATE TABLE \"Invoice\" (\\n\\t\"InvoiceId\" INTEGER NOT NULL, \\n\\t\"CustomerId\" INTEGER NOT NULL, \\n\\t\"InvoiceDate\" DATETIME NOT NULL, \\n\\t\"BillingAddress\" NVARCHAR(70), \\n\\t\"BillingCity\" NVARCHAR(40), \\n\\t\"BillingState\" NVARCHAR(40), \\n\\t\"BillingCountry\" NVARCHAR(40), \\n\\t\"BillingPostalCode\" NVARCHAR(10), \\n\\t\"Total\" NUMERIC(10, 2) NOT NULL, \\n\\tPRIMARY KEY (\"InvoiceId\"), \\n\\tFOREIGN KEY(\"CustomerId\") REFERENCES \"Customer\" (\"CustomerId\")\\n)\\n\\n/*\\n3 rows from Invoice table:\\nInvoiceId\\tCustomerId\\tInvoiceDate\\tBillingAddress\\tBillingCity\\tBillingState\\tBillingCountry\\tBillingPostalCode\\tTotal\\n1\\t2\\t2021-01-01 00:00:00\\tTheodor-Heuss-Straße 34\\tStuttgart\\tNone\\tGermany\\t70174\\t1.98\\n2\\t4\\t2021-01-02 00:00:00\\tUllevålsveien 14\\tOslo\\tNone\\tNorway\\t0171\\t3.96\\n3\\t8\\t2021-01-03 00:00:00\\tGrétrystraat 63\\tBrussels\\tNone\\tBelgium\\t1000\\t5.94\\n*/'),\n",
- " (AgentActionMessageLog(tool='sql_db_query', tool_input={'query': 'SELECT c.FirstName, c.LastName, SUM(i.Total) AS total_spent FROM Customer c JOIN Invoice i ON c.CustomerId = i.CustomerId GROUP BY c.CustomerId ORDER BY total_spent DESC LIMIT 1;'}, log=\"\\nI found that the 'Customer' table contains information about customers, including their names, addresses, and phone numbers. The 'Invoice' table contains information about invoices, including the customer ID, invoice date, and total amount. I will now write a new query to find the customer who has spent the most money.\\n{'tool_name': 'sql_db_query', 'parameters': {'query': 'SELECT c.FirstName, c.LastName, SUM(i.Total) AS total_spent FROM Customer c JOIN Invoice i ON c.CustomerId = i.CustomerId GROUP BY c.CustomerId ORDER BY total_spent DESC LIMIT 1;'}}\\n\", message_log=[AIMessage(content='\\nReflection: I found that the \\'Customer\\' table contains information about customers, including their names, addresses, and phone numbers. The \\'Invoice\\' table contains information about invoices, including the customer ID, invoice date, and total amount. I will now write a new query to find the customer who has spent the most money.\\nAction: ```json\\n[\\n {\\n \"tool_name\": \"sql_db_query\",\\n \"parameters\": {\\n \"query\": \"SELECT c.FirstName, c.LastName, SUM(i.Total) AS total_spent FROM Customer c JOIN Invoice i ON c.CustomerId = i.CustomerId GROUP BY c.CustomerId ORDER BY total_spent DESC LIMIT 1;\"\\n }\\n }\\n]\\n```')]),\n",
- " \"[('Helena', 'Holý', 49.62)]\")]}"
- ]
- },
- "execution_count": 18,
- "metadata": {},
- "output_type": "execute_result"
}
],
"source": [
- "agent_executor.invoke({\n",
+ "output=agent_executor.invoke({\n",
" \"input\": 'who is the best customer? The customer who has spent the most money is the best.',\n",
"})"
]
},
+ {
+ "cell_type": "code",
+ "execution_count": 30,
+ "id": "7274af42-6e58-4a2c-a43b-d1610c5d51e4",
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "The best customer is Helena Holý, who has spent a total of 49.62.\n"
+ ]
+ }
+ ],
+ "source": [
+ "print(output['output'])"
+ ]
+ },
{
"cell_type": "markdown",
"id": "14ca1ab1-1deb-4c25-bc2d-6f047bcb49d9",
"metadata": {},
"source": [
- "As you can see, the agent makes an error, but is able to rectify itself. It also manages to generate SQL query over two tables in the database"
+ "As you can see, the agent makes an error, but is able to rectify itself. It also manages to generate a SQL query over two tables in the database."
]
},
{
@@ -578,7 +557,7 @@
"id": "1c9ea4d9-8f92-4780-b33e-e348b2fa854f",
"metadata": {},
"source": [
- "\n",
+ "\n",
"# SQL Agent with context"
]
},
@@ -587,29 +566,249 @@
"id": "1883f0f6-c1b9-41d0-9049-68d2101f8a9e",
"metadata": {},
"source": [
- "From our experiments, we have found that passing in additional context to the preamble can help reduce the initial failures. This context is provided by the SQLDBToolkit and contains the first 3 rows of the tables in the Database"
+ "Generally, passing in additional context to the preamble can help reduce the initial failures. This context is provided by the SQLDBToolkit and contains the first 3 rows of the tables in the Database."
]
},
{
"cell_type": "code",
- "execution_count": 19,
+ "execution_count": 24,
"id": "fd781ff2-ee5f-4fef-b1b4-fde58562383c",
"metadata": {},
"outputs": [
{
- "data": {
- "text/plain": [
- "{'table_info': '\\nCREATE TABLE \"Album\" (\\n\\t\"AlbumId\" INTEGER NOT NULL, \\n\\t\"Title\" NVARCHAR(160) NOT NULL, \\n\\t\"ArtistId\" INTEGER NOT NULL, \\n\\tPRIMARY KEY (\"AlbumId\"), \\n\\tFOREIGN KEY(\"ArtistId\") REFERENCES \"Artist\" (\"ArtistId\")\\n)\\n\\n/*\\n3 rows from Album table:\\nAlbumId\\tTitle\\tArtistId\\n1\\tFor Those About To Rock We Salute You\\t1\\n2\\tBalls to the Wall\\t2\\n3\\tRestless and Wild\\t2\\n*/\\n\\n\\nCREATE TABLE \"Artist\" (\\n\\t\"ArtistId\" INTEGER NOT NULL, \\n\\t\"Name\" NVARCHAR(120), \\n\\tPRIMARY KEY (\"ArtistId\")\\n)\\n\\n/*\\n3 rows from Artist table:\\nArtistId\\tName\\n1\\tAC/DC\\n2\\tAccept\\n3\\tAerosmith\\n*/\\n\\n\\nCREATE TABLE \"Customer\" (\\n\\t\"CustomerId\" INTEGER NOT NULL, \\n\\t\"FirstName\" NVARCHAR(40) NOT NULL, \\n\\t\"LastName\" NVARCHAR(20) NOT NULL, \\n\\t\"Company\" NVARCHAR(80), \\n\\t\"Address\" NVARCHAR(70), \\n\\t\"City\" NVARCHAR(40), \\n\\t\"State\" NVARCHAR(40), \\n\\t\"Country\" NVARCHAR(40), \\n\\t\"PostalCode\" NVARCHAR(10), \\n\\t\"Phone\" NVARCHAR(24), \\n\\t\"Fax\" NVARCHAR(24), \\n\\t\"Email\" NVARCHAR(60) NOT NULL, \\n\\t\"SupportRepId\" INTEGER, \\n\\tPRIMARY KEY (\"CustomerId\"), \\n\\tFOREIGN KEY(\"SupportRepId\") REFERENCES \"Employee\" (\"EmployeeId\")\\n)\\n\\n/*\\n3 rows from Customer table:\\nCustomerId\\tFirstName\\tLastName\\tCompany\\tAddress\\tCity\\tState\\tCountry\\tPostalCode\\tPhone\\tFax\\tEmail\\tSupportRepId\\n1\\tLuís\\tGonçalves\\tEmbraer - Empresa Brasileira de Aeronáutica S.A.\\tAv. Brigadeiro Faria Lima, 2170\\tSão José dos Campos\\tSP\\tBrazil\\t12227-000\\t+55 (12) 3923-5555\\t+55 (12) 3923-5566\\tluisg@embraer.com.br\\t3\\n2\\tLeonie\\tKöhler\\tNone\\tTheodor-Heuss-Straße 34\\tStuttgart\\tNone\\tGermany\\t70174\\t+49 0711 2842222\\tNone\\tleonekohler@surfeu.de\\t5\\n3\\tFrançois\\tTremblay\\tNone\\t1498 rue Bélanger\\tMontréal\\tQC\\tCanada\\tH2G 1A7\\t+1 (514) 721-4711\\tNone\\tftremblay@gmail.com\\t3\\n*/\\n\\n\\nCREATE TABLE \"Employee\" (\\n\\t\"EmployeeId\" INTEGER NOT NULL, \\n\\t\"LastName\" NVARCHAR(20) NOT NULL, \\n\\t\"FirstName\" NVARCHAR(20) NOT NULL, \\n\\t\"Title\" NVARCHAR(30), \\n\\t\"ReportsTo\" INTEGER, \\n\\t\"BirthDate\" DATETIME, \\n\\t\"HireDate\" DATETIME, \\n\\t\"Address\" NVARCHAR(70), \\n\\t\"City\" NVARCHAR(40), \\n\\t\"State\" NVARCHAR(40), \\n\\t\"Country\" NVARCHAR(40), \\n\\t\"PostalCode\" NVARCHAR(10), \\n\\t\"Phone\" NVARCHAR(24), \\n\\t\"Fax\" NVARCHAR(24), \\n\\t\"Email\" NVARCHAR(60), \\n\\tPRIMARY KEY (\"EmployeeId\"), \\n\\tFOREIGN KEY(\"ReportsTo\") REFERENCES \"Employee\" (\"EmployeeId\")\\n)\\n\\n/*\\n3 rows from Employee table:\\nEmployeeId\\tLastName\\tFirstName\\tTitle\\tReportsTo\\tBirthDate\\tHireDate\\tAddress\\tCity\\tState\\tCountry\\tPostalCode\\tPhone\\tFax\\tEmail\\n1\\tAdams\\tAndrew\\tGeneral Manager\\tNone\\t1962-02-18 00:00:00\\t2002-08-14 00:00:00\\t11120 Jasper Ave NW\\tEdmonton\\tAB\\tCanada\\tT5K 2N1\\t+1 (780) 428-9482\\t+1 (780) 428-3457\\tandrew@chinookcorp.com\\n2\\tEdwards\\tNancy\\tSales Manager\\t1\\t1958-12-08 00:00:00\\t2002-05-01 00:00:00\\t825 8 Ave SW\\tCalgary\\tAB\\tCanada\\tT2P 2T3\\t+1 (403) 262-3443\\t+1 (403) 262-3322\\tnancy@chinookcorp.com\\n3\\tPeacock\\tJane\\tSales Support Agent\\t2\\t1973-08-29 00:00:00\\t2002-04-01 00:00:00\\t1111 6 Ave SW\\tCalgary\\tAB\\tCanada\\tT2P 5M5\\t+1 (403) 262-3443\\t+1 (403) 262-6712\\tjane@chinookcorp.com\\n*/\\n\\n\\nCREATE TABLE \"Genre\" (\\n\\t\"GenreId\" INTEGER NOT NULL, \\n\\t\"Name\" NVARCHAR(120), \\n\\tPRIMARY KEY (\"GenreId\")\\n)\\n\\n/*\\n3 rows from Genre table:\\nGenreId\\tName\\n1\\tRock\\n2\\tJazz\\n3\\tMetal\\n*/\\n\\n\\nCREATE TABLE \"Invoice\" (\\n\\t\"InvoiceId\" INTEGER NOT NULL, \\n\\t\"CustomerId\" INTEGER NOT NULL, \\n\\t\"InvoiceDate\" DATETIME NOT NULL, \\n\\t\"BillingAddress\" NVARCHAR(70), \\n\\t\"BillingCity\" NVARCHAR(40), \\n\\t\"BillingState\" NVARCHAR(40), \\n\\t\"BillingCountry\" NVARCHAR(40), \\n\\t\"BillingPostalCode\" NVARCHAR(10), \\n\\t\"Total\" NUMERIC(10, 2) NOT NULL, \\n\\tPRIMARY KEY (\"InvoiceId\"), \\n\\tFOREIGN KEY(\"CustomerId\") REFERENCES \"Customer\" (\"CustomerId\")\\n)\\n\\n/*\\n3 rows from Invoice table:\\nInvoiceId\\tCustomerId\\tInvoiceDate\\tBillingAddress\\tBillingCity\\tBillingState\\tBillingCountry\\tBillingPostalCode\\tTotal\\n1\\t2\\t2021-01-01 00:00:00\\tTheodor-Heuss-Straße 34\\tStuttgart\\tNone\\tGermany\\t70174\\t1.98\\n2\\t4\\t2021-01-02 00:00:00\\tUllevålsveien 14\\tOslo\\tNone\\tNorway\\t0171\\t3.96\\n3\\t8\\t2021-01-03 00:00:00\\tGrétrystraat 63\\tBrussels\\tNone\\tBelgium\\t1000\\t5.94\\n*/\\n\\n\\nCREATE TABLE \"InvoiceLine\" (\\n\\t\"InvoiceLineId\" INTEGER NOT NULL, \\n\\t\"InvoiceId\" INTEGER NOT NULL, \\n\\t\"TrackId\" INTEGER NOT NULL, \\n\\t\"UnitPrice\" NUMERIC(10, 2) NOT NULL, \\n\\t\"Quantity\" INTEGER NOT NULL, \\n\\tPRIMARY KEY (\"InvoiceLineId\"), \\n\\tFOREIGN KEY(\"TrackId\") REFERENCES \"Track\" (\"TrackId\"), \\n\\tFOREIGN KEY(\"InvoiceId\") REFERENCES \"Invoice\" (\"InvoiceId\")\\n)\\n\\n/*\\n3 rows from InvoiceLine table:\\nInvoiceLineId\\tInvoiceId\\tTrackId\\tUnitPrice\\tQuantity\\n1\\t1\\t2\\t0.99\\t1\\n2\\t1\\t4\\t0.99\\t1\\n3\\t2\\t6\\t0.99\\t1\\n*/\\n\\n\\nCREATE TABLE \"MediaType\" (\\n\\t\"MediaTypeId\" INTEGER NOT NULL, \\n\\t\"Name\" NVARCHAR(120), \\n\\tPRIMARY KEY (\"MediaTypeId\")\\n)\\n\\n/*\\n3 rows from MediaType table:\\nMediaTypeId\\tName\\n1\\tMPEG audio file\\n2\\tProtected AAC audio file\\n3\\tProtected MPEG-4 video file\\n*/\\n\\n\\nCREATE TABLE \"Playlist\" (\\n\\t\"PlaylistId\" INTEGER NOT NULL, \\n\\t\"Name\" NVARCHAR(120), \\n\\tPRIMARY KEY (\"PlaylistId\")\\n)\\n\\n/*\\n3 rows from Playlist table:\\nPlaylistId\\tName\\n1\\tMusic\\n2\\tMovies\\n3\\tTV Shows\\n*/\\n\\n\\nCREATE TABLE \"PlaylistTrack\" (\\n\\t\"PlaylistId\" INTEGER NOT NULL, \\n\\t\"TrackId\" INTEGER NOT NULL, \\n\\tPRIMARY KEY (\"PlaylistId\", \"TrackId\"), \\n\\tFOREIGN KEY(\"TrackId\") REFERENCES \"Track\" (\"TrackId\"), \\n\\tFOREIGN KEY(\"PlaylistId\") REFERENCES \"Playlist\" (\"PlaylistId\")\\n)\\n\\n/*\\n3 rows from PlaylistTrack table:\\nPlaylistId\\tTrackId\\n1\\t3402\\n1\\t3389\\n1\\t3390\\n*/\\n\\n\\nCREATE TABLE \"Track\" (\\n\\t\"TrackId\" INTEGER NOT NULL, \\n\\t\"Name\" NVARCHAR(200) NOT NULL, \\n\\t\"AlbumId\" INTEGER, \\n\\t\"MediaTypeId\" INTEGER NOT NULL, \\n\\t\"GenreId\" INTEGER, \\n\\t\"Composer\" NVARCHAR(220), \\n\\t\"Milliseconds\" INTEGER NOT NULL, \\n\\t\"Bytes\" INTEGER, \\n\\t\"UnitPrice\" NUMERIC(10, 2) NOT NULL, \\n\\tPRIMARY KEY (\"TrackId\"), \\n\\tFOREIGN KEY(\"MediaTypeId\") REFERENCES \"MediaType\" (\"MediaTypeId\"), \\n\\tFOREIGN KEY(\"GenreId\") REFERENCES \"Genre\" (\"GenreId\"), \\n\\tFOREIGN KEY(\"AlbumId\") REFERENCES \"Album\" (\"AlbumId\")\\n)\\n\\n/*\\n3 rows from Track table:\\nTrackId\\tName\\tAlbumId\\tMediaTypeId\\tGenreId\\tComposer\\tMilliseconds\\tBytes\\tUnitPrice\\n1\\tFor Those About To Rock (We Salute You)\\t1\\t1\\t1\\tAngus Young, Malcolm Young, Brian Johnson\\t343719\\t11170334\\t0.99\\n2\\tBalls to the Wall\\t2\\t2\\t1\\tU. Dirkschneider, W. Hoffmann, H. Frank, P. Baltes, S. Kaufmann, G. Hoffmann\\t342562\\t5510424\\t0.99\\n3\\tFast As a Shark\\t3\\t2\\t1\\tF. Baltes, S. Kaufman, U. Dirkscneider & W. Hoffman\\t230619\\t3990994\\t0.99\\n*/',\n",
- " 'table_names': 'Album, Artist, Customer, Employee, Genre, Invoice, InvoiceLine, MediaType, Playlist, PlaylistTrack, Track'}"
- ]
- },
- "execution_count": 19,
- "metadata": {},
- "output_type": "execute_result"
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "**Context to pass to LLM on tables**\n",
+ "Table Names\n",
+ "Album, Artist, Customer, Employee, Genre, Invoice, InvoiceLine, MediaType, Playlist, PlaylistTrack, Track\n",
+ "Table Schemas\n",
+ "\n",
+ "CREATE TABLE \"Album\" (\n",
+ "\t\"AlbumId\" INTEGER NOT NULL, \n",
+ "\t\"Title\" NVARCHAR(160) NOT NULL, \n",
+ "\t\"ArtistId\" INTEGER NOT NULL, \n",
+ "\tPRIMARY KEY (\"AlbumId\"), \n",
+ "\tFOREIGN KEY(\"ArtistId\") REFERENCES \"Artist\" (\"ArtistId\")\n",
+ ")\n",
+ "\n",
+ "/*\n",
+ "3 rows from Album table:\n",
+ "AlbumId\tTitle\tArtistId\n",
+ "1\tFor Those About To Rock We Salute You\t1\n",
+ "2\tBalls to the Wall\t2\n",
+ "3\tRestless and Wild\t2\n",
+ "*/\n",
+ "\n",
+ "\n",
+ "CREATE TABLE \"Artist\" (\n",
+ "\t\"ArtistId\" INTEGER NOT NULL, \n",
+ "\t\"Name\" NVARCHAR(120), \n",
+ "\tPRIMARY KEY (\"ArtistId\")\n",
+ ")\n",
+ "\n",
+ "/*\n",
+ "3 rows from Artist table:\n",
+ "ArtistId\tName\n",
+ "1\tAC/DC\n",
+ "2\tAccept\n",
+ "3\tAerosmith\n",
+ "*/\n",
+ "\n",
+ "\n",
+ "CREATE TABLE \"Customer\" (\n",
+ "\t\"CustomerId\" INTEGER NOT NULL, \n",
+ "\t\"FirstName\" NVARCHAR(40) NOT NULL, \n",
+ "\t\"LastName\" NVARCHAR(20) NOT NULL, \n",
+ "\t\"Company\" NVARCHAR(80), \n",
+ "\t\"Address\" NVARCHAR(70), \n",
+ "\t\"City\" NVARCHAR(40), \n",
+ "\t\"State\" NVARCHAR(40), \n",
+ "\t\"Country\" NVARCHAR(40), \n",
+ "\t\"PostalCode\" NVARCHAR(10), \n",
+ "\t\"Phone\" NVARCHAR(24), \n",
+ "\t\"Fax\" NVARCHAR(24), \n",
+ "\t\"Email\" NVARCHAR(60) NOT NULL, \n",
+ "\t\"SupportRepId\" INTEGER, \n",
+ "\tPRIMARY KEY (\"CustomerId\"), \n",
+ "\tFOREIGN KEY(\"SupportRepId\") REFERENCES \"Employee\" (\"EmployeeId\")\n",
+ ")\n",
+ "\n",
+ "/*\n",
+ "3 rows from Customer table:\n",
+ "CustomerId\tFirstName\tLastName\tCompany\tAddress\tCity\tState\tCountry\tPostalCode\tPhone\tFax\tEmail\tSupportRepId\n",
+ "1\tLuís\tGonçalves\tEmbraer - Empresa Brasileira de Aeronáutica S.A.\tAv. Brigadeiro Faria Lima, 2170\tSão José dos Campos\tSP\tBrazil\t12227-000\t+55 (12) 3923-5555\t+55 (12) 3923-5566\tluisg@embraer.com.br\t3\n",
+ "2\tLeonie\tKöhler\tNone\tTheodor-Heuss-Straße 34\tStuttgart\tNone\tGermany\t70174\t+49 0711 2842222\tNone\tleonekohler@surfeu.de\t5\n",
+ "3\tFrançois\tTremblay\tNone\t1498 rue Bélanger\tMontréal\tQC\tCanada\tH2G 1A7\t+1 (514) 721-4711\tNone\tftremblay@gmail.com\t3\n",
+ "*/\n",
+ "\n",
+ "\n",
+ "CREATE TABLE \"Employee\" (\n",
+ "\t\"EmployeeId\" INTEGER NOT NULL, \n",
+ "\t\"LastName\" NVARCHAR(20) NOT NULL, \n",
+ "\t\"FirstName\" NVARCHAR(20) NOT NULL, \n",
+ "\t\"Title\" NVARCHAR(30), \n",
+ "\t\"ReportsTo\" INTEGER, \n",
+ "\t\"BirthDate\" DATETIME, \n",
+ "\t\"HireDate\" DATETIME, \n",
+ "\t\"Address\" NVARCHAR(70), \n",
+ "\t\"City\" NVARCHAR(40), \n",
+ "\t\"State\" NVARCHAR(40), \n",
+ "\t\"Country\" NVARCHAR(40), \n",
+ "\t\"PostalCode\" NVARCHAR(10), \n",
+ "\t\"Phone\" NVARCHAR(24), \n",
+ "\t\"Fax\" NVARCHAR(24), \n",
+ "\t\"Email\" NVARCHAR(60), \n",
+ "\tPRIMARY KEY (\"EmployeeId\"), \n",
+ "\tFOREIGN KEY(\"ReportsTo\") REFERENCES \"Employee\" (\"EmployeeId\")\n",
+ ")\n",
+ "\n",
+ "/*\n",
+ "3 rows from Employee table:\n",
+ "EmployeeId\tLastName\tFirstName\tTitle\tReportsTo\tBirthDate\tHireDate\tAddress\tCity\tState\tCountry\tPostalCode\tPhone\tFax\tEmail\n",
+ "1\tAdams\tAndrew\tGeneral Manager\tNone\t1962-02-18 00:00:00\t2002-08-14 00:00:00\t11120 Jasper Ave NW\tEdmonton\tAB\tCanada\tT5K 2N1\t+1 (780) 428-9482\t+1 (780) 428-3457\tandrew@chinookcorp.com\n",
+ "2\tEdwards\tNancy\tSales Manager\t1\t1958-12-08 00:00:00\t2002-05-01 00:00:00\t825 8 Ave SW\tCalgary\tAB\tCanada\tT2P 2T3\t+1 (403) 262-3443\t+1 (403) 262-3322\tnancy@chinookcorp.com\n",
+ "3\tPeacock\tJane\tSales Support Agent\t2\t1973-08-29 00:00:00\t2002-04-01 00:00:00\t1111 6 Ave SW\tCalgary\tAB\tCanada\tT2P 5M5\t+1 (403) 262-3443\t+1 (403) 262-6712\tjane@chinookcorp.com\n",
+ "*/\n",
+ "\n",
+ "\n",
+ "CREATE TABLE \"Genre\" (\n",
+ "\t\"GenreId\" INTEGER NOT NULL, \n",
+ "\t\"Name\" NVARCHAR(120), \n",
+ "\tPRIMARY KEY (\"GenreId\")\n",
+ ")\n",
+ "\n",
+ "/*\n",
+ "3 rows from Genre table:\n",
+ "GenreId\tName\n",
+ "1\tRock\n",
+ "2\tJazz\n",
+ "3\tMetal\n",
+ "*/\n",
+ "\n",
+ "\n",
+ "CREATE TABLE \"Invoice\" (\n",
+ "\t\"InvoiceId\" INTEGER NOT NULL, \n",
+ "\t\"CustomerId\" INTEGER NOT NULL, \n",
+ "\t\"InvoiceDate\" DATETIME NOT NULL, \n",
+ "\t\"BillingAddress\" NVARCHAR(70), \n",
+ "\t\"BillingCity\" NVARCHAR(40), \n",
+ "\t\"BillingState\" NVARCHAR(40), \n",
+ "\t\"BillingCountry\" NVARCHAR(40), \n",
+ "\t\"BillingPostalCode\" NVARCHAR(10), \n",
+ "\t\"Total\" NUMERIC(10, 2) NOT NULL, \n",
+ "\tPRIMARY KEY (\"InvoiceId\"), \n",
+ "\tFOREIGN KEY(\"CustomerId\") REFERENCES \"Customer\" (\"CustomerId\")\n",
+ ")\n",
+ "\n",
+ "/*\n",
+ "3 rows from Invoice table:\n",
+ "InvoiceId\tCustomerId\tInvoiceDate\tBillingAddress\tBillingCity\tBillingState\tBillingCountry\tBillingPostalCode\tTotal\n",
+ "1\t2\t2021-01-01 00:00:00\tTheodor-Heuss-Straße 34\tStuttgart\tNone\tGermany\t70174\t1.98\n",
+ "2\t4\t2021-01-02 00:00:00\tUllevålsveien 14\tOslo\tNone\tNorway\t0171\t3.96\n",
+ "3\t8\t2021-01-03 00:00:00\tGrétrystraat 63\tBrussels\tNone\tBelgium\t1000\t5.94\n",
+ "*/\n",
+ "\n",
+ "\n",
+ "CREATE TABLE \"InvoiceLine\" (\n",
+ "\t\"InvoiceLineId\" INTEGER NOT NULL, \n",
+ "\t\"InvoiceId\" INTEGER NOT NULL, \n",
+ "\t\"TrackId\" INTEGER NOT NULL, \n",
+ "\t\"UnitPrice\" NUMERIC(10, 2) NOT NULL, \n",
+ "\t\"Quantity\" INTEGER NOT NULL, \n",
+ "\tPRIMARY KEY (\"InvoiceLineId\"), \n",
+ "\tFOREIGN KEY(\"TrackId\") REFERENCES \"Track\" (\"TrackId\"), \n",
+ "\tFOREIGN KEY(\"InvoiceId\") REFERENCES \"Invoice\" (\"InvoiceId\")\n",
+ ")\n",
+ "\n",
+ "/*\n",
+ "3 rows from InvoiceLine table:\n",
+ "InvoiceLineId\tInvoiceId\tTrackId\tUnitPrice\tQuantity\n",
+ "1\t1\t2\t0.99\t1\n",
+ "2\t1\t4\t0.99\t1\n",
+ "3\t2\t6\t0.99\t1\n",
+ "*/\n",
+ "\n",
+ "\n",
+ "CREATE TABLE \"MediaType\" (\n",
+ "\t\"MediaTypeId\" INTEGER NOT NULL, \n",
+ "\t\"Name\" NVARCHAR(120), \n",
+ "\tPRIMARY KEY (\"MediaTypeId\")\n",
+ ")\n",
+ "\n",
+ "/*\n",
+ "3 rows from MediaType table:\n",
+ "MediaTypeId\tName\n",
+ "1\tMPEG audio file\n",
+ "2\tProtected AAC audio file\n",
+ "3\tProtected MPEG-4 video file\n",
+ "*/\n",
+ "\n",
+ "\n",
+ "CREATE TABLE \"Playlist\" (\n",
+ "\t\"PlaylistId\" INTEGER NOT NULL, \n",
+ "\t\"Name\" NVARCHAR(120), \n",
+ "\tPRIMARY KEY (\"PlaylistId\")\n",
+ ")\n",
+ "\n",
+ "/*\n",
+ "3 rows from Playlist table:\n",
+ "PlaylistId\tName\n",
+ "1\tMusic\n",
+ "2\tMovies\n",
+ "3\tTV Shows\n",
+ "*/\n",
+ "\n",
+ "\n",
+ "CREATE TABLE \"PlaylistTrack\" (\n",
+ "\t\"PlaylistId\" INTEGER NOT NULL, \n",
+ "\t\"TrackId\" INTEGER NOT NULL, \n",
+ "\tPRIMARY KEY (\"PlaylistId\", \"TrackId\"), \n",
+ "\tFOREIGN KEY(\"TrackId\") REFERENCES \"Track\" (\"TrackId\"), \n",
+ "\tFOREIGN KEY(\"PlaylistId\") REFERENCES \"Playlist\" (\"PlaylistId\")\n",
+ ")\n",
+ "\n",
+ "/*\n",
+ "3 rows from PlaylistTrack table:\n",
+ "PlaylistId\tTrackId\n",
+ "1\t3402\n",
+ "1\t3389\n",
+ "1\t3390\n",
+ "*/\n",
+ "\n",
+ "\n",
+ "CREATE TABLE \"Track\" (\n",
+ "\t\"TrackId\" INTEGER NOT NULL, \n",
+ "\t\"Name\" NVARCHAR(200) NOT NULL, \n",
+ "\t\"AlbumId\" INTEGER, \n",
+ "\t\"MediaTypeId\" INTEGER NOT NULL, \n",
+ "\t\"GenreId\" INTEGER, \n",
+ "\t\"Composer\" NVARCHAR(220), \n",
+ "\t\"Milliseconds\" INTEGER NOT NULL, \n",
+ "\t\"Bytes\" INTEGER, \n",
+ "\t\"UnitPrice\" NUMERIC(10, 2) NOT NULL, \n",
+ "\tPRIMARY KEY (\"TrackId\"), \n",
+ "\tFOREIGN KEY(\"MediaTypeId\") REFERENCES \"MediaType\" (\"MediaTypeId\"), \n",
+ "\tFOREIGN KEY(\"GenreId\") REFERENCES \"Genre\" (\"GenreId\"), \n",
+ "\tFOREIGN KEY(\"AlbumId\") REFERENCES \"Album\" (\"AlbumId\")\n",
+ ")\n",
+ "\n",
+ "/*\n",
+ "3 rows from Track table:\n",
+ "TrackId\tName\tAlbumId\tMediaTypeId\tGenreId\tComposer\tMilliseconds\tBytes\tUnitPrice\n",
+ "1\tFor Those About To Rock (We Salute You)\t1\t1\t1\tAngus Young, Malcolm Young, Brian Johnson\t343719\t11170334\t0.99\n",
+ "2\tBalls to the Wall\t2\t2\t1\tU. Dirkschneider, W. Hoffmann, H. Frank, P. Baltes, S. Kaufmann, G. Hoffmann\t342562\t5510424\t0.99\n",
+ "3\tFast As a Shark\t3\t2\t1\tF. Baltes, S. Kaufman, U. Dirkscneider & W. Hoffman\t230619\t3990994\t0.99\n",
+ "*/\n"
+ ]
}
],
"source": [
- "context"
+ "print('**Context to pass to LLM on tables**')\n",
+ "print('Table Names')\n",
+ "print(context['table_names'])\n",
+ "print('Table Schemas')\n",
+ "print(context['table_info'])"
]
},
{
@@ -622,7 +821,7 @@
},
{
"cell_type": "code",
- "execution_count": 21,
+ "execution_count": 25,
"id": "06913698-60ee-4b10-b28b-6462c565ad63",
"metadata": {},
"outputs": [],
@@ -644,7 +843,7 @@
},
{
"cell_type": "code",
- "execution_count": 23,
+ "execution_count": 26,
"id": "ad461d18-6ae9-40b5-8c85-a48ab9aa6bc0",
"metadata": {},
"outputs": [
@@ -654,56 +853,51 @@
"text": [
"\n",
"\n",
- "\u001B[1m> Entering new AgentExecutor chain...\u001B[0m\n",
- "\u001B[32;1m\u001B[1;3m\n",
+ "\u001b[1m> Entering new AgentExecutor chain...\u001b[0m\n",
+ "\u001b[32;1m\u001b[1;3m\n",
"I will write a SQL query to find the customer who has spent the most money.\n",
- "{'tool_name': 'sql_db_query', 'parameters': {'query': 'SELECT c.CustomerId, c.FirstName, c.LastName, SUM(i.Total) AS TotalSpent\\nFROM Customer c\\nJOIN Invoice i ON c.CustomerId = i.CustomerId\\nGROUP BY c.CustomerId\\nORDER BY TotalSpent DESC\\nLIMIT 1;'}}\n",
- "\u001B[0m\u001B[36;1m\u001B[1;3m[(6, 'Helena', 'Holý', 49.62)]\u001B[0m\u001B[32;1m\u001B[1;3mRelevant Documents: 0\n",
+ "{'tool_name': 'sql_db_query', 'parameters': {'query': 'SELECT c.FirstName, c.LastName, SUM(i.Total) AS TotalSpent FROM Customer c JOIN Invoice i ON c.CustomerId = i.CustomerId GROUP BY c.CustomerId ORDER BY TotalSpent DESC LIMIT 1;'}}\n",
+ "\u001b[0m\u001b[36;1m\u001b[1;3m[('Helena', 'Holý', 49.62)]\u001b[0m\u001b[32;1m\u001b[1;3mRelevant Documents: 0\n",
"Cited Documents: 0\n",
"Answer: The customer who has spent the most money is Helena Holý.\n",
- "Grounded answer: The customer who has spent the most money is Helena Holý.\u001B[0m\n",
+ "Grounded answer: The customer who has spent the most money is Helena Holý.\u001b[0m\n",
"\n",
- "\u001B[1m> Finished chain.\u001B[0m\n"
+ "\u001b[1m> Finished chain.\u001b[0m\n"
]
- },
- {
- "data": {
- "text/plain": [
- "{'input': 'provide the name of the best customer? The customer who has spent the most money is the best.',\n",
- " 'preamble': '## Task And Context\\nYou use your advanced complex reasoning capabilities to help people by answering their questions and other requests interactively. You will be asked a very wide array of requests on all kinds of topics. You will be equipped with a wide range of search engines or similar tools to help you, which you use to research your answer. You may need to use multiple tools in parallel or sequentially to complete your task. You should focus on serving the user\\'s needs as best you can, which will be wide-ranging.\\n\\n## Style Guide\\nUnless the user asks for a different style of answer, you should answer in full sentences, using proper grammar and spelling.\\n\\n## Additional Information\\nYou are an expert who answers the user\\'s question by creating SQL queries and executing them.\\nYou are equipped with a number of relevant SQL tools.\\n\\nHere is information about the database:\\n{\\'table_info\\': \\'\\\\nCREATE TABLE \"Album\" (\\\\n\\\\t\"AlbumId\" INTEGER NOT NULL, \\\\n\\\\t\"Title\" NVARCHAR(160) NOT NULL, \\\\n\\\\t\"ArtistId\" INTEGER NOT NULL, \\\\n\\\\tPRIMARY KEY (\"AlbumId\"), \\\\n\\\\tFOREIGN KEY(\"ArtistId\") REFERENCES \"Artist\" (\"ArtistId\")\\\\n)\\\\n\\\\n/*\\\\n3 rows from Album table:\\\\nAlbumId\\\\tTitle\\\\tArtistId\\\\n1\\\\tFor Those About To Rock We Salute You\\\\t1\\\\n2\\\\tBalls to the Wall\\\\t2\\\\n3\\\\tRestless and Wild\\\\t2\\\\n*/\\\\n\\\\n\\\\nCREATE TABLE \"Artist\" (\\\\n\\\\t\"ArtistId\" INTEGER NOT NULL, \\\\n\\\\t\"Name\" NVARCHAR(120), \\\\n\\\\tPRIMARY KEY (\"ArtistId\")\\\\n)\\\\n\\\\n/*\\\\n3 rows from Artist table:\\\\nArtistId\\\\tName\\\\n1\\\\tAC/DC\\\\n2\\\\tAccept\\\\n3\\\\tAerosmith\\\\n*/\\\\n\\\\n\\\\nCREATE TABLE \"Customer\" (\\\\n\\\\t\"CustomerId\" INTEGER NOT NULL, \\\\n\\\\t\"FirstName\" NVARCHAR(40) NOT NULL, \\\\n\\\\t\"LastName\" NVARCHAR(20) NOT NULL, \\\\n\\\\t\"Company\" NVARCHAR(80), \\\\n\\\\t\"Address\" NVARCHAR(70), \\\\n\\\\t\"City\" NVARCHAR(40), \\\\n\\\\t\"State\" NVARCHAR(40), \\\\n\\\\t\"Country\" NVARCHAR(40), \\\\n\\\\t\"PostalCode\" NVARCHAR(10), \\\\n\\\\t\"Phone\" NVARCHAR(24), \\\\n\\\\t\"Fax\" NVARCHAR(24), \\\\n\\\\t\"Email\" NVARCHAR(60) NOT NULL, \\\\n\\\\t\"SupportRepId\" INTEGER, \\\\n\\\\tPRIMARY KEY (\"CustomerId\"), \\\\n\\\\tFOREIGN KEY(\"SupportRepId\") REFERENCES \"Employee\" (\"EmployeeId\")\\\\n)\\\\n\\\\n/*\\\\n3 rows from Customer table:\\\\nCustomerId\\\\tFirstName\\\\tLastName\\\\tCompany\\\\tAddress\\\\tCity\\\\tState\\\\tCountry\\\\tPostalCode\\\\tPhone\\\\tFax\\\\tEmail\\\\tSupportRepId\\\\n1\\\\tLuís\\\\tGonçalves\\\\tEmbraer - Empresa Brasileira de Aeronáutica S.A.\\\\tAv. Brigadeiro Faria Lima, 2170\\\\tSão José dos Campos\\\\tSP\\\\tBrazil\\\\t12227-000\\\\t+55 (12) 3923-5555\\\\t+55 (12) 3923-5566\\\\tluisg@embraer.com.br\\\\t3\\\\n2\\\\tLeonie\\\\tKöhler\\\\tNone\\\\tTheodor-Heuss-Straße 34\\\\tStuttgart\\\\tNone\\\\tGermany\\\\t70174\\\\t+49 0711 2842222\\\\tNone\\\\tleonekohler@surfeu.de\\\\t5\\\\n3\\\\tFrançois\\\\tTremblay\\\\tNone\\\\t1498 rue Bélanger\\\\tMontréal\\\\tQC\\\\tCanada\\\\tH2G 1A7\\\\t+1 (514) 721-4711\\\\tNone\\\\tftremblay@gmail.com\\\\t3\\\\n*/\\\\n\\\\n\\\\nCREATE TABLE \"Employee\" (\\\\n\\\\t\"EmployeeId\" INTEGER NOT NULL, \\\\n\\\\t\"LastName\" NVARCHAR(20) NOT NULL, \\\\n\\\\t\"FirstName\" NVARCHAR(20) NOT NULL, \\\\n\\\\t\"Title\" NVARCHAR(30), \\\\n\\\\t\"ReportsTo\" INTEGER, \\\\n\\\\t\"BirthDate\" DATETIME, \\\\n\\\\t\"HireDate\" DATETIME, \\\\n\\\\t\"Address\" NVARCHAR(70), \\\\n\\\\t\"City\" NVARCHAR(40), \\\\n\\\\t\"State\" NVARCHAR(40), \\\\n\\\\t\"Country\" NVARCHAR(40), \\\\n\\\\t\"PostalCode\" NVARCHAR(10), \\\\n\\\\t\"Phone\" NVARCHAR(24), \\\\n\\\\t\"Fax\" NVARCHAR(24), \\\\n\\\\t\"Email\" NVARCHAR(60), \\\\n\\\\tPRIMARY KEY (\"EmployeeId\"), \\\\n\\\\tFOREIGN KEY(\"ReportsTo\") REFERENCES \"Employee\" (\"EmployeeId\")\\\\n)\\\\n\\\\n/*\\\\n3 rows from Employee table:\\\\nEmployeeId\\\\tLastName\\\\tFirstName\\\\tTitle\\\\tReportsTo\\\\tBirthDate\\\\tHireDate\\\\tAddress\\\\tCity\\\\tState\\\\tCountry\\\\tPostalCode\\\\tPhone\\\\tFax\\\\tEmail\\\\n1\\\\tAdams\\\\tAndrew\\\\tGeneral Manager\\\\tNone\\\\t1962-02-18 00:00:00\\\\t2002-08-14 00:00:00\\\\t11120 Jasper Ave NW\\\\tEdmonton\\\\tAB\\\\tCanada\\\\tT5K 2N1\\\\t+1 (780) 428-9482\\\\t+1 (780) 428-3457\\\\tandrew@chinookcorp.com\\\\n2\\\\tEdwards\\\\tNancy\\\\tSales Manager\\\\t1\\\\t1958-12-08 00:00:00\\\\t2002-05-01 00:00:00\\\\t825 8 Ave SW\\\\tCalgary\\\\tAB\\\\tCanada\\\\tT2P 2T3\\\\t+1 (403) 262-3443\\\\t+1 (403) 262-3322\\\\tnancy@chinookcorp.com\\\\n3\\\\tPeacock\\\\tJane\\\\tSales Support Agent\\\\t2\\\\t1973-08-29 00:00:00\\\\t2002-04-01 00:00:00\\\\t1111 6 Ave SW\\\\tCalgary\\\\tAB\\\\tCanada\\\\tT2P 5M5\\\\t+1 (403) 262-3443\\\\t+1 (403) 262-6712\\\\tjane@chinookcorp.com\\\\n*/\\\\n\\\\n\\\\nCREATE TABLE \"Genre\" (\\\\n\\\\t\"GenreId\" INTEGER NOT NULL, \\\\n\\\\t\"Name\" NVARCHAR(120), \\\\n\\\\tPRIMARY KEY (\"GenreId\")\\\\n)\\\\n\\\\n/*\\\\n3 rows from Genre table:\\\\nGenreId\\\\tName\\\\n1\\\\tRock\\\\n2\\\\tJazz\\\\n3\\\\tMetal\\\\n*/\\\\n\\\\n\\\\nCREATE TABLE \"Invoice\" (\\\\n\\\\t\"InvoiceId\" INTEGER NOT NULL, \\\\n\\\\t\"CustomerId\" INTEGER NOT NULL, \\\\n\\\\t\"InvoiceDate\" DATETIME NOT NULL, \\\\n\\\\t\"BillingAddress\" NVARCHAR(70), \\\\n\\\\t\"BillingCity\" NVARCHAR(40), \\\\n\\\\t\"BillingState\" NVARCHAR(40), \\\\n\\\\t\"BillingCountry\" NVARCHAR(40), \\\\n\\\\t\"BillingPostalCode\" NVARCHAR(10), \\\\n\\\\t\"Total\" NUMERIC(10, 2) NOT NULL, \\\\n\\\\tPRIMARY KEY (\"InvoiceId\"), \\\\n\\\\tFOREIGN KEY(\"CustomerId\") REFERENCES \"Customer\" (\"CustomerId\")\\\\n)\\\\n\\\\n/*\\\\n3 rows from Invoice table:\\\\nInvoiceId\\\\tCustomerId\\\\tInvoiceDate\\\\tBillingAddress\\\\tBillingCity\\\\tBillingState\\\\tBillingCountry\\\\tBillingPostalCode\\\\tTotal\\\\n1\\\\t2\\\\t2021-01-01 00:00:00\\\\tTheodor-Heuss-Straße 34\\\\tStuttgart\\\\tNone\\\\tGermany\\\\t70174\\\\t1.98\\\\n2\\\\t4\\\\t2021-01-02 00:00:00\\\\tUllevålsveien 14\\\\tOslo\\\\tNone\\\\tNorway\\\\t0171\\\\t3.96\\\\n3\\\\t8\\\\t2021-01-03 00:00:00\\\\tGrétrystraat 63\\\\tBrussels\\\\tNone\\\\tBelgium\\\\t1000\\\\t5.94\\\\n*/\\\\n\\\\n\\\\nCREATE TABLE \"InvoiceLine\" (\\\\n\\\\t\"InvoiceLineId\" INTEGER NOT NULL, \\\\n\\\\t\"InvoiceId\" INTEGER NOT NULL, \\\\n\\\\t\"TrackId\" INTEGER NOT NULL, \\\\n\\\\t\"UnitPrice\" NUMERIC(10, 2) NOT NULL, \\\\n\\\\t\"Quantity\" INTEGER NOT NULL, \\\\n\\\\tPRIMARY KEY (\"InvoiceLineId\"), \\\\n\\\\tFOREIGN KEY(\"TrackId\") REFERENCES \"Track\" (\"TrackId\"), \\\\n\\\\tFOREIGN KEY(\"InvoiceId\") REFERENCES \"Invoice\" (\"InvoiceId\")\\\\n)\\\\n\\\\n/*\\\\n3 rows from InvoiceLine table:\\\\nInvoiceLineId\\\\tInvoiceId\\\\tTrackId\\\\tUnitPrice\\\\tQuantity\\\\n1\\\\t1\\\\t2\\\\t0.99\\\\t1\\\\n2\\\\t1\\\\t4\\\\t0.99\\\\t1\\\\n3\\\\t2\\\\t6\\\\t0.99\\\\t1\\\\n*/\\\\n\\\\n\\\\nCREATE TABLE \"MediaType\" (\\\\n\\\\t\"MediaTypeId\" INTEGER NOT NULL, \\\\n\\\\t\"Name\" NVARCHAR(120), \\\\n\\\\tPRIMARY KEY (\"MediaTypeId\")\\\\n)\\\\n\\\\n/*\\\\n3 rows from MediaType table:\\\\nMediaTypeId\\\\tName\\\\n1\\\\tMPEG audio file\\\\n2\\\\tProtected AAC audio file\\\\n3\\\\tProtected MPEG-4 video file\\\\n*/\\\\n\\\\n\\\\nCREATE TABLE \"Playlist\" (\\\\n\\\\t\"PlaylistId\" INTEGER NOT NULL, \\\\n\\\\t\"Name\" NVARCHAR(120), \\\\n\\\\tPRIMARY KEY (\"PlaylistId\")\\\\n)\\\\n\\\\n/*\\\\n3 rows from Playlist table:\\\\nPlaylistId\\\\tName\\\\n1\\\\tMusic\\\\n2\\\\tMovies\\\\n3\\\\tTV Shows\\\\n*/\\\\n\\\\n\\\\nCREATE TABLE \"PlaylistTrack\" (\\\\n\\\\t\"PlaylistId\" INTEGER NOT NULL, \\\\n\\\\t\"TrackId\" INTEGER NOT NULL, \\\\n\\\\tPRIMARY KEY (\"PlaylistId\", \"TrackId\"), \\\\n\\\\tFOREIGN KEY(\"TrackId\") REFERENCES \"Track\" (\"TrackId\"), \\\\n\\\\tFOREIGN KEY(\"PlaylistId\") REFERENCES \"Playlist\" (\"PlaylistId\")\\\\n)\\\\n\\\\n/*\\\\n3 rows from PlaylistTrack table:\\\\nPlaylistId\\\\tTrackId\\\\n1\\\\t3402\\\\n1\\\\t3389\\\\n1\\\\t3390\\\\n*/\\\\n\\\\n\\\\nCREATE TABLE \"Track\" (\\\\n\\\\t\"TrackId\" INTEGER NOT NULL, \\\\n\\\\t\"Name\" NVARCHAR(200) NOT NULL, \\\\n\\\\t\"AlbumId\" INTEGER, \\\\n\\\\t\"MediaTypeId\" INTEGER NOT NULL, \\\\n\\\\t\"GenreId\" INTEGER, \\\\n\\\\t\"Composer\" NVARCHAR(220), \\\\n\\\\t\"Milliseconds\" INTEGER NOT NULL, \\\\n\\\\t\"Bytes\" INTEGER, \\\\n\\\\t\"UnitPrice\" NUMERIC(10, 2) NOT NULL, \\\\n\\\\tPRIMARY KEY (\"TrackId\"), \\\\n\\\\tFOREIGN KEY(\"MediaTypeId\") REFERENCES \"MediaType\" (\"MediaTypeId\"), \\\\n\\\\tFOREIGN KEY(\"GenreId\") REFERENCES \"Genre\" (\"GenreId\"), \\\\n\\\\tFOREIGN KEY(\"AlbumId\") REFERENCES \"Album\" (\"AlbumId\")\\\\n)\\\\n\\\\n/*\\\\n3 rows from Track table:\\\\nTrackId\\\\tName\\\\tAlbumId\\\\tMediaTypeId\\\\tGenreId\\\\tComposer\\\\tMilliseconds\\\\tBytes\\\\tUnitPrice\\\\n1\\\\tFor Those About To Rock (We Salute You)\\\\t1\\\\t1\\\\t1\\\\tAngus Young, Malcolm Young, Brian Johnson\\\\t343719\\\\t11170334\\\\t0.99\\\\n2\\\\tBalls to the Wall\\\\t2\\\\t2\\\\t1\\\\tU. Dirkschneider, W. Hoffmann, H. Frank, P. Baltes, S. Kaufmann, G. Hoffmann\\\\t342562\\\\t5510424\\\\t0.99\\\\n3\\\\tFast As a Shark\\\\t3\\\\t2\\\\t1\\\\tF. Baltes, S. Kaufman, U. Dirkscneider & W. Hoffman\\\\t230619\\\\t3990994\\\\t0.99\\\\n*/\\', \\'table_names\\': \\'Album, Artist, Customer, Employee, Genre, Invoice, InvoiceLine, MediaType, Playlist, PlaylistTrack, Track\\'}\\n',\n",
- " 'output': 'The customer who has spent the most money is Helena Holý.',\n",
- " 'citations': [CohereCitation(start=45, end=56, text='Helena Holý', documents=[{'output': \"[(6, 'Helena', 'Holý', 49.62)]\"}])],\n",
- " 'intermediate_steps': [(AgentActionMessageLog(tool='sql_db_query', tool_input={'query': 'SELECT c.CustomerId, c.FirstName, c.LastName, SUM(i.Total) AS TotalSpent\\nFROM Customer c\\nJOIN Invoice i ON c.CustomerId = i.CustomerId\\nGROUP BY c.CustomerId\\nORDER BY TotalSpent DESC\\nLIMIT 1;'}, log=\"\\nI will write a SQL query to find the customer who has spent the most money.\\n{'tool_name': 'sql_db_query', 'parameters': {'query': 'SELECT c.CustomerId, c.FirstName, c.LastName, SUM(i.Total) AS TotalSpent\\\\nFROM Customer c\\\\nJOIN Invoice i ON c.CustomerId = i.CustomerId\\\\nGROUP BY c.CustomerId\\\\nORDER BY TotalSpent DESC\\\\nLIMIT 1;'}}\\n\", message_log=[AIMessage(content='\\nPlan: I will write a SQL query to find the customer who has spent the most money.\\nAction: ```json\\n[\\n {\\n \"tool_name\": \"sql_db_query\",\\n \"parameters\": {\\n \"query\": \"SELECT c.CustomerId, c.FirstName, c.LastName, SUM(i.Total) AS TotalSpent\\\\nFROM Customer c\\\\nJOIN Invoice i ON c.CustomerId = i.CustomerId\\\\nGROUP BY c.CustomerId\\\\nORDER BY TotalSpent DESC\\\\nLIMIT 1;\"\\n }\\n }\\n]\\n```')]),\n",
- " \"[(6, 'Helena', 'Holý', 49.62)]\")]}"
- ]
- },
- "execution_count": 23,
- "metadata": {},
- "output_type": "execute_result"
}
],
"source": [
- "agent_executor.invoke({\n",
+ "output=agent_executor.invoke({\n",
" \"input\": 'provide the name of the best customer? The customer who has spent the most money is the best.',\n",
" \"preamble\": preamble\n",
"})"
]
},
{
- "cell_type": "markdown",
- "id": "b1cfcaaa-b1c0-4ded-baaa-7bbdb097be6e",
+ "cell_type": "code",
+ "execution_count": 27,
+ "id": "ff38320e-b40b-4d03-bcd2-beb40eeed655",
"metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "The customer who has spent the most money is Helena Holý.\n"
+ ]
+ }
+ ],
"source": [
- "Great, we can see that passing that additional context actually avoids the error seen in the previous section and gets to the answer in one tool call. Of course this works as long as you have a few tables and a few columns per table. We will follow up with more techniques to improve stability and scalability in another notebook."
+ "print(output['output'])"
]
},
{
- "cell_type": "code",
- "execution_count": null,
- "id": "95a1feed-1480-497d-bacf-9ab2667b573c",
+ "cell_type": "markdown",
+ "id": "b1cfcaaa-b1c0-4ded-baaa-7bbdb097be6e",
"metadata": {},
- "outputs": [],
- "source": []
+ "source": [
+ "We can see that passing that additional context actually avoids the error seen in the previous section and gets to the answer in one tool call. This works as long as you have a few tables and a few columns per table. We will follow up with more techniques to improve stability and scalability in future work."
+ ]
}
],
"metadata": {
diff --git a/notebooks/guides/Analysis_of_Form_10_K_Using_Cohere_and_RAG.ipynb b/notebooks/guides/Analysis_of_Form_10_K_Using_Cohere_and_RAG.ipynb
index e83eb62e..f3e4b799 100644
--- a/notebooks/guides/Analysis_of_Form_10_K_Using_Cohere_and_RAG.ipynb
+++ b/notebooks/guides/Analysis_of_Form_10_K_Using_Cohere_and_RAG.ipynb
@@ -1,2845 +1,2846 @@
{
- "cells": [
- {
- "cell_type": "markdown",
- "metadata": {
- "id": "UzG0FP8fwmIZ"
- },
- "source": [
- "# **Analysis of Form 10-K/10-Q Using Cohere and RAG**"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {
- "id": "NXzl9w_9Zf5K"
- },
- "source": [
- "## **Getting Started**\n",
- "\n",
- "You may use this script to jumpstart financial analysis of 10-Ks or 10-Qs with Cohere's Command model.\n",
- "\n",
- "This cookbook relies on helpful tooling from LlamaIndex, as well as our Cohere SDK. If you're familiar with LlamaIndex, it should be easy to slot this process into your own productivity flows."
- ]
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "UzG0FP8fwmIZ"
+ },
+ "source": [
+ "# **Analysis of Form 10-K/10-Q Using Cohere and RAG**"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "NXzl9w_9Zf5K"
+ },
+ "source": [
+ "## **Getting Started**\n",
+ "\n",
+ "You may use this script to jumpstart financial analysis of 10-Ks or 10-Qs with Cohere's Command model.\n",
+ "\n",
+ "This cookbook relies on helpful tooling from LlamaIndex, as well as our Cohere SDK. If you're familiar with LlamaIndex, it should be easy to slot this process into your own productivity flows."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "eYFvTs4mVpU4"
+ },
+ "outputs": [],
+ "source": [
+ "%%capture\n",
+ "!sudo apt install tesseract-ocr poppler-utils\n",
+ "!pip install \"cohere<5\" langchain llama-index llama-index-embeddings-cohere llama-index-postprocessor-cohere-rerank pytesseract pdf2image"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "LYmEOVKCAuk7"
+ },
+ "outputs": [],
+ "source": [
+ "# Due to compatibility issues, we need to do imports like this\n",
+ "from llama_index.core.schema import TextNode\n",
+ "\n",
+ "%%capture\n",
+ "!pip install unstructured"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "colab": {
+ "base_uri": "https://localhost:8080/",
+ "height": 35
},
+ "id": "Lc8CGMajDV9b",
+ "outputId": "5efe06b3-7eb5-490c-e9d3-b95b926358ed"
+ },
+ "outputs": [
{
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "id": "eYFvTs4mVpU4"
- },
- "outputs": [],
- "source": [
- "%%capture\n",
- "!sudo apt install tesseract-ocr poppler-utils\n",
- "!pip install \"cohere<5\" langchain llama-index llama-index-embeddings-cohere llama-index-postprocessor-cohere-rerank pytesseract pdf2image"
+ "data": {
+ "text/html": [
+ "\n",
+ " \n",
+ " "
+ ],
+ "text/plain": [
+ ""
]
+ },
+ "metadata": {},
+ "output_type": "display_data"
},
{
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "id": "LYmEOVKCAuk7"
- },
- "outputs": [],
- "source": [
- "# Due to compatibility issues, we need to do imports like this\n",
- "from llama_index.core.schema import TextNode\n",
- "\n",
- "%%capture\n",
- "!pip install unstructured"
- ]
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "Enter your Cohere API key: ··········\n"
+ ]
+ }
+ ],
+ "source": [
+ "import cohere\n",
+ "from getpass import getpass\n",
+ "\n",
+ "# Set up Cohere client\n",
+ "COHERE_API_KEY = getpass(\"Enter your Cohere API key: \")\n",
+ "\n",
+ "# Instantiate a client to communicate with Cohere's API using our Python SDK\n",
+ "co = cohere.Client(COHERE_API_KEY)\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "FfCtNkl6Z-eP"
+ },
+ "source": [
+ "## **Step 1: Loading a 10-K**\n",
+ "\n",
+ "You may run the following cells to load a 10-K that has already been preprocessed with OCR.\n",
+ "\n",
+ "> 💡 If you'd like to run the OCR pipeline yourself, you can find more info in the section titled **PDF to Text using OCR and `pdf2image`**."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "colab": {
+ "base_uri": "https://localhost:8080/",
+ "height": 71
},
+ "id": "6gkZ67Eh7l1A",
+ "outputId": "84406883-d4d4-44b5-c071-837410ee0d5a"
+ },
+ "outputs": [
{
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "colab": {
- "base_uri": "https://localhost:8080/",
- "height": 35
- },
- "id": "Lc8CGMajDV9b",
- "outputId": "5efe06b3-7eb5-490c-e9d3-b95b926358ed"
- },
- "outputs": [
- {
- "data": {
- "text/html": [
- "\n",
- " \n",
- " "
- ],
- "text/plain": [
- ""
- ]
- },
- "metadata": {},
- "output_type": "display_data"
- },
- {
- "name": "stdout",
- "output_type": "stream",
- "text": [
- "Enter your Cohere API key: ··········\n"
- ]
- }
+ "data": {
+ "text/html": [
+ "\n",
+ " \n",
+ " "
],
- "source": [
- "import cohere\n",
- "from getpass import getpass\n",
- "\n",
- "# Set up Cohere client\n",
- "COHERE_API_KEY = getpass(\"Enter your Cohere API key: \")\n",
- "\n",
- "# Instantiate a client to communicate with Cohere's API using our Python SDK\n",
- "co = cohere.Client(COHERE_API_KEY)\n"
+ "text/plain": [
+ ""
]
+ },
+ "metadata": {},
+ "output_type": "display_data"
},
{
- "cell_type": "markdown",
- "metadata": {
- "id": "FfCtNkl6Z-eP"
- },
- "source": [
- "## **Step 1: Loading a 10-K**\n",
- "\n",
- "You may run the following cells to load a 10-K that has already been preprocessed with OCR.\n",
- "\n",
- "> 💡 If you'd like to run the OCR pipeline yourself, you can find more info in the section titled **PDF to Text using OCR and `pdf2image`**."
- ]
+ "name": "stderr",
+ "output_type": "stream",
+ "text": [
+ "[nltk_data] Downloading package averaged_perceptron_tagger to\n",
+ "[nltk_data] /root/nltk_data...\n",
+ "[nltk_data] Unzipping taggers/averaged_perceptron_tagger.zip.\n"
+ ]
+ }
+ ],
+ "source": [
+ "# Using langchain here since they have access to the Unstructured Data Loader powered by unstructured.io\n",
+ "from langchain_community.document_loaders import UnstructuredURLLoader\n",
+ "\n",
+ "# Load up Airbnb's 10-K from this past fiscal year (filed in 2024)\n",
+ "# Feel free to fill in some other EDGAR path\n",
+ "url = \"https://www.sec.gov/Archives/edgar/data/1559720/000155972024000006/abnb-20231231.htm\"\n",
+ "loader = UnstructuredURLLoader(urls=[url], headers={\"User-Agent\": \"cohere cohere@cohere.com\"})\n",
+ "documents = loader.load()\n",
+ "\n",
+ "edgar_10k = documents[0].page_content\n",
+ "\n",
+ "# Load the document(s) as simple text nodes, to be passed to the tokenization processor\n",
+ "nodes = [TextNode(text=document.page_content, id_=f\"doc_{i}\") for i, document in enumerate(documents)]"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "W2PStuqwPPUM"
+ },
+ "source": [
+ "We'll need to convert the text into chunks of a certain size in order for the Cohere embedding model to properly ingest them down the line.\n",
+ "\n",
+ "We choose to use LlamaIndex's `SentenceSplitter` in this case in order to get these chunks. We must pass a tokenization callable, which we can do using the `transformers` library.\n",
+ "\n",
+ "You may also apply further transformations from the LlamaIndex repo if you so choose. Take a look at the [docs](https://docs.llamaindex.ai/en/stable/understanding/loading/loading.html) for inspiration on what is possible with transformations."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "colab": {
+ "base_uri": "https://localhost:8080/",
+ "height": 552,
+ "referenced_widgets": [
+ "e2146d738e0d4fe39af19bfb22da2584",
+ "8670211888514256a54240d002135917",
+ "88b9a3c1bc78462ea149b94d1ca08e59",
+ "5a94eb68326b4466b7c19ad67f8f5ba6",
+ "ab441cef6118450f9bfbcc29f8f34d4f",
+ "016a793bd7684acc9733f1e3160bd4d6",
+ "c8330e6698cd41868efc23be3962db7d",
+ "7e9939b2a8394aa3ae4dd1a90d9517ce",
+ "24055747d0e44bf29f80017117e332cc",
+ "59f4d94a1be24ef7a1366b7220a9c9de",
+ "6ba07312eb854ae9a88e6cfb886b65c3",
+ "01d81bc2d81741e6bb0a82ba78dc2b21",
+ "76d51ab196f54b398dcdc574968331fa",
+ "b9f6e9e50e224cf89b902e419dae269a",
+ "1271483cc9da41b28637dce1a40fa538",
+ "78a1cb4ec6cc4044b2149cb84f34979c",
+ "403ebbf8e6da4c188c42e83ee01db07b",
+ "f8e3c447794448529391f9660d419632",
+ "a7b1f4318e3242e591b15147d699c656",
+ "7d7dd120240043c1aec77af517bf7add",
+ "204957d62b7a4f7db7fa195a3d042b42",
+ "930d287b503b4eeeb527c6facc97ca30",
+ "88fc0e784a71447abbba8273f6fcdace",
+ "bb120e29c7fb49e4823daf66f048e95b",
+ "012491a19a8143f2adc1a95c5d20e488",
+ "a9cd4375dfd94af4ab2c004e9dbe6fa7",
+ "21229dbb1f414913b8ad230648ab76e9",
+ "d102def58f554d48890008185d17af96",
+ "79a852590a0444bebad62f2c679e72c6",
+ "c0c90d8f84b64a11a48e81ba8dec7044",
+ "1167e72dd2184c6ca8c670b27c73a27d",
+ "af93bfd421504e3eada6c8c884869e2f",
+ "782b11c627be41b2b67e2774ee7fcf0b",
+ "159f81e1ea4c40a394b6d796527c7c4a",
+ "96997c9c8f3a4237b1d025252c3c2358",
+ "28b25e6395c54e8e9494f403953065bb",
+ "cf03a6a84e1345aab6d2d78f9ae5f34a",
+ "5a15923788d74d98a70646e1e921831e",
+ "81d523a8f29c491aaed45a199260b414",
+ "56cd150b53234f53b28dec80ebeb33a8",
+ "21c71aa09fae458fa483d58019506d46",
+ "b3cb567ec4934a6d935de606af921a61",
+ "af63d2b1e22c40629b2fe585813a2bf4",
+ "87d5a00cc4a2460e836ddd22cff918dd",
+ "a65823cef8a648cebad7e61f012de1f1",
+ "6643b53bdbe74a6491a3db6ec06446e9",
+ "93d9e20a78774bb485ceca31d4453204",
+ "41b58953474f49f0915d14801fb18174",
+ "17e8c3567f1a4d6aaa485fd1014977ae",
+ "0e300e8dbea143d688b08c32883d2d29",
+ "2e094e1165554834a0cc19ebffe93311",
+ "2a9c8f7bb54e4814825bddcbb10b3482",
+ "5f5e73e41dae4625a98a6eee6120a7d4",
+ "efb2bf269c1d4ace81ff4a6fabbfb3b6",
+ "fc96c077774241a58d24ac42dd52df7e"
+ ]
},
+ "id": "p_1mXiVZBZu2",
+ "outputId": "f0146d70-4a6f-4821-ed70-11200a6efd49"
+ },
+ "outputs": [
{
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "colab": {
- "base_uri": "https://localhost:8080/",
- "height": 71
- },
- "id": "6gkZ67Eh7l1A",
- "outputId": "84406883-d4d4-44b5-c071-837410ee0d5a"
- },
- "outputs": [
- {
- "data": {
- "text/html": [
- "\n",
- " \n",
- " "
- ],
- "text/plain": [
- ""
- ]
- },
- "metadata": {},
- "output_type": "display_data"
- },
- {
- "name": "stderr",
- "output_type": "stream",
- "text": [
- "[nltk_data] Downloading package averaged_perceptron_tagger to\n",
- "[nltk_data] /root/nltk_data...\n",
- "[nltk_data] Unzipping taggers/averaged_perceptron_tagger.zip.\n"
- ]
- }
+ "data": {
+ "text/html": [
+ "\n",
+ " \n",
+ " "
],
- "source": [
- "# Using langchain here since they have access to the Unstructured Data Loader powered by unstructured.io\n",
- "from langchain_community.document_loaders import UnstructuredURLLoader\n",
- "\n",
- "# Load up Airbnb's 10-K from this past fiscal year (filed in 2024)\n",
- "# Feel free to fill in some other EDGAR path\n",
- "url = \"https://www.sec.gov/Archives/edgar/data/1559720/000155972024000006/abnb-20231231.htm\"\n",
- "loader = UnstructuredURLLoader(urls=[url], headers={\"User-Agent\": \"cohere cohere@cohere.com\"})\n",
- "documents = loader.load()\n",
- "\n",
- "edgar_10k = documents[0].page_content\n",
- "\n",
- "# Load the document(s) as simple text nodes, to be passed to the tokenization processor\n",
- "nodes = [TextNode(text=document.page_content, id_=f\"doc_{i}\") for i, document in enumerate(documents)]"
+ "text/plain": [
+ ""
]
+ },
+ "metadata": {},
+ "output_type": "display_data"
},
{
- "cell_type": "markdown",
- "metadata": {
- "id": "W2PStuqwPPUM"
- },
- "source": [
- "We'll need to convert the text into chunks of a certain size in order for the Cohere embedding model to properly ingest them down the line.\n",
- "\n",
- "We choose to use LlamaIndex's `SentenceSplitter` in this case in order to get these chunks. We must pass a tokenization callable, which we can do using the `transformers` library.\n",
- "\n",
- "You may also apply further transformations from the LlamaIndex repo if you so choose. Take a look at the [docs](https://docs.llamaindex.ai/en/stable/understanding/loading/loading.html) for inspiration on what is possible with transformations."
- ]
+ "name": "stderr",
+ "output_type": "stream",
+ "text": [
+ "/usr/local/lib/python3.10/dist-packages/huggingface_hub/utils/_token.py:88: UserWarning: \n",
+ "The secret `HF_TOKEN` does not exist in your Colab secrets.\n",
+ "To authenticate with the Hugging Face Hub, create a token in your settings tab (https://huggingface.co/settings/tokens), set it as secret in your Google Colab and restart your session.\n",
+ "You will be able to reuse this secret in all of your notebooks.\n",
+ "Please note that authentication is recommended but still optional to access public models or datasets.\n",
+ " warnings.warn(\n"
+ ]
},
{
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "colab": {
- "base_uri": "https://localhost:8080/",
- "height": 552,
- "referenced_widgets": [
- "e2146d738e0d4fe39af19bfb22da2584",
- "8670211888514256a54240d002135917",
- "88b9a3c1bc78462ea149b94d1ca08e59",
- "5a94eb68326b4466b7c19ad67f8f5ba6",
- "ab441cef6118450f9bfbcc29f8f34d4f",
- "016a793bd7684acc9733f1e3160bd4d6",
- "c8330e6698cd41868efc23be3962db7d",
- "7e9939b2a8394aa3ae4dd1a90d9517ce",
- "24055747d0e44bf29f80017117e332cc",
- "59f4d94a1be24ef7a1366b7220a9c9de",
- "6ba07312eb854ae9a88e6cfb886b65c3",
- "01d81bc2d81741e6bb0a82ba78dc2b21",
- "76d51ab196f54b398dcdc574968331fa",
- "b9f6e9e50e224cf89b902e419dae269a",
- "1271483cc9da41b28637dce1a40fa538",
- "78a1cb4ec6cc4044b2149cb84f34979c",
- "403ebbf8e6da4c188c42e83ee01db07b",
- "f8e3c447794448529391f9660d419632",
- "a7b1f4318e3242e591b15147d699c656",
- "7d7dd120240043c1aec77af517bf7add",
- "204957d62b7a4f7db7fa195a3d042b42",
- "930d287b503b4eeeb527c6facc97ca30",
- "88fc0e784a71447abbba8273f6fcdace",
- "bb120e29c7fb49e4823daf66f048e95b",
- "012491a19a8143f2adc1a95c5d20e488",
- "a9cd4375dfd94af4ab2c004e9dbe6fa7",
- "21229dbb1f414913b8ad230648ab76e9",
- "d102def58f554d48890008185d17af96",
- "79a852590a0444bebad62f2c679e72c6",
- "c0c90d8f84b64a11a48e81ba8dec7044",
- "1167e72dd2184c6ca8c670b27c73a27d",
- "af93bfd421504e3eada6c8c884869e2f",
- "782b11c627be41b2b67e2774ee7fcf0b",
- "159f81e1ea4c40a394b6d796527c7c4a",
- "96997c9c8f3a4237b1d025252c3c2358",
- "28b25e6395c54e8e9494f403953065bb",
- "cf03a6a84e1345aab6d2d78f9ae5f34a",
- "5a15923788d74d98a70646e1e921831e",
- "81d523a8f29c491aaed45a199260b414",
- "56cd150b53234f53b28dec80ebeb33a8",
- "21c71aa09fae458fa483d58019506d46",
- "b3cb567ec4934a6d935de606af921a61",
- "af63d2b1e22c40629b2fe585813a2bf4",
- "87d5a00cc4a2460e836ddd22cff918dd",
- "a65823cef8a648cebad7e61f012de1f1",
- "6643b53bdbe74a6491a3db6ec06446e9",
- "93d9e20a78774bb485ceca31d4453204",
- "41b58953474f49f0915d14801fb18174",
- "17e8c3567f1a4d6aaa485fd1014977ae",
- "0e300e8dbea143d688b08c32883d2d29",
- "2e094e1165554834a0cc19ebffe93311",
- "2a9c8f7bb54e4814825bddcbb10b3482",
- "5f5e73e41dae4625a98a6eee6120a7d4",
- "efb2bf269c1d4ace81ff4a6fabbfb3b6",
- "fc96c077774241a58d24ac42dd52df7e"
- ]
- },
- "id": "p_1mXiVZBZu2",
- "outputId": "f0146d70-4a6f-4821-ed70-11200a6efd49"
+ "data": {
+ "application/vnd.jupyter.widget-view+json": {
+ "model_id": "e2146d738e0d4fe39af19bfb22da2584",
+ "version_major": 2,
+ "version_minor": 0
},
- "outputs": [
- {
- "data": {
- "text/html": [
- "\n",
- " \n",
- " "
- ],
- "text/plain": [
- ""
- ]
- },
- "metadata": {},
- "output_type": "display_data"
- },
- {
- "name": "stderr",
- "output_type": "stream",
- "text": [
- "/usr/local/lib/python3.10/dist-packages/huggingface_hub/utils/_token.py:88: UserWarning: \n",
- "The secret `HF_TOKEN` does not exist in your Colab secrets.\n",
- "To authenticate with the Hugging Face Hub, create a token in your settings tab (https://huggingface.co/settings/tokens), set it as secret in your Google Colab and restart your session.\n",
- "You will be able to reuse this secret in all of your notebooks.\n",
- "Please note that authentication is recommended but still optional to access public models or datasets.\n",
- " warnings.warn(\n"
- ]
- },
- {
- "data": {
- "application/vnd.jupyter.widget-view+json": {
- "model_id": "e2146d738e0d4fe39af19bfb22da2584",
- "version_major": 2,
- "version_minor": 0
- },
- "text/plain": [
- "tokenizer_config.json: 0%| | 0.00/7.92k [00:00 0 else []\n",
- "\n",
- "pipeline = IngestionPipeline(\n",
- " transformations=[\n",
- " SentenceSplitter(chunk_size=512, chunk_overlap=0, tokenizer=tokenizer_fn)\n",
- " ]\n",
- ")\n",
- "\n",
- "# Run the pipeline to transform the text\n",
- "nodes = pipeline.run(nodes=nodes)"
+ "text/plain": [
+ "tokenizer_config.json: 0%| | 0.00/7.92k [00:00\n",
- " pre {\n",
- " white-space: pre-wrap;\n",
- " }\n",
- " \n",
- " "
- ],
- "text/plain": [
- ""
- ]
- },
- "metadata": {},
- "output_type": "display_data"
- }
- ],
- "source": [
- "from llama_index.core import Settings, VectorStoreIndex\n",
- "\n",
- "from llama_index.postprocessor.cohere_rerank import CohereRerank\n",
- "\n",
- "from llama_index.embeddings.cohere import CohereEmbedding\n",
- "\n",
- "# Instantiate the embedding model\n",
- "embed_model = CohereEmbedding(cohere_api_key=COHERE_API_KEY)\n",
- "\n",
- "# Global settings\n",
- "Settings.chunk_size = 512\n",
- "Settings.embed_model = embed_model\n",
- "\n",
- "# Create the vector store\n",
- "index = VectorStoreIndex(nodes)\n",
- "\n",
- "retriever = index.as_retriever(similarity_top_k=30) # Change to whatever top_k you want\n",
- "\n",
- "# Instantiate the reranker\n",
- "rerank = CohereRerank(api_key=COHERE_API_KEY, top_n=15)\n",
- "\n",
- "# Function `retrieve` is ready, using both Cohere embeddings for similarity search as well as\n",
- "retrieve = lambda query: rerank.postprocess_nodes(retriever.retrieve(query), query_str=query)"
+ "text/plain": [
+ "configuration_cohere.py: 0%| | 0.00/7.37k [00:00\n",
- " pre {\n",
- " white-space: pre-wrap;\n",
- " }\n",
- " \n",
- " "
- ],
- "text/plain": [
- ""
- ]
- },
- "metadata": {},
- "output_type": "display_data"
- }
- ],
- "source": [
- "PROMPT = \"List the overall revenue numbers for 2021, 2022, and 2023 in the 10-K as bullet points, then explain the revenue growth trends.\"\n",
- "\n",
- "# Get queries to run against our index from the command-nightly model\n",
- "r = co.chat(PROMPT, model=\"command-r\", search_queries_only=True)\n",
- "if r.search_queries:\n",
- " queries = [q[\"text\"] for q in r.search_queries]\n",
- "else:\n",
- " print(\"No queries returned by the model\")"
+ "text/plain": [
+ "tokenizer.json: 0%| | 0.00/12.8M [00:00\n",
- " pre {\n",
- " white-space: pre-wrap;\n",
- " }\n",
- " \n",
- " "
- ],
- "text/plain": [
- ""
- ]
- },
- "metadata": {},
- "output_type": "display_data"
- }
- ],
- "source": [
- "# Convenience function for formatting documents\n",
- "def format_for_cohere_client(nodes_):\n",
- " return [\n",
- " {\n",
- " \"text\": node.node.text,\n",
- " \"llamaindex_id\": node.node.id_,\n",
- " }\n",
- " for node\n",
- " in nodes_\n",
- " ]\n",
- "\n",
- "\n",
- "documents = []\n",
- "# Retrieve a set of chunks from the vector index and append them to the list of\n",
- "# documents that should be included in the final RAG step\n",
- "for query in queries:\n",
- " ret_nodes = retrieve(query)\n",
- " documents.extend(format_for_cohere_client(ret_nodes))\n",
- "\n",
- "# One final dedpulication step in case multiple queries return the same chunk\n",
- "documents = [dict(t, id=f\"doc_{i}\") for i, t in enumerate({tuple(d.items()) for d in documents})]"
- ]
+ "name": "stderr",
+ "output_type": "stream",
+ "text": [
+ "Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.\n"
+ ]
+ }
+ ],
+ "source": [
+ "from llama_index.core.ingestion import IngestionPipeline\n",
+ "from llama_index.core.node_parser import SentenceSplitter\n",
+ "\n",
+ "from transformers import AutoTokenizer\n",
+ "\n",
+ "model_id = \"CohereForAI/c4ai-command-r-v01\"\n",
+ "tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)\n",
+ "\n",
+ "# TODO: replace with a HF implementation so this is much faster. We'll\n",
+ "# presumably release it when we OS the model\n",
+ "tokenizer_fn = lambda x: tokenizer(x).input_ids if len(x) > 0 else []\n",
+ "\n",
+ "pipeline = IngestionPipeline(\n",
+ " transformations=[\n",
+ " SentenceSplitter(chunk_size=512, chunk_overlap=0, tokenizer=tokenizer_fn)\n",
+ " ]\n",
+ ")\n",
+ "\n",
+ "# Run the pipeline to transform the text\n",
+ "nodes = pipeline.run(nodes=nodes)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "573Sl4Ijfura"
+ },
+ "source": [
+ "## **Step 2: Load document into a LlamaIndex vector store**\n",
+ "\n",
+ "Loading the document into a LlamaIndex vector store will allow us to use the Cohere embedding model and rerank model to retrieve the relevant parts of the form to pass into Command."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "colab": {
+ "base_uri": "https://localhost:8080/",
+ "height": 17
},
+ "id": "T3-T32hQ-cYl",
+ "outputId": "855d765f-8d45-4a99-ab76-0f30d8ada3e0"
+ },
+ "outputs": [
{
- "cell_type": "markdown",
- "metadata": {
- "id": "kJAlJgocRxBI"
- },
- "source": [
- "## **Step 4: Make a RAG request to Command using document mode**\n",
- "\n",
- "Now that we have our nicely formatted chunks from the 10-K, we can pass them directly into Command using the Cohere SDK. By passing the chunks into the `documents` kwarg, we enable document mode, which will perform grounded inference on the documents you pass in.\n",
- "\n",
- "You can see this for yourself by inspecting the `response.citations` field to check where the model is citing from.\n",
- "\n",
- "You can learn more about the `chat` endpoint by checking out the API reference [here](https://docs.cohere.com/reference/chat)."
+ "data": {
+ "text/html": [
+ "\n",
+ " \n",
+ " "
+ ],
+ "text/plain": [
+ ""
]
+ },
+ "metadata": {},
+ "output_type": "display_data"
+ }
+ ],
+ "source": [
+ "from llama_index.core import Settings, VectorStoreIndex\n",
+ "\n",
+ "from llama_index.postprocessor.cohere_rerank import CohereRerank\n",
+ "\n",
+ "from llama_index.embeddings.cohere import CohereEmbedding\n",
+ "\n",
+ "# Instantiate the embedding model\n",
+ "embed_model = CohereEmbedding(cohere_api_key=COHERE_API_KEY)\n",
+ "\n",
+ "# Global settings\n",
+ "Settings.chunk_size = 512\n",
+ "Settings.embed_model = embed_model\n",
+ "\n",
+ "# Create the vector store\n",
+ "index = VectorStoreIndex(nodes)\n",
+ "\n",
+ "retriever = index.as_retriever(similarity_top_k=30) # Change to whatever top_k you want\n",
+ "\n",
+ "# Instantiate the reranker\n",
+ "rerank = CohereRerank(api_key=COHERE_API_KEY, top_n=15)\n",
+ "\n",
+ "# Function `retrieve` is ready, using both Cohere embeddings for similarity search as well as\n",
+ "retrieve = lambda query: rerank.postprocess_nodes(retriever.retrieve(query), query_str=query)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "19QPS8pzQicf"
+ },
+ "source": [
+ "## **Step 3: Query generation and retrieval**\n",
+ "\n",
+ "In order to do RAG, we need a query or a set of queries to actually _do_ the retrieval step. As is standard in RAG settings, we'll use Command to generate those queries for us. Then, we'll use those queries along with the LlamaIndex retriever we built earlier to retrieve the most relevant pieces of the 10-K.\n",
+ "\n",
+ "To learn more about document mode and query generation, check out [our documentation](https://docs.cohere.com/docs/retrieval-augmented-generation-rag)."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "colab": {
+ "base_uri": "https://localhost:8080/",
+ "height": 17
},
+ "id": "XHCPxdvrFliD",
+ "outputId": "85cbbe95-cb88-49ec-d4ea-f7b5a15e1816"
+ },
+ "outputs": [
{
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "colab": {
- "base_uri": "https://localhost:8080/",
- "height": 267
- },
- "id": "NBfQoZcXYdFc",
- "outputId": "b3a4156b-2749-4aaa-8b07-7c331388183f"
- },
- "outputs": [
- {
- "data": {
- "text/html": [
- "\n",
- " \n",
- " "
- ],
- "text/plain": [
- ""
- ]
- },
- "metadata": {},
- "output_type": "display_data"
- },
- {
- "name": "stdout",
- "output_type": "stream",
- "text": [
- "Here are the overall revenue numbers for the years 2021, 2022, and 2023 as bullet points:\n",
- "- 2021: $5,992 million\n",
- "- 2022: $8,399 million\n",
- "- 2023: $9,917 million\n",
- "\n",
- "Revenue increased by 18% in 2023 compared to 2022, primarily due to a 14% increase in Nights and Experiences Booked, which reached 54.5 million. This, combined with higher average daily rates, resulted in a 16% increase in Gross Booking Value, which reached $10.0 billion. \n",
- "\n",
- "The revenue growth trend demonstrates sustained strong travel demand. On a constant-currency basis, revenue increased by 17% in 2023 compared to the previous year.\n",
- "\n",
- "Other factors influencing the company's financial performance are described outside of the revenue growth trends.\n"
- ]
- }
+ "data": {
+ "text/html": [
+ "\n",
+ " \n",
+ " "
],
- "source": [
- "# Make a request to the model\n",
- "response = co.chat(\n",
- " message=PROMPT,\n",
- " model=\"command-r\",\n",
- " temperature=0.3,\n",
- " documents=documents,\n",
- " prompt_truncation=\"AUTO\"\n",
- ")\n",
- "\n",
- "print(response.text)"
+ "text/plain": [
+ ""
]
+ },
+ "metadata": {},
+ "output_type": "display_data"
+ }
+ ],
+ "source": [
+ "PROMPT = \"List the overall revenue numbers for 2021, 2022, and 2023 in the 10-K as bullet points, then explain the revenue growth trends.\"\n",
+ "\n",
+ "# Get queries to run against our index from the command-nightly model\n",
+ "r = co.chat(PROMPT, model=\"command-r\", search_queries_only=True)\n",
+ "if r.search_queries:\n",
+ " queries = [q[\"text\"] for q in r.search_queries]\n",
+ "else:\n",
+ " print(\"No queries returned by the model\")"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "nyLVmUBART3U"
+ },
+ "source": [
+ "Now, with the queries in hand, we search against our vector index."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "colab": {
+ "base_uri": "https://localhost:8080/",
+ "height": 17
},
+ "id": "k4t8s4xUX51B",
+ "outputId": "e739b4a6-b7b4-4870-8d45-2c921b13bcf1"
+ },
+ "outputs": [
{
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "colab": {
- "base_uri": "https://localhost:8080/",
- "height": 267
- },
- "id": "LHi1PDFpWj5p",
- "outputId": "1d735808-b758-4535-a419-df91225b9bfb"
- },
- "outputs": [
- {
- "data": {
- "text/html": [
- "\n",
- " \n",
- " "
- ],
- "text/plain": [
- ""
- ]
- },
- "metadata": {},
- "output_type": "display_data"
- },
- {
- "name": "stdout",
- "output_type": "stream",
- "text": [
- "Here are the overall revenue numbers for the years 2021, 2022, and 2023 as bullet points:\n",
- "- 2021: $5,992 million [13]\n",
- "- 2022: $8,399 million [13]\n",
- "- 2023: $9,917 million [13]\n",
- "\n",
- "Revenue increased by 18% in 2023 [11] compared to 2022, primarily due to a 14% increase in Nights and Experiences Booked [11], which reached 54.5 million. [11] This, combined with higher average daily rates [11], resulted in a 16% increase in Gross Booking Value [11], which reached $10.0 billion. [11] \n",
- "\n",
- "The revenue growth trend demonstrates sustained strong travel demand. [11] On a constant-currency basis [11], revenue increased by 17% in 2023 [11] compared to the previous year.\n",
- "\n",
- "Other factors [8, 14] influencing the company's financial performance are described outside of the revenue growth trends. [8, 14]\n"
- ]
- }
+ "data": {
+ "text/html": [
+ "\n",
+ " \n",
+ " "
],
- "source": [
- "# Helper function for displaying response WITH citations\n",
- "def insert_citations(text: str, citations: list[dict]):\n",
- " \"\"\"\n",
- " A helper function to pretty print citations.\n",
- " \"\"\"\n",
- " offset = 0\n",
- " # Process citations in the order they were provided\n",
- " for citation in citations:\n",
- " # Adjust start/end with offset\n",
- " start, end = citation['start'] + offset, citation['end'] + offset\n",
- " cited_docs = [doc[4:] for doc in citation[\"document_ids\"]]\n",
- " # Shorten citations if they're too long for convenience\n",
- " if len(cited_docs) > 3:\n",
- " placeholder = \"[\" + \", \".join(cited_docs[:3]) + \"...]\"\n",
- " else:\n",
- " placeholder = \"[\" + \", \".join(cited_docs) + \"]\"\n",
- " # ^ doc[4:] removes the 'doc_' prefix, and leaves the quoted document\n",
- " modification = f'{text[start:end]} {placeholder}'\n",
- " # Replace the cited text with its bolded version + placeholder\n",
- " text = text[:start] + modification + text[end:]\n",
- " # Update the offset for subsequent replacements\n",
- " offset += len(modification) - (end - start)\n",
- "\n",
- " return text\n",
- "\n",
- "print(insert_citations(response.text, response.citations))"
+ "text/plain": [
+ ""
]
+ },
+ "metadata": {},
+ "output_type": "display_data"
+ }
+ ],
+ "source": [
+ "# Convenience function for formatting documents\n",
+ "def format_for_cohere_client(nodes_):\n",
+ " return [\n",
+ " {\n",
+ " \"text\": node.node.text,\n",
+ " \"llamaindex_id\": node.node.id_,\n",
+ " }\n",
+ " for node\n",
+ " in nodes_\n",
+ " ]\n",
+ "\n",
+ "\n",
+ "documents = []\n",
+ "# Retrieve a set of chunks from the vector index and append them to the list of\n",
+ "# documents that should be included in the final RAG step\n",
+ "for query in queries:\n",
+ " ret_nodes = retrieve(query)\n",
+ " documents.extend(format_for_cohere_client(ret_nodes))\n",
+ "\n",
+ "# One final dedpulication step in case multiple queries return the same chunk\n",
+ "documents = [dict(t, id=f\"doc_{i}\") for i, t in enumerate({tuple(d.items()) for d in documents})]"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "kJAlJgocRxBI"
+ },
+ "source": [
+ "## **Step 4: Make a RAG request to Command using document mode**\n",
+ "\n",
+ "Now that we have our nicely formatted chunks from the 10-K, we can pass them directly into Command using the Cohere SDK. By passing the chunks into the `documents` kwarg, we enable document mode, which will perform grounded inference on the documents you pass in.\n",
+ "\n",
+ "You can see this for yourself by inspecting the `response.citations` field to check where the model is citing from.\n",
+ "\n",
+ "You can learn more about the `chat` endpoint by checking out the API reference [here](https://docs.cohere.com/reference/chat)."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "colab": {
+ "base_uri": "https://localhost:8080/",
+ "height": 267
},
+ "id": "NBfQoZcXYdFc",
+ "outputId": "b3a4156b-2749-4aaa-8b07-7c331388183f"
+ },
+ "outputs": [
{
- "cell_type": "markdown",
- "metadata": {
- "id": "aI4mJqMMKE3N"
- },
- "source": [
- "# **Appendix**"
+ "data": {
+ "text/html": [
+ "\n",
+ " \n",
+ " "
+ ],
+ "text/plain": [
+ ""
]
+ },
+ "metadata": {},
+ "output_type": "display_data"
},
{
- "cell_type": "markdown",
- "metadata": {
- "id": "yOeXqm1-6vXh"
- },
- "source": [
- "## PDF to Text using OCR and `pdf2image`\n",
- "\n",
- "This method will be required for any PDFs you have that need to be converted to text.\n",
- "\n",
- "**WARNING**: this process can take a long time without the proper optimizations. We have provided a snippet for your use below, but use at your own risk."
- ]
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "Here are the overall revenue numbers for the years 2021, 2022, and 2023 as bullet points:\n",
+ "- 2021: $5,992 million\n",
+ "- 2022: $8,399 million\n",
+ "- 2023: $9,917 million\n",
+ "\n",
+ "Revenue increased by 18% in 2023 compared to 2022, primarily due to a 14% increase in Nights and Experiences Booked, which reached 54.5 million. This, combined with higher average daily rates, resulted in a 16% increase in Gross Booking Value, which reached $10.0 billion. \n",
+ "\n",
+ "The revenue growth trend demonstrates sustained strong travel demand. On a constant-currency basis, revenue increased by 17% in 2023 compared to the previous year.\n",
+ "\n",
+ "Other factors influencing the company's financial performance are described outside of the revenue growth trends.\n"
+ ]
+ }
+ ],
+ "source": [
+ "# Make a request to the model\n",
+ "response = co.chat(\n",
+ " message=PROMPT,\n",
+ " model=\"command-r\",\n",
+ " temperature=0.3,\n",
+ " documents=documents,\n",
+ " prompt_truncation=\"AUTO\"\n",
+ ")\n",
+ "\n",
+ "print(response.text)"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "colab": {
+ "base_uri": "https://localhost:8080/",
+ "height": 267
},
+ "id": "LHi1PDFpWj5p",
+ "outputId": "1d735808-b758-4535-a419-df91225b9bfb"
+ },
+ "outputs": [
{
- "cell_type": "markdown",
- "metadata": {
- "id": "hv91DjU77a3Q"
- },
- "source": [
- "To go from PDF to text with PyTesseract, there is an intermediary step of converting the PDF to an image first, then passing that image into the OCR package, as OCR is usually only available for images.\n",
- "\n",
- "To do this, we use `pdf2image`, which uses `poppler` behind the scenes to convert the PDF into a PNG. From there, we can pass the image (which is a PIL Image object) directly into the OCR tool."
+ "data": {
+ "text/html": [
+ "\n",
+ " \n",
+ " "
+ ],
+ "text/plain": [
+ ""
]
+ },
+ "metadata": {},
+ "output_type": "display_data"
},
{
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "id": "bK8sMxiTeRTB"
- },
- "outputs": [],
- "source": [
- "import pytesseract\n",
- "from pdf2image import convert_from_path\n",
- "\n",
- "# pdf2image extracts as a list of PIL.Image objects\n",
- "# TODO: host this PDF somewhere\n",
- "pages = convert_from_path(\"/content/uber_10k.pdf\")\n",
- "\n",
- "# We access the only page in this sample PDF by indexing at 0\n",
- "pages = [pytesseract.image_to_string(page) for page in pages]"
- ]
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "Here are the overall revenue numbers for the years 2021, 2022, and 2023 as bullet points:\n",
+ "- 2021: $5,992 million [13]\n",
+ "- 2022: $8,399 million [13]\n",
+ "- 2023: $9,917 million [13]\n",
+ "\n",
+ "Revenue increased by 18% in 2023 [11] compared to 2022, primarily due to a 14% increase in Nights and Experiences Booked [11], which reached 54.5 million. [11] This, combined with higher average daily rates [11], resulted in a 16% increase in Gross Booking Value [11], which reached $10.0 billion. [11] \n",
+ "\n",
+ "The revenue growth trend demonstrates sustained strong travel demand. [11] On a constant-currency basis [11], revenue increased by 17% in 2023 [11] compared to the previous year.\n",
+ "\n",
+ "Other factors [8, 14] influencing the company's financial performance are described outside of the revenue growth trends. [8, 14]\n"
+ ]
+ }
+ ],
+ "source": [
+ "# Helper function for displaying response WITH citations\n",
+ "def insert_citations(text: str, citations: list[dict]):\n",
+ " \"\"\"\n",
+ " A helper function to pretty print citations.\n",
+ " \"\"\"\n",
+ " offset = 0\n",
+ " # Process citations in the order they were provided\n",
+ " for citation in citations:\n",
+ " # Adjust start/end with offset\n",
+ " start, end = citation['start'] + offset, citation['end'] + offset\n",
+ " cited_docs = [doc[4:] for doc in citation[\"document_ids\"]]\n",
+ " # Shorten citations if they're too long for convenience\n",
+ " if len(cited_docs) > 3:\n",
+ " placeholder = \"[\" + \", \".join(cited_docs[:3]) + \"...]\"\n",
+ " else:\n",
+ " placeholder = \"[\" + \", \".join(cited_docs) + \"]\"\n",
+ " # ^ doc[4:] removes the 'doc_' prefix, and leaves the quoted document\n",
+ " modification = f'{text[start:end]} {placeholder}'\n",
+ " # Replace the cited text with its bolded version + placeholder\n",
+ " text = text[:start] + modification + text[end:]\n",
+ " # Update the offset for subsequent replacements\n",
+ " offset += len(modification) - (end - start)\n",
+ "\n",
+ " return text\n",
+ "\n",
+ "print(insert_citations(response.text, response.citations))"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "aI4mJqMMKE3N"
+ },
+ "source": [
+ "# **Appendix**"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "yOeXqm1-6vXh"
+ },
+ "source": [
+ "## PDF to Text using OCR and `pdf2image`\n",
+ "\n",
+ "This method will be required for any PDFs you have that need to be converted to text.\n",
+ "\n",
+ "**WARNING**: this process can take a long time without the proper optimizations. We have provided a snippet for your use below, but use at your own risk."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "hv91DjU77a3Q"
+ },
+ "source": [
+ "To go from PDF to text with PyTesseract, there is an intermediary step of converting the PDF to an image first, then passing that image into the OCR package, as OCR is usually only available for images.\n",
+ "\n",
+ "To do this, we use `pdf2image`, which uses `poppler` behind the scenes to convert the PDF into a PNG. From there, we can pass the image (which is a PIL Image object) directly into the OCR tool."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "bK8sMxiTeRTB"
+ },
+ "outputs": [],
+ "source": [
+ "import pytesseract\n",
+ "from pdf2image import convert_from_path\n",
+ "\n",
+ "# pdf2image extracts as a list of PIL.Image objects\n",
+ "# TODO: host this PDF somewhere\n",
+ "pages = convert_from_path(\"/content/uber_10k.pdf\")\n",
+ "\n",
+ "# We access the only page in this sample PDF by indexing at 0\n",
+ "pages = [pytesseract.image_to_string(page) for page in pages]"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "7w7HyFL9iaZI"
+ },
+ "source": [
+ "## Token count / price comparison and latency"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "colab": {
+ "base_uri": "https://localhost:8080/",
+ "height": 17
},
+ "id": "igUBVGF0cy0D",
+ "outputId": "73c4d4d5-f29c-4403-8b08-c2567c39b902"
+ },
+ "outputs": [
{
- "cell_type": "markdown",
- "metadata": {
- "id": "7w7HyFL9iaZI"
- },
- "source": [
- "## Token count / price comparison and latency"
+ "data": {
+ "text/html": [
+ "\n",
+ " \n",
+ " "
+ ],
+ "text/plain": [
+ ""
]
+ },
+ "metadata": {},
+ "output_type": "display_data"
+ }
+ ],
+ "source": [
+ "def get_response(prompt, rag):\n",
+ " if rag:\n",
+ " # Get queries to run against our index from the command-nightly model\n",
+ " r = co.chat(prompt, model=\"command-r\", search_queries_only=True)\n",
+ " if r.search_queries:\n",
+ " queries = [q[\"text\"] for q in r.search_queries]\n",
+ " else:\n",
+ " print(\"No queries returned by the model\")\n",
+ "\n",
+ " documents = []\n",
+ " # Retrieve a set of chunks from the vector index and append them to the list of\n",
+ " # documents that should be included in the final RAG step\n",
+ " for query in queries:\n",
+ " ret_nodes = retrieve(query)\n",
+ " documents.extend(format_for_cohere_client(ret_nodes))\n",
+ "\n",
+ " # One final dedpulication step in case multiple queries return the same chunk\n",
+ " documents = [dict(t) for t in {tuple(d.items()) for d in documents}]\n",
+ "\n",
+ " # Make a request to the model\n",
+ " response = co.chat(\n",
+ " message=prompt,\n",
+ " model=\"command-r\",\n",
+ " temperature=0.3,\n",
+ " documents=documents,\n",
+ " prompt_truncation=\"AUTO\"\n",
+ " )\n",
+ " else:\n",
+ " response = co.chat(\n",
+ " message=prompt,\n",
+ " model=\"command-r\",\n",
+ " temperature=0.3,\n",
+ " )\n",
+ "\n",
+ " return response"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "colab": {
+ "base_uri": "https://localhost:8080/",
+ "height": 17
},
+ "id": "lX0YByK2eIeF",
+ "outputId": "a29cdf89-8b88-4c37-901c-6b1ce88cfa5e"
+ },
+ "outputs": [
{
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "colab": {
- "base_uri": "https://localhost:8080/",
- "height": 17
- },
- "id": "igUBVGF0cy0D",
- "outputId": "73c4d4d5-f29c-4403-8b08-c2567c39b902"
- },
- "outputs": [
- {
- "data": {
- "text/html": [
- "\n",
- " \n",
- " "
- ],
- "text/plain": [
- ""
- ]
- },
- "metadata": {},
- "output_type": "display_data"
- }
+ "data": {
+ "text/html": [
+ "\n",
+ " \n",
+ " "
],
- "source": [
- "def get_response(prompt, rag):\n",
- " if rag:\n",
- " # Get queries to run against our index from the command-nightly model\n",
- " r = co.chat(prompt, model=\"command-r\", search_queries_only=True)\n",
- " if r.search_queries:\n",
- " queries = [q[\"text\"] for q in r.search_queries]\n",
- " else:\n",
- " print(\"No queries returned by the model\")\n",
- "\n",
- " documents = []\n",
- " # Retrieve a set of chunks from the vector index and append them to the list of\n",
- " # documents that should be included in the final RAG step\n",
- " for query in queries:\n",
- " ret_nodes = retrieve(query)\n",
- " documents.extend(format_for_cohere_client(ret_nodes))\n",
- "\n",
- " # One final dedpulication step in case multiple queries return the same chunk\n",
- " documents = [dict(t) for t in {tuple(d.items()) for d in documents}]\n",
- "\n",
- " # Make a request to the model\n",
- " response = co.chat(\n",
- " message=prompt,\n",
- " model=\"command-r\",\n",
- " temperature=0.3,\n",
- " documents=documents,\n",
- " prompt_truncation=\"AUTO\"\n",
- " )\n",
- " else:\n",
- " response = co.chat(\n",
- " message=prompt,\n",
- " model=\"command-r\",\n",
- " temperature=0.3,\n",
- " )\n",
- "\n",
- " return response"
+ "text/plain": [
+ ""
]
+ },
+ "metadata": {},
+ "output_type": "display_data"
+ }
+ ],
+ "source": [
+ "prompt_template = \"\"\"# financial form 10-K\n",
+ "{tenk}\n",
+ "\n",
+ "# question\n",
+ "{question}\"\"\"\n",
+ "\n",
+ "full_context_prompt = prompt_template.format(tenk=edgar_10k, question=PROMPT)"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "colab": {
+ "base_uri": "https://localhost:8080/",
+ "height": 17
},
+ "id": "_s_9t57wfRCy",
+ "outputId": "babf0cb3-a719-467d-ec5f-80e8477f2e40"
+ },
+ "outputs": [
{
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "colab": {
- "base_uri": "https://localhost:8080/",
- "height": 17
- },
- "id": "lX0YByK2eIeF",
- "outputId": "a29cdf89-8b88-4c37-901c-6b1ce88cfa5e"
- },
- "outputs": [
- {
- "data": {
- "text/html": [
- "\n",
- " \n",
- " "
- ],
- "text/plain": [
- ""
- ]
- },
- "metadata": {},
- "output_type": "display_data"
- }
+ "data": {
+ "text/html": [
+ "\n",
+ " \n",
+ " "
],
- "source": [
- "prompt_template = \"\"\"# financial form 10-K\n",
- "{tenk}\n",
- "\n",
- "# question\n",
- "{question}\"\"\"\n",
- "\n",
- "full_context_prompt = prompt_template.format(tenk=edgar_10k, question=PROMPT)"
+ "text/plain": [
+ ""
]
+ },
+ "metadata": {},
+ "output_type": "display_data"
+ }
+ ],
+ "source": [
+ "r1 = get_response(PROMPT, rag=True)\n",
+ "r2 = get_response(full_context_prompt, rag=False)"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "colab": {
+ "base_uri": "https://localhost:8080/",
+ "height": 17
},
+ "id": "INxf-xvdiOgF",
+ "outputId": "8632e4ce-e945-44d6-ed8d-7e6a4b923348"
+ },
+ "outputs": [
{
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "colab": {
- "base_uri": "https://localhost:8080/",
- "height": 17
- },
- "id": "_s_9t57wfRCy",
- "outputId": "babf0cb3-a719-467d-ec5f-80e8477f2e40"
- },
- "outputs": [
- {
- "data": {
- "text/html": [
- "\n",
- " \n",
- " "
- ],
- "text/plain": [
- ""
- ]
- },
- "metadata": {},
- "output_type": "display_data"
- }
+ "data": {
+ "text/html": [
+ "\n",
+ " \n",
+ " "
],
- "source": [
- "r1 = get_response(PROMPT, rag=True)\n",
- "r2 = get_response(full_context_prompt, rag=False)"
+ "text/plain": [
+ ""
]
+ },
+ "metadata": {},
+ "output_type": "display_data"
+ }
+ ],
+ "source": [
+ "def get_price(r):\n",
+ " return (r.token_count[\"prompt_tokens\"] * 0.5 / 10e6) + (r.token_count[\"response_tokens\"] * 1.5 / 10e6)"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "colab": {
+ "base_uri": "https://localhost:8080/",
+ "height": 35
},
+ "id": "uuUUZeewiSV0",
+ "outputId": "991125f1-5892-4744-e919-7b46930a0272"
+ },
+ "outputs": [
{
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "colab": {
- "base_uri": "https://localhost:8080/",
- "height": 17
- },
- "id": "INxf-xvdiOgF",
- "outputId": "8632e4ce-e945-44d6-ed8d-7e6a4b923348"
- },
- "outputs": [
- {
- "data": {
- "text/html": [
- "\n",
- " \n",
- " "
- ],
- "text/plain": [
- ""
- ]
- },
- "metadata": {},
- "output_type": "display_data"
- }
+ "data": {
+ "text/html": [
+ "\n",
+ " \n",
+ " "
],
- "source": [
- "def get_price(r):\n",
- " return (r.token_count[\"prompt_tokens\"] * 0.5 / 10e6) + (r.token_count[\"response_tokens\"] * 1.5 / 10e6)"
+ "text/plain": [
+ ""
]
+ },
+ "metadata": {},
+ "output_type": "display_data"
},
{
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "colab": {
- "base_uri": "https://localhost:8080/",
- "height": 35
- },
- "id": "uuUUZeewiSV0",
- "outputId": "991125f1-5892-4744-e919-7b46930a0272"
- },
- "outputs": [
- {
- "data": {
- "text/html": [
- "\n",
- " \n",
- " "
- ],
- "text/plain": [
- ""
- ]
- },
- "metadata": {},
- "output_type": "display_data"
- },
- {
- "name": "stdout",
- "output_type": "stream",
- "text": [
- "RAG is 93% cheaper than full context\n"
- ]
- }
- ],
- "source": [
- "rag_price = get_price(r1)\n",
- "full_context_price = get_price(r2)\n",
- "\n",
- "print(f\"RAG is {(full_context_price - rag_price) / full_context_price:.0%} cheaper than full context\")"
- ]
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "RAG is 93% cheaper than full context\n"
+ ]
+ }
+ ],
+ "source": [
+ "rag_price = get_price(r1)\n",
+ "full_context_price = get_price(r2)\n",
+ "\n",
+ "print(f\"RAG is {(full_context_price - rag_price) / full_context_price:.0%} cheaper than full context\")"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "colab": {
+ "base_uri": "https://localhost:8080/",
+ "height": 35
},
+ "id": "8sqRqK8ekKAH",
+ "outputId": "4d44b64c-50cc-42f0-e2f1-c38205e42c04"
+ },
+ "outputs": [
{
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "colab": {
- "base_uri": "https://localhost:8080/",
- "height": 35
- },
- "id": "8sqRqK8ekKAH",
- "outputId": "4d44b64c-50cc-42f0-e2f1-c38205e42c04"
- },
- "outputs": [
- {
- "data": {
- "text/html": [
- "\n",
- " \n",
- " "
- ],
- "text/plain": [
- ""
- ]
- },
- "metadata": {},
- "output_type": "display_data"
- },
- {
- "name": "stdout",
- "output_type": "stream",
- "text": [
- "14.9 s ± 1.4 s per loop (mean ± std. dev. of 7 runs, 1 loop each)\n"
- ]
- }
+ "data": {
+ "text/html": [
+ "\n",
+ " \n",
+ " "
],
- "source": [
- "%timeit get_response(PROMPT, rag=True)"
+ "text/plain": [
+ ""
]
+ },
+ "metadata": {},
+ "output_type": "display_data"
},
{
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "colab": {
- "base_uri": "https://localhost:8080/",
- "height": 35
- },
- "id": "r9Ck-k_gCNJ1",
- "outputId": "fe4cdd1a-a4ac-4e20-8c8d-a3ef05ab8bcc"
- },
- "outputs": [
- {
- "data": {
- "text/html": [
- "\n",
- " \n",
- " "
- ],
- "text/plain": [
- ""
- ]
- },
- "metadata": {},
- "output_type": "display_data"
- },
- {
- "name": "stdout",
- "output_type": "stream",
- "text": [
- "22.7 s ± 7.43 s per loop (mean ± std. dev. of 7 runs, 1 loop each)\n"
- ]
- }
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "14.9 s ± 1.4 s per loop (mean ± std. dev. of 7 runs, 1 loop each)\n"
+ ]
+ }
+ ],
+ "source": [
+ "%timeit get_response(PROMPT, rag=True)"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "colab": {
+ "base_uri": "https://localhost:8080/",
+ "height": 35
+ },
+ "id": "r9Ck-k_gCNJ1",
+ "outputId": "fe4cdd1a-a4ac-4e20-8c8d-a3ef05ab8bcc"
+ },
+ "outputs": [
+ {
+ "data": {
+ "text/html": [
+ "\n",
+ " \n",
+ " "
],
- "source": [
- "%timeit get_response(full_context_prompt, rag=False)"
+ "text/plain": [
+ ""
]
+ },
+ "metadata": {},
+ "output_type": "display_data"
},
{
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "id": "Y7sLarNgFGlV"
- },
- "outputs": [],
- "source": []
- }
- ],
- "metadata": {
- "colab": {
- "provenance": []
- },
- "kernelspec": {
- "display_name": "Python 3",
- "name": "python3"
- },
- "language_info": {
- "name": "python"
- },
- "widgets": {
- "application/vnd.jupyter.widget-state+json": {
- "012491a19a8143f2adc1a95c5d20e488": {
- "model_module": "@jupyter-widgets/controls",
- "model_module_version": "1.5.0",
- "model_name": "FloatProgressModel",
- "state": {
- "_dom_classes": [],
- "_model_module": "@jupyter-widgets/controls",
- "_model_module_version": "1.5.0",
- "_model_name": "FloatProgressModel",
- "_view_count": null,
- "_view_module": "@jupyter-widgets/controls",
- "_view_module_version": "1.5.0",
- "_view_name": "ProgressView",
- "bar_style": "success",
- "description": "",
- "description_tooltip": null,
- "layout": "IPY_MODEL_c0c90d8f84b64a11a48e81ba8dec7044",
- "max": 7366,
- "min": 0,
- "orientation": "horizontal",
- "style": "IPY_MODEL_1167e72dd2184c6ca8c670b27c73a27d",
- "value": 7366
- }
- },
- "016a793bd7684acc9733f1e3160bd4d6": {
- "model_module": "@jupyter-widgets/base",
- "model_module_version": "1.2.0",
- "model_name": "LayoutModel",
- "state": {
- "_model_module": "@jupyter-widgets/base",
- "_model_module_version": "1.2.0",
- "_model_name": "LayoutModel",
- "_view_count": null,
- "_view_module": "@jupyter-widgets/base",
- "_view_module_version": "1.2.0",
- "_view_name": "LayoutView",
- "align_content": null,
- "align_items": null,
- "align_self": null,
- "border": null,
- "bottom": null,
- "display": null,
- "flex": null,
- "flex_flow": null,
- "grid_area": null,
- "grid_auto_columns": null,
- "grid_auto_flow": null,
- "grid_auto_rows": null,
- "grid_column": null,
- "grid_gap": null,
- "grid_row": null,
- "grid_template_areas": null,
- "grid_template_columns": null,
- "grid_template_rows": null,
- "height": null,
- "justify_content": null,
- "justify_items": null,
- "left": null,
- "margin": null,
- "max_height": null,
- "max_width": null,
- "min_height": null,
- "min_width": null,
- "object_fit": null,
- "object_position": null,
- "order": null,
- "overflow": null,
- "overflow_x": null,
- "overflow_y": null,
- "padding": null,
- "right": null,
- "top": null,
- "visibility": null,
- "width": null
- }
- },
- "01d81bc2d81741e6bb0a82ba78dc2b21": {
- "model_module": "@jupyter-widgets/controls",
- "model_module_version": "1.5.0",
- "model_name": "HBoxModel",
- "state": {
- "_dom_classes": [],
- "_model_module": "@jupyter-widgets/controls",
- "_model_module_version": "1.5.0",
- "_model_name": "HBoxModel",
- "_view_count": null,
- "_view_module": "@jupyter-widgets/controls",
- "_view_module_version": "1.5.0",
- "_view_name": "HBoxView",
- "box_style": "",
- "children": [
- "IPY_MODEL_76d51ab196f54b398dcdc574968331fa",
- "IPY_MODEL_b9f6e9e50e224cf89b902e419dae269a",
- "IPY_MODEL_1271483cc9da41b28637dce1a40fa538"
- ],
- "layout": "IPY_MODEL_78a1cb4ec6cc4044b2149cb84f34979c"
- }
- },
- "0e300e8dbea143d688b08c32883d2d29": {
- "model_module": "@jupyter-widgets/base",
- "model_module_version": "1.2.0",
- "model_name": "LayoutModel",
- "state": {
- "_model_module": "@jupyter-widgets/base",
- "_model_module_version": "1.2.0",
- "_model_name": "LayoutModel",
- "_view_count": null,
- "_view_module": "@jupyter-widgets/base",
- "_view_module_version": "1.2.0",
- "_view_name": "LayoutView",
- "align_content": null,
- "align_items": null,
- "align_self": null,
- "border": null,
- "bottom": null,
- "display": null,
- "flex": null,
- "flex_flow": null,
- "grid_area": null,
- "grid_auto_columns": null,
- "grid_auto_flow": null,
- "grid_auto_rows": null,
- "grid_column": null,
- "grid_gap": null,
- "grid_row": null,
- "grid_template_areas": null,
- "grid_template_columns": null,
- "grid_template_rows": null,
- "height": null,
- "justify_content": null,
- "justify_items": null,
- "left": null,
- "margin": null,
- "max_height": null,
- "max_width": null,
- "min_height": null,
- "min_width": null,
- "object_fit": null,
- "object_position": null,
- "order": null,
- "overflow": null,
- "overflow_x": null,
- "overflow_y": null,
- "padding": null,
- "right": null,
- "top": null,
- "visibility": null,
- "width": null
- }
- },
- "1167e72dd2184c6ca8c670b27c73a27d": {
- "model_module": "@jupyter-widgets/controls",
- "model_module_version": "1.5.0",
- "model_name": "ProgressStyleModel",
- "state": {
- "_model_module": "@jupyter-widgets/controls",
- "_model_module_version": "1.5.0",
- "_model_name": "ProgressStyleModel",
- "_view_count": null,
- "_view_module": "@jupyter-widgets/base",
- "_view_module_version": "1.2.0",
- "_view_name": "StyleView",
- "bar_color": null,
- "description_width": ""
- }
- },
- "1271483cc9da41b28637dce1a40fa538": {
- "model_module": "@jupyter-widgets/controls",
- "model_module_version": "1.5.0",
- "model_name": "HTMLModel",
- "state": {
- "_dom_classes": [],
- "_model_module": "@jupyter-widgets/controls",
- "_model_module_version": "1.5.0",
- "_model_name": "HTMLModel",
- "_view_count": null,
- "_view_module": "@jupyter-widgets/controls",
- "_view_module_version": "1.5.0",
- "_view_name": "HTMLView",
- "description": "",
- "description_tooltip": null,
- "layout": "IPY_MODEL_204957d62b7a4f7db7fa195a3d042b42",
- "placeholder": "",
- "style": "IPY_MODEL_930d287b503b4eeeb527c6facc97ca30",
- "value": " 43.7k/43.7k [00:00<00:00, 1.97MB/s]"
- }
- },
- "159f81e1ea4c40a394b6d796527c7c4a": {
- "model_module": "@jupyter-widgets/controls",
- "model_module_version": "1.5.0",
- "model_name": "HBoxModel",
- "state": {
- "_dom_classes": [],
- "_model_module": "@jupyter-widgets/controls",
- "_model_module_version": "1.5.0",
- "_model_name": "HBoxModel",
- "_view_count": null,
- "_view_module": "@jupyter-widgets/controls",
- "_view_module_version": "1.5.0",
- "_view_name": "HBoxView",
- "box_style": "",
- "children": [
- "IPY_MODEL_96997c9c8f3a4237b1d025252c3c2358",
- "IPY_MODEL_28b25e6395c54e8e9494f403953065bb",
- "IPY_MODEL_cf03a6a84e1345aab6d2d78f9ae5f34a"
- ],
- "layout": "IPY_MODEL_5a15923788d74d98a70646e1e921831e"
- }
- },
- "17e8c3567f1a4d6aaa485fd1014977ae": {
- "model_module": "@jupyter-widgets/base",
- "model_module_version": "1.2.0",
- "model_name": "LayoutModel",
- "state": {
- "_model_module": "@jupyter-widgets/base",
- "_model_module_version": "1.2.0",
- "_model_name": "LayoutModel",
- "_view_count": null,
- "_view_module": "@jupyter-widgets/base",
- "_view_module_version": "1.2.0",
- "_view_name": "LayoutView",
- "align_content": null,
- "align_items": null,
- "align_self": null,
- "border": null,
- "bottom": null,
- "display": null,
- "flex": null,
- "flex_flow": null,
- "grid_area": null,
- "grid_auto_columns": null,
- "grid_auto_flow": null,
- "grid_auto_rows": null,
- "grid_column": null,
- "grid_gap": null,
- "grid_row": null,
- "grid_template_areas": null,
- "grid_template_columns": null,
- "grid_template_rows": null,
- "height": null,
- "justify_content": null,
- "justify_items": null,
- "left": null,
- "margin": null,
- "max_height": null,
- "max_width": null,
- "min_height": null,
- "min_width": null,
- "object_fit": null,
- "object_position": null,
- "order": null,
- "overflow": null,
- "overflow_x": null,
- "overflow_y": null,
- "padding": null,
- "right": null,
- "top": null,
- "visibility": null,
- "width": null
- }
- },
- "204957d62b7a4f7db7fa195a3d042b42": {
- "model_module": "@jupyter-widgets/base",
- "model_module_version": "1.2.0",
- "model_name": "LayoutModel",
- "state": {
- "_model_module": "@jupyter-widgets/base",
- "_model_module_version": "1.2.0",
- "_model_name": "LayoutModel",
- "_view_count": null,
- "_view_module": "@jupyter-widgets/base",
- "_view_module_version": "1.2.0",
- "_view_name": "LayoutView",
- "align_content": null,
- "align_items": null,
- "align_self": null,
- "border": null,
- "bottom": null,
- "display": null,
- "flex": null,
- "flex_flow": null,
- "grid_area": null,
- "grid_auto_columns": null,
- "grid_auto_flow": null,
- "grid_auto_rows": null,
- "grid_column": null,
- "grid_gap": null,
- "grid_row": null,
- "grid_template_areas": null,
- "grid_template_columns": null,
- "grid_template_rows": null,
- "height": null,
- "justify_content": null,
- "justify_items": null,
- "left": null,
- "margin": null,
- "max_height": null,
- "max_width": null,
- "min_height": null,
- "min_width": null,
- "object_fit": null,
- "object_position": null,
- "order": null,
- "overflow": null,
- "overflow_x": null,
- "overflow_y": null,
- "padding": null,
- "right": null,
- "top": null,
- "visibility": null,
- "width": null
- }
- },
- "21229dbb1f414913b8ad230648ab76e9": {
- "model_module": "@jupyter-widgets/base",
- "model_module_version": "1.2.0",
- "model_name": "LayoutModel",
- "state": {
- "_model_module": "@jupyter-widgets/base",
- "_model_module_version": "1.2.0",
- "_model_name": "LayoutModel",
- "_view_count": null,
- "_view_module": "@jupyter-widgets/base",
- "_view_module_version": "1.2.0",
- "_view_name": "LayoutView",
- "align_content": null,
- "align_items": null,
- "align_self": null,
- "border": null,
- "bottom": null,
- "display": null,
- "flex": null,
- "flex_flow": null,
- "grid_area": null,
- "grid_auto_columns": null,
- "grid_auto_flow": null,
- "grid_auto_rows": null,
- "grid_column": null,
- "grid_gap": null,
- "grid_row": null,
- "grid_template_areas": null,
- "grid_template_columns": null,
- "grid_template_rows": null,
- "height": null,
- "justify_content": null,
- "justify_items": null,
- "left": null,
- "margin": null,
- "max_height": null,
- "max_width": null,
- "min_height": null,
- "min_width": null,
- "object_fit": null,
- "object_position": null,
- "order": null,
- "overflow": null,
- "overflow_x": null,
- "overflow_y": null,
- "padding": null,
- "right": null,
- "top": null,
- "visibility": null,
- "width": null
- }
- },
- "21c71aa09fae458fa483d58019506d46": {
- "model_module": "@jupyter-widgets/base",
- "model_module_version": "1.2.0",
- "model_name": "LayoutModel",
- "state": {
- "_model_module": "@jupyter-widgets/base",
- "_model_module_version": "1.2.0",
- "_model_name": "LayoutModel",
- "_view_count": null,
- "_view_module": "@jupyter-widgets/base",
- "_view_module_version": "1.2.0",
- "_view_name": "LayoutView",
- "align_content": null,
- "align_items": null,
- "align_self": null,
- "border": null,
- "bottom": null,
- "display": null,
- "flex": null,
- "flex_flow": null,
- "grid_area": null,
- "grid_auto_columns": null,
- "grid_auto_flow": null,
- "grid_auto_rows": null,
- "grid_column": null,
- "grid_gap": null,
- "grid_row": null,
- "grid_template_areas": null,
- "grid_template_columns": null,
- "grid_template_rows": null,
- "height": null,
- "justify_content": null,
- "justify_items": null,
- "left": null,
- "margin": null,
- "max_height": null,
- "max_width": null,
- "min_height": null,
- "min_width": null,
- "object_fit": null,
- "object_position": null,
- "order": null,
- "overflow": null,
- "overflow_x": null,
- "overflow_y": null,
- "padding": null,
- "right": null,
- "top": null,
- "visibility": null,
- "width": null
- }
- },
- "24055747d0e44bf29f80017117e332cc": {
- "model_module": "@jupyter-widgets/controls",
- "model_module_version": "1.5.0",
- "model_name": "ProgressStyleModel",
- "state": {
- "_model_module": "@jupyter-widgets/controls",
- "_model_module_version": "1.5.0",
- "_model_name": "ProgressStyleModel",
- "_view_count": null,
- "_view_module": "@jupyter-widgets/base",
- "_view_module_version": "1.2.0",
- "_view_name": "StyleView",
- "bar_color": null,
- "description_width": ""
- }
- },
- "28b25e6395c54e8e9494f403953065bb": {
- "model_module": "@jupyter-widgets/controls",
- "model_module_version": "1.5.0",
- "model_name": "FloatProgressModel",
- "state": {
- "_dom_classes": [],
- "_model_module": "@jupyter-widgets/controls",
- "_model_module_version": "1.5.0",
- "_model_name": "FloatProgressModel",
- "_view_count": null,
- "_view_module": "@jupyter-widgets/controls",
- "_view_module_version": "1.5.0",
- "_view_name": "ProgressView",
- "bar_style": "success",
- "description": "",
- "description_tooltip": null,
- "layout": "IPY_MODEL_21c71aa09fae458fa483d58019506d46",
- "max": 12777406,
- "min": 0,
- "orientation": "horizontal",
- "style": "IPY_MODEL_b3cb567ec4934a6d935de606af921a61",
- "value": 12777406
- }
- },
- "2a9c8f7bb54e4814825bddcbb10b3482": {
- "model_module": "@jupyter-widgets/base",
- "model_module_version": "1.2.0",
- "model_name": "LayoutModel",
- "state": {
- "_model_module": "@jupyter-widgets/base",
- "_model_module_version": "1.2.0",
- "_model_name": "LayoutModel",
- "_view_count": null,
- "_view_module": "@jupyter-widgets/base",
- "_view_module_version": "1.2.0",
- "_view_name": "LayoutView",
- "align_content": null,
- "align_items": null,
- "align_self": null,
- "border": null,
- "bottom": null,
- "display": null,
- "flex": null,
- "flex_flow": null,
- "grid_area": null,
- "grid_auto_columns": null,
- "grid_auto_flow": null,
- "grid_auto_rows": null,
- "grid_column": null,
- "grid_gap": null,
- "grid_row": null,
- "grid_template_areas": null,
- "grid_template_columns": null,
- "grid_template_rows": null,
- "height": null,
- "justify_content": null,
- "justify_items": null,
- "left": null,
- "margin": null,
- "max_height": null,
- "max_width": null,
- "min_height": null,
- "min_width": null,
- "object_fit": null,
- "object_position": null,
- "order": null,
- "overflow": null,
- "overflow_x": null,
- "overflow_y": null,
- "padding": null,
- "right": null,
- "top": null,
- "visibility": null,
- "width": null
- }
- },
- "2e094e1165554834a0cc19ebffe93311": {
- "model_module": "@jupyter-widgets/controls",
- "model_module_version": "1.5.0",
- "model_name": "DescriptionStyleModel",
- "state": {
- "_model_module": "@jupyter-widgets/controls",
- "_model_module_version": "1.5.0",
- "_model_name": "DescriptionStyleModel",
- "_view_count": null,
- "_view_module": "@jupyter-widgets/base",
- "_view_module_version": "1.2.0",
- "_view_name": "StyleView",
- "description_width": ""
- }
- },
- "403ebbf8e6da4c188c42e83ee01db07b": {
- "model_module": "@jupyter-widgets/base",
- "model_module_version": "1.2.0",
- "model_name": "LayoutModel",
- "state": {
- "_model_module": "@jupyter-widgets/base",
- "_model_module_version": "1.2.0",
- "_model_name": "LayoutModel",
- "_view_count": null,
- "_view_module": "@jupyter-widgets/base",
- "_view_module_version": "1.2.0",
- "_view_name": "LayoutView",
- "align_content": null,
- "align_items": null,
- "align_self": null,
- "border": null,
- "bottom": null,
- "display": null,
- "flex": null,
- "flex_flow": null,
- "grid_area": null,
- "grid_auto_columns": null,
- "grid_auto_flow": null,
- "grid_auto_rows": null,
- "grid_column": null,
- "grid_gap": null,
- "grid_row": null,
- "grid_template_areas": null,
- "grid_template_columns": null,
- "grid_template_rows": null,
- "height": null,
- "justify_content": null,
- "justify_items": null,
- "left": null,
- "margin": null,
- "max_height": null,
- "max_width": null,
- "min_height": null,
- "min_width": null,
- "object_fit": null,
- "object_position": null,
- "order": null,
- "overflow": null,
- "overflow_x": null,
- "overflow_y": null,
- "padding": null,
- "right": null,
- "top": null,
- "visibility": null,
- "width": null
- }
- },
- "41b58953474f49f0915d14801fb18174": {
- "model_module": "@jupyter-widgets/controls",
- "model_module_version": "1.5.0",
- "model_name": "HTMLModel",
- "state": {
- "_dom_classes": [],
- "_model_module": "@jupyter-widgets/controls",
- "_model_module_version": "1.5.0",
- "_model_name": "HTMLModel",
- "_view_count": null,
- "_view_module": "@jupyter-widgets/controls",
- "_view_module_version": "1.5.0",
- "_view_name": "HTMLView",
- "description": "",
- "description_tooltip": null,
- "layout": "IPY_MODEL_efb2bf269c1d4ace81ff4a6fabbfb3b6",
- "placeholder": "",
- "style": "IPY_MODEL_fc96c077774241a58d24ac42dd52df7e",
- "value": " 429/429 [00:00<00:00, 20.7kB/s]"
- }
- },
- "56cd150b53234f53b28dec80ebeb33a8": {
- "model_module": "@jupyter-widgets/controls",
- "model_module_version": "1.5.0",
- "model_name": "DescriptionStyleModel",
- "state": {
- "_model_module": "@jupyter-widgets/controls",
- "_model_module_version": "1.5.0",
- "_model_name": "DescriptionStyleModel",
- "_view_count": null,
- "_view_module": "@jupyter-widgets/base",
- "_view_module_version": "1.2.0",
- "_view_name": "StyleView",
- "description_width": ""
- }
- },
- "59f4d94a1be24ef7a1366b7220a9c9de": {
- "model_module": "@jupyter-widgets/base",
- "model_module_version": "1.2.0",
- "model_name": "LayoutModel",
- "state": {
- "_model_module": "@jupyter-widgets/base",
- "_model_module_version": "1.2.0",
- "_model_name": "LayoutModel",
- "_view_count": null,
- "_view_module": "@jupyter-widgets/base",
- "_view_module_version": "1.2.0",
- "_view_name": "LayoutView",
- "align_content": null,
- "align_items": null,
- "align_self": null,
- "border": null,
- "bottom": null,
- "display": null,
- "flex": null,
- "flex_flow": null,
- "grid_area": null,
- "grid_auto_columns": null,
- "grid_auto_flow": null,
- "grid_auto_rows": null,
- "grid_column": null,
- "grid_gap": null,
- "grid_row": null,
- "grid_template_areas": null,
- "grid_template_columns": null,
- "grid_template_rows": null,
- "height": null,
- "justify_content": null,
- "justify_items": null,
- "left": null,
- "margin": null,
- "max_height": null,
- "max_width": null,
- "min_height": null,
- "min_width": null,
- "object_fit": null,
- "object_position": null,
- "order": null,
- "overflow": null,
- "overflow_x": null,
- "overflow_y": null,
- "padding": null,
- "right": null,
- "top": null,
- "visibility": null,
- "width": null
- }
- },
- "5a15923788d74d98a70646e1e921831e": {
- "model_module": "@jupyter-widgets/base",
- "model_module_version": "1.2.0",
- "model_name": "LayoutModel",
- "state": {
- "_model_module": "@jupyter-widgets/base",
- "_model_module_version": "1.2.0",
- "_model_name": "LayoutModel",
- "_view_count": null,
- "_view_module": "@jupyter-widgets/base",
- "_view_module_version": "1.2.0",
- "_view_name": "LayoutView",
- "align_content": null,
- "align_items": null,
- "align_self": null,
- "border": null,
- "bottom": null,
- "display": null,
- "flex": null,
- "flex_flow": null,
- "grid_area": null,
- "grid_auto_columns": null,
- "grid_auto_flow": null,
- "grid_auto_rows": null,
- "grid_column": null,
- "grid_gap": null,
- "grid_row": null,
- "grid_template_areas": null,
- "grid_template_columns": null,
- "grid_template_rows": null,
- "height": null,
- "justify_content": null,
- "justify_items": null,
- "left": null,
- "margin": null,
- "max_height": null,
- "max_width": null,
- "min_height": null,
- "min_width": null,
- "object_fit": null,
- "object_position": null,
- "order": null,
- "overflow": null,
- "overflow_x": null,
- "overflow_y": null,
- "padding": null,
- "right": null,
- "top": null,
- "visibility": null,
- "width": null
- }
- },
- "5a94eb68326b4466b7c19ad67f8f5ba6": {
- "model_module": "@jupyter-widgets/controls",
- "model_module_version": "1.5.0",
- "model_name": "HTMLModel",
- "state": {
- "_dom_classes": [],
- "_model_module": "@jupyter-widgets/controls",
- "_model_module_version": "1.5.0",
- "_model_name": "HTMLModel",
- "_view_count": null,
- "_view_module": "@jupyter-widgets/controls",
- "_view_module_version": "1.5.0",
- "_view_name": "HTMLView",
- "description": "",
- "description_tooltip": null,
- "layout": "IPY_MODEL_59f4d94a1be24ef7a1366b7220a9c9de",
- "placeholder": "",
- "style": "IPY_MODEL_6ba07312eb854ae9a88e6cfb886b65c3",
- "value": " 7.92k/7.92k [00:00<00:00, 366kB/s]"
- }
- },
- "5f5e73e41dae4625a98a6eee6120a7d4": {
- "model_module": "@jupyter-widgets/controls",
- "model_module_version": "1.5.0",
- "model_name": "ProgressStyleModel",
- "state": {
- "_model_module": "@jupyter-widgets/controls",
- "_model_module_version": "1.5.0",
- "_model_name": "ProgressStyleModel",
- "_view_count": null,
- "_view_module": "@jupyter-widgets/base",
- "_view_module_version": "1.2.0",
- "_view_name": "StyleView",
- "bar_color": null,
- "description_width": ""
- }
- },
- "6643b53bdbe74a6491a3db6ec06446e9": {
- "model_module": "@jupyter-widgets/controls",
- "model_module_version": "1.5.0",
- "model_name": "HTMLModel",
- "state": {
- "_dom_classes": [],
- "_model_module": "@jupyter-widgets/controls",
- "_model_module_version": "1.5.0",
- "_model_name": "HTMLModel",
- "_view_count": null,
- "_view_module": "@jupyter-widgets/controls",
- "_view_module_version": "1.5.0",
- "_view_name": "HTMLView",
- "description": "",
- "description_tooltip": null,
- "layout": "IPY_MODEL_0e300e8dbea143d688b08c32883d2d29",
- "placeholder": "",
- "style": "IPY_MODEL_2e094e1165554834a0cc19ebffe93311",
- "value": "special_tokens_map.json: 100%"
- }
- },
- "6ba07312eb854ae9a88e6cfb886b65c3": {
- "model_module": "@jupyter-widgets/controls",
- "model_module_version": "1.5.0",
- "model_name": "DescriptionStyleModel",
- "state": {
- "_model_module": "@jupyter-widgets/controls",
- "_model_module_version": "1.5.0",
- "_model_name": "DescriptionStyleModel",
- "_view_count": null,
- "_view_module": "@jupyter-widgets/base",
- "_view_module_version": "1.2.0",
- "_view_name": "StyleView",
- "description_width": ""
- }
- },
- "76d51ab196f54b398dcdc574968331fa": {
- "model_module": "@jupyter-widgets/controls",
- "model_module_version": "1.5.0",
- "model_name": "HTMLModel",
- "state": {
- "_dom_classes": [],
- "_model_module": "@jupyter-widgets/controls",
- "_model_module_version": "1.5.0",
- "_model_name": "HTMLModel",
- "_view_count": null,
- "_view_module": "@jupyter-widgets/controls",
- "_view_module_version": "1.5.0",
- "_view_name": "HTMLView",
- "description": "",
- "description_tooltip": null,
- "layout": "IPY_MODEL_403ebbf8e6da4c188c42e83ee01db07b",
- "placeholder": "",
- "style": "IPY_MODEL_f8e3c447794448529391f9660d419632",
- "value": "tokenization_cohere_fast.py: 100%"
- }
- },
- "782b11c627be41b2b67e2774ee7fcf0b": {
- "model_module": "@jupyter-widgets/controls",
- "model_module_version": "1.5.0",
- "model_name": "DescriptionStyleModel",
- "state": {
- "_model_module": "@jupyter-widgets/controls",
- "_model_module_version": "1.5.0",
- "_model_name": "DescriptionStyleModel",
- "_view_count": null,
- "_view_module": "@jupyter-widgets/base",
- "_view_module_version": "1.2.0",
- "_view_name": "StyleView",
- "description_width": ""
- }
- },
- "78a1cb4ec6cc4044b2149cb84f34979c": {
- "model_module": "@jupyter-widgets/base",
- "model_module_version": "1.2.0",
- "model_name": "LayoutModel",
- "state": {
- "_model_module": "@jupyter-widgets/base",
- "_model_module_version": "1.2.0",
- "_model_name": "LayoutModel",
- "_view_count": null,
- "_view_module": "@jupyter-widgets/base",
- "_view_module_version": "1.2.0",
- "_view_name": "LayoutView",
- "align_content": null,
- "align_items": null,
- "align_self": null,
- "border": null,
- "bottom": null,
- "display": null,
- "flex": null,
- "flex_flow": null,
- "grid_area": null,
- "grid_auto_columns": null,
- "grid_auto_flow": null,
- "grid_auto_rows": null,
- "grid_column": null,
- "grid_gap": null,
- "grid_row": null,
- "grid_template_areas": null,
- "grid_template_columns": null,
- "grid_template_rows": null,
- "height": null,
- "justify_content": null,
- "justify_items": null,
- "left": null,
- "margin": null,
- "max_height": null,
- "max_width": null,
- "min_height": null,
- "min_width": null,
- "object_fit": null,
- "object_position": null,
- "order": null,
- "overflow": null,
- "overflow_x": null,
- "overflow_y": null,
- "padding": null,
- "right": null,
- "top": null,
- "visibility": null,
- "width": null
- }
- },
- "79a852590a0444bebad62f2c679e72c6": {
- "model_module": "@jupyter-widgets/controls",
- "model_module_version": "1.5.0",
- "model_name": "DescriptionStyleModel",
- "state": {
- "_model_module": "@jupyter-widgets/controls",
- "_model_module_version": "1.5.0",
- "_model_name": "DescriptionStyleModel",
- "_view_count": null,
- "_view_module": "@jupyter-widgets/base",
- "_view_module_version": "1.2.0",
- "_view_name": "StyleView",
- "description_width": ""
- }
- },
- "7d7dd120240043c1aec77af517bf7add": {
- "model_module": "@jupyter-widgets/controls",
- "model_module_version": "1.5.0",
- "model_name": "ProgressStyleModel",
- "state": {
- "_model_module": "@jupyter-widgets/controls",
- "_model_module_version": "1.5.0",
- "_model_name": "ProgressStyleModel",
- "_view_count": null,
- "_view_module": "@jupyter-widgets/base",
- "_view_module_version": "1.2.0",
- "_view_name": "StyleView",
- "bar_color": null,
- "description_width": ""
- }
- },
- "7e9939b2a8394aa3ae4dd1a90d9517ce": {
- "model_module": "@jupyter-widgets/base",
- "model_module_version": "1.2.0",
- "model_name": "LayoutModel",
- "state": {
- "_model_module": "@jupyter-widgets/base",
- "_model_module_version": "1.2.0",
- "_model_name": "LayoutModel",
- "_view_count": null,
- "_view_module": "@jupyter-widgets/base",
- "_view_module_version": "1.2.0",
- "_view_name": "LayoutView",
- "align_content": null,
- "align_items": null,
- "align_self": null,
- "border": null,
- "bottom": null,
- "display": null,
- "flex": null,
- "flex_flow": null,
- "grid_area": null,
- "grid_auto_columns": null,
- "grid_auto_flow": null,
- "grid_auto_rows": null,
- "grid_column": null,
- "grid_gap": null,
- "grid_row": null,
- "grid_template_areas": null,
- "grid_template_columns": null,
- "grid_template_rows": null,
- "height": null,
- "justify_content": null,
- "justify_items": null,
- "left": null,
- "margin": null,
- "max_height": null,
- "max_width": null,
- "min_height": null,
- "min_width": null,
- "object_fit": null,
- "object_position": null,
- "order": null,
- "overflow": null,
- "overflow_x": null,
- "overflow_y": null,
- "padding": null,
- "right": null,
- "top": null,
- "visibility": null,
- "width": null
- }
- },
- "81d523a8f29c491aaed45a199260b414": {
- "model_module": "@jupyter-widgets/base",
- "model_module_version": "1.2.0",
- "model_name": "LayoutModel",
- "state": {
- "_model_module": "@jupyter-widgets/base",
- "_model_module_version": "1.2.0",
- "_model_name": "LayoutModel",
- "_view_count": null,
- "_view_module": "@jupyter-widgets/base",
- "_view_module_version": "1.2.0",
- "_view_name": "LayoutView",
- "align_content": null,
- "align_items": null,
- "align_self": null,
- "border": null,
- "bottom": null,
- "display": null,
- "flex": null,
- "flex_flow": null,
- "grid_area": null,
- "grid_auto_columns": null,
- "grid_auto_flow": null,
- "grid_auto_rows": null,
- "grid_column": null,
- "grid_gap": null,
- "grid_row": null,
- "grid_template_areas": null,
- "grid_template_columns": null,
- "grid_template_rows": null,
- "height": null,
- "justify_content": null,
- "justify_items": null,
- "left": null,
- "margin": null,
- "max_height": null,
- "max_width": null,
- "min_height": null,
- "min_width": null,
- "object_fit": null,
- "object_position": null,
- "order": null,
- "overflow": null,
- "overflow_x": null,
- "overflow_y": null,
- "padding": null,
- "right": null,
- "top": null,
- "visibility": null,
- "width": null
- }
- },
- "8670211888514256a54240d002135917": {
- "model_module": "@jupyter-widgets/controls",
- "model_module_version": "1.5.0",
- "model_name": "HTMLModel",
- "state": {
- "_dom_classes": [],
- "_model_module": "@jupyter-widgets/controls",
- "_model_module_version": "1.5.0",
- "_model_name": "HTMLModel",
- "_view_count": null,
- "_view_module": "@jupyter-widgets/controls",
- "_view_module_version": "1.5.0",
- "_view_name": "HTMLView",
- "description": "",
- "description_tooltip": null,
- "layout": "IPY_MODEL_016a793bd7684acc9733f1e3160bd4d6",
- "placeholder": "",
- "style": "IPY_MODEL_c8330e6698cd41868efc23be3962db7d",
- "value": "tokenizer_config.json: 100%"
- }
- },
- "87d5a00cc4a2460e836ddd22cff918dd": {
- "model_module": "@jupyter-widgets/controls",
- "model_module_version": "1.5.0",
- "model_name": "DescriptionStyleModel",
- "state": {
- "_model_module": "@jupyter-widgets/controls",
- "_model_module_version": "1.5.0",
- "_model_name": "DescriptionStyleModel",
- "_view_count": null,
- "_view_module": "@jupyter-widgets/base",
- "_view_module_version": "1.2.0",
- "_view_name": "StyleView",
- "description_width": ""
- }
- },
- "88b9a3c1bc78462ea149b94d1ca08e59": {
- "model_module": "@jupyter-widgets/controls",
- "model_module_version": "1.5.0",
- "model_name": "FloatProgressModel",
- "state": {
- "_dom_classes": [],
- "_model_module": "@jupyter-widgets/controls",
- "_model_module_version": "1.5.0",
- "_model_name": "FloatProgressModel",
- "_view_count": null,
- "_view_module": "@jupyter-widgets/controls",
- "_view_module_version": "1.5.0",
- "_view_name": "ProgressView",
- "bar_style": "success",
- "description": "",
- "description_tooltip": null,
- "layout": "IPY_MODEL_7e9939b2a8394aa3ae4dd1a90d9517ce",
- "max": 7916,
- "min": 0,
- "orientation": "horizontal",
- "style": "IPY_MODEL_24055747d0e44bf29f80017117e332cc",
- "value": 7916
- }
- },
- "88fc0e784a71447abbba8273f6fcdace": {
- "model_module": "@jupyter-widgets/controls",
- "model_module_version": "1.5.0",
- "model_name": "HBoxModel",
- "state": {
- "_dom_classes": [],
- "_model_module": "@jupyter-widgets/controls",
- "_model_module_version": "1.5.0",
- "_model_name": "HBoxModel",
- "_view_count": null,
- "_view_module": "@jupyter-widgets/controls",
- "_view_module_version": "1.5.0",
- "_view_name": "HBoxView",
- "box_style": "",
- "children": [
- "IPY_MODEL_bb120e29c7fb49e4823daf66f048e95b",
- "IPY_MODEL_012491a19a8143f2adc1a95c5d20e488",
- "IPY_MODEL_a9cd4375dfd94af4ab2c004e9dbe6fa7"
- ],
- "layout": "IPY_MODEL_21229dbb1f414913b8ad230648ab76e9"
- }
- },
- "930d287b503b4eeeb527c6facc97ca30": {
- "model_module": "@jupyter-widgets/controls",
- "model_module_version": "1.5.0",
- "model_name": "DescriptionStyleModel",
- "state": {
- "_model_module": "@jupyter-widgets/controls",
- "_model_module_version": "1.5.0",
- "_model_name": "DescriptionStyleModel",
- "_view_count": null,
- "_view_module": "@jupyter-widgets/base",
- "_view_module_version": "1.2.0",
- "_view_name": "StyleView",
- "description_width": ""
- }
- },
- "93d9e20a78774bb485ceca31d4453204": {
- "model_module": "@jupyter-widgets/controls",
- "model_module_version": "1.5.0",
- "model_name": "FloatProgressModel",
- "state": {
- "_dom_classes": [],
- "_model_module": "@jupyter-widgets/controls",
- "_model_module_version": "1.5.0",
- "_model_name": "FloatProgressModel",
- "_view_count": null,
- "_view_module": "@jupyter-widgets/controls",
- "_view_module_version": "1.5.0",
- "_view_name": "ProgressView",
- "bar_style": "success",
- "description": "",
- "description_tooltip": null,
- "layout": "IPY_MODEL_2a9c8f7bb54e4814825bddcbb10b3482",
- "max": 429,
- "min": 0,
- "orientation": "horizontal",
- "style": "IPY_MODEL_5f5e73e41dae4625a98a6eee6120a7d4",
- "value": 429
- }
- },
- "96997c9c8f3a4237b1d025252c3c2358": {
- "model_module": "@jupyter-widgets/controls",
- "model_module_version": "1.5.0",
- "model_name": "HTMLModel",
- "state": {
- "_dom_classes": [],
- "_model_module": "@jupyter-widgets/controls",
- "_model_module_version": "1.5.0",
- "_model_name": "HTMLModel",
- "_view_count": null,
- "_view_module": "@jupyter-widgets/controls",
- "_view_module_version": "1.5.0",
- "_view_name": "HTMLView",
- "description": "",
- "description_tooltip": null,
- "layout": "IPY_MODEL_81d523a8f29c491aaed45a199260b414",
- "placeholder": "",
- "style": "IPY_MODEL_56cd150b53234f53b28dec80ebeb33a8",
- "value": "tokenizer.json: 100%"
- }
- },
- "a65823cef8a648cebad7e61f012de1f1": {
- "model_module": "@jupyter-widgets/controls",
- "model_module_version": "1.5.0",
- "model_name": "HBoxModel",
- "state": {
- "_dom_classes": [],
- "_model_module": "@jupyter-widgets/controls",
- "_model_module_version": "1.5.0",
- "_model_name": "HBoxModel",
- "_view_count": null,
- "_view_module": "@jupyter-widgets/controls",
- "_view_module_version": "1.5.0",
- "_view_name": "HBoxView",
- "box_style": "",
- "children": [
- "IPY_MODEL_6643b53bdbe74a6491a3db6ec06446e9",
- "IPY_MODEL_93d9e20a78774bb485ceca31d4453204",
- "IPY_MODEL_41b58953474f49f0915d14801fb18174"
- ],
- "layout": "IPY_MODEL_17e8c3567f1a4d6aaa485fd1014977ae"
- }
- },
- "a7b1f4318e3242e591b15147d699c656": {
- "model_module": "@jupyter-widgets/base",
- "model_module_version": "1.2.0",
- "model_name": "LayoutModel",
- "state": {
- "_model_module": "@jupyter-widgets/base",
- "_model_module_version": "1.2.0",
- "_model_name": "LayoutModel",
- "_view_count": null,
- "_view_module": "@jupyter-widgets/base",
- "_view_module_version": "1.2.0",
- "_view_name": "LayoutView",
- "align_content": null,
- "align_items": null,
- "align_self": null,
- "border": null,
- "bottom": null,
- "display": null,
- "flex": null,
- "flex_flow": null,
- "grid_area": null,
- "grid_auto_columns": null,
- "grid_auto_flow": null,
- "grid_auto_rows": null,
- "grid_column": null,
- "grid_gap": null,
- "grid_row": null,
- "grid_template_areas": null,
- "grid_template_columns": null,
- "grid_template_rows": null,
- "height": null,
- "justify_content": null,
- "justify_items": null,
- "left": null,
- "margin": null,
- "max_height": null,
- "max_width": null,
- "min_height": null,
- "min_width": null,
- "object_fit": null,
- "object_position": null,
- "order": null,
- "overflow": null,
- "overflow_x": null,
- "overflow_y": null,
- "padding": null,
- "right": null,
- "top": null,
- "visibility": null,
- "width": null
- }
- },
- "a9cd4375dfd94af4ab2c004e9dbe6fa7": {
- "model_module": "@jupyter-widgets/controls",
- "model_module_version": "1.5.0",
- "model_name": "HTMLModel",
- "state": {
- "_dom_classes": [],
- "_model_module": "@jupyter-widgets/controls",
- "_model_module_version": "1.5.0",
- "_model_name": "HTMLModel",
- "_view_count": null,
- "_view_module": "@jupyter-widgets/controls",
- "_view_module_version": "1.5.0",
- "_view_name": "HTMLView",
- "description": "",
- "description_tooltip": null,
- "layout": "IPY_MODEL_af93bfd421504e3eada6c8c884869e2f",
- "placeholder": "",
- "style": "IPY_MODEL_782b11c627be41b2b67e2774ee7fcf0b",
- "value": " 7.37k/7.37k [00:00<00:00, 373kB/s]"
- }
- },
- "ab441cef6118450f9bfbcc29f8f34d4f": {
- "model_module": "@jupyter-widgets/base",
- "model_module_version": "1.2.0",
- "model_name": "LayoutModel",
- "state": {
- "_model_module": "@jupyter-widgets/base",
- "_model_module_version": "1.2.0",
- "_model_name": "LayoutModel",
- "_view_count": null,
- "_view_module": "@jupyter-widgets/base",
- "_view_module_version": "1.2.0",
- "_view_name": "LayoutView",
- "align_content": null,
- "align_items": null,
- "align_self": null,
- "border": null,
- "bottom": null,
- "display": null,
- "flex": null,
- "flex_flow": null,
- "grid_area": null,
- "grid_auto_columns": null,
- "grid_auto_flow": null,
- "grid_auto_rows": null,
- "grid_column": null,
- "grid_gap": null,
- "grid_row": null,
- "grid_template_areas": null,
- "grid_template_columns": null,
- "grid_template_rows": null,
- "height": null,
- "justify_content": null,
- "justify_items": null,
- "left": null,
- "margin": null,
- "max_height": null,
- "max_width": null,
- "min_height": null,
- "min_width": null,
- "object_fit": null,
- "object_position": null,
- "order": null,
- "overflow": null,
- "overflow_x": null,
- "overflow_y": null,
- "padding": null,
- "right": null,
- "top": null,
- "visibility": null,
- "width": null
- }
- },
- "af63d2b1e22c40629b2fe585813a2bf4": {
- "model_module": "@jupyter-widgets/base",
- "model_module_version": "1.2.0",
- "model_name": "LayoutModel",
- "state": {
- "_model_module": "@jupyter-widgets/base",
- "_model_module_version": "1.2.0",
- "_model_name": "LayoutModel",
- "_view_count": null,
- "_view_module": "@jupyter-widgets/base",
- "_view_module_version": "1.2.0",
- "_view_name": "LayoutView",
- "align_content": null,
- "align_items": null,
- "align_self": null,
- "border": null,
- "bottom": null,
- "display": null,
- "flex": null,
- "flex_flow": null,
- "grid_area": null,
- "grid_auto_columns": null,
- "grid_auto_flow": null,
- "grid_auto_rows": null,
- "grid_column": null,
- "grid_gap": null,
- "grid_row": null,
- "grid_template_areas": null,
- "grid_template_columns": null,
- "grid_template_rows": null,
- "height": null,
- "justify_content": null,
- "justify_items": null,
- "left": null,
- "margin": null,
- "max_height": null,
- "max_width": null,
- "min_height": null,
- "min_width": null,
- "object_fit": null,
- "object_position": null,
- "order": null,
- "overflow": null,
- "overflow_x": null,
- "overflow_y": null,
- "padding": null,
- "right": null,
- "top": null,
- "visibility": null,
- "width": null
- }
- },
- "af93bfd421504e3eada6c8c884869e2f": {
- "model_module": "@jupyter-widgets/base",
- "model_module_version": "1.2.0",
- "model_name": "LayoutModel",
- "state": {
- "_model_module": "@jupyter-widgets/base",
- "_model_module_version": "1.2.0",
- "_model_name": "LayoutModel",
- "_view_count": null,
- "_view_module": "@jupyter-widgets/base",
- "_view_module_version": "1.2.0",
- "_view_name": "LayoutView",
- "align_content": null,
- "align_items": null,
- "align_self": null,
- "border": null,
- "bottom": null,
- "display": null,
- "flex": null,
- "flex_flow": null,
- "grid_area": null,
- "grid_auto_columns": null,
- "grid_auto_flow": null,
- "grid_auto_rows": null,
- "grid_column": null,
- "grid_gap": null,
- "grid_row": null,
- "grid_template_areas": null,
- "grid_template_columns": null,
- "grid_template_rows": null,
- "height": null,
- "justify_content": null,
- "justify_items": null,
- "left": null,
- "margin": null,
- "max_height": null,
- "max_width": null,
- "min_height": null,
- "min_width": null,
- "object_fit": null,
- "object_position": null,
- "order": null,
- "overflow": null,
- "overflow_x": null,
- "overflow_y": null,
- "padding": null,
- "right": null,
- "top": null,
- "visibility": null,
- "width": null
- }
- },
- "b3cb567ec4934a6d935de606af921a61": {
- "model_module": "@jupyter-widgets/controls",
- "model_module_version": "1.5.0",
- "model_name": "ProgressStyleModel",
- "state": {
- "_model_module": "@jupyter-widgets/controls",
- "_model_module_version": "1.5.0",
- "_model_name": "ProgressStyleModel",
- "_view_count": null,
- "_view_module": "@jupyter-widgets/base",
- "_view_module_version": "1.2.0",
- "_view_name": "StyleView",
- "bar_color": null,
- "description_width": ""
- }
- },
- "b9f6e9e50e224cf89b902e419dae269a": {
- "model_module": "@jupyter-widgets/controls",
- "model_module_version": "1.5.0",
- "model_name": "FloatProgressModel",
- "state": {
- "_dom_classes": [],
- "_model_module": "@jupyter-widgets/controls",
- "_model_module_version": "1.5.0",
- "_model_name": "FloatProgressModel",
- "_view_count": null,
- "_view_module": "@jupyter-widgets/controls",
- "_view_module_version": "1.5.0",
- "_view_name": "ProgressView",
- "bar_style": "success",
- "description": "",
- "description_tooltip": null,
- "layout": "IPY_MODEL_a7b1f4318e3242e591b15147d699c656",
- "max": 43727,
- "min": 0,
- "orientation": "horizontal",
- "style": "IPY_MODEL_7d7dd120240043c1aec77af517bf7add",
- "value": 43727
- }
- },
- "bb120e29c7fb49e4823daf66f048e95b": {
- "model_module": "@jupyter-widgets/controls",
- "model_module_version": "1.5.0",
- "model_name": "HTMLModel",
- "state": {
- "_dom_classes": [],
- "_model_module": "@jupyter-widgets/controls",
- "_model_module_version": "1.5.0",
- "_model_name": "HTMLModel",
- "_view_count": null,
- "_view_module": "@jupyter-widgets/controls",
- "_view_module_version": "1.5.0",
- "_view_name": "HTMLView",
- "description": "",
- "description_tooltip": null,
- "layout": "IPY_MODEL_d102def58f554d48890008185d17af96",
- "placeholder": "",
- "style": "IPY_MODEL_79a852590a0444bebad62f2c679e72c6",
- "value": "configuration_cohere.py: 100%"
- }
- },
- "c0c90d8f84b64a11a48e81ba8dec7044": {
- "model_module": "@jupyter-widgets/base",
- "model_module_version": "1.2.0",
- "model_name": "LayoutModel",
- "state": {
- "_model_module": "@jupyter-widgets/base",
- "_model_module_version": "1.2.0",
- "_model_name": "LayoutModel",
- "_view_count": null,
- "_view_module": "@jupyter-widgets/base",
- "_view_module_version": "1.2.0",
- "_view_name": "LayoutView",
- "align_content": null,
- "align_items": null,
- "align_self": null,
- "border": null,
- "bottom": null,
- "display": null,
- "flex": null,
- "flex_flow": null,
- "grid_area": null,
- "grid_auto_columns": null,
- "grid_auto_flow": null,
- "grid_auto_rows": null,
- "grid_column": null,
- "grid_gap": null,
- "grid_row": null,
- "grid_template_areas": null,
- "grid_template_columns": null,
- "grid_template_rows": null,
- "height": null,
- "justify_content": null,
- "justify_items": null,
- "left": null,
- "margin": null,
- "max_height": null,
- "max_width": null,
- "min_height": null,
- "min_width": null,
- "object_fit": null,
- "object_position": null,
- "order": null,
- "overflow": null,
- "overflow_x": null,
- "overflow_y": null,
- "padding": null,
- "right": null,
- "top": null,
- "visibility": null,
- "width": null
- }
- },
- "c8330e6698cd41868efc23be3962db7d": {
- "model_module": "@jupyter-widgets/controls",
- "model_module_version": "1.5.0",
- "model_name": "DescriptionStyleModel",
- "state": {
- "_model_module": "@jupyter-widgets/controls",
- "_model_module_version": "1.5.0",
- "_model_name": "DescriptionStyleModel",
- "_view_count": null,
- "_view_module": "@jupyter-widgets/base",
- "_view_module_version": "1.2.0",
- "_view_name": "StyleView",
- "description_width": ""
- }
- },
- "cf03a6a84e1345aab6d2d78f9ae5f34a": {
- "model_module": "@jupyter-widgets/controls",
- "model_module_version": "1.5.0",
- "model_name": "HTMLModel",
- "state": {
- "_dom_classes": [],
- "_model_module": "@jupyter-widgets/controls",
- "_model_module_version": "1.5.0",
- "_model_name": "HTMLModel",
- "_view_count": null,
- "_view_module": "@jupyter-widgets/controls",
- "_view_module_version": "1.5.0",
- "_view_name": "HTMLView",
- "description": "",
- "description_tooltip": null,
- "layout": "IPY_MODEL_af63d2b1e22c40629b2fe585813a2bf4",
- "placeholder": "",
- "style": "IPY_MODEL_87d5a00cc4a2460e836ddd22cff918dd",
- "value": " 12.8M/12.8M [00:00<00:00, 51.7MB/s]"
- }
- },
- "d102def58f554d48890008185d17af96": {
- "model_module": "@jupyter-widgets/base",
- "model_module_version": "1.2.0",
- "model_name": "LayoutModel",
- "state": {
- "_model_module": "@jupyter-widgets/base",
- "_model_module_version": "1.2.0",
- "_model_name": "LayoutModel",
- "_view_count": null,
- "_view_module": "@jupyter-widgets/base",
- "_view_module_version": "1.2.0",
- "_view_name": "LayoutView",
- "align_content": null,
- "align_items": null,
- "align_self": null,
- "border": null,
- "bottom": null,
- "display": null,
- "flex": null,
- "flex_flow": null,
- "grid_area": null,
- "grid_auto_columns": null,
- "grid_auto_flow": null,
- "grid_auto_rows": null,
- "grid_column": null,
- "grid_gap": null,
- "grid_row": null,
- "grid_template_areas": null,
- "grid_template_columns": null,
- "grid_template_rows": null,
- "height": null,
- "justify_content": null,
- "justify_items": null,
- "left": null,
- "margin": null,
- "max_height": null,
- "max_width": null,
- "min_height": null,
- "min_width": null,
- "object_fit": null,
- "object_position": null,
- "order": null,
- "overflow": null,
- "overflow_x": null,
- "overflow_y": null,
- "padding": null,
- "right": null,
- "top": null,
- "visibility": null,
- "width": null
- }
- },
- "e2146d738e0d4fe39af19bfb22da2584": {
- "model_module": "@jupyter-widgets/controls",
- "model_module_version": "1.5.0",
- "model_name": "HBoxModel",
- "state": {
- "_dom_classes": [],
- "_model_module": "@jupyter-widgets/controls",
- "_model_module_version": "1.5.0",
- "_model_name": "HBoxModel",
- "_view_count": null,
- "_view_module": "@jupyter-widgets/controls",
- "_view_module_version": "1.5.0",
- "_view_name": "HBoxView",
- "box_style": "",
- "children": [
- "IPY_MODEL_8670211888514256a54240d002135917",
- "IPY_MODEL_88b9a3c1bc78462ea149b94d1ca08e59",
- "IPY_MODEL_5a94eb68326b4466b7c19ad67f8f5ba6"
- ],
- "layout": "IPY_MODEL_ab441cef6118450f9bfbcc29f8f34d4f"
- }
- },
- "efb2bf269c1d4ace81ff4a6fabbfb3b6": {
- "model_module": "@jupyter-widgets/base",
- "model_module_version": "1.2.0",
- "model_name": "LayoutModel",
- "state": {
- "_model_module": "@jupyter-widgets/base",
- "_model_module_version": "1.2.0",
- "_model_name": "LayoutModel",
- "_view_count": null,
- "_view_module": "@jupyter-widgets/base",
- "_view_module_version": "1.2.0",
- "_view_name": "LayoutView",
- "align_content": null,
- "align_items": null,
- "align_self": null,
- "border": null,
- "bottom": null,
- "display": null,
- "flex": null,
- "flex_flow": null,
- "grid_area": null,
- "grid_auto_columns": null,
- "grid_auto_flow": null,
- "grid_auto_rows": null,
- "grid_column": null,
- "grid_gap": null,
- "grid_row": null,
- "grid_template_areas": null,
- "grid_template_columns": null,
- "grid_template_rows": null,
- "height": null,
- "justify_content": null,
- "justify_items": null,
- "left": null,
- "margin": null,
- "max_height": null,
- "max_width": null,
- "min_height": null,
- "min_width": null,
- "object_fit": null,
- "object_position": null,
- "order": null,
- "overflow": null,
- "overflow_x": null,
- "overflow_y": null,
- "padding": null,
- "right": null,
- "top": null,
- "visibility": null,
- "width": null
- }
- },
- "f8e3c447794448529391f9660d419632": {
- "model_module": "@jupyter-widgets/controls",
- "model_module_version": "1.5.0",
- "model_name": "DescriptionStyleModel",
- "state": {
- "_model_module": "@jupyter-widgets/controls",
- "_model_module_version": "1.5.0",
- "_model_name": "DescriptionStyleModel",
- "_view_count": null,
- "_view_module": "@jupyter-widgets/base",
- "_view_module_version": "1.2.0",
- "_view_name": "StyleView",
- "description_width": ""
- }
- },
- "fc96c077774241a58d24ac42dd52df7e": {
- "model_module": "@jupyter-widgets/controls",
- "model_module_version": "1.5.0",
- "model_name": "DescriptionStyleModel",
- "state": {
- "_model_module": "@jupyter-widgets/controls",
- "_model_module_version": "1.5.0",
- "_model_name": "DescriptionStyleModel",
- "_view_count": null,
- "_view_module": "@jupyter-widgets/base",
- "_view_module_version": "1.2.0",
- "_view_name": "StyleView",
- "description_width": ""
- }
- }
- }
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "22.7 s ± 7.43 s per loop (mean ± std. dev. of 7 runs, 1 loop each)\n"
+ ]
}
+ ],
+ "source": [
+ "%timeit get_response(full_context_prompt, rag=False)"
+ ]
+ }
+ ],
+ "metadata": {
+ "colab": {
+ "provenance": []
+ },
+ "kernelspec": {
+ "display_name": "Python 3 (ipykernel)",
+ "language": "python",
+ "name": "python3"
},
- "nbformat": 4,
- "nbformat_minor": 0
+ "language_info": {
+ "codemirror_mode": {
+ "name": "ipython",
+ "version": 3
+ },
+ "file_extension": ".py",
+ "mimetype": "text/x-python",
+ "name": "python",
+ "nbconvert_exporter": "python",
+ "pygments_lexer": "ipython3",
+ "version": "3.8.16"
+ },
+ "widgets": {
+ "application/vnd.jupyter.widget-state+json": {
+ "012491a19a8143f2adc1a95c5d20e488": {
+ "model_module": "@jupyter-widgets/controls",
+ "model_module_version": "1.5.0",
+ "model_name": "FloatProgressModel",
+ "state": {
+ "_dom_classes": [],
+ "_model_module": "@jupyter-widgets/controls",
+ "_model_module_version": "1.5.0",
+ "_model_name": "FloatProgressModel",
+ "_view_count": null,
+ "_view_module": "@jupyter-widgets/controls",
+ "_view_module_version": "1.5.0",
+ "_view_name": "ProgressView",
+ "bar_style": "success",
+ "description": "",
+ "description_tooltip": null,
+ "layout": "IPY_MODEL_c0c90d8f84b64a11a48e81ba8dec7044",
+ "max": 7366,
+ "min": 0,
+ "orientation": "horizontal",
+ "style": "IPY_MODEL_1167e72dd2184c6ca8c670b27c73a27d",
+ "value": 7366
+ }
+ },
+ "016a793bd7684acc9733f1e3160bd4d6": {
+ "model_module": "@jupyter-widgets/base",
+ "model_module_version": "1.2.0",
+ "model_name": "LayoutModel",
+ "state": {
+ "_model_module": "@jupyter-widgets/base",
+ "_model_module_version": "1.2.0",
+ "_model_name": "LayoutModel",
+ "_view_count": null,
+ "_view_module": "@jupyter-widgets/base",
+ "_view_module_version": "1.2.0",
+ "_view_name": "LayoutView",
+ "align_content": null,
+ "align_items": null,
+ "align_self": null,
+ "border": null,
+ "bottom": null,
+ "display": null,
+ "flex": null,
+ "flex_flow": null,
+ "grid_area": null,
+ "grid_auto_columns": null,
+ "grid_auto_flow": null,
+ "grid_auto_rows": null,
+ "grid_column": null,
+ "grid_gap": null,
+ "grid_row": null,
+ "grid_template_areas": null,
+ "grid_template_columns": null,
+ "grid_template_rows": null,
+ "height": null,
+ "justify_content": null,
+ "justify_items": null,
+ "left": null,
+ "margin": null,
+ "max_height": null,
+ "max_width": null,
+ "min_height": null,
+ "min_width": null,
+ "object_fit": null,
+ "object_position": null,
+ "order": null,
+ "overflow": null,
+ "overflow_x": null,
+ "overflow_y": null,
+ "padding": null,
+ "right": null,
+ "top": null,
+ "visibility": null,
+ "width": null
+ }
+ },
+ "01d81bc2d81741e6bb0a82ba78dc2b21": {
+ "model_module": "@jupyter-widgets/controls",
+ "model_module_version": "1.5.0",
+ "model_name": "HBoxModel",
+ "state": {
+ "_dom_classes": [],
+ "_model_module": "@jupyter-widgets/controls",
+ "_model_module_version": "1.5.0",
+ "_model_name": "HBoxModel",
+ "_view_count": null,
+ "_view_module": "@jupyter-widgets/controls",
+ "_view_module_version": "1.5.0",
+ "_view_name": "HBoxView",
+ "box_style": "",
+ "children": [
+ "IPY_MODEL_76d51ab196f54b398dcdc574968331fa",
+ "IPY_MODEL_b9f6e9e50e224cf89b902e419dae269a",
+ "IPY_MODEL_1271483cc9da41b28637dce1a40fa538"
+ ],
+ "layout": "IPY_MODEL_78a1cb4ec6cc4044b2149cb84f34979c"
+ }
+ },
+ "0e300e8dbea143d688b08c32883d2d29": {
+ "model_module": "@jupyter-widgets/base",
+ "model_module_version": "1.2.0",
+ "model_name": "LayoutModel",
+ "state": {
+ "_model_module": "@jupyter-widgets/base",
+ "_model_module_version": "1.2.0",
+ "_model_name": "LayoutModel",
+ "_view_count": null,
+ "_view_module": "@jupyter-widgets/base",
+ "_view_module_version": "1.2.0",
+ "_view_name": "LayoutView",
+ "align_content": null,
+ "align_items": null,
+ "align_self": null,
+ "border": null,
+ "bottom": null,
+ "display": null,
+ "flex": null,
+ "flex_flow": null,
+ "grid_area": null,
+ "grid_auto_columns": null,
+ "grid_auto_flow": null,
+ "grid_auto_rows": null,
+ "grid_column": null,
+ "grid_gap": null,
+ "grid_row": null,
+ "grid_template_areas": null,
+ "grid_template_columns": null,
+ "grid_template_rows": null,
+ "height": null,
+ "justify_content": null,
+ "justify_items": null,
+ "left": null,
+ "margin": null,
+ "max_height": null,
+ "max_width": null,
+ "min_height": null,
+ "min_width": null,
+ "object_fit": null,
+ "object_position": null,
+ "order": null,
+ "overflow": null,
+ "overflow_x": null,
+ "overflow_y": null,
+ "padding": null,
+ "right": null,
+ "top": null,
+ "visibility": null,
+ "width": null
+ }
+ },
+ "1167e72dd2184c6ca8c670b27c73a27d": {
+ "model_module": "@jupyter-widgets/controls",
+ "model_module_version": "1.5.0",
+ "model_name": "ProgressStyleModel",
+ "state": {
+ "_model_module": "@jupyter-widgets/controls",
+ "_model_module_version": "1.5.0",
+ "_model_name": "ProgressStyleModel",
+ "_view_count": null,
+ "_view_module": "@jupyter-widgets/base",
+ "_view_module_version": "1.2.0",
+ "_view_name": "StyleView",
+ "bar_color": null,
+ "description_width": ""
+ }
+ },
+ "1271483cc9da41b28637dce1a40fa538": {
+ "model_module": "@jupyter-widgets/controls",
+ "model_module_version": "1.5.0",
+ "model_name": "HTMLModel",
+ "state": {
+ "_dom_classes": [],
+ "_model_module": "@jupyter-widgets/controls",
+ "_model_module_version": "1.5.0",
+ "_model_name": "HTMLModel",
+ "_view_count": null,
+ "_view_module": "@jupyter-widgets/controls",
+ "_view_module_version": "1.5.0",
+ "_view_name": "HTMLView",
+ "description": "",
+ "description_tooltip": null,
+ "layout": "IPY_MODEL_204957d62b7a4f7db7fa195a3d042b42",
+ "placeholder": "",
+ "style": "IPY_MODEL_930d287b503b4eeeb527c6facc97ca30",
+ "value": " 43.7k/43.7k [00:00<00:00, 1.97MB/s]"
+ }
+ },
+ "159f81e1ea4c40a394b6d796527c7c4a": {
+ "model_module": "@jupyter-widgets/controls",
+ "model_module_version": "1.5.0",
+ "model_name": "HBoxModel",
+ "state": {
+ "_dom_classes": [],
+ "_model_module": "@jupyter-widgets/controls",
+ "_model_module_version": "1.5.0",
+ "_model_name": "HBoxModel",
+ "_view_count": null,
+ "_view_module": "@jupyter-widgets/controls",
+ "_view_module_version": "1.5.0",
+ "_view_name": "HBoxView",
+ "box_style": "",
+ "children": [
+ "IPY_MODEL_96997c9c8f3a4237b1d025252c3c2358",
+ "IPY_MODEL_28b25e6395c54e8e9494f403953065bb",
+ "IPY_MODEL_cf03a6a84e1345aab6d2d78f9ae5f34a"
+ ],
+ "layout": "IPY_MODEL_5a15923788d74d98a70646e1e921831e"
+ }
+ },
+ "17e8c3567f1a4d6aaa485fd1014977ae": {
+ "model_module": "@jupyter-widgets/base",
+ "model_module_version": "1.2.0",
+ "model_name": "LayoutModel",
+ "state": {
+ "_model_module": "@jupyter-widgets/base",
+ "_model_module_version": "1.2.0",
+ "_model_name": "LayoutModel",
+ "_view_count": null,
+ "_view_module": "@jupyter-widgets/base",
+ "_view_module_version": "1.2.0",
+ "_view_name": "LayoutView",
+ "align_content": null,
+ "align_items": null,
+ "align_self": null,
+ "border": null,
+ "bottom": null,
+ "display": null,
+ "flex": null,
+ "flex_flow": null,
+ "grid_area": null,
+ "grid_auto_columns": null,
+ "grid_auto_flow": null,
+ "grid_auto_rows": null,
+ "grid_column": null,
+ "grid_gap": null,
+ "grid_row": null,
+ "grid_template_areas": null,
+ "grid_template_columns": null,
+ "grid_template_rows": null,
+ "height": null,
+ "justify_content": null,
+ "justify_items": null,
+ "left": null,
+ "margin": null,
+ "max_height": null,
+ "max_width": null,
+ "min_height": null,
+ "min_width": null,
+ "object_fit": null,
+ "object_position": null,
+ "order": null,
+ "overflow": null,
+ "overflow_x": null,
+ "overflow_y": null,
+ "padding": null,
+ "right": null,
+ "top": null,
+ "visibility": null,
+ "width": null
+ }
+ },
+ "204957d62b7a4f7db7fa195a3d042b42": {
+ "model_module": "@jupyter-widgets/base",
+ "model_module_version": "1.2.0",
+ "model_name": "LayoutModel",
+ "state": {
+ "_model_module": "@jupyter-widgets/base",
+ "_model_module_version": "1.2.0",
+ "_model_name": "LayoutModel",
+ "_view_count": null,
+ "_view_module": "@jupyter-widgets/base",
+ "_view_module_version": "1.2.0",
+ "_view_name": "LayoutView",
+ "align_content": null,
+ "align_items": null,
+ "align_self": null,
+ "border": null,
+ "bottom": null,
+ "display": null,
+ "flex": null,
+ "flex_flow": null,
+ "grid_area": null,
+ "grid_auto_columns": null,
+ "grid_auto_flow": null,
+ "grid_auto_rows": null,
+ "grid_column": null,
+ "grid_gap": null,
+ "grid_row": null,
+ "grid_template_areas": null,
+ "grid_template_columns": null,
+ "grid_template_rows": null,
+ "height": null,
+ "justify_content": null,
+ "justify_items": null,
+ "left": null,
+ "margin": null,
+ "max_height": null,
+ "max_width": null,
+ "min_height": null,
+ "min_width": null,
+ "object_fit": null,
+ "object_position": null,
+ "order": null,
+ "overflow": null,
+ "overflow_x": null,
+ "overflow_y": null,
+ "padding": null,
+ "right": null,
+ "top": null,
+ "visibility": null,
+ "width": null
+ }
+ },
+ "21229dbb1f414913b8ad230648ab76e9": {
+ "model_module": "@jupyter-widgets/base",
+ "model_module_version": "1.2.0",
+ "model_name": "LayoutModel",
+ "state": {
+ "_model_module": "@jupyter-widgets/base",
+ "_model_module_version": "1.2.0",
+ "_model_name": "LayoutModel",
+ "_view_count": null,
+ "_view_module": "@jupyter-widgets/base",
+ "_view_module_version": "1.2.0",
+ "_view_name": "LayoutView",
+ "align_content": null,
+ "align_items": null,
+ "align_self": null,
+ "border": null,
+ "bottom": null,
+ "display": null,
+ "flex": null,
+ "flex_flow": null,
+ "grid_area": null,
+ "grid_auto_columns": null,
+ "grid_auto_flow": null,
+ "grid_auto_rows": null,
+ "grid_column": null,
+ "grid_gap": null,
+ "grid_row": null,
+ "grid_template_areas": null,
+ "grid_template_columns": null,
+ "grid_template_rows": null,
+ "height": null,
+ "justify_content": null,
+ "justify_items": null,
+ "left": null,
+ "margin": null,
+ "max_height": null,
+ "max_width": null,
+ "min_height": null,
+ "min_width": null,
+ "object_fit": null,
+ "object_position": null,
+ "order": null,
+ "overflow": null,
+ "overflow_x": null,
+ "overflow_y": null,
+ "padding": null,
+ "right": null,
+ "top": null,
+ "visibility": null,
+ "width": null
+ }
+ },
+ "21c71aa09fae458fa483d58019506d46": {
+ "model_module": "@jupyter-widgets/base",
+ "model_module_version": "1.2.0",
+ "model_name": "LayoutModel",
+ "state": {
+ "_model_module": "@jupyter-widgets/base",
+ "_model_module_version": "1.2.0",
+ "_model_name": "LayoutModel",
+ "_view_count": null,
+ "_view_module": "@jupyter-widgets/base",
+ "_view_module_version": "1.2.0",
+ "_view_name": "LayoutView",
+ "align_content": null,
+ "align_items": null,
+ "align_self": null,
+ "border": null,
+ "bottom": null,
+ "display": null,
+ "flex": null,
+ "flex_flow": null,
+ "grid_area": null,
+ "grid_auto_columns": null,
+ "grid_auto_flow": null,
+ "grid_auto_rows": null,
+ "grid_column": null,
+ "grid_gap": null,
+ "grid_row": null,
+ "grid_template_areas": null,
+ "grid_template_columns": null,
+ "grid_template_rows": null,
+ "height": null,
+ "justify_content": null,
+ "justify_items": null,
+ "left": null,
+ "margin": null,
+ "max_height": null,
+ "max_width": null,
+ "min_height": null,
+ "min_width": null,
+ "object_fit": null,
+ "object_position": null,
+ "order": null,
+ "overflow": null,
+ "overflow_x": null,
+ "overflow_y": null,
+ "padding": null,
+ "right": null,
+ "top": null,
+ "visibility": null,
+ "width": null
+ }
+ },
+ "24055747d0e44bf29f80017117e332cc": {
+ "model_module": "@jupyter-widgets/controls",
+ "model_module_version": "1.5.0",
+ "model_name": "ProgressStyleModel",
+ "state": {
+ "_model_module": "@jupyter-widgets/controls",
+ "_model_module_version": "1.5.0",
+ "_model_name": "ProgressStyleModel",
+ "_view_count": null,
+ "_view_module": "@jupyter-widgets/base",
+ "_view_module_version": "1.2.0",
+ "_view_name": "StyleView",
+ "bar_color": null,
+ "description_width": ""
+ }
+ },
+ "28b25e6395c54e8e9494f403953065bb": {
+ "model_module": "@jupyter-widgets/controls",
+ "model_module_version": "1.5.0",
+ "model_name": "FloatProgressModel",
+ "state": {
+ "_dom_classes": [],
+ "_model_module": "@jupyter-widgets/controls",
+ "_model_module_version": "1.5.0",
+ "_model_name": "FloatProgressModel",
+ "_view_count": null,
+ "_view_module": "@jupyter-widgets/controls",
+ "_view_module_version": "1.5.0",
+ "_view_name": "ProgressView",
+ "bar_style": "success",
+ "description": "",
+ "description_tooltip": null,
+ "layout": "IPY_MODEL_21c71aa09fae458fa483d58019506d46",
+ "max": 12777406,
+ "min": 0,
+ "orientation": "horizontal",
+ "style": "IPY_MODEL_b3cb567ec4934a6d935de606af921a61",
+ "value": 12777406
+ }
+ },
+ "2a9c8f7bb54e4814825bddcbb10b3482": {
+ "model_module": "@jupyter-widgets/base",
+ "model_module_version": "1.2.0",
+ "model_name": "LayoutModel",
+ "state": {
+ "_model_module": "@jupyter-widgets/base",
+ "_model_module_version": "1.2.0",
+ "_model_name": "LayoutModel",
+ "_view_count": null,
+ "_view_module": "@jupyter-widgets/base",
+ "_view_module_version": "1.2.0",
+ "_view_name": "LayoutView",
+ "align_content": null,
+ "align_items": null,
+ "align_self": null,
+ "border": null,
+ "bottom": null,
+ "display": null,
+ "flex": null,
+ "flex_flow": null,
+ "grid_area": null,
+ "grid_auto_columns": null,
+ "grid_auto_flow": null,
+ "grid_auto_rows": null,
+ "grid_column": null,
+ "grid_gap": null,
+ "grid_row": null,
+ "grid_template_areas": null,
+ "grid_template_columns": null,
+ "grid_template_rows": null,
+ "height": null,
+ "justify_content": null,
+ "justify_items": null,
+ "left": null,
+ "margin": null,
+ "max_height": null,
+ "max_width": null,
+ "min_height": null,
+ "min_width": null,
+ "object_fit": null,
+ "object_position": null,
+ "order": null,
+ "overflow": null,
+ "overflow_x": null,
+ "overflow_y": null,
+ "padding": null,
+ "right": null,
+ "top": null,
+ "visibility": null,
+ "width": null
+ }
+ },
+ "2e094e1165554834a0cc19ebffe93311": {
+ "model_module": "@jupyter-widgets/controls",
+ "model_module_version": "1.5.0",
+ "model_name": "DescriptionStyleModel",
+ "state": {
+ "_model_module": "@jupyter-widgets/controls",
+ "_model_module_version": "1.5.0",
+ "_model_name": "DescriptionStyleModel",
+ "_view_count": null,
+ "_view_module": "@jupyter-widgets/base",
+ "_view_module_version": "1.2.0",
+ "_view_name": "StyleView",
+ "description_width": ""
+ }
+ },
+ "403ebbf8e6da4c188c42e83ee01db07b": {
+ "model_module": "@jupyter-widgets/base",
+ "model_module_version": "1.2.0",
+ "model_name": "LayoutModel",
+ "state": {
+ "_model_module": "@jupyter-widgets/base",
+ "_model_module_version": "1.2.0",
+ "_model_name": "LayoutModel",
+ "_view_count": null,
+ "_view_module": "@jupyter-widgets/base",
+ "_view_module_version": "1.2.0",
+ "_view_name": "LayoutView",
+ "align_content": null,
+ "align_items": null,
+ "align_self": null,
+ "border": null,
+ "bottom": null,
+ "display": null,
+ "flex": null,
+ "flex_flow": null,
+ "grid_area": null,
+ "grid_auto_columns": null,
+ "grid_auto_flow": null,
+ "grid_auto_rows": null,
+ "grid_column": null,
+ "grid_gap": null,
+ "grid_row": null,
+ "grid_template_areas": null,
+ "grid_template_columns": null,
+ "grid_template_rows": null,
+ "height": null,
+ "justify_content": null,
+ "justify_items": null,
+ "left": null,
+ "margin": null,
+ "max_height": null,
+ "max_width": null,
+ "min_height": null,
+ "min_width": null,
+ "object_fit": null,
+ "object_position": null,
+ "order": null,
+ "overflow": null,
+ "overflow_x": null,
+ "overflow_y": null,
+ "padding": null,
+ "right": null,
+ "top": null,
+ "visibility": null,
+ "width": null
+ }
+ },
+ "41b58953474f49f0915d14801fb18174": {
+ "model_module": "@jupyter-widgets/controls",
+ "model_module_version": "1.5.0",
+ "model_name": "HTMLModel",
+ "state": {
+ "_dom_classes": [],
+ "_model_module": "@jupyter-widgets/controls",
+ "_model_module_version": "1.5.0",
+ "_model_name": "HTMLModel",
+ "_view_count": null,
+ "_view_module": "@jupyter-widgets/controls",
+ "_view_module_version": "1.5.0",
+ "_view_name": "HTMLView",
+ "description": "",
+ "description_tooltip": null,
+ "layout": "IPY_MODEL_efb2bf269c1d4ace81ff4a6fabbfb3b6",
+ "placeholder": "",
+ "style": "IPY_MODEL_fc96c077774241a58d24ac42dd52df7e",
+ "value": " 429/429 [00:00<00:00, 20.7kB/s]"
+ }
+ },
+ "56cd150b53234f53b28dec80ebeb33a8": {
+ "model_module": "@jupyter-widgets/controls",
+ "model_module_version": "1.5.0",
+ "model_name": "DescriptionStyleModel",
+ "state": {
+ "_model_module": "@jupyter-widgets/controls",
+ "_model_module_version": "1.5.0",
+ "_model_name": "DescriptionStyleModel",
+ "_view_count": null,
+ "_view_module": "@jupyter-widgets/base",
+ "_view_module_version": "1.2.0",
+ "_view_name": "StyleView",
+ "description_width": ""
+ }
+ },
+ "59f4d94a1be24ef7a1366b7220a9c9de": {
+ "model_module": "@jupyter-widgets/base",
+ "model_module_version": "1.2.0",
+ "model_name": "LayoutModel",
+ "state": {
+ "_model_module": "@jupyter-widgets/base",
+ "_model_module_version": "1.2.0",
+ "_model_name": "LayoutModel",
+ "_view_count": null,
+ "_view_module": "@jupyter-widgets/base",
+ "_view_module_version": "1.2.0",
+ "_view_name": "LayoutView",
+ "align_content": null,
+ "align_items": null,
+ "align_self": null,
+ "border": null,
+ "bottom": null,
+ "display": null,
+ "flex": null,
+ "flex_flow": null,
+ "grid_area": null,
+ "grid_auto_columns": null,
+ "grid_auto_flow": null,
+ "grid_auto_rows": null,
+ "grid_column": null,
+ "grid_gap": null,
+ "grid_row": null,
+ "grid_template_areas": null,
+ "grid_template_columns": null,
+ "grid_template_rows": null,
+ "height": null,
+ "justify_content": null,
+ "justify_items": null,
+ "left": null,
+ "margin": null,
+ "max_height": null,
+ "max_width": null,
+ "min_height": null,
+ "min_width": null,
+ "object_fit": null,
+ "object_position": null,
+ "order": null,
+ "overflow": null,
+ "overflow_x": null,
+ "overflow_y": null,
+ "padding": null,
+ "right": null,
+ "top": null,
+ "visibility": null,
+ "width": null
+ }
+ },
+ "5a15923788d74d98a70646e1e921831e": {
+ "model_module": "@jupyter-widgets/base",
+ "model_module_version": "1.2.0",
+ "model_name": "LayoutModel",
+ "state": {
+ "_model_module": "@jupyter-widgets/base",
+ "_model_module_version": "1.2.0",
+ "_model_name": "LayoutModel",
+ "_view_count": null,
+ "_view_module": "@jupyter-widgets/base",
+ "_view_module_version": "1.2.0",
+ "_view_name": "LayoutView",
+ "align_content": null,
+ "align_items": null,
+ "align_self": null,
+ "border": null,
+ "bottom": null,
+ "display": null,
+ "flex": null,
+ "flex_flow": null,
+ "grid_area": null,
+ "grid_auto_columns": null,
+ "grid_auto_flow": null,
+ "grid_auto_rows": null,
+ "grid_column": null,
+ "grid_gap": null,
+ "grid_row": null,
+ "grid_template_areas": null,
+ "grid_template_columns": null,
+ "grid_template_rows": null,
+ "height": null,
+ "justify_content": null,
+ "justify_items": null,
+ "left": null,
+ "margin": null,
+ "max_height": null,
+ "max_width": null,
+ "min_height": null,
+ "min_width": null,
+ "object_fit": null,
+ "object_position": null,
+ "order": null,
+ "overflow": null,
+ "overflow_x": null,
+ "overflow_y": null,
+ "padding": null,
+ "right": null,
+ "top": null,
+ "visibility": null,
+ "width": null
+ }
+ },
+ "5a94eb68326b4466b7c19ad67f8f5ba6": {
+ "model_module": "@jupyter-widgets/controls",
+ "model_module_version": "1.5.0",
+ "model_name": "HTMLModel",
+ "state": {
+ "_dom_classes": [],
+ "_model_module": "@jupyter-widgets/controls",
+ "_model_module_version": "1.5.0",
+ "_model_name": "HTMLModel",
+ "_view_count": null,
+ "_view_module": "@jupyter-widgets/controls",
+ "_view_module_version": "1.5.0",
+ "_view_name": "HTMLView",
+ "description": "",
+ "description_tooltip": null,
+ "layout": "IPY_MODEL_59f4d94a1be24ef7a1366b7220a9c9de",
+ "placeholder": "",
+ "style": "IPY_MODEL_6ba07312eb854ae9a88e6cfb886b65c3",
+ "value": " 7.92k/7.92k [00:00<00:00, 366kB/s]"
+ }
+ },
+ "5f5e73e41dae4625a98a6eee6120a7d4": {
+ "model_module": "@jupyter-widgets/controls",
+ "model_module_version": "1.5.0",
+ "model_name": "ProgressStyleModel",
+ "state": {
+ "_model_module": "@jupyter-widgets/controls",
+ "_model_module_version": "1.5.0",
+ "_model_name": "ProgressStyleModel",
+ "_view_count": null,
+ "_view_module": "@jupyter-widgets/base",
+ "_view_module_version": "1.2.0",
+ "_view_name": "StyleView",
+ "bar_color": null,
+ "description_width": ""
+ }
+ },
+ "6643b53bdbe74a6491a3db6ec06446e9": {
+ "model_module": "@jupyter-widgets/controls",
+ "model_module_version": "1.5.0",
+ "model_name": "HTMLModel",
+ "state": {
+ "_dom_classes": [],
+ "_model_module": "@jupyter-widgets/controls",
+ "_model_module_version": "1.5.0",
+ "_model_name": "HTMLModel",
+ "_view_count": null,
+ "_view_module": "@jupyter-widgets/controls",
+ "_view_module_version": "1.5.0",
+ "_view_name": "HTMLView",
+ "description": "",
+ "description_tooltip": null,
+ "layout": "IPY_MODEL_0e300e8dbea143d688b08c32883d2d29",
+ "placeholder": "",
+ "style": "IPY_MODEL_2e094e1165554834a0cc19ebffe93311",
+ "value": "special_tokens_map.json: 100%"
+ }
+ },
+ "6ba07312eb854ae9a88e6cfb886b65c3": {
+ "model_module": "@jupyter-widgets/controls",
+ "model_module_version": "1.5.0",
+ "model_name": "DescriptionStyleModel",
+ "state": {
+ "_model_module": "@jupyter-widgets/controls",
+ "_model_module_version": "1.5.0",
+ "_model_name": "DescriptionStyleModel",
+ "_view_count": null,
+ "_view_module": "@jupyter-widgets/base",
+ "_view_module_version": "1.2.0",
+ "_view_name": "StyleView",
+ "description_width": ""
+ }
+ },
+ "76d51ab196f54b398dcdc574968331fa": {
+ "model_module": "@jupyter-widgets/controls",
+ "model_module_version": "1.5.0",
+ "model_name": "HTMLModel",
+ "state": {
+ "_dom_classes": [],
+ "_model_module": "@jupyter-widgets/controls",
+ "_model_module_version": "1.5.0",
+ "_model_name": "HTMLModel",
+ "_view_count": null,
+ "_view_module": "@jupyter-widgets/controls",
+ "_view_module_version": "1.5.0",
+ "_view_name": "HTMLView",
+ "description": "",
+ "description_tooltip": null,
+ "layout": "IPY_MODEL_403ebbf8e6da4c188c42e83ee01db07b",
+ "placeholder": "",
+ "style": "IPY_MODEL_f8e3c447794448529391f9660d419632",
+ "value": "tokenization_cohere_fast.py: 100%"
+ }
+ },
+ "782b11c627be41b2b67e2774ee7fcf0b": {
+ "model_module": "@jupyter-widgets/controls",
+ "model_module_version": "1.5.0",
+ "model_name": "DescriptionStyleModel",
+ "state": {
+ "_model_module": "@jupyter-widgets/controls",
+ "_model_module_version": "1.5.0",
+ "_model_name": "DescriptionStyleModel",
+ "_view_count": null,
+ "_view_module": "@jupyter-widgets/base",
+ "_view_module_version": "1.2.0",
+ "_view_name": "StyleView",
+ "description_width": ""
+ }
+ },
+ "78a1cb4ec6cc4044b2149cb84f34979c": {
+ "model_module": "@jupyter-widgets/base",
+ "model_module_version": "1.2.0",
+ "model_name": "LayoutModel",
+ "state": {
+ "_model_module": "@jupyter-widgets/base",
+ "_model_module_version": "1.2.0",
+ "_model_name": "LayoutModel",
+ "_view_count": null,
+ "_view_module": "@jupyter-widgets/base",
+ "_view_module_version": "1.2.0",
+ "_view_name": "LayoutView",
+ "align_content": null,
+ "align_items": null,
+ "align_self": null,
+ "border": null,
+ "bottom": null,
+ "display": null,
+ "flex": null,
+ "flex_flow": null,
+ "grid_area": null,
+ "grid_auto_columns": null,
+ "grid_auto_flow": null,
+ "grid_auto_rows": null,
+ "grid_column": null,
+ "grid_gap": null,
+ "grid_row": null,
+ "grid_template_areas": null,
+ "grid_template_columns": null,
+ "grid_template_rows": null,
+ "height": null,
+ "justify_content": null,
+ "justify_items": null,
+ "left": null,
+ "margin": null,
+ "max_height": null,
+ "max_width": null,
+ "min_height": null,
+ "min_width": null,
+ "object_fit": null,
+ "object_position": null,
+ "order": null,
+ "overflow": null,
+ "overflow_x": null,
+ "overflow_y": null,
+ "padding": null,
+ "right": null,
+ "top": null,
+ "visibility": null,
+ "width": null
+ }
+ },
+ "79a852590a0444bebad62f2c679e72c6": {
+ "model_module": "@jupyter-widgets/controls",
+ "model_module_version": "1.5.0",
+ "model_name": "DescriptionStyleModel",
+ "state": {
+ "_model_module": "@jupyter-widgets/controls",
+ "_model_module_version": "1.5.0",
+ "_model_name": "DescriptionStyleModel",
+ "_view_count": null,
+ "_view_module": "@jupyter-widgets/base",
+ "_view_module_version": "1.2.0",
+ "_view_name": "StyleView",
+ "description_width": ""
+ }
+ },
+ "7d7dd120240043c1aec77af517bf7add": {
+ "model_module": "@jupyter-widgets/controls",
+ "model_module_version": "1.5.0",
+ "model_name": "ProgressStyleModel",
+ "state": {
+ "_model_module": "@jupyter-widgets/controls",
+ "_model_module_version": "1.5.0",
+ "_model_name": "ProgressStyleModel",
+ "_view_count": null,
+ "_view_module": "@jupyter-widgets/base",
+ "_view_module_version": "1.2.0",
+ "_view_name": "StyleView",
+ "bar_color": null,
+ "description_width": ""
+ }
+ },
+ "7e9939b2a8394aa3ae4dd1a90d9517ce": {
+ "model_module": "@jupyter-widgets/base",
+ "model_module_version": "1.2.0",
+ "model_name": "LayoutModel",
+ "state": {
+ "_model_module": "@jupyter-widgets/base",
+ "_model_module_version": "1.2.0",
+ "_model_name": "LayoutModel",
+ "_view_count": null,
+ "_view_module": "@jupyter-widgets/base",
+ "_view_module_version": "1.2.0",
+ "_view_name": "LayoutView",
+ "align_content": null,
+ "align_items": null,
+ "align_self": null,
+ "border": null,
+ "bottom": null,
+ "display": null,
+ "flex": null,
+ "flex_flow": null,
+ "grid_area": null,
+ "grid_auto_columns": null,
+ "grid_auto_flow": null,
+ "grid_auto_rows": null,
+ "grid_column": null,
+ "grid_gap": null,
+ "grid_row": null,
+ "grid_template_areas": null,
+ "grid_template_columns": null,
+ "grid_template_rows": null,
+ "height": null,
+ "justify_content": null,
+ "justify_items": null,
+ "left": null,
+ "margin": null,
+ "max_height": null,
+ "max_width": null,
+ "min_height": null,
+ "min_width": null,
+ "object_fit": null,
+ "object_position": null,
+ "order": null,
+ "overflow": null,
+ "overflow_x": null,
+ "overflow_y": null,
+ "padding": null,
+ "right": null,
+ "top": null,
+ "visibility": null,
+ "width": null
+ }
+ },
+ "81d523a8f29c491aaed45a199260b414": {
+ "model_module": "@jupyter-widgets/base",
+ "model_module_version": "1.2.0",
+ "model_name": "LayoutModel",
+ "state": {
+ "_model_module": "@jupyter-widgets/base",
+ "_model_module_version": "1.2.0",
+ "_model_name": "LayoutModel",
+ "_view_count": null,
+ "_view_module": "@jupyter-widgets/base",
+ "_view_module_version": "1.2.0",
+ "_view_name": "LayoutView",
+ "align_content": null,
+ "align_items": null,
+ "align_self": null,
+ "border": null,
+ "bottom": null,
+ "display": null,
+ "flex": null,
+ "flex_flow": null,
+ "grid_area": null,
+ "grid_auto_columns": null,
+ "grid_auto_flow": null,
+ "grid_auto_rows": null,
+ "grid_column": null,
+ "grid_gap": null,
+ "grid_row": null,
+ "grid_template_areas": null,
+ "grid_template_columns": null,
+ "grid_template_rows": null,
+ "height": null,
+ "justify_content": null,
+ "justify_items": null,
+ "left": null,
+ "margin": null,
+ "max_height": null,
+ "max_width": null,
+ "min_height": null,
+ "min_width": null,
+ "object_fit": null,
+ "object_position": null,
+ "order": null,
+ "overflow": null,
+ "overflow_x": null,
+ "overflow_y": null,
+ "padding": null,
+ "right": null,
+ "top": null,
+ "visibility": null,
+ "width": null
+ }
+ },
+ "8670211888514256a54240d002135917": {
+ "model_module": "@jupyter-widgets/controls",
+ "model_module_version": "1.5.0",
+ "model_name": "HTMLModel",
+ "state": {
+ "_dom_classes": [],
+ "_model_module": "@jupyter-widgets/controls",
+ "_model_module_version": "1.5.0",
+ "_model_name": "HTMLModel",
+ "_view_count": null,
+ "_view_module": "@jupyter-widgets/controls",
+ "_view_module_version": "1.5.0",
+ "_view_name": "HTMLView",
+ "description": "",
+ "description_tooltip": null,
+ "layout": "IPY_MODEL_016a793bd7684acc9733f1e3160bd4d6",
+ "placeholder": "",
+ "style": "IPY_MODEL_c8330e6698cd41868efc23be3962db7d",
+ "value": "tokenizer_config.json: 100%"
+ }
+ },
+ "87d5a00cc4a2460e836ddd22cff918dd": {
+ "model_module": "@jupyter-widgets/controls",
+ "model_module_version": "1.5.0",
+ "model_name": "DescriptionStyleModel",
+ "state": {
+ "_model_module": "@jupyter-widgets/controls",
+ "_model_module_version": "1.5.0",
+ "_model_name": "DescriptionStyleModel",
+ "_view_count": null,
+ "_view_module": "@jupyter-widgets/base",
+ "_view_module_version": "1.2.0",
+ "_view_name": "StyleView",
+ "description_width": ""
+ }
+ },
+ "88b9a3c1bc78462ea149b94d1ca08e59": {
+ "model_module": "@jupyter-widgets/controls",
+ "model_module_version": "1.5.0",
+ "model_name": "FloatProgressModel",
+ "state": {
+ "_dom_classes": [],
+ "_model_module": "@jupyter-widgets/controls",
+ "_model_module_version": "1.5.0",
+ "_model_name": "FloatProgressModel",
+ "_view_count": null,
+ "_view_module": "@jupyter-widgets/controls",
+ "_view_module_version": "1.5.0",
+ "_view_name": "ProgressView",
+ "bar_style": "success",
+ "description": "",
+ "description_tooltip": null,
+ "layout": "IPY_MODEL_7e9939b2a8394aa3ae4dd1a90d9517ce",
+ "max": 7916,
+ "min": 0,
+ "orientation": "horizontal",
+ "style": "IPY_MODEL_24055747d0e44bf29f80017117e332cc",
+ "value": 7916
+ }
+ },
+ "88fc0e784a71447abbba8273f6fcdace": {
+ "model_module": "@jupyter-widgets/controls",
+ "model_module_version": "1.5.0",
+ "model_name": "HBoxModel",
+ "state": {
+ "_dom_classes": [],
+ "_model_module": "@jupyter-widgets/controls",
+ "_model_module_version": "1.5.0",
+ "_model_name": "HBoxModel",
+ "_view_count": null,
+ "_view_module": "@jupyter-widgets/controls",
+ "_view_module_version": "1.5.0",
+ "_view_name": "HBoxView",
+ "box_style": "",
+ "children": [
+ "IPY_MODEL_bb120e29c7fb49e4823daf66f048e95b",
+ "IPY_MODEL_012491a19a8143f2adc1a95c5d20e488",
+ "IPY_MODEL_a9cd4375dfd94af4ab2c004e9dbe6fa7"
+ ],
+ "layout": "IPY_MODEL_21229dbb1f414913b8ad230648ab76e9"
+ }
+ },
+ "930d287b503b4eeeb527c6facc97ca30": {
+ "model_module": "@jupyter-widgets/controls",
+ "model_module_version": "1.5.0",
+ "model_name": "DescriptionStyleModel",
+ "state": {
+ "_model_module": "@jupyter-widgets/controls",
+ "_model_module_version": "1.5.0",
+ "_model_name": "DescriptionStyleModel",
+ "_view_count": null,
+ "_view_module": "@jupyter-widgets/base",
+ "_view_module_version": "1.2.0",
+ "_view_name": "StyleView",
+ "description_width": ""
+ }
+ },
+ "93d9e20a78774bb485ceca31d4453204": {
+ "model_module": "@jupyter-widgets/controls",
+ "model_module_version": "1.5.0",
+ "model_name": "FloatProgressModel",
+ "state": {
+ "_dom_classes": [],
+ "_model_module": "@jupyter-widgets/controls",
+ "_model_module_version": "1.5.0",
+ "_model_name": "FloatProgressModel",
+ "_view_count": null,
+ "_view_module": "@jupyter-widgets/controls",
+ "_view_module_version": "1.5.0",
+ "_view_name": "ProgressView",
+ "bar_style": "success",
+ "description": "",
+ "description_tooltip": null,
+ "layout": "IPY_MODEL_2a9c8f7bb54e4814825bddcbb10b3482",
+ "max": 429,
+ "min": 0,
+ "orientation": "horizontal",
+ "style": "IPY_MODEL_5f5e73e41dae4625a98a6eee6120a7d4",
+ "value": 429
+ }
+ },
+ "96997c9c8f3a4237b1d025252c3c2358": {
+ "model_module": "@jupyter-widgets/controls",
+ "model_module_version": "1.5.0",
+ "model_name": "HTMLModel",
+ "state": {
+ "_dom_classes": [],
+ "_model_module": "@jupyter-widgets/controls",
+ "_model_module_version": "1.5.0",
+ "_model_name": "HTMLModel",
+ "_view_count": null,
+ "_view_module": "@jupyter-widgets/controls",
+ "_view_module_version": "1.5.0",
+ "_view_name": "HTMLView",
+ "description": "",
+ "description_tooltip": null,
+ "layout": "IPY_MODEL_81d523a8f29c491aaed45a199260b414",
+ "placeholder": "",
+ "style": "IPY_MODEL_56cd150b53234f53b28dec80ebeb33a8",
+ "value": "tokenizer.json: 100%"
+ }
+ },
+ "a65823cef8a648cebad7e61f012de1f1": {
+ "model_module": "@jupyter-widgets/controls",
+ "model_module_version": "1.5.0",
+ "model_name": "HBoxModel",
+ "state": {
+ "_dom_classes": [],
+ "_model_module": "@jupyter-widgets/controls",
+ "_model_module_version": "1.5.0",
+ "_model_name": "HBoxModel",
+ "_view_count": null,
+ "_view_module": "@jupyter-widgets/controls",
+ "_view_module_version": "1.5.0",
+ "_view_name": "HBoxView",
+ "box_style": "",
+ "children": [
+ "IPY_MODEL_6643b53bdbe74a6491a3db6ec06446e9",
+ "IPY_MODEL_93d9e20a78774bb485ceca31d4453204",
+ "IPY_MODEL_41b58953474f49f0915d14801fb18174"
+ ],
+ "layout": "IPY_MODEL_17e8c3567f1a4d6aaa485fd1014977ae"
+ }
+ },
+ "a7b1f4318e3242e591b15147d699c656": {
+ "model_module": "@jupyter-widgets/base",
+ "model_module_version": "1.2.0",
+ "model_name": "LayoutModel",
+ "state": {
+ "_model_module": "@jupyter-widgets/base",
+ "_model_module_version": "1.2.0",
+ "_model_name": "LayoutModel",
+ "_view_count": null,
+ "_view_module": "@jupyter-widgets/base",
+ "_view_module_version": "1.2.0",
+ "_view_name": "LayoutView",
+ "align_content": null,
+ "align_items": null,
+ "align_self": null,
+ "border": null,
+ "bottom": null,
+ "display": null,
+ "flex": null,
+ "flex_flow": null,
+ "grid_area": null,
+ "grid_auto_columns": null,
+ "grid_auto_flow": null,
+ "grid_auto_rows": null,
+ "grid_column": null,
+ "grid_gap": null,
+ "grid_row": null,
+ "grid_template_areas": null,
+ "grid_template_columns": null,
+ "grid_template_rows": null,
+ "height": null,
+ "justify_content": null,
+ "justify_items": null,
+ "left": null,
+ "margin": null,
+ "max_height": null,
+ "max_width": null,
+ "min_height": null,
+ "min_width": null,
+ "object_fit": null,
+ "object_position": null,
+ "order": null,
+ "overflow": null,
+ "overflow_x": null,
+ "overflow_y": null,
+ "padding": null,
+ "right": null,
+ "top": null,
+ "visibility": null,
+ "width": null
+ }
+ },
+ "a9cd4375dfd94af4ab2c004e9dbe6fa7": {
+ "model_module": "@jupyter-widgets/controls",
+ "model_module_version": "1.5.0",
+ "model_name": "HTMLModel",
+ "state": {
+ "_dom_classes": [],
+ "_model_module": "@jupyter-widgets/controls",
+ "_model_module_version": "1.5.0",
+ "_model_name": "HTMLModel",
+ "_view_count": null,
+ "_view_module": "@jupyter-widgets/controls",
+ "_view_module_version": "1.5.0",
+ "_view_name": "HTMLView",
+ "description": "",
+ "description_tooltip": null,
+ "layout": "IPY_MODEL_af93bfd421504e3eada6c8c884869e2f",
+ "placeholder": "",
+ "style": "IPY_MODEL_782b11c627be41b2b67e2774ee7fcf0b",
+ "value": " 7.37k/7.37k [00:00<00:00, 373kB/s]"
+ }
+ },
+ "ab441cef6118450f9bfbcc29f8f34d4f": {
+ "model_module": "@jupyter-widgets/base",
+ "model_module_version": "1.2.0",
+ "model_name": "LayoutModel",
+ "state": {
+ "_model_module": "@jupyter-widgets/base",
+ "_model_module_version": "1.2.0",
+ "_model_name": "LayoutModel",
+ "_view_count": null,
+ "_view_module": "@jupyter-widgets/base",
+ "_view_module_version": "1.2.0",
+ "_view_name": "LayoutView",
+ "align_content": null,
+ "align_items": null,
+ "align_self": null,
+ "border": null,
+ "bottom": null,
+ "display": null,
+ "flex": null,
+ "flex_flow": null,
+ "grid_area": null,
+ "grid_auto_columns": null,
+ "grid_auto_flow": null,
+ "grid_auto_rows": null,
+ "grid_column": null,
+ "grid_gap": null,
+ "grid_row": null,
+ "grid_template_areas": null,
+ "grid_template_columns": null,
+ "grid_template_rows": null,
+ "height": null,
+ "justify_content": null,
+ "justify_items": null,
+ "left": null,
+ "margin": null,
+ "max_height": null,
+ "max_width": null,
+ "min_height": null,
+ "min_width": null,
+ "object_fit": null,
+ "object_position": null,
+ "order": null,
+ "overflow": null,
+ "overflow_x": null,
+ "overflow_y": null,
+ "padding": null,
+ "right": null,
+ "top": null,
+ "visibility": null,
+ "width": null
+ }
+ },
+ "af63d2b1e22c40629b2fe585813a2bf4": {
+ "model_module": "@jupyter-widgets/base",
+ "model_module_version": "1.2.0",
+ "model_name": "LayoutModel",
+ "state": {
+ "_model_module": "@jupyter-widgets/base",
+ "_model_module_version": "1.2.0",
+ "_model_name": "LayoutModel",
+ "_view_count": null,
+ "_view_module": "@jupyter-widgets/base",
+ "_view_module_version": "1.2.0",
+ "_view_name": "LayoutView",
+ "align_content": null,
+ "align_items": null,
+ "align_self": null,
+ "border": null,
+ "bottom": null,
+ "display": null,
+ "flex": null,
+ "flex_flow": null,
+ "grid_area": null,
+ "grid_auto_columns": null,
+ "grid_auto_flow": null,
+ "grid_auto_rows": null,
+ "grid_column": null,
+ "grid_gap": null,
+ "grid_row": null,
+ "grid_template_areas": null,
+ "grid_template_columns": null,
+ "grid_template_rows": null,
+ "height": null,
+ "justify_content": null,
+ "justify_items": null,
+ "left": null,
+ "margin": null,
+ "max_height": null,
+ "max_width": null,
+ "min_height": null,
+ "min_width": null,
+ "object_fit": null,
+ "object_position": null,
+ "order": null,
+ "overflow": null,
+ "overflow_x": null,
+ "overflow_y": null,
+ "padding": null,
+ "right": null,
+ "top": null,
+ "visibility": null,
+ "width": null
+ }
+ },
+ "af93bfd421504e3eada6c8c884869e2f": {
+ "model_module": "@jupyter-widgets/base",
+ "model_module_version": "1.2.0",
+ "model_name": "LayoutModel",
+ "state": {
+ "_model_module": "@jupyter-widgets/base",
+ "_model_module_version": "1.2.0",
+ "_model_name": "LayoutModel",
+ "_view_count": null,
+ "_view_module": "@jupyter-widgets/base",
+ "_view_module_version": "1.2.0",
+ "_view_name": "LayoutView",
+ "align_content": null,
+ "align_items": null,
+ "align_self": null,
+ "border": null,
+ "bottom": null,
+ "display": null,
+ "flex": null,
+ "flex_flow": null,
+ "grid_area": null,
+ "grid_auto_columns": null,
+ "grid_auto_flow": null,
+ "grid_auto_rows": null,
+ "grid_column": null,
+ "grid_gap": null,
+ "grid_row": null,
+ "grid_template_areas": null,
+ "grid_template_columns": null,
+ "grid_template_rows": null,
+ "height": null,
+ "justify_content": null,
+ "justify_items": null,
+ "left": null,
+ "margin": null,
+ "max_height": null,
+ "max_width": null,
+ "min_height": null,
+ "min_width": null,
+ "object_fit": null,
+ "object_position": null,
+ "order": null,
+ "overflow": null,
+ "overflow_x": null,
+ "overflow_y": null,
+ "padding": null,
+ "right": null,
+ "top": null,
+ "visibility": null,
+ "width": null
+ }
+ },
+ "b3cb567ec4934a6d935de606af921a61": {
+ "model_module": "@jupyter-widgets/controls",
+ "model_module_version": "1.5.0",
+ "model_name": "ProgressStyleModel",
+ "state": {
+ "_model_module": "@jupyter-widgets/controls",
+ "_model_module_version": "1.5.0",
+ "_model_name": "ProgressStyleModel",
+ "_view_count": null,
+ "_view_module": "@jupyter-widgets/base",
+ "_view_module_version": "1.2.0",
+ "_view_name": "StyleView",
+ "bar_color": null,
+ "description_width": ""
+ }
+ },
+ "b9f6e9e50e224cf89b902e419dae269a": {
+ "model_module": "@jupyter-widgets/controls",
+ "model_module_version": "1.5.0",
+ "model_name": "FloatProgressModel",
+ "state": {
+ "_dom_classes": [],
+ "_model_module": "@jupyter-widgets/controls",
+ "_model_module_version": "1.5.0",
+ "_model_name": "FloatProgressModel",
+ "_view_count": null,
+ "_view_module": "@jupyter-widgets/controls",
+ "_view_module_version": "1.5.0",
+ "_view_name": "ProgressView",
+ "bar_style": "success",
+ "description": "",
+ "description_tooltip": null,
+ "layout": "IPY_MODEL_a7b1f4318e3242e591b15147d699c656",
+ "max": 43727,
+ "min": 0,
+ "orientation": "horizontal",
+ "style": "IPY_MODEL_7d7dd120240043c1aec77af517bf7add",
+ "value": 43727
+ }
+ },
+ "bb120e29c7fb49e4823daf66f048e95b": {
+ "model_module": "@jupyter-widgets/controls",
+ "model_module_version": "1.5.0",
+ "model_name": "HTMLModel",
+ "state": {
+ "_dom_classes": [],
+ "_model_module": "@jupyter-widgets/controls",
+ "_model_module_version": "1.5.0",
+ "_model_name": "HTMLModel",
+ "_view_count": null,
+ "_view_module": "@jupyter-widgets/controls",
+ "_view_module_version": "1.5.0",
+ "_view_name": "HTMLView",
+ "description": "",
+ "description_tooltip": null,
+ "layout": "IPY_MODEL_d102def58f554d48890008185d17af96",
+ "placeholder": "",
+ "style": "IPY_MODEL_79a852590a0444bebad62f2c679e72c6",
+ "value": "configuration_cohere.py: 100%"
+ }
+ },
+ "c0c90d8f84b64a11a48e81ba8dec7044": {
+ "model_module": "@jupyter-widgets/base",
+ "model_module_version": "1.2.0",
+ "model_name": "LayoutModel",
+ "state": {
+ "_model_module": "@jupyter-widgets/base",
+ "_model_module_version": "1.2.0",
+ "_model_name": "LayoutModel",
+ "_view_count": null,
+ "_view_module": "@jupyter-widgets/base",
+ "_view_module_version": "1.2.0",
+ "_view_name": "LayoutView",
+ "align_content": null,
+ "align_items": null,
+ "align_self": null,
+ "border": null,
+ "bottom": null,
+ "display": null,
+ "flex": null,
+ "flex_flow": null,
+ "grid_area": null,
+ "grid_auto_columns": null,
+ "grid_auto_flow": null,
+ "grid_auto_rows": null,
+ "grid_column": null,
+ "grid_gap": null,
+ "grid_row": null,
+ "grid_template_areas": null,
+ "grid_template_columns": null,
+ "grid_template_rows": null,
+ "height": null,
+ "justify_content": null,
+ "justify_items": null,
+ "left": null,
+ "margin": null,
+ "max_height": null,
+ "max_width": null,
+ "min_height": null,
+ "min_width": null,
+ "object_fit": null,
+ "object_position": null,
+ "order": null,
+ "overflow": null,
+ "overflow_x": null,
+ "overflow_y": null,
+ "padding": null,
+ "right": null,
+ "top": null,
+ "visibility": null,
+ "width": null
+ }
+ },
+ "c8330e6698cd41868efc23be3962db7d": {
+ "model_module": "@jupyter-widgets/controls",
+ "model_module_version": "1.5.0",
+ "model_name": "DescriptionStyleModel",
+ "state": {
+ "_model_module": "@jupyter-widgets/controls",
+ "_model_module_version": "1.5.0",
+ "_model_name": "DescriptionStyleModel",
+ "_view_count": null,
+ "_view_module": "@jupyter-widgets/base",
+ "_view_module_version": "1.2.0",
+ "_view_name": "StyleView",
+ "description_width": ""
+ }
+ },
+ "cf03a6a84e1345aab6d2d78f9ae5f34a": {
+ "model_module": "@jupyter-widgets/controls",
+ "model_module_version": "1.5.0",
+ "model_name": "HTMLModel",
+ "state": {
+ "_dom_classes": [],
+ "_model_module": "@jupyter-widgets/controls",
+ "_model_module_version": "1.5.0",
+ "_model_name": "HTMLModel",
+ "_view_count": null,
+ "_view_module": "@jupyter-widgets/controls",
+ "_view_module_version": "1.5.0",
+ "_view_name": "HTMLView",
+ "description": "",
+ "description_tooltip": null,
+ "layout": "IPY_MODEL_af63d2b1e22c40629b2fe585813a2bf4",
+ "placeholder": "",
+ "style": "IPY_MODEL_87d5a00cc4a2460e836ddd22cff918dd",
+ "value": " 12.8M/12.8M [00:00<00:00, 51.7MB/s]"
+ }
+ },
+ "d102def58f554d48890008185d17af96": {
+ "model_module": "@jupyter-widgets/base",
+ "model_module_version": "1.2.0",
+ "model_name": "LayoutModel",
+ "state": {
+ "_model_module": "@jupyter-widgets/base",
+ "_model_module_version": "1.2.0",
+ "_model_name": "LayoutModel",
+ "_view_count": null,
+ "_view_module": "@jupyter-widgets/base",
+ "_view_module_version": "1.2.0",
+ "_view_name": "LayoutView",
+ "align_content": null,
+ "align_items": null,
+ "align_self": null,
+ "border": null,
+ "bottom": null,
+ "display": null,
+ "flex": null,
+ "flex_flow": null,
+ "grid_area": null,
+ "grid_auto_columns": null,
+ "grid_auto_flow": null,
+ "grid_auto_rows": null,
+ "grid_column": null,
+ "grid_gap": null,
+ "grid_row": null,
+ "grid_template_areas": null,
+ "grid_template_columns": null,
+ "grid_template_rows": null,
+ "height": null,
+ "justify_content": null,
+ "justify_items": null,
+ "left": null,
+ "margin": null,
+ "max_height": null,
+ "max_width": null,
+ "min_height": null,
+ "min_width": null,
+ "object_fit": null,
+ "object_position": null,
+ "order": null,
+ "overflow": null,
+ "overflow_x": null,
+ "overflow_y": null,
+ "padding": null,
+ "right": null,
+ "top": null,
+ "visibility": null,
+ "width": null
+ }
+ },
+ "e2146d738e0d4fe39af19bfb22da2584": {
+ "model_module": "@jupyter-widgets/controls",
+ "model_module_version": "1.5.0",
+ "model_name": "HBoxModel",
+ "state": {
+ "_dom_classes": [],
+ "_model_module": "@jupyter-widgets/controls",
+ "_model_module_version": "1.5.0",
+ "_model_name": "HBoxModel",
+ "_view_count": null,
+ "_view_module": "@jupyter-widgets/controls",
+ "_view_module_version": "1.5.0",
+ "_view_name": "HBoxView",
+ "box_style": "",
+ "children": [
+ "IPY_MODEL_8670211888514256a54240d002135917",
+ "IPY_MODEL_88b9a3c1bc78462ea149b94d1ca08e59",
+ "IPY_MODEL_5a94eb68326b4466b7c19ad67f8f5ba6"
+ ],
+ "layout": "IPY_MODEL_ab441cef6118450f9bfbcc29f8f34d4f"
+ }
+ },
+ "efb2bf269c1d4ace81ff4a6fabbfb3b6": {
+ "model_module": "@jupyter-widgets/base",
+ "model_module_version": "1.2.0",
+ "model_name": "LayoutModel",
+ "state": {
+ "_model_module": "@jupyter-widgets/base",
+ "_model_module_version": "1.2.0",
+ "_model_name": "LayoutModel",
+ "_view_count": null,
+ "_view_module": "@jupyter-widgets/base",
+ "_view_module_version": "1.2.0",
+ "_view_name": "LayoutView",
+ "align_content": null,
+ "align_items": null,
+ "align_self": null,
+ "border": null,
+ "bottom": null,
+ "display": null,
+ "flex": null,
+ "flex_flow": null,
+ "grid_area": null,
+ "grid_auto_columns": null,
+ "grid_auto_flow": null,
+ "grid_auto_rows": null,
+ "grid_column": null,
+ "grid_gap": null,
+ "grid_row": null,
+ "grid_template_areas": null,
+ "grid_template_columns": null,
+ "grid_template_rows": null,
+ "height": null,
+ "justify_content": null,
+ "justify_items": null,
+ "left": null,
+ "margin": null,
+ "max_height": null,
+ "max_width": null,
+ "min_height": null,
+ "min_width": null,
+ "object_fit": null,
+ "object_position": null,
+ "order": null,
+ "overflow": null,
+ "overflow_x": null,
+ "overflow_y": null,
+ "padding": null,
+ "right": null,
+ "top": null,
+ "visibility": null,
+ "width": null
+ }
+ },
+ "f8e3c447794448529391f9660d419632": {
+ "model_module": "@jupyter-widgets/controls",
+ "model_module_version": "1.5.0",
+ "model_name": "DescriptionStyleModel",
+ "state": {
+ "_model_module": "@jupyter-widgets/controls",
+ "_model_module_version": "1.5.0",
+ "_model_name": "DescriptionStyleModel",
+ "_view_count": null,
+ "_view_module": "@jupyter-widgets/base",
+ "_view_module_version": "1.2.0",
+ "_view_name": "StyleView",
+ "description_width": ""
+ }
+ },
+ "fc96c077774241a58d24ac42dd52df7e": {
+ "model_module": "@jupyter-widgets/controls",
+ "model_module_version": "1.5.0",
+ "model_name": "DescriptionStyleModel",
+ "state": {
+ "_model_module": "@jupyter-widgets/controls",
+ "_model_module_version": "1.5.0",
+ "_model_name": "DescriptionStyleModel",
+ "_view_count": null,
+ "_view_module": "@jupyter-widgets/base",
+ "_view_module_version": "1.2.0",
+ "_view_name": "StyleView",
+ "description_width": ""
+ }
+ }
+ }
+ }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 4
}
diff --git a/notebooks/guides/Basic_Semantic_Search.ipynb b/notebooks/guides/Basic_Semantic_Search.ipynb
index 681311e9..37deac0d 100644
--- a/notebooks/guides/Basic_Semantic_Search.ipynb
+++ b/notebooks/guides/Basic_Semantic_Search.ipynb
@@ -1,7 +1,6 @@
{
"cells": [
{
- "attachments": {},
"cell_type": "markdown",
"metadata": {
"id": "65_Qg86Sl2eF"
@@ -36,12 +35,10 @@
"source": [
"# Install Cohere for embeddings, Umap to reduce embeddings to 2 dimensions, \n",
"# Altair for visualization, Annoy for approximate nearest neighbor search\n",
- "# TODO: upgrade to \"cohere>5\"",
-"!pip install \"cohere<5\" umap-learn altair annoy datasets tqdm"
+ "# TODO: upgrade to \"cohere>5\"!pip install \"cohere<5\" umap-learn altair annoy datasets tqdm"
]
},
{
- "attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
@@ -70,7 +67,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
- "## 1. Getting Set up"
+ "## 1. Getting Set Up"
]
},
{
@@ -99,7 +96,6 @@
]
},
{
- "attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
@@ -124,7 +120,6 @@
]
},
{
- "attachments": {},
"cell_type": "markdown",
"metadata": {
"id": "2LZ7SBKSU4bx"
@@ -359,7 +354,6 @@
]
},
{
- "attachments": {},
"cell_type": "markdown",
"metadata": {
"id": "zrajiGkNVw7E"
@@ -370,7 +364,6 @@
]
},
{
- "attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
@@ -427,7 +420,6 @@
]
},
{
- "attachments": {},
"cell_type": "markdown",
"metadata": {
"id": "lzcDFCDePVxA"
@@ -474,7 +466,6 @@
]
},
{
- "attachments": {},
"cell_type": "markdown",
"metadata": {
"id": "GfIDv25K0A-0"
@@ -613,7 +604,6 @@
]
},
{
- "attachments": {},
"cell_type": "markdown",
"metadata": {
"id": "bI4kGnnPcu90"
@@ -776,7 +766,6 @@
]
},
{
- "attachments": {},
"cell_type": "markdown",
"metadata": {
"id": "4tN7fGDst7rh"
@@ -892,7 +881,6 @@
]
},
{
- "attachments": {},
"cell_type": "markdown",
"metadata": {
"id": "4ZEYkKsSfQne"
@@ -905,12 +893,6 @@
"\n",
"We can’t wait to see what you start building! Share your projects or find support on [Discord](https://discord.com/invite/co-mmunity).\n"
]
- },
- {
- "attachments": {},
- "cell_type": "markdown",
- "metadata": {},
- "source": []
}
],
"metadata": {
@@ -934,7 +916,7 @@
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
- "version": "3.9.7"
+ "version": "3.8.16"
},
"widgets": {
"application/vnd.jupyter.widget-state+json": {
diff --git a/notebooks/guides/Basic_Summarization_Notebook.ipynb b/notebooks/guides/Basic_Summarization_Notebook.ipynb
index 87d3b645..7cbad4cd 100644
--- a/notebooks/guides/Basic_Summarization_Notebook.ipynb
+++ b/notebooks/guides/Basic_Summarization_Notebook.ipynb
@@ -16,23 +16,6 @@
"`\"\". In summary: \"\"`."
]
},
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "colab": {
- "base_uri": "https://localhost:8080/"
- },
- "id": "EqdDiovEBRLw",
- "outputId": "b72dc9ea-d73f-4291-bc59-7d3b1d8490cd"
- },
- "outputs": [],
- "source": [
- "# Let's first install Cohere's python SDK\n",
- "# TODO: upgrade to \"cohere>5\"",
-"!pip install \"cohere<5\""
- ]
- },
{
"cell_type": "code",
"execution_count": 1,
@@ -256,18 +239,11 @@
"id": "M3HM7go_tebY"
},
"source": [
- "In a lot of cases, better generations can be reached by creating multiple generations then ranking and filtering them. In this case we're ranking the generations by their average likelihoods. \n",
+ "In a lot of cases, better generations can be reached by creating multiple generations then ranking and filtering them. In this case, we're ranking the generations by their average likelihoods. \n",
"\n",
"## Hyperparameters\n",
"It's worth spending some time learning the various hyperparameters of the generation endpoint. For example, [temperature](https://docs.cohere.ai/temperature-wiki) tunes the degree of randomness in the generations. Other parameters include [top-k and top-p](https://docs.cohere.ai/token-picking) as well as `frequency_penalty` and `presence_penalty` which can reduce the amount of repetition in the output of the model. See the [API reference of the generate endpoint](https://docs.cohere.ai/generate-reference) for more details on all the parameters."
]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {},
- "outputs": [],
- "source": []
}
],
"metadata": {
@@ -291,7 +267,7 @@
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
- "version": "3.9.7"
+ "version": "3.8.16"
}
},
"nbformat": 4,
diff --git a/notebooks/guides/Chunking_strategies.ipynb b/notebooks/guides/Chunking_strategies.ipynb
index 26ce93e6..41e7487e 100644
--- a/notebooks/guides/Chunking_strategies.ipynb
+++ b/notebooks/guides/Chunking_strategies.ipynb
@@ -1,17 +1,8 @@
{
"cells": [
- {
- "cell_type": "markdown",
- "source": [
- "**"
- ],
- "metadata": {
- "id": "KA0rmEN-XNsd"
- }
- },
{
"cell_type": "code",
- "execution_count": null,
+ "execution_count": 1,
"metadata": {
"id": "F_jICWZXSVA8"
},
@@ -52,12 +43,7 @@
},
{
"cell_type": "code",
- "source": [
- "# Set up Cohere client\n",
- "co_model = 'command-r'\n",
- "co_api_key = getpass(\"Enter Cohere API key: \")\n",
- "co = cohere.Client(api_key=co_api_key)"
- ],
+ "execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
@@ -65,7 +51,6 @@
"id": "wznWW57-BAML",
"outputId": "56de0fa7-7321-4718-8133-8740d405e78b"
},
- "execution_count": null,
"outputs": [
{
"name": "stdout",
@@ -74,39 +59,47 @@
"Enter Cohere API key: ··········\n"
]
}
+ ],
+ "source": [
+ "# Set up Cohere client\n",
+ "co_model = 'command-r'\n",
+ "co_api_key = getpass(\"Enter Cohere API key: \")\n",
+ "co = cohere.Client(api_key=co_api_key)"
]
},
{
"cell_type": "markdown",
+ "metadata": {
+ "id": "pIJH7u4dL7mZ"
+ },
"source": [
- "\n",
- "# Introduction\n",
+ "## Introduction\n",
"\n",
"Chunking is an essential component of any RAG-based system. This cookbook aims to demonstrate how different chunking strategies affect the results of LLM-generated output. There are multiple considerations that need to be taken into account when designing chunking strategy. Therefore, we begin by providing a framework for these strategies and then jump into a practical example. We will focus our example on transcript calls, which create a unique challenge because of their rich content and the change of people speaking throughout the text.\n",
"\n",
- "# Table of content\n",
+ "## Table of contents\n",
"\n",
- "1. [Chunking Strategies Framework](#framework)\n",
+ "1. [Chunking strategies framework](#chunking-strategies-framework)\n",
"2. [Getting started](#getting-started)\n",
"3. [Example 1: Chunking using content-independent strategies](#example-1)\n",
"4. [Example 2: Chunking using content-dependent strategies](#example-2)\n",
"5. [Discussion](#discussion)"
- ],
- "metadata": {
- "id": "pIJH7u4dL7mZ"
- }
+ ]
},
{
"cell_type": "markdown",
+ "metadata": {
+ "id": "Yuse4PR_9saD"
+ },
"source": [
- "\n",
- "# Chunking Strategies Framework\n",
+ "\n",
+ "## Chunking strategies framework\n",
"\n",
- "## Document splitting\n",
+ "### Document splitting\n",
"\n",
"By document splitting, we mean deciding on the conditions under which we will break the text. At this stage, we should ask, *\"Are there any parts of consecutive text we want to ensure we do not break?\"*. If the answer is \"no\", then, the content-independent splitting strategies are helpful. On the other hand, in scenarios like transcripts or meeting notes, we probably would like to keep the content of one speaker together, which might require us to deploy content-dependent strategies.\n",
"\n",
- "### Content-independent splitting strategies\n",
+ "#### Content-independent splitting strategies\n",
"\n",
"We split the document based on some content-independent conditions, among the most popular ones are:\n",
"- splitting by the number of characters,\n",
@@ -115,49 +108,41 @@
"\n",
"The advantage of this scenario is that we do not need to make any assumptions about the text. However, some considerations remain, like whether we want to preserve some semantic structure, for example, sentences or paragraphs. Sentence splitting is better suited if we are looking for small chunks to ensure accuracy. Conversely, paragraphs preserve more context and might be more useful in open-ended questions.\n",
"\n",
- "### Content-dependent splitting strategies\n",
+ "#### Content-dependent splitting strategies\n",
"\n",
"On the other hand, there are scenarios in which we care about preserving some text structure. Then, we develop custom splitting strategies based on the document's content. A prime example is call transcripts. In such scenarios, we aim to ensure that one person's speech is fully contained within a chunk.\n",
"\n",
- "## Creating chunks from the document splits\n",
+ "### Creating chunks from the document splits\n",
"\n",
"After the document is split, we need to decide on the desired **size** of our chunks (the split only defines how we break the document, but we can create bigger chunks from multiple splits).\n",
"\n",
"Smaller chunks support more accurate retrieval. However, they might lack context. On the other hand, larger chunks offer more context, but they reduce the effectiveness of the retrieval. It is important to experiment with different settings to find the optimal balance.\n",
"\n",
- "## Overlapping chunks\n",
+ "### Overlapping chunks\n",
"\n",
"Overlapping chunks is a useful technique to have in the toolbox. Especially when we employ content-independent splitting strategies, it helps us mitigate some of the pitfalls of breaking the document without fully understanding the text. Overlapping guarantees that there is always some buffer between the chunks, and even if an important piece of information might be split in the original splitting strategy, it is more probable that the full information will be captured in the next chunk. The disadvantage of this method is that it creates redundancy."
- ],
- "metadata": {
- "id": "Yuse4PR_9saD"
- }
+ ]
},
{
"cell_type": "markdown",
- "source": [
- "\n",
- "# Getting started\n",
- "\n",
- "Designing a robust chunking strategy is as much a science as an art. There are no straightforward answers; the most effective strategies often emerge through experimentation. Therefore, let's dive straight into an example to illustrate this concept.\n",
- "\n",
- "\n",
- "\n",
- "\n",
- "\n"
- ],
"metadata": {
"id": "eKdDiizv_QtR"
- }
+ },
+ "source": [
+ "\n",
+ "## Getting started\n",
+ "\n",
+ "Designing a robust chunking strategy is as much a science as an art. There are no straightforward answers; the most effective strategies often emerge through experimentation. Therefore, let's dive straight into an example to illustrate this concept."
+ ]
},
{
"cell_type": "markdown",
- "source": [
- "## Utils"
- ],
"metadata": {
"id": "1ivVb-oMCAaz"
- }
+ },
+ "source": [
+ "## Utils"
+ ]
},
{
"cell_type": "code",
@@ -172,11 +157,7 @@
},
"outputs": [
{
- "output_type": "display_data",
"data": {
- "text/plain": [
- ""
- ],
"text/html": [
"\n",
" \n",
" "
+ ],
+ "text/plain": [
+ ""
]
},
- "metadata": {}
+ "metadata": {},
+ "output_type": "display_data"
}
],
"source": [
@@ -217,11 +202,7 @@
},
"outputs": [
{
- "output_type": "display_data",
"data": {
- "text/plain": [
- ""
- ],
"text/html": [
"\n",
" \n",
" "
+ ],
+ "text/plain": [
+ ""
]
},
- "metadata": {}
+ "metadata": {},
+ "output_type": "display_data"
}
],
"source": [
@@ -255,7 +240,7 @@
"\n",
" return text\n",
"\n",
- "def build_retreiver(documents, top_n=5):\n",
+ "def build_retriever(documents, top_n=5):\n",
" # Create the embedding model\n",
" embed_model = CohereEmbedding(\n",
" cohere_api_key=co_api_key,\n",
@@ -279,31 +264,18 @@
},
{
"cell_type": "markdown",
+ "metadata": {
+ "id": "AUXfWr13CyFE"
+ },
"source": [
- "##Load the data\n",
+ "## Load the data\n",
"\n",
"In this example we will work with an 2023 Tesla earning call transcript."
- ],
- "metadata": {
- "id": "AUXfWr13CyFE"
- }
+ ]
},
{
"cell_type": "code",
- "source": [
- "# Get all investement memos (19) in bvp repository\n",
- "url_path = 'https://www.fool.com/earnings/call-transcripts/2024/01/24/tesla-tsla-q4-2023-earnings-call-transcript/'\n",
- "response = requests.get(url_path)\n",
- "soup = BeautifulSoup(response.content, 'html.parser')\n",
- "\n",
- "target_divs = soup.find(\"div\", {\"class\": \"article-body\"}).find_all(\"p\")[2:]\n",
- "print('Length of the script: ', len(target_divs))\n",
- "\n",
- "print()\n",
- "print('Example of processed text:')\n",
- "text = '\\n\\n'.join([div.get_text() for div in target_divs])\n",
- "print(text[:500])"
- ],
+ "execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
@@ -312,14 +284,9 @@
"id": "NeSVXvO_e1DH",
"outputId": "5042cbef-d3e7-4a78-b921-aa269e60570e"
},
- "execution_count": null,
"outputs": [
{
- "output_type": "display_data",
"data": {
- "text/plain": [
- ""
- ],
"text/html": [
"\n",
" \n",
" "
+ ],
+ "text/plain": [
+ ""
]
},
- "metadata": {}
+ "metadata": {},
+ "output_type": "display_data"
},
{
- "output_type": "stream",
"name": "stdout",
+ "output_type": "stream",
"text": [
"Length of the script: 385\n",
"\n",
@@ -346,15 +317,32 @@
"During this call, we will discuss our business outlook and make forward-looking statements. These comments are based on our predictions and \n"
]
}
+ ],
+ "source": [
+ "# Get all investement memos (19) in bvp repository\n",
+ "url_path = 'https://www.fool.com/earnings/call-transcripts/2024/01/24/tesla-tsla-q4-2023-earnings-call-transcript/'\n",
+ "response = requests.get(url_path)\n",
+ "soup = BeautifulSoup(response.content, 'html.parser')\n",
+ "\n",
+ "target_divs = soup.find(\"div\", {\"class\": \"article-body\"}).find_all(\"p\")[2:]\n",
+ "print('Length of the script: ', len(target_divs))\n",
+ "\n",
+ "print()\n",
+ "print('Example of processed text:')\n",
+ "text = '\\n\\n'.join([div.get_text() for div in target_divs])\n",
+ "print(text[:500])"
]
},
{
"cell_type": "markdown",
+ "metadata": {
+ "id": "UhlsdHPmhoMT"
+ },
"source": [
- "\n",
- "# Example 1: Chunking using content-independent strategies\n",
+ "\n",
+ "## Example 1: Chunking using content-independent strategies\n",
"\n",
- "Let's begin with a simple content-independent strategy. We aim to answer the question, `Who mentiones Jonathan Nolan?`. We chose this question as it is easily verifiable and it requires to identify the speaker. The answer to this questions can be found in the dowloaded transcscript, here is the relevant passage:\n",
+ "Let's begin with a simple content-independent strategy. We aim to answer the question, `Who mentions Jonathan Nolan?`. We chose this question as it is easily verifiable and it requires to identify the speaker. The answer to this question can be found in the downloaded transcript, here is the relevant passage:\n",
"\n",
"\n",
"```\n",
@@ -362,17 +350,11 @@
"\n",
"Yeah. The creators of Westworld, Jonathan Nolan, Lisa Joy Nolan, are friends -- are all friends of mine, actually. And I invited them to come see the lab and, like, well, come see it, hopefully soon. It's pretty well -- especially the sort of subsystem test stands where you've just got like one leg on a test stand just doing repetitive exercises and one arm on a test stand pretty well.\n",
"```"
- ],
- "metadata": {
- "id": "UhlsdHPmhoMT"
- }
+ ]
},
{
"cell_type": "code",
- "source": [
- "# Define the question\n",
- "question = \"Who mentiones Jonathan Nolan?\""
- ],
+ "execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
@@ -381,14 +363,9 @@
"id": "P4xy_imcEj10",
"outputId": "a2904832-788a-41d8-ae10-c9049677547c"
},
- "execution_count": null,
"outputs": [
{
- "output_type": "display_data",
"data": {
- "text/plain": [
- ""
- ],
"text/html": [
"\n",
" \n",
" "
+ ],
+ "text/plain": [
+ ""
]
},
- "metadata": {}
+ "metadata": {},
+ "output_type": "display_data"
}
+ ],
+ "source": [
+ "# Define the question\n",
+ "question = \"Who mentions Jonathan Nolan?\""
]
},
{
"cell_type": "markdown",
+ "metadata": {
+ "id": "cy8IXqA4H2ET"
+ },
"source": [
"In this case, we are more concerned about accuracy than a verbose answer, so we **focus on keeping the chunks small**. To ensure that the desired size is not exceeded, we will randomly split the list of characters, in our case `[\"\\n\\n\", \"\\n\", \" \", \"\"]`.\n",
"\n",
"We employ the `RecursiveCharacterTextSplitter` from [LangChain](https://python.langchain.com/docs/get_started/introduction) for this task."
- ],
- "metadata": {
- "id": "cy8IXqA4H2ET"
- }
+ ]
},
{
"cell_type": "code",
- "source": [
- "# Define the chunking function\n",
- "def get_chunks(text, chunk_size, chunk_overlap):\n",
- " text_splitter = RecursiveCharacterTextSplitter(\n",
- " chunk_size=chunk_size,\n",
- " chunk_overlap=chunk_overlap,\n",
- " length_function=len,\n",
- " is_separator_regex=False,\n",
- " )\n",
- "\n",
- " documents = text_splitter.create_documents([text])\n",
- " documents = [Document(text=doc.page_content) for doc in documents]\n",
- "\n",
- " return documents"
- ],
+ "execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
@@ -439,14 +410,9 @@
"id": "xy69i0i7xIy1",
"outputId": "08c4a509-bab3-4f9b-9c98-cb2ad71ee59b"
},
- "execution_count": null,
"outputs": [
{
- "output_type": "display_data",
"data": {
- "text/plain": [
- ""
- ],
"text/html": [
"\n",
" \n",
" "
+ ],
+ "text/plain": [
+ ""
]
},
- "metadata": {}
+ "metadata": {},
+ "output_type": "display_data"
}
+ ],
+ "source": [
+ "# Define the chunking function\n",
+ "def get_chunks(text, chunk_size, chunk_overlap):\n",
+ " text_splitter = RecursiveCharacterTextSplitter(\n",
+ " chunk_size=chunk_size,\n",
+ " chunk_overlap=chunk_overlap,\n",
+ " length_function=len,\n",
+ " is_separator_regex=False,\n",
+ " )\n",
+ "\n",
+ " documents = text_splitter.create_documents([text])\n",
+ " documents = [Document(text=doc.page_content) for doc in documents]\n",
+ "\n",
+ " return documents"
]
},
{
"cell_type": "markdown",
+ "metadata": {
+ "id": "4YXsoVMMGCnC"
+ },
"source": [
"### Experiment 1 - no overlap\n",
"In our first experiment we define the chunk size as 500 and allow **no overlap between consecutive chunks**.\n",
"\n",
"Subsequently, we implement the standard RAG pipeline. We feed the chunks into a retriever, selecting the `top_n` most pertinent to the query chunks, and supply them as context to the generation model. Throughout this pipeline, we leverage [Cohere's endpoints](https://docs.cohere.com/reference/about), specifically, `co.embed`, `co.re.rank`, and finally, `co.chat`."
- ],
- "metadata": {
- "id": "4YXsoVMMGCnC"
- }
+ ]
},
{
"cell_type": "code",
- "source": [
- "chunk_size = 500\n",
- "chunk_overlap = 0\n",
- "documents = get_chunks(text, chunk_size, chunk_overlap)\n",
- "retriever = build_retreiver(documents)\n",
- "\n",
- "source_nodes = retriever.retrieve(question)\n",
- "print('Number of docuemnts: ',len(source_nodes))\n",
- "source_nodes= [{\"text\": ni.get_content()}for ni in source_nodes]\n",
- "\n",
- "\n",
- "response = co.chat(\n",
- " message=question,\n",
- " documents=source_nodes,\n",
- " model=co_model\n",
- ")\n",
- "response = response\n",
- "print(response.text)"
- ],
+ "execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
@@ -502,14 +469,9 @@
"id": "HAG0ybf8xbcB",
"outputId": "6a5f2b54-f4aa-4674-8684-7601d47bcd10"
},
- "execution_count": null,
"outputs": [
{
- "output_type": "display_data",
"data": {
- "text/plain": [
- ""
- ],
"text/html": [
"\n",
" \n",
" "
+ ],
+ "text/plain": [
+ ""
]
},
- "metadata": {}
+ "metadata": {},
+ "output_type": "display_data"
},
{
- "output_type": "stream",
"name": "stdout",
+ "output_type": "stream",
"text": [
"Number of docuemnts: 5\n",
"An unknown speaker mentions Jonathan Nolan in a conversation about the creators of Westworld. They mention that Jonathan Nolan and Lisa Joy Nolan are friends of theirs, and that they have invited them to visit the lab.\n"
]
}
+ ],
+ "source": [
+ "chunk_size = 500\n",
+ "chunk_overlap = 0\n",
+ "documents = get_chunks(text, chunk_size, chunk_overlap)\n",
+ "retriever = build_retriever(documents)\n",
+ "\n",
+ "source_nodes = retriever.retrieve(question)\n",
+ "print('Number of docuemnts: ',len(source_nodes))\n",
+ "source_nodes= [{\"text\": ni.get_content()}for ni in source_nodes]\n",
+ "\n",
+ "\n",
+ "response = co.chat(\n",
+ " message=question,\n",
+ " documents=source_nodes,\n",
+ " model=co_model\n",
+ ")\n",
+ "response = response\n",
+ "print(response.text)"
]
},
{
"cell_type": "markdown",
- "source": [
- "A notable feature of [`co.chat`](https://docs.cohere.com/reference/chat) is its ability to ground the model's answer within the context. This means we can identify which chunks were used to generate the answer. Below, we show the previous output of the model together with the citation reference, where `[num]` represents the index of the chunk."
- ],
"metadata": {
"id": "zbXfrN125I-i"
- }
+ },
+ "source": [
+ "A notable feature of [`co.chat`](https://docs.cohere.com/reference/chat) is its ability to ground the model's answer within the context. This means we can identify which chunks were used to generate the answer. Below, we show the previous output of the model together with the citation reference, where `[num]` represents the index of the chunk."
+ ]
},
{
"cell_type": "code",
- "source": [
- "print(insert_citations(response.text, response.citations))"
- ],
+ "execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
@@ -554,14 +537,9 @@
"id": "pyLy-OwBGcut",
"outputId": "0e4a63b0-2cf2-4021-8cb7-4292a7813644"
},
- "execution_count": null,
"outputs": [
{
- "output_type": "display_data",
"data": {
- "text/plain": [
- ""
- ],
"text/html": [
"\n",
" \n",
" "
+ ],
+ "text/plain": [
+ ""
]
},
- "metadata": {}
+ "metadata": {},
+ "output_type": "display_data"
},
{
- "output_type": "stream",
"name": "stdout",
+ "output_type": "stream",
"text": [
"An unknown speaker [0] mentions Jonathan Nolan in a conversation about the creators of Westworld. [0] They mention that Jonathan Nolan and Lisa Joy Nolan [0] are friends [0] of theirs, and that they have invited them to visit the lab. [0]\n"
]
}
+ ],
+ "source": [
+ "print(insert_citations(response.text, response.citations))"
]
},
{
"cell_type": "markdown",
- "source": [
- "Indeed, by printing the cited chunk, we can validate that the text was divided so that the generation model could not provide the correct response. Notably, the speaker's name is not included in the context, which is why the model refes to an `unknown speaker`."
- ],
"metadata": {
"id": "X07wAmqLG3Mf"
- }
+ },
+ "source": [
+ "Indeed, by printing the cited chunk, we can validate that the text was divided so that the generation model could not provide the correct response. Notably, the speaker's name is not included in the context, which is why the model refes to an `unknown speaker`."
+ ]
},
{
"cell_type": "code",
- "source": [
- "print(source_nodes[0])"
- ],
+ "execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
@@ -605,14 +588,9 @@
"id": "StbIRRQWGei3",
"outputId": "856b8735-661a-4bdb-e859-67082a56f6af"
},
- "execution_count": null,
"outputs": [
{
- "output_type": "display_data",
"data": {
- "text/plain": [
- ""
- ],
"text/html": [
"\n",
" \n",
" "
+ ],
+ "text/plain": [
+ ""
]
},
- "metadata": {}
+ "metadata": {},
+ "output_type": "display_data"
},
{
- "output_type": "stream",
"name": "stdout",
+ "output_type": "stream",
"text": [
"{'text': \"Yeah. The creators of Westworld, Jonathan Nolan, Lisa Joy Nolan, are friends -- are all friends of mine, actually. And I invited them to come see the lab and, like, well, come see it, hopefully soon. It's pretty well -- especially the sort of subsystem test stands where you've just got like one leg on a test stand just doing repetitive exercises and one arm on a test stand pretty well.\\n\\nYeah.\\n\\nUnknown speaker\\n\\nWe're not entering Westworld anytime soon.\"}\n"
]
}
+ ],
+ "source": [
+ "print(source_nodes[0])"
]
},
{
"cell_type": "markdown",
+ "metadata": {
+ "id": "Iu91q_IhHZoh"
+ },
"source": [
"### Experiment 2 - allow overlap\n",
"In the previous experiment, we discovered that the chunks were generated in a way that made it impossible to generate the correct answer. The name of the speaker was not included in the relevant chunk.\n",
"\n",
"Therefore, this time to mitigate this issue, we **allow for overlap between consecutive chunks**."
- ],
- "metadata": {
- "id": "Iu91q_IhHZoh"
- }
+ ]
},
{
"cell_type": "code",
- "source": [
- "chunk_size = 500\n",
- "chunk_overlap = 100\n",
- "documents = get_chunks(text,chunk_size, chunk_overlap)\n",
- "retriever = build_retreiver(documents)\n",
- "\n",
- "source_nodes = retriever.retrieve(question)\n",
- "print('Number of docuemnts: ',len(source_nodes))\n",
- "source_nodes= [{\"text\": ni.get_content()}for ni in source_nodes]\n",
- "\n",
- "\n",
- "response = co.chat(\n",
- " message=question,\n",
- " documents=source_nodes,\n",
- " model=co_model\n",
- ")\n",
- "response = response\n",
- "print(response.text)"
- ],
+ "execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
@@ -675,14 +642,9 @@
"id": "Za9HW8z_xog_",
"outputId": "f3722475-44f2-40b1-c3d7-07cf603b4994"
},
- "execution_count": null,
"outputs": [
{
- "output_type": "display_data",
"data": {
- "text/plain": [
- ""
- ],
"text/html": [
"\n",
" \n",
" "
+ ],
+ "text/plain": [
+ ""
]
},
- "metadata": {}
+ "metadata": {},
+ "output_type": "display_data"
},
{
- "output_type": "stream",
"name": "stdout",
+ "output_type": "stream",
"text": [
"Number of docuemnts: 5\n",
"Elon Musk mentions Jonathan Nolan. Musk is the CEO and Product Architect of the lab that resembles the set of Westworld, a show created by Jonathan Nolan and Lisa Joy Nolan.\n"
]
}
+ ],
+ "source": [
+ "chunk_size = 500\n",
+ "chunk_overlap = 100\n",
+ "documents = get_chunks(text,chunk_size, chunk_overlap)\n",
+ "retriever = build_retriever(documents)\n",
+ "\n",
+ "source_nodes = retriever.retrieve(question)\n",
+ "print('Number of docuemnts: ',len(source_nodes))\n",
+ "source_nodes= [{\"text\": ni.get_content()}for ni in source_nodes]\n",
+ "\n",
+ "\n",
+ "response = co.chat(\n",
+ " message=question,\n",
+ " documents=source_nodes,\n",
+ " model=co_model\n",
+ ")\n",
+ "response = response\n",
+ "print(response.text)"
]
},
{
"cell_type": "markdown",
- "source": [
- "Again, we can print the text along with the citations."
- ],
"metadata": {
"id": "G1Ia5b-q5UKg"
- }
+ },
+ "source": [
+ "Again, we can print the text along with the citations."
+ ]
},
{
"cell_type": "code",
- "source": [
- "print(insert_citations(response.text, response.citations))"
- ],
+ "execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
@@ -727,14 +710,9 @@
"id": "yluOCX4I9WcV",
"outputId": "c5bb53b7-8029-4f4c-b48c-fc2d5f4df466"
},
- "execution_count": null,
"outputs": [
{
- "output_type": "display_data",
"data": {
- "text/plain": [
- ""
- ],
"text/html": [
"\n",
" \n",
" "
+ ],
+ "text/plain": [
+ ""
]
},
- "metadata": {}
+ "metadata": {},
+ "output_type": "display_data"
},
{
- "output_type": "stream",
"name": "stdout",
+ "output_type": "stream",
"text": [
"Elon Musk [0] mentions Jonathan Nolan. Musk is the CEO and Product Architect [0] of the lab [0] that resembles the set of Westworld [0], a show created by Jonathan Nolan [0] and Lisa Joy Nolan. [0]\n"
]
}
+ ],
+ "source": [
+ "print(insert_citations(response.text, response.citations))"
]
},
{
"cell_type": "markdown",
+ "metadata": {
+ "id": "pMleTdDCH22o"
+ },
"source": [
"And investigate the chunks which were used as context to answer the query.\n",
"\n"
- ],
- "metadata": {
- "id": "pMleTdDCH22o"
- }
+ ]
},
{
"cell_type": "code",
- "source": [
- "source_nodes[0]"
- ],
+ "execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
@@ -779,14 +762,9 @@
"id": "WdQP7ghK4GLx",
"outputId": "ab3bd918-5c7b-4411-da3a-654b87b8fa0c"
},
- "execution_count": null,
"outputs": [
{
- "output_type": "display_data",
"data": {
- "text/plain": [
- ""
- ],
"text/html": [
"\n",
" \n",
" "
+ ],
+ "text/plain": [
+ ""
]
},
- "metadata": {}
+ "metadata": {},
+ "output_type": "display_data"
},
{
- "output_type": "execute_result",
"data": {
"text/plain": [
"{'text': \"Yeah, not the best reference.\\n\\nElon Musk -- Chief Executive Officer and Product Architect\\n\\nYeah. The creators of Westworld, Jonathan Nolan, Lisa Joy Nolan, are friends -- are all friends of mine, actually. And I invited them to come see the lab and, like, well, come see it, hopefully soon. It's pretty well -- especially the sort of subsystem test stands where you've just got like one leg on a test stand just doing repetitive exercises and one arm on a test stand pretty well.\\n\\nYeah.\"}"
]
},
+ "execution_count": 14,
"metadata": {},
- "execution_count": 14
+ "output_type": "execute_result"
}
+ ],
+ "source": [
+ "source_nodes[0]"
]
},
{
"cell_type": "markdown",
- "source": [
- "As we can see, by allowing overlap we managed to get the correct answer to our question."
- ],
"metadata": {
"id": "ppL1npzNLSpQ"
- }
+ },
+ "source": [
+ "As we can see, by allowing overlap we managed to get the correct answer to our question."
+ ]
},
{
"cell_type": "markdown",
+ "metadata": {
+ "id": "1IaTId1GzwMQ"
+ },
"source": [
- "\n",
- "# Example 2: Chunking using content-dependent strategies\n",
+ "\n",
+ "## Example 2: Chunking using content-dependent strategies\n",
"\n",
"In the previous experiment, we provided an example of how using or not using overlapping can affect a model's performance, particularly in documents such as call transcripts where subjects change frequently. Ensuring that each chunk contains all relevant information is crucial. While we managed to retrieve the correct information by introducing overlapping into the chunking strategy, this might still not be the optimal approach for transcripts with longer speaker speeches.\n",
"\n",
"Therefore, in this experiment, we will adopt a content-dependent strategy.\n",
"\n",
"Our proposed approach entails segmenting the text whenever a new speaker begins speaking, which requires preprocessing the text accordingly."
- ],
- "metadata": {
- "id": "1IaTId1GzwMQ"
- }
+ ]
},
{
"cell_type": "markdown",
+ "metadata": {
+ "id": "RqRTHKg204yV"
+ },
"source": [
"### Preprocess the text\n",
"\n",
"Firstly, let's observe that in the HTML text, each time the speaker changes, their name is enclosed within `
Name
` tags, denoting the speaker's name in bold letters.\n",
"\n",
"To facilitate our text chunking process, we'll use the above observation and introduce a unique character sequence `###`, which we'll utilize as a marker for splitting the text."
- ],
- "metadata": {
- "id": "RqRTHKg204yV"
- }
+ ]
},
{
"cell_type": "code",
- "source": [
- "print('HTML text')\n",
- "print(target_divs[:3])\n",
- "print('-------------------\\n')\n",
- "\n",
- "text_custom = []\n",
- "for div in target_divs:\n",
- " if div.get_text() is None:\n",
- " continue\n",
- " if str(div).startswith('
Good afternoon, everyone, and welcome to Tesla's fourth-quarter 2023 Q&A webcast. My name is Martin Viecha, VP of investor relations, and I'm joined today by Elon Musk, Vaibhav Taneja, and a number of other executives. Our Q4 results were announced at about 3 p.m. Central Time in the update that we published at the same link as this webcast.
,
During this call, we will discuss our business outlook and make forward-looking statements. These comments are based on our predictions and expectations as of today. Actual events or results could differ materially due to a number of risks and uncertainties, including those mentioned in our most recent filings with the SEC. [Operator instructions] But before we jump into Q&A, Elon has some opening remarks.
]\n",
@@ -909,49 +877,37 @@
"During this call, we will discuss our business outlook and make forward-looking statements. These comments are based on our predictions an\n"
]
}
+ ],
+ "source": [
+ "print('HTML text')\n",
+ "print(target_divs[:3])\n",
+ "print('-------------------\\n')\n",
+ "\n",
+ "text_custom = []\n",
+ "for div in target_divs:\n",
+ " if div.get_text() is None:\n",
+ " continue\n",
+ " if str(div).startswith('
'):\n",
+ " text_custom.append(f'### {div.get_text()}')\n",
+ " else:\n",
+ " text_custom.append(div.get_text())\n",
+ "\n",
+ "text_custom = '\\n'.join(text_custom)\n",
+ "print(text_custom[:500])"
]
},
{
"cell_type": "markdown",
- "source": [
- "In this approach, we prioritize splitting the text at the appropriate separator, `###.` To ensure this behavior, we'll use `CharacterTextSplitter` from [LangChain](https://python.langchain.com/docs/get_started/introduction), guaranteeing such behavior. From our analysis of the text and the fact that we aim to preserve entire speaker speeches intact, we anticipate that most of them will exceed a length of 500. Hence, we'll increase the chunk size to 1000."
- ],
"metadata": {
"id": "6CU0gC4y7Rvv"
- }
+ },
+ "source": [
+ "In this approach, we prioritize splitting the text at the appropriate separator, `###.` To ensure this behavior, we'll use `CharacterTextSplitter` from [LangChain](https://python.langchain.com/docs/get_started/introduction), guaranteeing such behavior. From our analysis of the text and the fact that we aim to preserve entire speaker speeches intact, we anticipate that most of them will exceed a length of 500. Hence, we'll increase the chunk size to 1000."
+ ]
},
{
"cell_type": "code",
- "source": [
- "separator = \"###\"\n",
- "chunk_size = 1000\n",
- "chunk_overlap = 0\n",
- "\n",
- "text_splitter = CharacterTextSplitter(\n",
- " separator = separator,\n",
- " chunk_size=chunk_size,\n",
- " chunk_overlap=chunk_overlap,\n",
- " length_function=len,\n",
- " is_separator_regex=False,\n",
- ")\n",
- "\n",
- "documents = text_splitter.create_documents([text_custom])\n",
- "documents = [Document(text=doc.page_content) for doc in documents]\n",
- "\n",
- "retriever = build_retreiver(documents)\n",
- "\n",
- "source_nodes = retriever.retrieve(question)\n",
- "print('Number of docuemnts: ',len(source_nodes))\n",
- "source_nodes= [{\"text\": ni.get_content()}for ni in source_nodes]\n",
- "\n",
- "response = co.chat(\n",
- " message=question,\n",
- " documents=source_nodes,\n",
- " model=co_model\n",
- ")\n",
- "response = response\n",
- "print(response.text)"
- ],
+ "execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
@@ -960,14 +916,9 @@
"id": "ZFvigf6Szzev",
"outputId": "99a9e061-7cad-4308-e809-1cd38ed443e1"
},
- "execution_count": null,
"outputs": [
{
- "output_type": "display_data",
"data": {
- "text/plain": [
- ""
- ],
"text/html": [
"\n",
" \n",
" "
+ ],
+ "text/plain": [
+ ""
]
},
- "metadata": {}
+ "metadata": {},
+ "output_type": "display_data"
},
{
- "output_type": "stream",
"name": "stderr",
+ "output_type": "stream",
"text": [
"WARNING:langchain_text_splitters.base:Created a chunk of size 5946, which is longer than the specified 1000\n",
"WARNING:langchain_text_splitters.base:Created a chunk of size 4092, which is longer than the specified 1000\n",
@@ -997,29 +952,57 @@
]
},
{
- "output_type": "stream",
"name": "stdout",
+ "output_type": "stream",
"text": [
"Number of docuemnts: 5\n",
"Elon Musk mentions Jonathan Nolan. Musk is friends with the creators of Westworld, Jonathan Nolan and Lisa Joy Nolan.\n"
]
}
+ ],
+ "source": [
+ "separator = \"###\"\n",
+ "chunk_size = 1000\n",
+ "chunk_overlap = 0\n",
+ "\n",
+ "text_splitter = CharacterTextSplitter(\n",
+ " separator = separator,\n",
+ " chunk_size=chunk_size,\n",
+ " chunk_overlap=chunk_overlap,\n",
+ " length_function=len,\n",
+ " is_separator_regex=False,\n",
+ ")\n",
+ "\n",
+ "documents = text_splitter.create_documents([text_custom])\n",
+ "documents = [Document(text=doc.page_content) for doc in documents]\n",
+ "\n",
+ "retriever = build_retriever(documents)\n",
+ "\n",
+ "source_nodes = retriever.retrieve(question)\n",
+ "print('Number of docuemnts: ',len(source_nodes))\n",
+ "source_nodes= [{\"text\": ni.get_content()}for ni in source_nodes]\n",
+ "\n",
+ "response = co.chat(\n",
+ " message=question,\n",
+ " documents=source_nodes,\n",
+ " model=co_model\n",
+ ")\n",
+ "response = response\n",
+ "print(response.text)"
]
},
{
"cell_type": "markdown",
- "source": [
- "Below we validate the answer using citations."
- ],
"metadata": {
"id": "TH62W6ek7Iom"
- }
+ },
+ "source": [
+ "Below we validate the answer using citations."
+ ]
},
{
"cell_type": "code",
- "source": [
- "print(insert_citations(response.text, response.citations))"
- ],
+ "execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
@@ -1028,14 +1011,9 @@
"id": "1wPQZRMr1aX2",
"outputId": "939f6b3a-d969-487e-83d9-f64074a538e6"
},
- "execution_count": null,
"outputs": [
{
- "output_type": "display_data",
"data": {
- "text/plain": [
- ""
- ],
"text/html": [
"\n",
" \n",
" "
+ ],
+ "text/plain": [
+ ""
]
},
- "metadata": {}
+ "metadata": {},
+ "output_type": "display_data"
},
{
- "output_type": "stream",
"name": "stdout",
+ "output_type": "stream",
"text": [
"Elon Musk [0] mentions Jonathan Nolan. [0] Musk is friends [0] with the creators of Westworld [0], Jonathan Nolan [0] and Lisa Joy Nolan. [0]\n"
]
}
+ ],
+ "source": [
+ "print(insert_citations(response.text, response.citations))"
]
},
{
"cell_type": "code",
- "source": [
- "source_nodes[0]"
- ],
+ "execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
@@ -1070,14 +1053,9 @@
"id": "D9vm42za2uhE",
"outputId": "eeaa7e51-19b7-4ac5-a08a-5e9f603f40fd"
},
- "execution_count": null,
"outputs": [
{
- "output_type": "display_data",
"data": {
- "text/plain": [
- ""
- ],
"text/html": [
"\n",
" \n",
" "
+ ],
+ "text/plain": [
+ ""
]
},
- "metadata": {}
+ "metadata": {},
+ "output_type": "display_data"
},
{
- "output_type": "execute_result",
"data": {
"text/plain": [
"{'text': \"Elon Musk -- Chief Executive Officer and Product Architect\\nYeah. The creators of Westworld, Jonathan Nolan, Lisa Joy Nolan, are friends -- are all friends of mine, actually. And I invited them to come see the lab and, like, well, come see it, hopefully soon. It's pretty well -- especially the sort of subsystem test stands where you've just got like one leg on a test stand just doing repetitive exercises and one arm on a test stand pretty well.\\nYeah.\\n### Unknown speaker\\nWe're not entering Westworld anytime soon.\\n### Elon Musk -- Chief Executive Officer and Product Architect\\nRight, right. Yeah. I take -- take safety very very seriously.\\n### Martin Viecha\\nThank you. The next question from Norman is: How many Cybertruck orders are in the queue? And when do you anticipate to be able to fulfill existing orders?\"}"
]
},
+ "execution_count": 18,
"metadata": {},
- "execution_count": 18
+ "output_type": "execute_result"
}
+ ],
+ "source": [
+ "source_nodes[0]"
]
},
{
"cell_type": "markdown",
+ "metadata": {
+ "id": "uhTrS95m9x-7"
+ },
"source": [
- "\n",
- "# Discussion\n",
+ "\n",
+ "## Discussion\n",
"\n",
"This example highlights some of the concerns that arise when implementing chunking strategies. This is a field of ongoing research, and many exciting surveys have been published in domain-specific applications. For example, this [paper](https://arxiv.org/pdf/2402.05131.pdf) examines different chunking strategies in finance."
- ],
- "metadata": {
- "id": "uhTrS95m9x-7"
- }
+ ]
}
],
"metadata": {
"colab": {
- "provenance": [],
"collapsed_sections": [
"1ivVb-oMCAaz"
- ]
+ ],
+ "provenance": []
},
"kernelspec": {
"display_name": "Python 3",
"name": "python3"
},
"language_info": {
- "name": "python"
+ "codemirror_mode": {
+ "name": "ipython",
+ "version": 3
+ },
+ "file_extension": ".py",
+ "mimetype": "text/x-python",
+ "name": "python",
+ "nbconvert_exporter": "python",
+ "pygments_lexer": "ipython3",
+ "version": "3.10.13"
}
},
"nbformat": 4,
"nbformat_minor": 0
-}
\ No newline at end of file
+}
diff --git a/notebooks/guides/Creating_a_QA_bot_from_technical_documentation.ipynb b/notebooks/guides/Creating_a_QA_bot_from_technical_documentation.ipynb
index 6b01c2d4..eab362ee 100644
--- a/notebooks/guides/Creating_a_QA_bot_from_technical_documentation.ipynb
+++ b/notebooks/guides/Creating_a_QA_bot_from_technical_documentation.ipynb
@@ -1,634 +1,635 @@
{
- "nbformat": 4,
- "nbformat_minor": 0,
- "metadata": {
- "colab": {
- "provenance": []
- },
- "kernelspec": {
- "name": "python3",
- "display_name": "Python 3"
- },
- "language_info": {
- "name": "python"
- }
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "d5-gWUEqG4-T"
+ },
+ "source": [
+ "# Creating a QA bot from technical documentation\n",
+ "\n",
+ "This notebook demonstrates how to create a chatbot (single turn) that answers user questions based on technical documentation made available to the model.\n",
+ "\n",
+ "We use the `aws-documentation` dataset ([link](https://github.com/siagholami/aws-documentation/tree/main)) for representativeness. This dataset contains 26k+ AWS documentation pages, preprocessed into 120k+ chunks, and 100 questions based on real user questions.\n",
+ "\n",
+ "We proceed as follows:\n",
+ "1. Embed the AWS documentation into a vector database using Cohere embeddings and `llama_index`\n",
+ "2. Build a retriever using Cohere's `rerank` for better accuracy, lower inference costs and lower latency\n",
+ "3. Create model answers for the eval set of 100 questions\n",
+ "4. Evaluate the model answers against the golden answers of the eval set\n"
+ ]
},
- "cells": [
- {
- "cell_type": "markdown",
- "source": [
- "# Creating a QA bot from technical documentation\n",
- "\n",
- "This notebook demonstrates how to create a chatbot (single turn) that answers user questions based on technical documentation made available to the model.\n",
- "\n",
- "We use the `aws-documentation` dataset ([link](https://github.com/siagholami/aws-documentation/tree/main)) for representativeness. This dataset contains 26k+ AWS documentation pages, preprocessed into 120k+ chunks, and 100 questions based on real user questions.\n",
- "\n",
- "We proceed as follows:\n",
- "1. Embed the AWS documentation into a vector database using Cohere embeddings and `llama_index`\n",
- "2. Build a retriever using Cohere's `rerank` for better accuracy, lower inference costs and lower latency\n",
- "3. Create model answers for the eval set of 100 questions\n",
- "4. Evaluate the model answers against the golden answers of the eval set\n"
- ],
- "metadata": {
- "id": "d5-gWUEqG4-T"
- }
- },
- {
- "cell_type": "markdown",
- "source": [
- "## Setup"
- ],
- "metadata": {
- "id": "JY2wHt3AG7X0"
- }
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "id": "wajdtJmNG0Uy"
- },
- "outputs": [],
- "source": [
- "%%capture\n",
- "!pip install cohere datasets llama_index llama-index-llms-cohere llama-index-embeddings-cohere"
- ]
- },
- {
- "cell_type": "code",
- "source": [
- "import cohere\n",
- "import datasets\n",
- "from llama_index.core import StorageContext, VectorStoreIndex, load_index_from_storage\n",
- "from llama_index.core.schema import TextNode\n",
- "from llama_index.embeddings.cohere import CohereEmbedding\n",
- "import pandas as pd\n",
- "\n",
- "import json\n",
- "from pathlib import Path\n",
- "from tqdm import tqdm\n",
- "from typing import List\n"
- ],
- "metadata": {
- "id": "kJ_9dyJxG0zA"
- },
- "execution_count": null,
- "outputs": []
- },
- {
- "cell_type": "code",
- "source": [
- "# Set up Cohere client\n",
- "api_key = \"\" # \n",
- "co = cohere.Client(api_key=api_key)"
- ],
- "metadata": {
- "id": "EECuI7OdIy8M"
- },
- "execution_count": null,
- "outputs": []
- },
- {
- "cell_type": "markdown",
- "source": [
- "## 1. Embed technical documentation and store as vector database\n",
- "\n",
- "* Load the dataset from HuggingFace\n",
- "* Compute embeddings using Cohere's implementation in LlamaIndex, `CohereEmbedding`\n",
- "* Store inside a vector database, `VectorStoreIndex` from LlamaIndex\n",
- "\n",
- "\n",
- "Because this process is lengthy (~2h for all documents on a MacBookPro), we store the index to disc for future reuse. We also provide a (commented) code snippet to index only a subset of the data. If you use this snippet, bear in mind that many documents will become unavailable to the model and, as a result, performance will suffer!"
- ],
- "metadata": {
- "id": "u_UfRVoBIHmD"
- }
- },
- {
- "cell_type": "code",
- "source": [
- "data = datasets.load_dataset(\"sauravjoshi23/aws-documentation-chunked\")\n",
- "print(data)\n",
- "# The data comes prechunked. We keep the data as-is in this notebook.\n",
- "# For more information on optimal preprocessing strategies, please check\n",
- "# our other notebooks!\n",
- "\n",
- "# Build a mapping from sample id to index inside data (will be useful for retrieval later)\n",
- "map_id2index = {sample[\"id\"]: index for index, sample in enumerate(data[\"train\"])}\n"
- ],
- "metadata": {
- "colab": {
- "base_uri": "https://localhost:8080/"
- },
- "id": "NoXezqvoG02f",
- "outputId": "cabfccf3-3c15-4955-dab9-e2734fa65a7d"
- },
- "execution_count": null,
- "outputs": [
- {
- "output_type": "stream",
- "name": "stderr",
- "text": [
- "/usr/local/lib/python3.10/dist-packages/huggingface_hub/utils/_token.py:88: UserWarning: \n",
- "The secret `HF_TOKEN` does not exist in your Colab secrets.\n",
- "To authenticate with the Hugging Face Hub, create a token in your settings tab (https://huggingface.co/settings/tokens), set it as secret in your Google Colab and restart your session.\n",
- "You will be able to reuse this secret in all of your notebooks.\n",
- "Please note that authentication is recommended but still optional to access public models or datasets.\n",
- " warnings.warn(\n"
- ]
- },
- {
- "output_type": "stream",
- "name": "stdout",
- "text": [
- "DatasetDict({\n",
- " train: Dataset({\n",
- " features: ['id', 'text', 'source'],\n",
- " num_rows: 187147\n",
- " })\n",
- "})\n"
- ]
- }
- ]
- },
- {
- "cell_type": "code",
- "source": [
- "# Create index in vector database, and persist it for later reuse\n",
- "# Note: this cell takes about ~2h on a MacBookPro\n",
- "\n",
- "overwrite = True # only compute index if it doesn't exist\n",
- "path_index = Path(\".\") / \"aws-documentation_index_cohere\"\n",
- "\n",
- "# Select Cohere's new `embed-english-v3.0` as the engine to compute embeddings\n",
- "embed_model = CohereEmbedding(\n",
- " cohere_api_key=api_key,\n",
- " model_name=\"embed-english-v3.0\",\n",
- ")\n",
- "\n",
- "if not path_index.exists() or overwrite:\n",
- " # Documents are prechunked. Keep them as-is for now\n",
- " stub_len = len(\"https://github.com/siagholami/aws-documentation/tree/main/documents/\")\n",
- " documents = [\n",
- " # -- for indexing full dataset --\n",
- " TextNode(\n",
- " text=sample[\"text\"],\n",
- " title=sample[\"source\"][stub_len:], # save source minus stub\n",
- " id_=sample[\"id\"],\n",
- " ) for sample in data[\"train\"]\n",
- " # -- for testing on subset --\n",
- " # TextNode(\n",
- " # text=data[\"train\"][index][\"text\"],\n",
- " # title=data[\"train\"][index][\"source\"][stub_len:],\n",
- " # id_=data[\"train\"][index][\"id\"],\n",
- " # ) for index in range(1_000)\n",
- " ]\n",
- " index = VectorStoreIndex(documents, embed_model=embed_model)\n",
- " index.storage_context.persist(path_index)\n",
- "\n",
- "else:\n",
- " storage_context = StorageContext.from_defaults(persist_dir=path_index)\n",
- " index = load_index_from_storage(storage_context, embed_model=embed_model)\n"
- ],
- "metadata": {
- "id": "tvgtKDBTG05h"
- },
- "execution_count": null,
- "outputs": []
- },
- {
- "cell_type": "markdown",
- "source": [
- "## 2. Build a retriever using Cohere's `rerank`\n",
- "\n",
- "The vector database we built using `VectorStoreIndex` comes with an in-built retriever. We can call that retriever to fetch the top $k$ documents most relevant to the user question with:\n",
- "\n",
- "```python\n",
- "retriever = index.as_retriever(similarity_top_k=top_k)\n",
- "```\n",
- "\n",
- "We recently released [Rerank-3](https://txt.cohere.com/rerank-3/) (April '24), which we can use to improve the quality of retrieval, as well as reduce latency and the cost of inference. To use the retriever with `rerank`, we create a thin wrapper around `index.as_retriever` as follows:"
- ],
- "metadata": {
- "id": "sS2yzRwOKHTv"
- }
- },
- {
- "cell_type": "code",
- "source": [
- "class RetrieverWithRerank:\n",
- " def __init__(self, retriever, api_key):\n",
- " self.retriever = retriever\n",
- " self.co = cohere.Client(api_key=api_key)\n",
- "\n",
- " def retrieve(self, query: str, top_n: int):\n",
- " # First call to the retriever fetches the closest indices\n",
- " nodes = self.retriever.retrieve(query)\n",
- " nodes = [\n",
- " {\n",
- " \"text\": node.node.text,\n",
- " \"llamaindex_id\": node.node.id_,\n",
- " }\n",
- " for node\n",
- " in nodes\n",
- " ]\n",
- " # Call co.rerank to improve the relevance of retrieved documents\n",
- " reranked = self.co.rerank(query=query, documents=nodes, model=\"rerank-english-v3.0\", top_n=top_n)\n",
- " nodes = [nodes[node.index] for node in reranked.results]\n",
- " return nodes\n",
- "\n",
- "\n",
- "top_k = 60 # how many documents to fetch on first pass\n",
- "top_n = 20 # how many documents to sub-select with rerank\n",
- "\n",
- "# Instantiate retriver\n",
- "retriever = RetrieverWithRerank(\n",
- " index.as_retriever(similarity_top_k=top_k),\n",
- " api_key=api_key,\n",
- ")\n"
- ],
- "metadata": {
- "id": "Wy_BGGbGG08w"
- },
- "execution_count": null,
- "outputs": []
- },
- {
- "cell_type": "code",
- "source": [
- "# Test the retriever on a single question!\n",
- "query = \"What happens to my Amazon EC2 instances if I delete my Auto Scaling group?\"\n",
- "\n",
- "# Retrieving relevant documents with rerank now fits in one line\n",
- "documents = retriever.retrieve(query, top_n=top_n)\n",
- "\n",
- "# Call Cohere's RAG pipeline with co.chat and the `documents` argument\n",
- "resp = co.chat(message=query, model=\"command-r\", temperature=0., documents=documents)\n",
- "print(resp.text)\n"
- ],
- "metadata": {
- "id": "ODihI1YCG0_5"
- },
- "execution_count": null,
- "outputs": []
- },
- {
- "cell_type": "markdown",
- "source": [
- "This works! With `co.chat`, you get the additional benefit that citations are returned for every span of text. Here's a simple function to display the citations inside square brackets."
- ],
- "metadata": {
- "id": "tHTKtvMTMLhz"
- }
- },
- {
- "cell_type": "code",
- "source": [
- "def build_answer_with_citations(response):\n",
- " \"\"\" \"\"\"\n",
- " text = response.text\n",
- " citations = response.citations\n",
- "\n",
- " # Construct text_with_citations adding citation spans as we iterate through citations\n",
- " end = 0\n",
- " text_with_citations = \"\"\n",
- "\n",
- " for citation in citations:\n",
- " # Add snippet between last citatiton and current citation\n",
- " start = citation.start\n",
- " text_with_citations += text[end : start]\n",
- " end = citation.end # overwrite\n",
- " citation_blocks = \" [\" + \", \".join([stub[4:] for stub in citation.document_ids]) + \"] \"\n",
- " text_with_citations += text[start : end] + citation_blocks\n",
- " # Add any left-over\n",
- " text_with_citations += text[end:]\n",
- "\n",
- " return text_with_citations\n",
- "\n",
- "grounded_answer = build_answer_with_citations(resp)\n",
- "print(grounded_answer)\n"
- ],
- "metadata": {
- "id": "7fH5mv2PG1DI"
- },
- "execution_count": null,
- "outputs": []
- },
- {
- "cell_type": "markdown",
- "source": [
- "## 3. Create model answers for 100 QA pairs\n",
- "\n",
- "Now that we have a running pipeline, we need to assess its performance.\n",
- "\n",
- "The author of the repository provides 100 QA pairs that we can test the model on. Let's download these questions, then run inference on all 100 questions. Later, we will use Command-R+ -- Cohere's largest and most powerful model -- to measure performance."
- ],
- "metadata": {
- "id": "f1mJdBQYRI4k"
- }
- },
- {
- "cell_type": "code",
- "source": [
- "# Load data from github\n",
- "url = \"https://github.com/siagholami/aws-documentation/blob/main/QA_true.csv?raw=true\"\n",
- "qa_pairs = pd.read_csv(url)\n",
- "qa_pairs.sample(2)\n"
- ],
- "metadata": {
- "id": "jWkBVKSvMlip"
- },
- "execution_count": null,
- "outputs": []
- },
- {
- "cell_type": "markdown",
- "source": [
- "We'll use the fields as follows:\n",
- "* `Question`: the user question, passed to `co.chat` to generate the answer\n",
- "* `Answer_True`: treat as the ground gruth; compare to the model-generated answer to determine its correctness\n",
- "* `Document_True`: treat as the (single) golden document; check the rank of this document inside the model's retrieved documents"
- ],
- "metadata": {
- "id": "eS_IDKnvcI40"
- }
- },
- {
- "cell_type": "markdown",
- "source": [
- "We'll loop over each question and generate our model answer. We'll also complete two steps that will be useful for evaluating our model next:\n",
- "1. We compute the rank of the golden document amid the retrieved documents -- this will inform how well our retrieval system performs\n",
- "2. We prepare the grading prompts -- these will be sent to an LLM scorer to compute the goodness of responses"
- ],
- "metadata": {
- "id": "bi4gH235eMzA"
- }
- },
- {
- "cell_type": "code",
- "source": [
- "# Define the LLM eval prompt\n",
- "# We request a score and a reason for assigning that score to 'trigger CoT' and\n",
- "# improve the model response\n",
- "\n",
- "LLM_EVAL_TEMPLATE = \"\"\"## References\n",
- "{references}\n",
- "\n",
- "QUESTION: based on the above reference documents, answer the following question: {question}\n",
- "ANSWER: {answer}\n",
- "STUDENT RESPONSE: {completion}\n",
- "\n",
- "Based on the question and answer above, grade the studen't reponse. A correct response will contain exactly \\\n",
- "the same information as in the answer, even if it is worded differently. If the student's reponse is correct, \\\n",
- "give it a score of 1. Otherwise, give it a score of 0. Let's think step by step. Return your answer as \\\n",
- "as a compilable JSON with the following structure:\n",
- "{{\n",
- " \"reasoning\": ,\n",
- " \"score: ,\n",
- "}}\"\"\"\n",
- "\n",
- "\n",
- "def get_rank_of_golden_within_retrieved(golden: str, retrieved: List[dict]) -> int:\n",
- " \"\"\"\n",
- " Returns the rank that the golden document (single) has within the retrieved documents\n",
- " * `golden` contains the source of the document, e.g. 'amazon-ec2-user-guide/EBSEncryption.md'\n",
- " * `retrieved` has a list of responses with key 'llamaindex_id', which links back to document sources\n",
- " \"\"\"\n",
- " # Create {document: rank} map using llamaindex_id (count first occurrence of any document; they can\n",
- " # appear multiple times because they're chunked)\n",
- " doc_to_rank = {}\n",
- " for rank, doc in enumerate(retrieved):\n",
- " # retrieve source of document\n",
- " _id = doc[\"llamaindex_id\"]\n",
- " source = data[\"train\"][map_id2index[_id]][\"source\"]\n",
- " # format as in dataset\n",
- " source = source[stub_len:] # remove stub\n",
- " source = source.replace(\"/doc_source\", \"\") # remove /doc_source/\n",
- " if source not in doc_to_rank:\n",
- " doc_to_rank[source] = rank + 1\n",
- "\n",
- " # Return rank of `golden`, defaulting to len(retrieved) + 1 if it's absent\n",
- " return doc_to_rank.get(golden, len(retrieved) + 1)\n"
- ],
- "metadata": {
- "id": "rR67DcP5epAV"
- },
- "execution_count": null,
- "outputs": []
- },
- {
- "cell_type": "code",
- "source": [
- "from tqdm import tqdm\n",
- "\n",
- "answers = []\n",
- "golden_answers = []\n",
- "ranks = []\n",
- "grading_prompts = [] # best computed in batch\n",
- "\n",
- "for _, row in tqdm(qa_pairs.iterrows(), total=len(qa_pairs)):\n",
- " query, golden_answer, golden_doc = row[\"Question\"], row[\"Answer_True\"], row[\"Document_True\"]\n",
- " golden_answers.append(golden_answer)\n",
- "\n",
- " # --- Produce answer using retriever ---\n",
- " documents = retriever.retrieve(query, top_n=top_n)\n",
- " resp = co.chat(message=query, model=\"command-r\", temperature=0., documents=documents)\n",
- " answer = resp.text\n",
- " answers.append(answer)\n",
- "\n",
- " # --- Do some prework for evaluation later ---\n",
- " # Rank\n",
- " rank = get_rank_of_golden_within_retrieved(golden_doc, documents)\n",
- " ranks.append(rank)\n",
- " # Score: construct the grading prompts for LLM evals, then evaluate in batch\n",
- " # Need to reformat documents slightly\n",
- " documents = [{\"index\": str(i), \"text\": doc[\"text\"]} for i, doc in enumerate(documents)]\n",
- " references_text = \"\\n\\n\".join(\"\\n\".join([f\"{k}: {v}\" for k, v in doc.items()]) for doc in documents)\n",
- " # ^ snippet looks complicated, but all it does it unpack all kwargs from `documents`\n",
- " # into text separated by \\n\\n\n",
- " grading_prompt = LLM_EVAL_TEMPLATE.format(\n",
- " references=references_text, question=query, answer=golden_answer, completion=answer,\n",
- " )\n",
- " grading_prompts.append(grading_prompt)\n"
- ],
- "metadata": {
- "id": "tAg3MTOcMll4"
- },
- "execution_count": null,
- "outputs": []
- },
- {
- "cell_type": "markdown",
- "source": [
- "## 4. Evaluate model performance\n",
- "\n",
- "We want to test our model performance on two dimensions:\n",
- "1. How good is the final answer? We'll compare our model answer to the golden answer using Command-R+ as a judge.\n",
- "2. How good is the retrieval? We'll use the rank of the golden document within the retrieved documents to this end.\n",
- "\n",
- "Note that this pipeline is for illustration only. To measure performance in practice, we would want to run more in-depths tests on a broader, representative dataset."
- ],
- "metadata": {
- "id": "2BCs7ozyO1vL"
- }
- },
- {
- "cell_type": "code",
- "source": [
- "# For simplicity, prepare a DataFrame with the results\n",
- "results = pd.DataFrame()\n",
- "results[\"answer\"] = answers\n",
- "results[\"golden_answer\"] = qa_pairs[\"Answer_True\"]\n",
- "results[\"rank\"] = ranks\n"
- ],
- "metadata": {
- "id": "1qBvMpYmQDTK"
- },
- "execution_count": null,
- "outputs": []
- },
- {
- "cell_type": "markdown",
- "source": [
- "### 4.1 Compare answer to golden answer\n",
- "\n",
- "We'll use Command-R+ as a judge of whether the answers produced by our model convey the same information as the golden answers. Since we've defined the grading prompts earlier, we can simply ask our LLM judge to evaluate that grading prompt. After a little bit of postprocessing, we can then extract our model scores."
- ],
- "metadata": {
- "id": "8Wglx0dsQtgX"
- }
- },
- {
- "cell_type": "code",
- "source": [
- "scores = []\n",
- "reasonings = []\n",
- "\n",
- "def remove_backticks(text: str) -> str:\n",
- " \"\"\"\n",
- " Some models are trained to output JSON in Markdown formatting:\n",
- " ```json {json object}```\n",
- " Remove the backticks from those model responses so that they become\n",
- " parasable by json.loads.\n",
- " \"\"\"\n",
- " if text.startswith(\"```json\"):\n",
- " text = text[7:]\n",
- " if text.endswith(\"```\"):\n",
- " text = text[:-3]\n",
- " return text\n",
- "\n",
- "\n",
- "for prompt in tqdm(grading_prompts, total=len(grading_prompts)):\n",
- " resp = co.chat(message=prompt, model=\"command-r-plus\", temperature=0.)\n",
- " # Convert response to JSON to extract the `score` and `reasoning` fields\n",
- " # We remove backticks for compatibility with different LLMs\n",
- " parsed = json.loads(remove_backticks(resp.text))\n",
- " scores.append(parsed[\"score\"])\n",
- " reasonings.append(parsed[\"reasoning\"])\n"
- ],
- "metadata": {
- "id": "_F3V4E56Q1CE"
- },
- "execution_count": null,
- "outputs": []
- },
- {
- "cell_type": "code",
- "source": [
- "# Add scores to our DataFrame\n",
- "results[\"score\"] = scores\n",
- "results[\"reasoning\"] = reasonings"
- ],
- "metadata": {
- "id": "1xksywOeQ1IJ"
- },
- "execution_count": null,
- "outputs": []
- },
- {
- "cell_type": "code",
- "source": [
- "print(f\"Average score: {results['score'].mean():.3f}\")\n"
- ],
- "metadata": {
- "id": "l8Z2w1ERSeHZ"
- },
- "execution_count": null,
- "outputs": []
- },
- {
- "cell_type": "markdown",
- "source": [
- "### 4.2 Compute rank\n",
- "\n",
- "We've already computed the rank of the golden documents using `get_rank_of_golden_within_retrieved`. Here, we'll plot the histogram of ranks, using blue when the answer scored a 1, and red when the answer scored a 0."
- ],
- "metadata": {
- "id": "ob-Km6dAPeHJ"
- }
- },
- {
- "cell_type": "code",
- "source": [
- "import matplotlib.pyplot as plt\n",
- "import seaborn as sns\n",
- "\n",
- "sns.set_theme(style=\"darkgrid\", rc={\"grid.color\": \".8\"})\n",
- "\n",
- "# create offsets for better vis\n",
- "results[\"rank_shifted_left\"] = results[\"rank\"] - 0.1\n",
- "results[\"rank_shifted_right\"] = results[\"rank\"] + 0.1\n",
- "\n",
- "f, ax = plt.subplots(figsize=(5, 3))\n",
- "sns.histplot(data=results.loc[results[\"score\"] == 1], x=\"rank_shifted_left\", color=\"skyblue\", label=\"Correct answer\", binwidth=1)\n",
- "sns.histplot(data=results.loc[results[\"score\"] == 0], x=\"rank_shifted_right\", color=\"red\", label=\"False answer\", binwidth=1)\n",
- "\n",
- "ax.set_xticks([1, 5, 0, 10, 15, 20])\n",
- "ax.set_title(\"Rank of golden document (max means golden doc. wasn't retrieved)\")\n",
- "ax.set_xlabel(\"Rank\")\n",
- "ax.legend();\n"
- ],
- "metadata": {
- "id": "SLn3s3n_MlpO"
- },
- "execution_count": null,
- "outputs": []
- },
- {
- "cell_type": "markdown",
- "source": [
- "We see that retrieval works well overall: for 80% of questions, the golden document is within the top 5 documents. However, we also notice that approx. half the false answers come from instances where the golden document wasn't retrieved (`rank = top_k = 20`). This should be improved, e.g. by adding metadata to the documents such as their section headings, or altering the chunking strategy.\n",
- "\n",
- "There is also a non-negligible instance of false answers where the top document was retrieved. On closer inspection, many of these are due to the model phrasing its answers more verbosely than the (very laconic) golden documents. This highlights the importance of checking eval results before jumping to conclusions about model performance."
- ],
- "metadata": {
- "id": "PZeTt7ijVKxo"
- }
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "JY2wHt3AG7X0"
+ },
+ "source": [
+ "## Setup"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "wajdtJmNG0Uy"
+ },
+ "outputs": [],
+ "source": [
+ "%%capture\n",
+ "!pip install cohere datasets llama_index llama-index-llms-cohere llama-index-embeddings-cohere"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "kJ_9dyJxG0zA"
+ },
+ "outputs": [],
+ "source": [
+ "import cohere\n",
+ "import datasets\n",
+ "from llama_index.core import StorageContext, VectorStoreIndex, load_index_from_storage\n",
+ "from llama_index.core.schema import TextNode\n",
+ "from llama_index.embeddings.cohere import CohereEmbedding\n",
+ "import pandas as pd\n",
+ "\n",
+ "import json\n",
+ "from pathlib import Path\n",
+ "from tqdm import tqdm\n",
+ "from typing import List\n"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "EECuI7OdIy8M"
+ },
+ "outputs": [],
+ "source": [
+ "# Set up Cohere client\n",
+ "api_key = \"\" # \n",
+ "co = cohere.Client(api_key=api_key)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "u_UfRVoBIHmD"
+ },
+ "source": [
+ "## 1. Embed technical documentation and store as vector database\n",
+ "\n",
+ "* Load the dataset from HuggingFace\n",
+ "* Compute embeddings using Cohere's implementation in LlamaIndex, `CohereEmbedding`\n",
+ "* Store inside a vector database, `VectorStoreIndex` from LlamaIndex\n",
+ "\n",
+ "\n",
+ "Because this process is lengthy (~2h for all documents on a MacBookPro), we store the index to disc for future reuse. We also provide a (commented) code snippet to index only a subset of the data. If you use this snippet, bear in mind that many documents will become unavailable to the model and, as a result, performance will suffer!"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "colab": {
+ "base_uri": "https://localhost:8080/"
},
+ "id": "NoXezqvoG02f",
+ "outputId": "cabfccf3-3c15-4955-dab9-e2734fa65a7d"
+ },
+ "outputs": [
{
- "cell_type": "markdown",
- "source": [
- "## Conclusions\n",
- "\n",
- "In this notebook, we've built a QA bot that answers user questions based on technical documentation. We've learnt:\n",
- "\n",
- "1. How to embed the technical documentation into a vector database using Cohere embeddings and `llama_index`\n",
- "2. How to build a custom retriever that leverages Cohere's `rerank`\n",
- "3. How to evaluate model performance against a predetermined set of golden QA pairs\n",
- "\n"
- ],
- "metadata": {
- "id": "4Q5i8EYDV_da"
- }
+ "name": "stderr",
+ "output_type": "stream",
+ "text": [
+ "/usr/local/lib/python3.10/dist-packages/huggingface_hub/utils/_token.py:88: UserWarning: \n",
+ "The secret `HF_TOKEN` does not exist in your Colab secrets.\n",
+ "To authenticate with the Hugging Face Hub, create a token in your settings tab (https://huggingface.co/settings/tokens), set it as secret in your Google Colab and restart your session.\n",
+ "You will be able to reuse this secret in all of your notebooks.\n",
+ "Please note that authentication is recommended but still optional to access public models or datasets.\n",
+ " warnings.warn(\n"
+ ]
},
{
- "cell_type": "code",
- "source": [],
- "metadata": {
- "id": "UC2FKrkSWcPn"
- },
- "execution_count": null,
- "outputs": []
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "DatasetDict({\n",
+ " train: Dataset({\n",
+ " features: ['id', 'text', 'source'],\n",
+ " num_rows: 187147\n",
+ " })\n",
+ "})\n"
+ ]
}
- ]
-}
\ No newline at end of file
+ ],
+ "source": [
+ "data = datasets.load_dataset(\"sauravjoshi23/aws-documentation-chunked\")\n",
+ "print(data)\n",
+ "# The data comes prechunked. We keep the data as-is in this notebook.\n",
+ "# For more information on optimal preprocessing strategies, please check\n",
+ "# our other notebooks!\n",
+ "\n",
+ "# Build a mapping from sample id to index inside data (will be useful for retrieval later)\n",
+ "map_id2index = {sample[\"id\"]: index for index, sample in enumerate(data[\"train\"])}\n"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "tvgtKDBTG05h"
+ },
+ "outputs": [],
+ "source": [
+ "# Create index in vector database, and persist it for later reuse\n",
+ "# Note: this cell takes about ~2h on a MacBookPro\n",
+ "\n",
+ "overwrite = True # only compute index if it doesn't exist\n",
+ "path_index = Path(\".\") / \"aws-documentation_index_cohere\"\n",
+ "\n",
+ "# Select Cohere's new `embed-english-v3.0` as the engine to compute embeddings\n",
+ "embed_model = CohereEmbedding(\n",
+ " cohere_api_key=api_key,\n",
+ " model_name=\"embed-english-v3.0\",\n",
+ ")\n",
+ "\n",
+ "if not path_index.exists() or overwrite:\n",
+ " # Documents are prechunked. Keep them as-is for now\n",
+ " stub_len = len(\"https://github.com/siagholami/aws-documentation/tree/main/documents/\")\n",
+ " documents = [\n",
+ " # -- for indexing full dataset --\n",
+ " TextNode(\n",
+ " text=sample[\"text\"],\n",
+ " title=sample[\"source\"][stub_len:], # save source minus stub\n",
+ " id_=sample[\"id\"],\n",
+ " ) for sample in data[\"train\"]\n",
+ " # -- for testing on subset --\n",
+ " # TextNode(\n",
+ " # text=data[\"train\"][index][\"text\"],\n",
+ " # title=data[\"train\"][index][\"source\"][stub_len:],\n",
+ " # id_=data[\"train\"][index][\"id\"],\n",
+ " # ) for index in range(1_000)\n",
+ " ]\n",
+ " index = VectorStoreIndex(documents, embed_model=embed_model)\n",
+ " index.storage_context.persist(path_index)\n",
+ "\n",
+ "else:\n",
+ " storage_context = StorageContext.from_defaults(persist_dir=path_index)\n",
+ " index = load_index_from_storage(storage_context, embed_model=embed_model)\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "sS2yzRwOKHTv"
+ },
+ "source": [
+ "## 2. Build a retriever using Cohere's `rerank`\n",
+ "\n",
+ "The vector database we built using `VectorStoreIndex` comes with an in-built retriever. We can call that retriever to fetch the top $k$ documents most relevant to the user question with:\n",
+ "\n",
+ "```python\n",
+ "retriever = index.as_retriever(similarity_top_k=top_k)\n",
+ "```\n",
+ "\n",
+ "We recently released [Rerank-3](https://txt.cohere.com/rerank-3/) (April '24), which we can use to improve the quality of retrieval, as well as reduce latency and the cost of inference. To use the retriever with `rerank`, we create a thin wrapper around `index.as_retriever` as follows:"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "Wy_BGGbGG08w"
+ },
+ "outputs": [],
+ "source": [
+ "class RetrieverWithRerank:\n",
+ " def __init__(self, retriever, api_key):\n",
+ " self.retriever = retriever\n",
+ " self.co = cohere.Client(api_key=api_key)\n",
+ "\n",
+ " def retrieve(self, query: str, top_n: int):\n",
+ " # First call to the retriever fetches the closest indices\n",
+ " nodes = self.retriever.retrieve(query)\n",
+ " nodes = [\n",
+ " {\n",
+ " \"text\": node.node.text,\n",
+ " \"llamaindex_id\": node.node.id_,\n",
+ " }\n",
+ " for node\n",
+ " in nodes\n",
+ " ]\n",
+ " # Call co.rerank to improve the relevance of retrieved documents\n",
+ " reranked = self.co.rerank(query=query, documents=nodes, model=\"rerank-english-v3.0\", top_n=top_n)\n",
+ " nodes = [nodes[node.index] for node in reranked.results]\n",
+ " return nodes\n",
+ "\n",
+ "\n",
+ "top_k = 60 # how many documents to fetch on first pass\n",
+ "top_n = 20 # how many documents to sub-select with rerank\n",
+ "\n",
+ "# Instantiate retriver\n",
+ "retriever = RetrieverWithRerank(\n",
+ " index.as_retriever(similarity_top_k=top_k),\n",
+ " api_key=api_key,\n",
+ ")\n"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "ODihI1YCG0_5"
+ },
+ "outputs": [],
+ "source": [
+ "# Test the retriever on a single question!\n",
+ "query = \"What happens to my Amazon EC2 instances if I delete my Auto Scaling group?\"\n",
+ "\n",
+ "# Retrieving relevant documents with rerank now fits in one line\n",
+ "documents = retriever.retrieve(query, top_n=top_n)\n",
+ "\n",
+ "# Call Cohere's RAG pipeline with co.chat and the `documents` argument\n",
+ "resp = co.chat(message=query, model=\"command-r\", temperature=0., documents=documents)\n",
+ "print(resp.text)\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "tHTKtvMTMLhz"
+ },
+ "source": [
+ "This works! With `co.chat`, you get the additional benefit that citations are returned for every span of text. Here's a simple function to display the citations inside square brackets."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "7fH5mv2PG1DI"
+ },
+ "outputs": [],
+ "source": [
+ "def build_answer_with_citations(response):\n",
+ " \"\"\" \"\"\"\n",
+ " text = response.text\n",
+ " citations = response.citations\n",
+ "\n",
+ " # Construct text_with_citations adding citation spans as we iterate through citations\n",
+ " end = 0\n",
+ " text_with_citations = \"\"\n",
+ "\n",
+ " for citation in citations:\n",
+ " # Add snippet between last citatiton and current citation\n",
+ " start = citation.start\n",
+ " text_with_citations += text[end : start]\n",
+ " end = citation.end # overwrite\n",
+ " citation_blocks = \" [\" + \", \".join([stub[4:] for stub in citation.document_ids]) + \"] \"\n",
+ " text_with_citations += text[start : end] + citation_blocks\n",
+ " # Add any left-over\n",
+ " text_with_citations += text[end:]\n",
+ "\n",
+ " return text_with_citations\n",
+ "\n",
+ "grounded_answer = build_answer_with_citations(resp)\n",
+ "print(grounded_answer)\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "f1mJdBQYRI4k"
+ },
+ "source": [
+ "## 3. Create model answers for 100 QA pairs\n",
+ "\n",
+ "Now that we have a running pipeline, we need to assess its performance.\n",
+ "\n",
+ "The author of the repository provides 100 QA pairs that we can test the model on. Let's download these questions, then run inference on all 100 questions. Later, we will use Command-R+ -- Cohere's largest and most powerful model -- to measure performance."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "jWkBVKSvMlip"
+ },
+ "outputs": [],
+ "source": [
+ "# Load data from github\n",
+ "url = \"https://github.com/siagholami/aws-documentation/blob/main/QA_true.csv?raw=true\"\n",
+ "qa_pairs = pd.read_csv(url)\n",
+ "qa_pairs.sample(2)\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "eS_IDKnvcI40"
+ },
+ "source": [
+ "We'll use the fields as follows:\n",
+ "* `Question`: the user question, passed to `co.chat` to generate the answer\n",
+ "* `Answer_True`: treat as the ground gruth; compare to the model-generated answer to determine its correctness\n",
+ "* `Document_True`: treat as the (single) golden document; check the rank of this document inside the model's retrieved documents"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "bi4gH235eMzA"
+ },
+ "source": [
+ "We'll loop over each question and generate our model answer. We'll also complete two steps that will be useful for evaluating our model next:\n",
+ "1. We compute the rank of the golden document amid the retrieved documents -- this will inform how well our retrieval system performs\n",
+ "2. We prepare the grading prompts -- these will be sent to an LLM scorer to compute the goodness of responses"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "rR67DcP5epAV"
+ },
+ "outputs": [],
+ "source": [
+ "# Define the LLM eval prompt\n",
+ "# We request a score and a reason for assigning that score to 'trigger CoT' and\n",
+ "# improve the model response\n",
+ "\n",
+ "LLM_EVAL_TEMPLATE = \"\"\"## References\n",
+ "{references}\n",
+ "\n",
+ "QUESTION: based on the above reference documents, answer the following question: {question}\n",
+ "ANSWER: {answer}\n",
+ "STUDENT RESPONSE: {completion}\n",
+ "\n",
+ "Based on the question and answer above, grade the studen't reponse. A correct response will contain exactly \\\n",
+ "the same information as in the answer, even if it is worded differently. If the student's reponse is correct, \\\n",
+ "give it a score of 1. Otherwise, give it a score of 0. Let's think step by step. Return your answer as \\\n",
+ "as a compilable JSON with the following structure:\n",
+ "{{\n",
+ " \"reasoning\": ,\n",
+ " \"score: ,\n",
+ "}}\"\"\"\n",
+ "\n",
+ "\n",
+ "def get_rank_of_golden_within_retrieved(golden: str, retrieved: List[dict]) -> int:\n",
+ " \"\"\"\n",
+ " Returns the rank that the golden document (single) has within the retrieved documents\n",
+ " * `golden` contains the source of the document, e.g. 'amazon-ec2-user-guide/EBSEncryption.md'\n",
+ " * `retrieved` has a list of responses with key 'llamaindex_id', which links back to document sources\n",
+ " \"\"\"\n",
+ " # Create {document: rank} map using llamaindex_id (count first occurrence of any document; they can\n",
+ " # appear multiple times because they're chunked)\n",
+ " doc_to_rank = {}\n",
+ " for rank, doc in enumerate(retrieved):\n",
+ " # retrieve source of document\n",
+ " _id = doc[\"llamaindex_id\"]\n",
+ " source = data[\"train\"][map_id2index[_id]][\"source\"]\n",
+ " # format as in dataset\n",
+ " source = source[stub_len:] # remove stub\n",
+ " source = source.replace(\"/doc_source\", \"\") # remove /doc_source/\n",
+ " if source not in doc_to_rank:\n",
+ " doc_to_rank[source] = rank + 1\n",
+ "\n",
+ " # Return rank of `golden`, defaulting to len(retrieved) + 1 if it's absent\n",
+ " return doc_to_rank.get(golden, len(retrieved) + 1)\n"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "tAg3MTOcMll4"
+ },
+ "outputs": [],
+ "source": [
+ "from tqdm import tqdm\n",
+ "\n",
+ "answers = []\n",
+ "golden_answers = []\n",
+ "ranks = []\n",
+ "grading_prompts = [] # best computed in batch\n",
+ "\n",
+ "for _, row in tqdm(qa_pairs.iterrows(), total=len(qa_pairs)):\n",
+ " query, golden_answer, golden_doc = row[\"Question\"], row[\"Answer_True\"], row[\"Document_True\"]\n",
+ " golden_answers.append(golden_answer)\n",
+ "\n",
+ " # --- Produce answer using retriever ---\n",
+ " documents = retriever.retrieve(query, top_n=top_n)\n",
+ " resp = co.chat(message=query, model=\"command-r\", temperature=0., documents=documents)\n",
+ " answer = resp.text\n",
+ " answers.append(answer)\n",
+ "\n",
+ " # --- Do some prework for evaluation later ---\n",
+ " # Rank\n",
+ " rank = get_rank_of_golden_within_retrieved(golden_doc, documents)\n",
+ " ranks.append(rank)\n",
+ " # Score: construct the grading prompts for LLM evals, then evaluate in batch\n",
+ " # Need to reformat documents slightly\n",
+ " documents = [{\"index\": str(i), \"text\": doc[\"text\"]} for i, doc in enumerate(documents)]\n",
+ " references_text = \"\\n\\n\".join(\"\\n\".join([f\"{k}: {v}\" for k, v in doc.items()]) for doc in documents)\n",
+ " # ^ snippet looks complicated, but all it does it unpack all kwargs from `documents`\n",
+ " # into text separated by \\n\\n\n",
+ " grading_prompt = LLM_EVAL_TEMPLATE.format(\n",
+ " references=references_text, question=query, answer=golden_answer, completion=answer,\n",
+ " )\n",
+ " grading_prompts.append(grading_prompt)\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "2BCs7ozyO1vL"
+ },
+ "source": [
+ "## 4. Evaluate model performance\n",
+ "\n",
+ "We want to test our model performance on two dimensions:\n",
+ "1. How good is the final answer? We'll compare our model answer to the golden answer using Command-R+ as a judge.\n",
+ "2. How good is the retrieval? We'll use the rank of the golden document within the retrieved documents to this end.\n",
+ "\n",
+ "Note that this pipeline is for illustration only. To measure performance in practice, we would want to run more in-depths tests on a broader, representative dataset."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "1qBvMpYmQDTK"
+ },
+ "outputs": [],
+ "source": [
+ "# For simplicity, prepare a DataFrame with the results\n",
+ "results = pd.DataFrame()\n",
+ "results[\"answer\"] = answers\n",
+ "results[\"golden_answer\"] = qa_pairs[\"Answer_True\"]\n",
+ "results[\"rank\"] = ranks\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "8Wglx0dsQtgX"
+ },
+ "source": [
+ "### 4.1 Compare answer to golden answer\n",
+ "\n",
+ "We'll use Command-R+ as a judge of whether the answers produced by our model convey the same information as the golden answers. Since we've defined the grading prompts earlier, we can simply ask our LLM judge to evaluate that grading prompt. After a little bit of postprocessing, we can then extract our model scores."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "_F3V4E56Q1CE"
+ },
+ "outputs": [],
+ "source": [
+ "scores = []\n",
+ "reasonings = []\n",
+ "\n",
+ "def remove_backticks(text: str) -> str:\n",
+ " \"\"\"\n",
+ " Some models are trained to output JSON in Markdown formatting:\n",
+ " ```json {json object}```\n",
+ " Remove the backticks from those model responses so that they become\n",
+ " parasable by json.loads.\n",
+ " \"\"\"\n",
+ " if text.startswith(\"```json\"):\n",
+ " text = text[7:]\n",
+ " if text.endswith(\"```\"):\n",
+ " text = text[:-3]\n",
+ " return text\n",
+ "\n",
+ "\n",
+ "for prompt in tqdm(grading_prompts, total=len(grading_prompts)):\n",
+ " resp = co.chat(message=prompt, model=\"command-r-plus\", temperature=0.)\n",
+ " # Convert response to JSON to extract the `score` and `reasoning` fields\n",
+ " # We remove backticks for compatibility with different LLMs\n",
+ " parsed = json.loads(remove_backticks(resp.text))\n",
+ " scores.append(parsed[\"score\"])\n",
+ " reasonings.append(parsed[\"reasoning\"])\n"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "1xksywOeQ1IJ"
+ },
+ "outputs": [],
+ "source": [
+ "# Add scores to our DataFrame\n",
+ "results[\"score\"] = scores\n",
+ "results[\"reasoning\"] = reasonings"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "l8Z2w1ERSeHZ"
+ },
+ "outputs": [],
+ "source": [
+ "print(f\"Average score: {results['score'].mean():.3f}\")\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "ob-Km6dAPeHJ"
+ },
+ "source": [
+ "### 4.2 Compute rank\n",
+ "\n",
+ "We've already computed the rank of the golden documents using `get_rank_of_golden_within_retrieved`. Here, we'll plot the histogram of ranks, using blue when the answer scored a 1, and red when the answer scored a 0."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "SLn3s3n_MlpO"
+ },
+ "outputs": [],
+ "source": [
+ "import matplotlib.pyplot as plt\n",
+ "import seaborn as sns\n",
+ "\n",
+ "sns.set_theme(style=\"darkgrid\", rc={\"grid.color\": \".8\"})\n",
+ "\n",
+ "# create offsets for better vis\n",
+ "results[\"rank_shifted_left\"] = results[\"rank\"] - 0.1\n",
+ "results[\"rank_shifted_right\"] = results[\"rank\"] + 0.1\n",
+ "\n",
+ "f, ax = plt.subplots(figsize=(5, 3))\n",
+ "sns.histplot(data=results.loc[results[\"score\"] == 1], x=\"rank_shifted_left\", color=\"skyblue\", label=\"Correct answer\", binwidth=1)\n",
+ "sns.histplot(data=results.loc[results[\"score\"] == 0], x=\"rank_shifted_right\", color=\"red\", label=\"False answer\", binwidth=1)\n",
+ "\n",
+ "ax.set_xticks([1, 5, 0, 10, 15, 20])\n",
+ "ax.set_title(\"Rank of golden document (max means golden doc. wasn't retrieved)\")\n",
+ "ax.set_xlabel(\"Rank\")\n",
+ "ax.legend();\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "PZeTt7ijVKxo"
+ },
+ "source": [
+ "We see that retrieval works well overall: for 80% of questions, the golden document is within the top 5 documents. However, we also notice that approx. half the false answers come from instances where the golden document wasn't retrieved (`rank = top_k = 20`). This should be improved, e.g. by adding metadata to the documents such as their section headings, or altering the chunking strategy.\n",
+ "\n",
+ "There is also a non-negligible instance of false answers where the top document was retrieved. On closer inspection, many of these are due to the model phrasing its answers more verbosely than the (very laconic) golden documents. This highlights the importance of checking eval results before jumping to conclusions about model performance."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "4Q5i8EYDV_da"
+ },
+ "source": [
+ "## Conclusions\n",
+ "\n",
+ "In this notebook, we've built a QA bot that answers user questions based on technical documentation. We've learnt:\n",
+ "\n",
+ "1. How to embed the technical documentation into a vector database using Cohere embeddings and `llama_index`\n",
+ "2. How to build a custom retriever that leverages Cohere's `rerank`\n",
+ "3. How to evaluate model performance against a predetermined set of golden QA pairs\n",
+ "\n"
+ ]
+ }
+ ],
+ "metadata": {
+ "colab": {
+ "provenance": []
+ },
+ "kernelspec": {
+ "display_name": "Python 3 (ipykernel)",
+ "language": "python",
+ "name": "python3"
+ },
+ "language_info": {
+ "codemirror_mode": {
+ "name": "ipython",
+ "version": 3
+ },
+ "file_extension": ".py",
+ "mimetype": "text/x-python",
+ "name": "python",
+ "nbconvert_exporter": "python",
+ "pygments_lexer": "ipython3",
+ "version": "3.8.16"
+ }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 4
+}
diff --git a/notebooks/guides/Deep_dive_into_RAG_evaluation.ipynb b/notebooks/guides/Deep_dive_into_RAG_evaluation.ipynb
index 73690b63..31eefc58 100644
--- a/notebooks/guides/Deep_dive_into_RAG_evaluation.ipynb
+++ b/notebooks/guides/Deep_dive_into_RAG_evaluation.ipynb
@@ -1,44 +1,34 @@
{
- "nbformat": 4,
- "nbformat_minor": 0,
- "metadata": {
- "colab": {
- "provenance": []
- },
- "kernelspec": {
- "name": "python3",
- "display_name": "Python 3"
- },
- "language_info": {
- "name": "python"
- }
- },
"cells": [
{
"cell_type": "markdown",
+ "metadata": {
+ "id": "eXdiZhdNJU6N",
+ "tags": []
+ },
"source": [
"# Deep dive into RAG Evaluation\n",
"\n",
"In this notebook, we'll show you how to evaluate the output of a RAG system. The high-level RAG flow is depicted in the diagram below.\n",
""
- ],
- "metadata": {
- "id": "eXdiZhdNJU6N"
- }
+ ]
},
{
"cell_type": "markdown",
+ "metadata": {
+ "id": "Vi4P1tqgxJxH"
+ },
"source": [
- "We will focus on the evaluation of **Retrive** and **Response** (or **Generation**), and present a set of metrics for each phase. We will deep dive into each metric, to give you a full understanding of how we evaluate models and why we do it this way, and provide code so you can repdroduce on your own data.\n",
+ "We will focus on the evaluation of **Retrieve** and **Response** (or **Generation**), and present a set of metrics for each phase. We will deep dive into each metric, to give you a full understanding of how we evaluate models and why we do it this way, and provide code so you can repdroduce on your own data.\n",
"\n",
"To demonstrate the metrics, we will use data from the [Docugami's KG-RAG](https://github.com/docugami/KG-RAG-datasets/tree/main/sec-10-q/data/v1) dataset, a RAG dataset for financial 10Q filing reports. We will focus only on evaluation, without performing the actual Retrieval and response Generation steps."
- ],
- "metadata": {
- "id": "Vi4P1tqgxJxH"
- }
+ ]
},
{
"cell_type": "markdown",
+ "metadata": {
+ "id": "5bnEyZUl_KmS"
+ },
"source": [
"# Table of content\n",
"\n",
@@ -46,70 +36,73 @@
"2. [Retrieval Evaluation](#retrieval-evaluation)\n",
"3. [Generation Evaluation](#generation-evaluation)\n",
"4. [Final Comments](#final-comments)"
- ],
- "metadata": {
- "id": "5bnEyZUl_KmS"
- }
+ ]
},
{
"cell_type": "markdown",
+ "metadata": {
+ "id": "rVLK7Bhux8Bs"
+ },
"source": [
"\n",
"## Getting Started\n",
"\n",
"Let's start by setting the environment and downloading the dataset."
- ],
- "metadata": {
- "id": "rVLK7Bhux8Bs"
- }
+ ]
},
{
"cell_type": "code",
+ "execution_count": 4,
+ "metadata": {
+ "id": "VmGk_rOco3m5"
+ },
+ "outputs": [],
"source": [
"%%capture\n",
"!pip install llama-index cohere openai\n",
"!pip install mistralai"
- ],
- "metadata": {
- "id": "VmGk_rOco3m5"
- },
- "execution_count": null,
- "outputs": []
+ ]
},
{
"cell_type": "code",
+ "execution_count": 5,
+ "metadata": {
+ "id": "SIpPuVxfo_vz"
+ },
+ "outputs": [],
"source": [
"# required imports\n",
- "import cohere\n",
"from getpass import getpass\n",
"import os\n",
"import re\n",
- "import json\n",
"import numpy as np\n",
- "import pandas as pd\n",
"from llama_index.core import SimpleDirectoryReader\n",
"from llama_index.core.llama_dataset import download_llama_dataset, LabelledRagDataset\n",
"from openai import Client\n",
"from mistralai.client import MistralClient"
- ],
- "metadata": {
- "id": "SIpPuVxfo_vz"
- },
- "execution_count": null,
- "outputs": []
+ ]
},
{
"cell_type": "markdown",
+ "metadata": {
+ "id": "J0ZO3ki_yIJE"
+ },
"source": [
"For Response evaluation, we will use an LLM as a judge.\n",
"Any LLM can be used for this goal, but because evaluation is a very challenging task, we recommend using powerful LLMs, possibly as an ensemble of models. In [previous work](https://arxiv.org/pdf/2303.16634.pdf), it has been shown that models tend to assign higher scores to their own output. Since we generated the answers in this notebook using `command-r`, we will not use it for evaluation. We will provide two alternatives, `gpt-4` and `mistral`. We set `gpt-4` as the default model because, as mentioned above, evaluation is challenging, and `gpt-4` is powerful enough to efficiently perform the task."
- ],
- "metadata": {
- "id": "J0ZO3ki_yIJE"
- }
+ ]
},
{
"cell_type": "code",
+ "execution_count": 6,
+ "metadata": {
+ "colab": {
+ "base_uri": "https://localhost:8080/"
+ },
+ "id": "-SdZGWLTtLok",
+ "outputId": "46d48939-5547-4e53-de78-9ae16440107d"
+ },
+ "outputs": [],
"source": [
"# Get keys\n",
"openai_api_key = getpass(\"Enter your OpenAI API Key: \")\n",
@@ -120,41 +113,29 @@
"model = \"gpt-4\"\n",
"# uncomment if you want to use mistral\n",
"#model = \"mistral-large-latest\"\n"
- ],
- "metadata": {
- "id": "-SdZGWLTtLok",
- "colab": {
- "base_uri": "https://localhost:8080/"
- },
- "outputId": "46d48939-5547-4e53-de78-9ae16440107d"
- },
- "execution_count": null,
- "outputs": [
- {
- "name": "stdout",
- "output_type": "stream",
- "text": [
- "Enter your OpenAI API Key: ··········\n"
- ]
- }
]
},
{
"cell_type": "code",
+ "execution_count": 7,
+ "metadata": {
+ "id": "Zk02dD_9mc7B"
+ },
+ "outputs": [],
"source": [
"if model == \"gpt-4\":\n",
" client = Client(api_key=openai_api_key)\n",
"else:\n",
" client = MistralClient(api_key=mistral_api_key)"
- ],
- "metadata": {
- "id": "Zk02dD_9mc7B"
- },
- "execution_count": null,
- "outputs": []
+ ]
},
{
"cell_type": "code",
+ "execution_count": 8,
+ "metadata": {
+ "id": "Kx4cxDGw6LI_"
+ },
+ "outputs": [],
"source": [
"# let's define a function to get the model's response for a given input\n",
"def get_response(model, client, prompt):\n",
@@ -163,15 +144,19 @@
" messages=[{\"role\": \"user\", \"content\": prompt}],\n",
" temperature=0)\n",
" return response.choices[0].message.content"
- ],
- "metadata": {
- "id": "Kx4cxDGw6LI_"
- },
- "execution_count": null,
- "outputs": []
+ ]
},
{
"cell_type": "code",
+ "execution_count": 9,
+ "metadata": {
+ "colab": {
+ "base_uri": "https://localhost:8080/"
+ },
+ "id": "voJk7dPvuSdN",
+ "outputId": "1dd2c527-7d4a-4278-e14b-2c9479ad5caf"
+ },
+ "outputs": [],
"source": [
"# load the DocugamiKgRagSec10Q dataset\n",
"if os.path.exists(\"./data/source_files\") and os.path.exists(\"./data/rag_dataset.json\"):\n",
@@ -179,27 +164,13 @@
" documents = SimpleDirectoryReader(input_dir=\"./data/source_files\").load_data(show_progress=True)\n",
"else:\n",
" rag_dataset, documents = download_llama_dataset(\"DocugamiKgRagSec10Q\", \"./data\")"
- ],
- "metadata": {
- "id": "voJk7dPvuSdN",
- "colab": {
- "base_uri": "https://localhost:8080/"
- },
- "outputId": "1dd2c527-7d4a-4278-e14b-2c9479ad5caf"
- },
- "execution_count": null,
- "outputs": [
- {
- "output_type": "stream",
- "name": "stderr",
- "text": [
- "Loading files: 100%|██████████| 20/20 [01:44<00:00, 5.21s/file]\n"
- ]
- }
]
},
{
"cell_type": "markdown",
+ "metadata": {
+ "id": "lB6vO4JvMEkT"
+ },
"source": [
"\n",
"## Retrieval Evaluation\n",
@@ -213,13 +184,15 @@
"* **Mean Average Precision** (**MAP**): measures the capability of the retriever to return relevant documents at the top of the list\n",
"\n",
"We implement these three metrics in the class below:"
- ],
- "metadata": {
- "id": "lB6vO4JvMEkT"
- }
+ ]
},
{
"cell_type": "code",
+ "execution_count": 17,
+ "metadata": {
+ "id": "CooNq035eU6f"
+ },
+ "outputs": [],
"source": [
"class RetrievalEvaluator:\n",
"\n",
@@ -245,27 +218,40 @@
" results = {'precision': [precision],\n",
" 'recall': [recall],\n",
" 'map': [map]}\n",
- " results = pd.DataFrame(results)\n",
- " return results\n",
- "\n"
- ],
- "metadata": {
- "id": "CooNq035eU6f"
- },
- "execution_count": null,
- "outputs": []
+ " for k,v in results.items():\n",
+ " print(f\"{k}: {v[0]}\")\n"
+ ]
},
{
"cell_type": "markdown",
- "source": [
- "Let's now see how to use the class above to compute the results on a single datapoint."
- ],
"metadata": {
"id": "MWW2VM-Tj8iQ"
- }
+ },
+ "source": [
+ "Let's now see how to use the class above to compute the results on a single datapoint."
+ ]
},
{
"cell_type": "code",
+ "execution_count": 18,
+ "metadata": {
+ "colab": {
+ "base_uri": "https://localhost:8080/"
+ },
+ "id": "QyMGDqaqe7fg",
+ "outputId": "e95242c7-dcdd-48bb-9da2-52e25a2a2d41"
+ },
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "Query: How has Apple's total net sales changed over time?\n",
+ "Golden docs: ['2022 Q3 AAPL.pdf', '2023 Q1 AAPL.pdf', '2023 Q2 AAPL.pdf', '2023 Q3 AAPL.pdf']\n",
+ "Retrieved docs: ['2022 Q3 AAPL.pdf', '2023 Q1 MSFT.pdf', '2023 Q1 AAPL.pdf']\n"
+ ]
+ }
+ ],
"source": [
"# select the index of a single datapoint - the first one in the dataset\n",
"idx = 0\n",
@@ -282,183 +268,43 @@
"print(f'Query: {query}')\n",
"print(f'Golden docs: {golden_docs}')\n",
"print(f'Retrieved docs: {retrieved_docs}')"
- ],
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 19,
"metadata": {
"colab": {
- "base_uri": "https://localhost:8080/"
+ "base_uri": "https://localhost:8080/",
+ "height": 81
},
- "id": "QyMGDqaqe7fg",
- "outputId": "e95242c7-dcdd-48bb-9da2-52e25a2a2d41"
+ "id": "Ron8lm3Z1t1M",
+ "outputId": "a37eee30-c318-4a2f-8dec-bfd8f4d8c8f2"
},
- "execution_count": null,
"outputs": [
{
- "output_type": "stream",
"name": "stdout",
+ "output_type": "stream",
"text": [
- "Query: How has Apple's total net sales changed over time?\n",
- "Golden docs: ['2022 Q3 AAPL.pdf', '2023 Q1 AAPL.pdf', '2023 Q2 AAPL.pdf', '2023 Q3 AAPL.pdf']\n",
- "Retrieved docs: ['2022 Q3 AAPL.pdf', '2023 Q1 MSFT.pdf', '2023 Q1 AAPL.pdf']\n"
+ "precision: 0.67\n",
+ "recall: 0.5\n",
+ "map: 0.83\n"
]
}
- ]
- },
- {
- "cell_type": "code",
+ ],
"source": [
"# we can now instantiate the evaluator\n",
"evaluate_retrieval = RetrievalEvaluator()\n",
"\n",
"# and run the evaluation\n",
"evaluate_retrieval.run_evals(retrieved_docs,golden_docs)\n"
- ],
- "metadata": {
- "colab": {
- "base_uri": "https://localhost:8080/",
- "height": 81
- },
- "id": "Ron8lm3Z1t1M",
- "outputId": "a37eee30-c318-4a2f-8dec-bfd8f4d8c8f2"
- },
- "execution_count": null,
- "outputs": [
- {
- "output_type": "execute_result",
- "data": {
- "text/plain": [
- " precision recall map\n",
- "0 0.67 0.5 0.83"
- ],
- "text/html": [
- "\n",
- "
\n"
- ],
- "application/vnd.google.colaboratory.intrinsic+json": {
- "type": "dataframe",
- "summary": "{\n \"name\": \"evaluate_retrieval\",\n \"rows\": 1,\n \"fields\": [\n {\n \"column\": \"precision\",\n \"properties\": {\n \"dtype\": \"number\",\n \"std\": null,\n \"min\": 0.67,\n \"max\": 0.67,\n \"num_unique_values\": 1,\n \"samples\": [\n 0.67\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"recall\",\n \"properties\": {\n \"dtype\": \"number\",\n \"std\": null,\n \"min\": 0.5,\n \"max\": 0.5,\n \"num_unique_values\": 1,\n \"samples\": [\n 0.5\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"map\",\n \"properties\": {\n \"dtype\": \"number\",\n \"std\": null,\n \"min\": 0.83,\n \"max\": 0.83,\n \"num_unique_values\": 1,\n \"samples\": [\n 0.83\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n }\n ]\n}"
- }
- },
- "metadata": {},
- "execution_count": 9
- }
]
},
{
"cell_type": "markdown",
+ "metadata": {
+ "id": "sL-sIiSl2vw9"
+ },
"source": [
"What are the figures above telling us?\n",
"\n",
@@ -467,13 +313,14 @@
"* MAP (0.83) is computed as the average of 1/1 (the highest ranked doc is correct) and 2/3 (the 2nd ranked doc is wrong, the 3rd is correct).\n",
"\n",
"While the example here focuses on a single datapoint, you can easily apply the same metrics to all your dataset and get the overall performance of your Retrieve phase."
- ],
- "metadata": {
- "id": "sL-sIiSl2vw9"
- }
+ ]
},
{
"cell_type": "markdown",
+ "metadata": {
+ "id": "mwEyE9IaklPL",
+ "tags": []
+ },
"source": [
"\n",
"## Generation Evaluation\n",
@@ -491,24 +338,28 @@
"\n",
"Note that Faithfulness and Correctness share the exact same approach, the difference being that the former checks the claims against the supporting docs, while the latter against the golden answer.\n",
"Also, while Correctness is measuring the precision of the claims in the response, Coverage can be seen as complementary, as it measures recall."
- ],
- "metadata": {
- "id": "mwEyE9IaklPL"
- }
+ ]
},
{
"cell_type": "markdown",
+ "metadata": {
+ "id": "ySgduJrTTjvK",
+ "jp-MarkdownHeadingCollapsed": true,
+ "tags": []
+ },
"source": [
"### Claim Extraction\n",
"\n",
"Let's now see how we implement the evaluation described above using LLMs. Let's start with **claim extraction**."
- ],
- "metadata": {
- "id": "ySgduJrTTjvK"
- }
+ ]
},
{
"cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "X2fQa0Vbuy3W"
+ },
+ "outputs": [],
"source": [
"# first, let's define a function which extracts the claims from a response\n",
"def extract_claims(query, response, model, client):\n",
@@ -523,25 +374,11 @@
" claims = get_response(model, client, prompt)\n",
"\n",
" return claims\n"
- ],
- "metadata": {
- "id": "X2fQa0Vbuy3W"
- },
- "execution_count": null,
- "outputs": []
+ ]
},
{
"cell_type": "code",
- "source": [
- "# now, let's consider this answer, which we previously generated with command-r\n",
- "response = \"Apple's total net sales experienced a decline over the last year. The three-month period ended July 1, 2023, saw a total net sale of $81,797 million, which was a 1% decrease from the same period in 2022. The nine-month period ended July 1, 2023, fared slightly better, with a 3% decrease in net sales compared to the first nine months of 2022.\\nThis downward trend continued into the three and six-month periods ending April 1, 2023. Apple's total net sales decreased by 3% and 4% respectively, compared to the same periods in 2022.\"\n",
- "\n",
- "# let's extract the claims\n",
- "claims = extract_claims(query, response, model, client)\n",
- "\n",
- "# and see what the model returns\n",
- "print(f\"List of claims extracted from the model's response:\\n\\n{claims}\")"
- ],
+ "execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
@@ -549,11 +386,10 @@
"id": "y_0NXZEfu0AJ",
"outputId": "3fa32ab2-6898-449f-f062-187df7aea41e"
},
- "execution_count": null,
"outputs": [
{
- "output_type": "stream",
"name": "stdout",
+ "output_type": "stream",
"text": [
"List of claims extracted from the model's response:\n",
"\n",
@@ -565,21 +401,36 @@
"- Apple's total net sales decreased by 3% and 4% respectively, compared to the same periods in 2022.\n"
]
}
+ ],
+ "source": [
+ "# now, let's consider this answer, which we previously generated with command-r\n",
+ "response = \"Apple's total net sales experienced a decline over the last year. The three-month period ended July 1, 2023, saw a total net sale of $81,797 million, which was a 1% decrease from the same period in 2022. The nine-month period ended July 1, 2023, fared slightly better, with a 3% decrease in net sales compared to the first nine months of 2022.\\nThis downward trend continued into the three and six-month periods ending April 1, 2023. Apple's total net sales decreased by 3% and 4% respectively, compared to the same periods in 2022.\"\n",
+ "\n",
+ "# let's extract the claims\n",
+ "claims = extract_claims(query, response, model, client)\n",
+ "\n",
+ "# and see what the model returns\n",
+ "print(f\"List of claims extracted from the model's response:\\n\\n{claims}\")"
]
},
{
"cell_type": "markdown",
+ "metadata": {
+ "id": "50QRlCVe7dZf"
+ },
"source": [
"### Claim Assessment\n",
"\n",
"Nice! now that we have the list of claims, we can go ahead and **assess the validity** of each claim."
- ],
- "metadata": {
- "id": "50QRlCVe7dZf"
- }
+ ]
},
{
"cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "jrg9-dlMTRpB"
+ },
+ "outputs": [],
"source": [
"# Let's create a function that checks each claim against a reference text,\n",
"# which here we will call \"context\". As you will see, we will use different contexts,\n",
@@ -601,40 +452,20 @@
" assessment = get_response(model, client, prompt)\n",
"\n",
" return assessment"
- ],
- "metadata": {
- "id": "jrg9-dlMTRpB"
- },
- "execution_count": null,
- "outputs": []
+ ]
},
{
"cell_type": "markdown",
- "source": [
- "### Faithfulness"
- ],
"metadata": {
"id": "1OmD9pKxArk7"
- }
+ },
+ "source": [
+ "### Faithfulness"
+ ]
},
{
"cell_type": "code",
- "source": [
- "# Let's start with Faithfulness: in this case, we want to assess the claims\n",
- "# in the response against the retrieved documents (i.e., context = retrieved documents)\n",
- "\n",
- "# for the sake of clarity, we report the actual text of the retrieved documents\n",
- "retrieved_documents = ['Products and Services Performance\\nThe following table shows net sales by category for the three- and six-month periods ended April 1, 2023 and March 26, 2022 (dollars in millions):\\nThree Months Ended Six Months Ended\\nApril 1,\\n2023March 26,\\n2022 ChangeApril 1,\\n2023March 26,\\n2022 Change\\nNet sales by category:\\niPhone $ 51,334 $ 50,570 2 %$ 117,109 $ 122,198 (4)%\\nMac 7,168 10,435 (31)% 14,903 21,287 (30)%\\niPad 6,670 7,646 (13)% 16,066 14,894 8 %\\nWearables, Home and Accessories 8,757 8,806 (1)% 22,239 23,507 (5)%\\nServices 20,907 19,821 5 % 41,673 39,337 6 %\\nTotal net sales $ 94,836 $ 97,278 (3)%$ 211,990 $ 221,223 (4)%\\niPhone\\niPhone net sales were relatively flat during the second quarter of 2023 compared to the secon d quarter of 2022. Year-over-year iPhone net sales decreased\\nduring the first six months of 2023 due primarily to lower net sales from the Company’ s new iPhone models launched in the fourth quarter of 2022.\\nMac\\nMac net sales decreased during the second quarter and first six months of 2023 compared to the same periods in 2022 due primarily to lower net sales of\\nMacBook Pro.\\niPad\\niPad net sales decreased during the second quarter of 2023 compared to the second quarter of 2022 due primarily to lower net sales of iPad Pro and iPad Air.\\nYear-over-year iPad net sales increased during the first six months of 2023 due primarily to higher net sales of iPad, partially offset by lower net sales of iPad\\nmini .\\nWearables, Home and Accessories\\nWearables, Home and Accessories net sales were relatively flat during the second quarter of 2023 compared to the second quarter of 2022. Year-over-year\\nWearables, Home and Accessories net sales decreased during the first six months of 2023 due primarily to lower net sales of AirPods .\\nServices\\nServices net sales increased during the second quarter and first six months of 2023 compared to the same periods in 2022 due primarily to higher net sales from\\ncloud services, music and advertising.® ®\\n®\\n®\\nApple Inc. | Q2 2023 Form 10-Q | 16', 'Products and Services Performance\\nThe following table shows net sales by category for the three- and nine-month periods ended July 1, 2023 and June 25, 2022 (dollars in millions):\\nThree Months Ended Nine Months Ended\\nJuly 1,\\n2023June 25,\\n2022 ChangeJuly 1,\\n2023June 25,\\n2022 Change\\nNet sales by category:\\niPhone $ 39,669 $ 40,665 (2)%$ 156,778 $ 162,863 (4)%\\nMac 6,840 7,382 (7)% 21,743 28,669 (24)%\\niPad 5,791 7,224 (20)% 21,857 22,118 (1)%\\nWearables, Home and Accessories 8,284 8,084 2 % 30,523 31,591 (3)%\\nServices 21,213 19,604 8 % 62,886 58,941 7 %\\nTotal net sales $ 81,797 $ 82,959 (1)%$ 293,787 $ 304,182 (3)%\\niPhone\\niPhone net sales decreased during the third quarter and first nine months of 2023 compared to the same periods in 2022 due primarily to lower net sales from\\ncertain iPhone models, partially of fset by higher net sales of iPhone 14 Pro models.\\nMac\\nMac net sales decreased during the third quarter and first nine months of 2023 compared to the same periods in 2022 due primarily to lower net sales of laptops.\\niPad\\niPad net sales decreased during the third quarter of 2023 compared to the third quarter of 2022 due primarily to lower net sales across most iPad models. Year-\\nover-year iPad net sales were relatively flat during the first nine months of 2023.\\nWearables, Home and Accessories\\nWearables, Home and Accessories net sales increased during the third quarter of 2023 compare d to the third quarter of 2022 due primarily to higher net sales of\\nWearables, which includes AirPods , Apple Watch and Beats products, partially offset by lower net sales of accessories. Year-over-year Wearables, Home\\nand Accessories net sales decreased during the first nine months of 2023 due primarily to lower net sales of W earables and accessories.\\nServices\\nServices net sales increased during the third quarter of 2023 compared to the third quarter of 2022 due primarily to higher net sales from advertising, cloud\\nservices and the App Store . Year-over-year Services net sales increased during the first nine months of 2023 due primarily to higher net sales from cloud\\nservices, advertising and music.® ® ®\\n®\\nApple Inc. | Q3 2023 Form 10-Q | 16']\n",
- "\n",
- "# get the Faithfulness assessment for each claim\n",
- "assessed_claims_faithfulness = assess_claims(query=query,\n",
- " claims=claims,\n",
- " context=retrieved_documents,\n",
- " model=model,\n",
- " client=client)\n",
- "\n",
- "print(f\"Assessment of the claims extracted from the model's response:\\n\\n{assessed_claims_faithfulness}\")"
- ],
+ "execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
@@ -642,11 +473,10 @@
"id": "5VmLE4yCwEMe",
"outputId": "10874986-1f91-45a3-c081-dc2aae3fd618"
},
- "execution_count": null,
"outputs": [
{
- "output_type": "stream",
"name": "stdout",
+ "output_type": "stream",
"text": [
"Assessment of the claims extracted from the model's response:\n",
"\n",
@@ -658,19 +488,40 @@
"- Apple's total net sales decreased by 3% and 4% respectively, compared to the same periods in 2022. SUPPORTED=1\n"
]
}
+ ],
+ "source": [
+ "# Let's start with Faithfulness: in this case, we want to assess the claims\n",
+ "# in the response against the retrieved documents (i.e., context = retrieved documents)\n",
+ "\n",
+ "# for the sake of clarity, we report the actual text of the retrieved documents\n",
+ "retrieved_documents = ['Products and Services Performance\\nThe following table shows net sales by category for the three- and six-month periods ended April 1, 2023 and March 26, 2022 (dollars in millions):\\nThree Months Ended Six Months Ended\\nApril 1,\\n2023March 26,\\n2022 ChangeApril 1,\\n2023March 26,\\n2022 Change\\nNet sales by category:\\niPhone $ 51,334 $ 50,570 2 %$ 117,109 $ 122,198 (4)%\\nMac 7,168 10,435 (31)% 14,903 21,287 (30)%\\niPad 6,670 7,646 (13)% 16,066 14,894 8 %\\nWearables, Home and Accessories 8,757 8,806 (1)% 22,239 23,507 (5)%\\nServices 20,907 19,821 5 % 41,673 39,337 6 %\\nTotal net sales $ 94,836 $ 97,278 (3)%$ 211,990 $ 221,223 (4)%\\niPhone\\niPhone net sales were relatively flat during the second quarter of 2023 compared to the secon d quarter of 2022. Year-over-year iPhone net sales decreased\\nduring the first six months of 2023 due primarily to lower net sales from the Company’ s new iPhone models launched in the fourth quarter of 2022.\\nMac\\nMac net sales decreased during the second quarter and first six months of 2023 compared to the same periods in 2022 due primarily to lower net sales of\\nMacBook Pro.\\niPad\\niPad net sales decreased during the second quarter of 2023 compared to the second quarter of 2022 due primarily to lower net sales of iPad Pro and iPad Air.\\nYear-over-year iPad net sales increased during the first six months of 2023 due primarily to higher net sales of iPad, partially offset by lower net sales of iPad\\nmini .\\nWearables, Home and Accessories\\nWearables, Home and Accessories net sales were relatively flat during the second quarter of 2023 compared to the second quarter of 2022. Year-over-year\\nWearables, Home and Accessories net sales decreased during the first six months of 2023 due primarily to lower net sales of AirPods .\\nServices\\nServices net sales increased during the second quarter and first six months of 2023 compared to the same periods in 2022 due primarily to higher net sales from\\ncloud services, music and advertising.® ®\\n®\\n®\\nApple Inc. | Q2 2023 Form 10-Q | 16', 'Products and Services Performance\\nThe following table shows net sales by category for the three- and nine-month periods ended July 1, 2023 and June 25, 2022 (dollars in millions):\\nThree Months Ended Nine Months Ended\\nJuly 1,\\n2023June 25,\\n2022 ChangeJuly 1,\\n2023June 25,\\n2022 Change\\nNet sales by category:\\niPhone $ 39,669 $ 40,665 (2)%$ 156,778 $ 162,863 (4)%\\nMac 6,840 7,382 (7)% 21,743 28,669 (24)%\\niPad 5,791 7,224 (20)% 21,857 22,118 (1)%\\nWearables, Home and Accessories 8,284 8,084 2 % 30,523 31,591 (3)%\\nServices 21,213 19,604 8 % 62,886 58,941 7 %\\nTotal net sales $ 81,797 $ 82,959 (1)%$ 293,787 $ 304,182 (3)%\\niPhone\\niPhone net sales decreased during the third quarter and first nine months of 2023 compared to the same periods in 2022 due primarily to lower net sales from\\ncertain iPhone models, partially of fset by higher net sales of iPhone 14 Pro models.\\nMac\\nMac net sales decreased during the third quarter and first nine months of 2023 compared to the same periods in 2022 due primarily to lower net sales of laptops.\\niPad\\niPad net sales decreased during the third quarter of 2023 compared to the third quarter of 2022 due primarily to lower net sales across most iPad models. Year-\\nover-year iPad net sales were relatively flat during the first nine months of 2023.\\nWearables, Home and Accessories\\nWearables, Home and Accessories net sales increased during the third quarter of 2023 compare d to the third quarter of 2022 due primarily to higher net sales of\\nWearables, which includes AirPods , Apple Watch and Beats products, partially offset by lower net sales of accessories. Year-over-year Wearables, Home\\nand Accessories net sales decreased during the first nine months of 2023 due primarily to lower net sales of W earables and accessories.\\nServices\\nServices net sales increased during the third quarter of 2023 compared to the third quarter of 2022 due primarily to higher net sales from advertising, cloud\\nservices and the App Store . Year-over-year Services net sales increased during the first nine months of 2023 due primarily to higher net sales from cloud\\nservices, advertising and music.® ® ®\\n®\\nApple Inc. | Q3 2023 Form 10-Q | 16']\n",
+ "\n",
+ "# get the Faithfulness assessment for each claim\n",
+ "assessed_claims_faithfulness = assess_claims(query=query,\n",
+ " claims=claims,\n",
+ " context=retrieved_documents,\n",
+ " model=model,\n",
+ " client=client)\n",
+ "\n",
+ "print(f\"Assessment of the claims extracted from the model's response:\\n\\n{assessed_claims_faithfulness}\")"
]
},
{
"cell_type": "markdown",
- "source": [
- "Great, we now have an assessment for each of the claims: in the last step, we just need to use these assessments to define the final score."
- ],
"metadata": {
"id": "vBbLuSkO-iN7"
- }
+ },
+ "source": [
+ "Great, we now have an assessment for each of the claims: in the last step, we just need to use these assessments to define the final score."
+ ]
},
{
"cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "tFk-IxK_wDVT"
+ },
+ "outputs": [],
"source": [
"# given the list of claims and their label, compute the final score\n",
"# as the proportion of correct claims over the full list of claims\n",
@@ -679,19 +530,11 @@
" non_supported = len(re.findall(\"SUPPORTED=0\", claims_list))\n",
" score = supported / (supported+non_supported)\n",
" return round(score, 2)"
- ],
- "metadata": {
- "id": "tFk-IxK_wDVT"
- },
- "execution_count": null,
- "outputs": []
+ ]
},
{
"cell_type": "code",
- "source": [
- "score_faithfulness = get_final_score(assessed_claims_faithfulness)\n",
- "print(f'Faithfulness: {score_faithfulness}')"
- ],
+ "execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
@@ -699,49 +542,34 @@
"id": "Z680DvFOW2Ea",
"outputId": "1666057d-7981-43d3-f4bb-c10fdc861cae"
},
- "execution_count": null,
"outputs": [
{
- "output_type": "stream",
"name": "stdout",
+ "output_type": "stream",
"text": [
"Faithfulness: 1.0\n"
]
}
+ ],
+ "source": [
+ "score_faithfulness = get_final_score(assessed_claims_faithfulness)\n",
+ "print(f'Faithfulness: {score_faithfulness}')"
]
},
{
"cell_type": "markdown",
+ "metadata": {
+ "id": "3_Ks8ELQrdQO"
+ },
"source": [
"The final Faithfulness score is 1, which means that the model's response is fully grounded in the retrieved documents: that's a very good news :)\n",
"\n",
"Before moving on, let's modify the model's response by adding a piece of information which is **not** grounded in any document, and re-compute Faithfulness."
- ],
- "metadata": {
- "id": "3_Ks8ELQrdQO"
- }
+ ]
},
{
"cell_type": "code",
- "source": [
- "# let's mess up the century, changing 2022 to 1922\n",
- "modified_response = response.replace('2022', '1922')\n",
- "\n",
- "# extract the claims from the modified response\n",
- "modified_claims = extract_claims(query, modified_response, model, client)\n",
- "\n",
- "# and get assess the modified claims\n",
- "assessed_modified_claims = assess_claims(query=query,\n",
- " claims=modified_claims,\n",
- " context=retrieved_documents,\n",
- " model=model,\n",
- " client=client)\n",
- "\n",
- "print(f\"Assessment of the modified claims:\\n\\n{assessed_modified_claims}\\n\")\n",
- "\n",
- "score_faithfulness_modified_claims = get_final_score(assessed_modified_claims)\n",
- "print(f'Faithfulness: {score_faithfulness_modified_claims}')"
- ],
+ "execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
@@ -749,11 +577,10 @@
"id": "YEZUA8XMAxLE",
"outputId": "9b6e59b1-a4f9-46bf-cdf8-a4f520462d12"
},
- "execution_count": null,
"outputs": [
{
- "output_type": "stream",
"name": "stdout",
+ "output_type": "stream",
"text": [
"Assessment of the modified claims:\n",
"\n",
@@ -767,46 +594,50 @@
"Faithfulness: 0.5\n"
]
}
+ ],
+ "source": [
+ "# let's mess up the century, changing 2022 to 1922\n",
+ "modified_response = response.replace('2022', '1922')\n",
+ "\n",
+ "# extract the claims from the modified response\n",
+ "modified_claims = extract_claims(query, modified_response, model, client)\n",
+ "\n",
+ "# and get assess the modified claims\n",
+ "assessed_modified_claims = assess_claims(query=query,\n",
+ " claims=modified_claims,\n",
+ " context=retrieved_documents,\n",
+ " model=model,\n",
+ " client=client)\n",
+ "\n",
+ "print(f\"Assessment of the modified claims:\\n\\n{assessed_modified_claims}\\n\")\n",
+ "\n",
+ "score_faithfulness_modified_claims = get_final_score(assessed_modified_claims)\n",
+ "print(f'Faithfulness: {score_faithfulness_modified_claims}')"
]
},
{
"cell_type": "markdown",
- "source": [
- "As you can see, by assessing claims one by one, we are able to spot **hallucinations**, that is, the (corrupted) cases in which the information provided by the model is not grounded in any of the retrieved documents."
- ],
"metadata": {
"id": "TccmzE9TCQsN"
- }
+ },
+ "source": [
+ "As you can see, by assessing claims one by one, we are able to spot **hallucinations**, that is, the (corrupted) cases in which the information provided by the model is not grounded in any of the retrieved documents."
+ ]
},
{
"cell_type": "markdown",
+ "metadata": {
+ "id": "bPK-MND1riQD"
+ },
"source": [
"### Correctness\n",
"\n",
"As said, Faithfulness and Correctness share the same logic, the only difference being that we will check the claims against the gold answer. We can therefore repeat the process above, and just substitute the `context`."
- ],
- "metadata": {
- "id": "bPK-MND1riQD"
- }
+ ]
},
{
"cell_type": "code",
- "source": [
- "# let's get the gold answer from the dataset\n",
- "golden_answer = rag_dataset[idx].reference_answer\n",
- "\n",
- "# and check the claims in the response against the gold.\n",
- "# note that assess_claims takes exactly the same args as with Faithfulness\n",
- "# except for the context, that now is the golden_answer\n",
- "assessed_claims_correctness = assess_claims(query=query,\n",
- " claims=claims,\n",
- " context=golden_answer, # note the different context\n",
- " model=model,\n",
- " client=client)\n",
- "\n",
- "\n",
- "print(f\"Assess the claims extracted from the model's response against the golden answer:\\n\\n{assessed_claims_correctness}\")"
- ],
+ "execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
@@ -814,11 +645,10 @@
"id": "qVeQDVDYEr45",
"outputId": "a14a5b09-b8f7-4563-ade5-fe2cd1384cfb"
},
- "execution_count": null,
"outputs": [
{
- "output_type": "stream",
"name": "stdout",
+ "output_type": "stream",
"text": [
"Assess the claims extracted from the model's response against the golden answer:\n",
"\n",
@@ -830,24 +660,36 @@
"- Apple's total net sales decreased by 3% and 4% respectively, compared to the same periods in 2022. SUPPORTED=0\n"
]
}
+ ],
+ "source": [
+ "# let's get the gold answer from the dataset\n",
+ "golden_answer = rag_dataset[idx].reference_answer\n",
+ "\n",
+ "# and check the claims in the response against the gold.\n",
+ "# note that assess_claims takes exactly the same args as with Faithfulness\n",
+ "# except for the context, that now is the golden_answer\n",
+ "assessed_claims_correctness = assess_claims(query=query,\n",
+ " claims=claims,\n",
+ " context=golden_answer, # note the different context\n",
+ " model=model,\n",
+ " client=client)\n",
+ "\n",
+ "\n",
+ "print(f\"Assess the claims extracted from the model's response against the golden answer:\\n\\n{assessed_claims_correctness}\")"
]
},
{
"cell_type": "markdown",
- "source": [
- "As mentioned above, automatic evaluation is a hard task, and even when using powerful models, claim assessment can present problems: for example, the third claim is labelled as 0, even if it might be inferred from the information in the gold answer."
- ],
"metadata": {
"id": "V4pCRJ0qPQWT"
- }
+ },
+ "source": [
+ "As mentioned above, automatic evaluation is a hard task, and even when using powerful models, claim assessment can present problems: for example, the third claim is labelled as 0, even if it might be inferred from the information in the gold answer."
+ ]
},
{
"cell_type": "code",
- "source": [
- "# we can now compute the final Correctness score\n",
- "score_correctness = get_final_score(assessed_claims_correctness)\n",
- "print(f'Correctness: {score_correctness}')"
- ],
+ "execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
@@ -855,45 +697,44 @@
"id": "nObA30PzxQha",
"outputId": "2e354428-ea91-4c7e-94b6-31433018058c"
},
- "execution_count": null,
"outputs": [
{
- "output_type": "stream",
"name": "stdout",
+ "output_type": "stream",
"text": [
"Correctness: 0.5\n"
]
}
+ ],
+ "source": [
+ "# we can now compute the final Correctness score\n",
+ "score_correctness = get_final_score(assessed_claims_correctness)\n",
+ "print(f'Correctness: {score_correctness}')"
]
},
{
"cell_type": "markdown",
- "source": [
- "For Correctness, we found that only half of the claims in the generated response are found in the gold answer. Note that this is not necessarily an issue: reference answers are often non-exhaustive, especially in dataset including open-ended questions, like the one we are considering in this post, and *both* the generated and golden answer can include relevant information.\n"
- ],
"metadata": {
"id": "4Nd0bQ3rR1c3"
- }
+ },
+ "source": [
+ "For Correctness, we found that only half of the claims in the generated response are found in the gold answer. Note that this is not necessarily an issue: reference answers are often non-exhaustive, especially in dataset including open-ended questions, like the one we are considering in this post, and *both* the generated and golden answer can include relevant information.\n"
+ ]
},
{
"cell_type": "markdown",
+ "metadata": {
+ "id": "YhsEJDNXR8OY"
+ },
"source": [
"### Coverage\n",
"\n",
"We finally move to Coverage. Remember that, in this case, we want to check how many of the claims *in the gold answer* are included in the generated response. To do it, we first need to extract the claims from the gold answer."
- ],
- "metadata": {
- "id": "YhsEJDNXR8OY"
- }
+ ]
},
{
"cell_type": "code",
- "source": [
- "# let's extract the golden claims\n",
- "gold_claims = extract_claims(query, golden_answer, model, client)\n",
- "\n",
- "print(f\"List of claims extracted from the gold answer:\\n\\n{gold_claims}\")"
- ],
+ "execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
@@ -901,11 +742,10 @@
"id": "wJeHJhiAxQjy",
"outputId": "8da8edf0-5605-4548-aab4-002d0202ee88"
},
- "execution_count": null,
"outputs": [
{
- "output_type": "stream",
"name": "stdout",
+ "output_type": "stream",
"text": [
"List of claims extracted from the gold answer:\n",
"\n",
@@ -917,30 +757,26 @@
"- There was a decrease in total net sales in the quarters ended April 1, 2023, and July 1, 2023.\n"
]
}
+ ],
+ "source": [
+ "# let's extract the golden claims\n",
+ "gold_claims = extract_claims(query, golden_answer, model, client)\n",
+ "\n",
+ "print(f\"List of claims extracted from the gold answer:\\n\\n{gold_claims}\")"
]
},
{
"cell_type": "markdown",
- "source": [
- "Then, we check which of these claims is present in the response generated by the model."
- ],
"metadata": {
"id": "6VXxxrL1SuFv"
- }
+ },
+ "source": [
+ "Then, we check which of these claims is present in the response generated by the model."
+ ]
},
{
"cell_type": "code",
- "source": [
- "# note that in, this case, the context is the model's response\n",
- "assessed_claims_coverage = assess_claims(query=query,\n",
- " claims=gold_claims,\n",
- " context=response,\n",
- " model=model,\n",
- " client=client)\n",
- "\n",
- "\n",
- "print(f\"Assess which of the gold claims is in the model's response:\\n\\n{assessed_claims_coverage}\")"
- ],
+ "execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
@@ -948,11 +784,10 @@
"id": "Ro-sYsp5SnOo",
"outputId": "d334a72e-a87a-4900-b9e6-9a6c2140dffb"
},
- "execution_count": null,
"outputs": [
{
- "output_type": "stream",
"name": "stdout",
+ "output_type": "stream",
"text": [
"Assess which of the gold claims is in the model's response:\n",
"\n",
@@ -964,15 +799,22 @@
"- There was a decrease in total net sales in the quarters ended April 1, 2023, and July 1, 2023. SUPPORTED=1\n"
]
}
+ ],
+ "source": [
+ "# note that in, this case, the context is the model's response\n",
+ "assessed_claims_coverage = assess_claims(query=query,\n",
+ " claims=gold_claims,\n",
+ " context=response,\n",
+ " model=model,\n",
+ " client=client)\n",
+ "\n",
+ "\n",
+ "print(f\"Assess which of the gold claims is in the model's response:\\n\\n{assessed_claims_coverage}\")"
]
},
{
"cell_type": "code",
- "source": [
- "# we compute the final Coverage score\n",
- "score_coverage = get_final_score(assessed_claims_coverage)\n",
- "print(f'Coverage: {score_coverage}')"
- ],
+ "execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
@@ -980,39 +822,67 @@
"id": "FUvVUN8qSnG2",
"outputId": "e854432a-f13b-4596-fb7c-f9b51ee870ec"
},
- "execution_count": null,
"outputs": [
{
- "output_type": "stream",
"name": "stdout",
+ "output_type": "stream",
"text": [
"Coverage: 0.33\n"
]
}
+ ],
+ "source": [
+ "# we compute the final Coverage score\n",
+ "score_coverage = get_final_score(assessed_claims_coverage)\n",
+ "print(f'Coverage: {score_coverage}')"
]
},
{
"cell_type": "markdown",
+ "metadata": {
+ "id": "pduSBr2-U_R7"
+ },
"source": [
"The Coverage score is telling us that 1/3 of the information in the gold answer is present in the generated answer. This is a useful information, that, similarly to what said above regarding Correctness, can raise further questions, such as: is it acceptable to have diverging information in the generated answer? Is any crucial piece of information missing in the generated answer?\n",
"\n",
"The answer to these questions is use case-specific, and has to be made by the end user: The claim-based approach implemented here supports the user by providing a clear and detailed view on what the model is assessing and how."
- ],
- "metadata": {
- "id": "pduSBr2-U_R7"
- }
+ ]
},
{
"cell_type": "markdown",
+ "metadata": {
+ "id": "0-1FsN2dHzMS"
+ },
"source": [
"\n",
"## Final Comments\n",
"\n",
"RAG evaluation is a hard task, especially the evaluation of the generated response. In this notebook we offer a clear, robust and replicable approach to evaluation, on which you can build on to build your evaluation pipeline."
- ],
- "metadata": {
- "id": "0-1FsN2dHzMS"
- }
+ ]
}
- ]
+ ],
+ "metadata": {
+ "colab": {
+ "provenance": []
+ },
+ "kernelspec": {
+ "name": "general_venv",
+ "language": "python",
+ "display_name": "general_venv"
+ },
+ "language_info": {
+ "codemirror_mode": {
+ "name": "ipython",
+ "version": 3
+ },
+ "file_extension": ".py",
+ "mimetype": "text/x-python",
+ "name": "python",
+ "nbconvert_exporter": "python",
+ "pygments_lexer": "ipython3",
+ "version": "3.8.16"
+ }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 4
}
diff --git a/notebooks/guides/Document_Parsing_For_Enterprises.ipynb b/notebooks/guides/Document_Parsing_For_Enterprises.ipynb
index d5e4c51b..0076c295 100644
--- a/notebooks/guides/Document_Parsing_For_Enterprises.ipynb
+++ b/notebooks/guides/Document_Parsing_For_Enterprises.ipynb
@@ -18,22 +18,17 @@
"## Introduction\n",
"\n",
"The bread and butter of natural language processing technology is text. Once we can reduce a set of data into text, we can do all kinds of things with it: question answering, summarization, classification, sentiment analysis, searching and indexing, and more.\n",
- " \n",
- " \n",
+ "\n",
"In the context of enterprise Retrieval Augmented Generation (RAG), the information is often locked in complex file types such as PDFs. These formats are made for sharing information between humans, but not so much with language models.\n",
- " \n",
- " \n",
+ "\n",
"In this notebook, we will use a real-world pharmaceutical drug label to test out various performant approaches to parsing PDFs. This will allow us to use [Cohere's Command-R model](https://txt.cohere.com/command-r/) in a RAG setting to answer questions and asks about this label, such as \"I need a succinct summary of the compound name, indication, route of administration, and mechanism of action of\" a given pharmaceutical.\n",
- " \n",
- " \n",
- ""
+ "\n",
+ ""
]
},
{
"cell_type": "markdown",
- "metadata": {
- "id": "f-mq1FCojI2p"
- },
+ "metadata": {},
"source": [
"\n",
"## PDF Parsing\n",
@@ -47,7 +42,7 @@
"\n",
"By way of example, we will be parsing a [21-page PDF](https://www.accessdata.fda.gov/drugsatfda_docs/label/2023/215500s000lbl.pdf) containing the label for a recent FDA drug approval, the beginning of which is shown below. Then, we will perform a series of basic RAG tasks with our different parsings and evaluate their performance.\n",
"\n",
- ""
+ ""
]
},
{
@@ -212,7 +207,7 @@
"### Solution 1: Google Cloud Document AI [[Back to Solutions]](#top)\n",
"\n",
"Document AI helps developers create high-accuracy processors to extract, classify, and split documents.\n",
- " \n",
+ "\n",
"External documentation: https://cloud.google.com/document-ai"
]
},
@@ -224,13 +219,13 @@
"source": [
"#### Parsing the document\n",
"\n",
- "The following block can be executed in one of two ways\n",
- "1. Inside a Google Vertex AI environment\n",
- " - No authentication is needed\n",
- "2. Inside the notebook\n",
- " - Authentication needed\n",
- " - There are pointers inside the code on which lines to uncomment in order to achieve that\n",
- " \n",
+ "The following block can be executed in one of two ways:\n",
+ "- Inside a Google Vertex AI environment\n",
+ " - No authentication needed\n",
+ "- From this notebook\n",
+ " - Authentication is needed\n",
+ " - There are pointers inside the code on which lines to uncomment in order to make this work\n",
+ "\n",
"**Note: You can skip to the next block if you want to use the pre-existing parsed version.**"
]
},
@@ -364,47 +359,35 @@
"id": "CT4QwMTwLPad",
"outputId": "1c85c3aa-a3b5-48c4-bb6e-5033c2c604a0"
},
- "outputs": [
- {
- "data": {
- "application/vnd.google.colaboratory.intrinsic+json": {
- "type": "string"
- },
- "text/plain": [
- "'\\nPost process parsed document and store it locally.\\nMake sure to run in Google Vertex AI environment or include a credentials file.\\n'"
- ]
- },
- "execution_count": 10,
- "metadata": {},
- "output_type": "execute_result"
- }
- ],
+ "outputs": [],
"source": [
"\"\"\"\n",
"Post process parsed document and store it locally.\n",
- "Make sure to run in Google Vertex AI environment or include a credentials file.\n",
+ "Make sure to run this in a Google Vertex AI environment or include a credentials file.\n",
"\"\"\"\n",
"\n",
- "# from pathlib import Path\n",
- "# from collections import defaultdict\n",
+ "\"\"\"\n",
+ "from pathlib import Path\n",
+ "from collections import defaultdict\n",
"\n",
- "# parsed_documents = []\n",
- "# combined_versioned_parsed_documents = defaultdict(list)\n",
+ "parsed_documents = []\n",
+ "combined_versioned_parsed_documents = defaultdict(list)\n",
"\n",
- "# # Assemble versioned documents together ({\"doc_name\": [(0, doc_content_0), (1, doc_content_1), ...]}).\n",
- "# for filename, doc_content in versioned_parsed_documents:\n",
- "# filename, version = \"-\".join(filename.split(\"-\")[:-1]), filename.split(\"-\")[-1]\n",
- "# combined_versioned_parsed_documents[filename].append((version, doc_content))\n",
+ "# Assemble versioned documents together ({\"doc_name\": [(0, doc_content_0), (1, doc_content_1), ...]}).\n",
+ "for filename, doc_content in versioned_parsed_documents:\n",
+ " filename, version = \"-\".join(filename.split(\"-\")[:-1]), filename.split(\"-\")[-1]\n",
+ " combined_versioned_parsed_documents[filename].append((version, doc_content))\n",
"\n",
- "# # Sort documents by version and join the content together.\n",
- "# for filename, docs in combined_versioned_parsed_documents.items():\n",
- "# doc_content = \" \".join([x[1] for x in sorted(docs, key=lambda x: x[0])])\n",
- "# parsed_documents.append((filename, doc_content))\n",
+ "# Sort documents by version and join the content together.\n",
+ "for filename, docs in combined_versioned_parsed_documents.items():\n",
+ " doc_content = \" \".join([x[1] for x in sorted(docs, key=lambda x: x[0])])\n",
+ " parsed_documents.append((filename, doc_content))\n",
"\n",
- "# # Store parsed documents in local storage.\n",
- "# for filename, doc_content in parsed_documents:\n",
- "# file_path = \"{}/{}-parsed-{}.txt\".format(data_dir, \"gcp\", source_filename)\n",
- "# store_document(file_path, doc_content)"
+ "# Store parsed documents in local storage.\n",
+ "for filename, doc_content in parsed_documents:\n",
+ " file_path = \"{}/{}-parsed-{}.txt\".format(data_dir, \"gcp\", source_filename)\n",
+ " store_document(file_path, doc_content)\n",
+ "\"\"\""
]
},
{
@@ -440,9 +423,7 @@
"\n",
"### Solution 2: AWS Textract [[Back to Solutions]](#top)\n",
"\n",
- "[Amazon Textract](https://aws.amazon.com/textract/) is an OCR service offered by AWS. It can detect text, forms, tables, and more in PDFs and images. In this section, we go over how to use Textract's asynchronous API.\n",
- " \n",
- " "
+ "[Amazon Textract](https://aws.amazon.com/textract/) is an OCR service offered by AWS. It can detect text, forms, tables, and more in PDFs and images. In this section, we go over how to use Textract's asynchronous API."
]
},
{
@@ -460,7 +441,7 @@
"id": "vFBasw782Gho"
},
"source": [
- "We assume you are working within the AWS ecosystem (from a SageMaker notebook, EC2 instance, a Lambda function, ...) with valid credentials. Much of the code here is from supplemental materials created by AWS and offered here:\n",
+ "We assume that you are working within the AWS ecosystem (from a SageMaker notebook, EC2 instance, a Lambda function, etc.) with valid credentials. Much of the code here is from supplemental materials created by AWS and offered here:\n",
"\n",
"- https://github.com/awsdocs/aws-doc-sdk-examples/tree/main/python/example_code/textract\n",
"- https://github.com/aws-samples/textract-paragraph-identification/tree/main\n",
@@ -769,8 +750,7 @@
"id": "kpF9UagY01-l"
},
"source": [
- "Next, we set up Textract and S3, and provide this to an instance of `TextractWrapper`.\n",
- "\n"
+ "Next, we set up Textract and S3, and provide this to an instance of `TextractWrapper`."
]
},
{
@@ -987,7 +967,7 @@
"### Solution 3: Unstructured.io [[Back to Solutions]](#top)\n",
"\n",
"Unstructured.io provides libraries with open-source components for pre-processing text documents such as PDFs, HTML and Word Documents.\n",
- " \n",
+ "\n",
"External documentation: https://github.com/Unstructured-IO/unstructured-api"
]
},
@@ -1002,7 +982,7 @@
"The guide assumes an endpoint exists that hosts this service. The API is offered in two forms\n",
"1. [a hosted version](https://unstructured.io/)\n",
"2. [an OSS docker image](https://github.com/Unstructured-IO/unstructured-api?tab=readme-ov-file#dizzy-instructions-for-using-the-docker-image)\n",
- " \n",
+ "\n",
"**Note: You can skip to the next block if you want to use the pre-existing parsed version.**"
]
},
@@ -1092,7 +1072,7 @@
"### Solution 4: LlamaParse [[Back to Solutions]](#top)\n",
"\n",
"LlamaParse is an API created by LlamaIndex to efficiently parse and represent files for efficient retrieval and context augmentation using LlamaIndex frameworks.\n",
- " \n",
+ "\n",
"External documentation: https://github.com/run-llama/llama_parse"
]
},
@@ -1105,11 +1085,10 @@
"#### Parsing the document\n",
"\n",
"The following block uses the LlamaParse cloud offering. You can learn more and fetch a respective API key for the service [here](https://cloud.llamaindex.ai/parse).\n",
- " \n",
+ "\n",
"Parsing documents with LlamaParse offers an option for two output modes both of which we will explore and compare below\n",
"- Text\n",
"- Markdown\n",
- " \n",
"\n",
"**Note: You can skip to the next block if you want to use the pre-existing parsed version.**"
]
diff --git a/notebooks/guides/Fueling_Generative_Content_with_Keyword_Research.ipynb b/notebooks/guides/Fueling_Generative_Content_with_Keyword_Research.ipynb
index 3c45e59b..08d2dc86 100644
--- a/notebooks/guides/Fueling_Generative_Content_with_Keyword_Research.ipynb
+++ b/notebooks/guides/Fueling_Generative_Content_with_Keyword_Research.ipynb
@@ -1,1071 +1,1063 @@
{
- "cells": [
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "\n",
- " \n",
- ""
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {
- "id": "5HkhthoeyQnV"
- },
- "source": [
- "# Fueling Generative Content with Keyword Research\n",
- "\n",
- "Generative models have proven extremely useful in content idea generation. But they don’t take into account user search demand and trends. In this notebook, let’s see how we can solve that by adding keyword research into the equation.\n",
- "\n",
- "Read the accompanying [blog post here](https://txt.cohere.ai/generative-content-keyword-research/)."
- ]
- },
- {
- "cell_type": "code",
- "execution_count": 2,
- "metadata": {},
- "outputs": [],
- "source": [
- "# Install packages\n",
- "! pip install cohere -q"
- ]
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "\n",
+ " \n",
+ ""
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "5HkhthoeyQnV"
+ },
+ "source": [
+ "# Fueling Generative Content with Keyword Research\n",
+ "\n",
+ "Generative models have proven extremely useful in content idea generation. But they don’t take into account user search demand and trends. In this notebook, let’s see how we can solve that by adding keyword research into the equation.\n",
+ "\n",
+ "Read the accompanying [blog post here](https://txt.cohere.ai/generative-content-keyword-research/)."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 2,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "# Install packages\n",
+ "! pip install cohere -q"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 10,
+ "metadata": {
+ "colab": {
+ "base_uri": "https://localhost:8080/",
+ "height": 17
},
+ "id": "4awUCb_3jw-v",
+ "outputId": "ab8e8e13-99a8-47fc-8736-49a698476075"
+ },
+ "outputs": [
{
- "cell_type": "code",
- "execution_count": 10,
- "metadata": {
- "colab": {
- "base_uri": "https://localhost:8080/",
- "height": 17
- },
- "id": "4awUCb_3jw-v",
- "outputId": "ab8e8e13-99a8-47fc-8736-49a698476075"
- },
- "outputs": [
- {
- "data": {
- "text/html": [
- "\n",
- " \n",
- " "
- ],
- "text/plain": [
- ""
- ]
- },
- "metadata": {},
- "output_type": "display_data"
- }
+ "data": {
+ "text/html": [
+ "\n",
+ " \n",
+ " "
],
- "source": [
- "import cohere\n",
- "import numpy as np\n",
- "import pandas as pd\n",
- "from sklearn.cluster import KMeans\n",
- "\n",
- "import cohere\n",
- "co = cohere.Client(\"COHERE_API_KEY\") # Get your API key: https://dashboard.cohere.com/api-keys"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": 2,
- "metadata": {},
- "outputs": [],
- "source": [
- "#@title Enable text wrapping in Google Colab\n",
- "\n",
- "from IPython.display import HTML, display\n",
- "\n",
- "def set_css():\n",
- " display(HTML('''\n",
- " \n",
- " '''))\n",
- "get_ipython().events.register('pre_run_cell', set_css)"
+ "text/plain": [
+ ""
]
- },
+ },
+ "metadata": {},
+ "output_type": "display_data"
+ }
+ ],
+ "source": [
+ "import cohere\n",
+ "import numpy as np\n",
+ "import pandas as pd\n",
+ "from sklearn.cluster import KMeans\n",
+ "\n",
+ "import cohere\n",
+ "co = cohere.Client(\"COHERE_API_KEY\") # Get your API key: https://dashboard.cohere.com/api-keys"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 2,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "#@title Enable text wrapping in Google Colab\n",
+ "\n",
+ "from IPython.display import HTML, display\n",
+ "\n",
+ "def set_css():\n",
+ " display(HTML('''\n",
+ " \n",
+ " '''))\n",
+ "get_ipython().events.register('pre_run_cell', set_css)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "9YsKrQFsyZIE"
+ },
+ "source": [
+ "# Step 1: Get a list of High-performing Keywords "
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "First, we need to get a supply of high-traffic keywords for a given topic. We can get this via keyword research tools, of which are many available. We’ll use Google Keyword Planner, which is free to use."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 3,
+ "metadata": {},
+ "outputs": [
{
- "cell_type": "markdown",
- "metadata": {
- "id": "9YsKrQFsyZIE"
- },
- "source": [
- "# Step 1: Get a list of High-performing Keywords "
+ "data": {
+ "text/html": [
+ "\n",
+ " \n",
+ " "
+ ],
+ "text/plain": [
+ ""
]
+ },
+ "metadata": {},
+ "output_type": "display_data"
},
{
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "First, we need to get a supply of high-traffic keywords for a given topic. We can get this via keyword research tools, of which are many available. We’ll use Google Keyword Planner, which is free to use."
+ "data": {
+ "text/plain": [
+ "'remote_teams.csv'"
]
+ },
+ "execution_count": 3,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "# Download the pre-created dataset (feel free to replace with your CSV file, containing two columns - \"keyword\" and \"volume\")\n",
+ "\n",
+ "import wget\n",
+ "wget.download(\"https://raw.githubusercontent.com/cohere-ai/notebooks/main/notebooks/data/remote_teams.csv\", \"remote_teams.csv\")"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 4,
+ "metadata": {
+ "colab": {
+ "base_uri": "https://localhost:8080/",
+ "height": 206
},
+ "id": "_JdLS_FVyjfr",
+ "outputId": "211c8b07-da89-4269-a652-2c164d09338d"
+ },
+ "outputs": [
{
- "cell_type": "code",
- "execution_count": 3,
- "metadata": {},
- "outputs": [
- {
- "data": {
- "text/html": [
- "\n",
- " \n",
- " "
- ],
- "text/plain": [
- ""
- ]
- },
- "metadata": {},
- "output_type": "display_data"
- },
- {
- "data": {
- "text/plain": [
- "'remote_teams.csv'"
- ]
- },
- "execution_count": 3,
- "metadata": {},
- "output_type": "execute_result"
- }
+ "data": {
+ "text/html": [
+ "\n",
+ " \n",
+ " "
],
- "source": [
- "# Download the pre-created dataset (feel free to replace with your CSV file, containing two columns - \"keyword\" and \"volume\")\n",
- "\n",
- "import wget\n",
- "wget.download(\"https://raw.githubusercontent.com/cohere-ai/notebooks/main/notebooks/data/remote_teams.csv\", \"remote_teams.csv\")"
+ "text/plain": [
+ ""
]
+ },
+ "metadata": {},
+ "output_type": "display_data"
},
{
- "cell_type": "code",
- "execution_count": 4,
- "metadata": {
- "colab": {
- "base_uri": "https://localhost:8080/",
- "height": 206
- },
- "id": "_JdLS_FVyjfr",
- "outputId": "211c8b07-da89-4269-a652-2c164d09338d"
- },
- "outputs": [
- {
- "data": {
- "text/html": [
- "\n",
- " \n",
- " "
- ],
- "text/plain": [
- ""
- ]
- },
- "metadata": {},
- "output_type": "display_data"
- },
- {
- "data": {
- "text/html": [
- "
\n",
- "\n",
- "
\n",
- " \n",
- "
\n",
- "
\n",
- "
keyword
\n",
- "
volume
\n",
- "
\n",
- " \n",
- " \n",
- "
\n",
- "
0
\n",
- "
managing remote teams
\n",
- "
1000
\n",
- "
\n",
- "
\n",
- "
1
\n",
- "
remote teams
\n",
- "
390
\n",
- "
\n",
- "
\n",
- "
2
\n",
- "
collaboration tools for remote teams
\n",
- "
320
\n",
- "
\n",
- "
\n",
- "
3
\n",
- "
online games for remote teams
\n",
- "
320
\n",
- "
\n",
- "
\n",
- "
4
\n",
- "
how to manage remote teams
\n",
- "
260
\n",
- "
\n",
- " \n",
- "
\n",
- "
"
- ],
- "text/plain": [
- " keyword volume\n",
- "0 managing remote teams 1000\n",
- "1 remote teams 390\n",
- "2 collaboration tools for remote teams 320\n",
- "3 online games for remote teams 320\n",
- "4 how to manage remote teams 260"
- ]
- },
- "execution_count": 4,
- "metadata": {},
- "output_type": "execute_result"
- }
+ "data": {
+ "text/html": [
+ "
\n",
+ "\n",
+ "
\n",
+ " \n",
+ "
\n",
+ "
\n",
+ "
keyword
\n",
+ "
volume
\n",
+ "
\n",
+ " \n",
+ " \n",
+ "
\n",
+ "
0
\n",
+ "
managing remote teams
\n",
+ "
1000
\n",
+ "
\n",
+ "
\n",
+ "
1
\n",
+ "
remote teams
\n",
+ "
390
\n",
+ "
\n",
+ "
\n",
+ "
2
\n",
+ "
collaboration tools for remote teams
\n",
+ "
320
\n",
+ "
\n",
+ "
\n",
+ "
3
\n",
+ "
online games for remote teams
\n",
+ "
320
\n",
+ "
\n",
+ "
\n",
+ "
4
\n",
+ "
how to manage remote teams
\n",
+ "
260
\n",
+ "
\n",
+ " \n",
+ "
\n",
+ "
"
],
- "source": [
- "# Create a dataframe\n",
- "df = pd.read_csv('remote_teams.csv')\n",
- "df.columns = [\"keyword\",\"volume\"]\n",
- "df.head()"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {
- "id": "whqG2M2Dylxu"
- },
- "source": [
- "# Step 2: Group the Keywords into Topics "
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "We now have a list of keywords, but this list is still raw. For example, “managing remote teams” is the top-ranking keyword in this list. But at the same time, there are many similar keywords further down in the list, such as “how to effectively manage remote teams.”\n",
- "\n",
- "We can do that by clustering them into topics. For this, we’ll leverage Cohere’s Embed endpoint and scikit-learn."
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {
- "id": "qJbbahQA0BJb"
- },
- "source": [
- "### Embed the Keywords with Cohere Embed"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "The Cohere Embed endpoint turns a text input into a text embedding."
+ "text/plain": [
+ " keyword volume\n",
+ "0 managing remote teams 1000\n",
+ "1 remote teams 390\n",
+ "2 collaboration tools for remote teams 320\n",
+ "3 online games for remote teams 320\n",
+ "4 how to manage remote teams 260"
]
+ },
+ "execution_count": 4,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "# Create a dataframe\n",
+ "df = pd.read_csv('remote_teams.csv')\n",
+ "df.columns = [\"keyword\",\"volume\"]\n",
+ "df.head()"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "whqG2M2Dylxu"
+ },
+ "source": [
+ "# Step 2: Group the Keywords into Topics "
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "We now have a list of keywords, but this list is still raw. For example, “managing remote teams” is the top-ranking keyword in this list. But at the same time, there are many similar keywords further down in the list, such as “how to effectively manage remote teams.”\n",
+ "\n",
+ "We can do that by clustering them into topics. For this, we’ll leverage Cohere’s Embed endpoint and scikit-learn."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "qJbbahQA0BJb"
+ },
+ "source": [
+ "### Embed the Keywords with Cohere Embed"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "The Cohere Embed endpoint turns a text input into a text embedding."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 6,
+ "metadata": {
+ "colab": {
+ "base_uri": "https://localhost:8080/",
+ "height": 17
},
+ "id": "O7vz9gkXjGMI",
+ "outputId": "d7f53d3d-e426-4433-a5be-1bb9c7d2b566"
+ },
+ "outputs": [
{
- "cell_type": "code",
- "execution_count": 6,
- "metadata": {
- "colab": {
- "base_uri": "https://localhost:8080/",
- "height": 17
- },
- "id": "O7vz9gkXjGMI",
- "outputId": "d7f53d3d-e426-4433-a5be-1bb9c7d2b566"
- },
- "outputs": [
- {
- "data": {
- "text/html": [
- "\n",
- " \n",
- " "
- ],
- "text/plain": [
- ""
- ]
- },
- "metadata": {},
- "output_type": "display_data"
- }
+ "data": {
+ "text/html": [
+ "\n",
+ " \n",
+ " "
],
- "source": [
- "def embed_text(texts):\n",
- " output = co.embed(\n",
- " texts=texts,\n",
- " model='embed-english-v3.0',\n",
- " input_type=\"search_document\",\n",
- " )\n",
- " return output.embeddings\n",
- "\n",
- "embeds = np.array(embed_text(df['keyword'].tolist()))"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {
- "id": "GKnEI1E40Djn"
- },
- "source": [
- "### Cluster the Keywords into Topics with scikit-learn"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "We then use these embeddings to cluster the keywords. A common term used for this exercise is “topic modeling.” Here, we can leverage scikit-learn’s KMeans module, a machine learning algorithm for clustering."
+ "text/plain": [
+ ""
]
+ },
+ "metadata": {},
+ "output_type": "display_data"
+ }
+ ],
+ "source": [
+ "def embed_text(texts):\n",
+ " output = co.embed(\n",
+ " texts=texts,\n",
+ " model='embed-english-v3.0',\n",
+ " input_type=\"search_document\",\n",
+ " )\n",
+ " return output.embeddings\n",
+ "\n",
+ "embeds = np.array(embed_text(df['keyword'].tolist()))"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "GKnEI1E40Djn"
+ },
+ "source": [
+ "### Cluster the Keywords into Topics with scikit-learn"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "We then use these embeddings to cluster the keywords. A common term used for this exercise is “topic modeling.” Here, we can leverage scikit-learn’s KMeans module, a machine learning algorithm for clustering."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 7,
+ "metadata": {
+ "colab": {
+ "base_uri": "https://localhost:8080/",
+ "height": 206
},
+ "id": "jUAZueb0luKN",
+ "outputId": "b05799e5-665d-4dcf-fb30-7eb7df1ed03f"
+ },
+ "outputs": [
{
- "cell_type": "code",
- "execution_count": 7,
- "metadata": {
- "colab": {
- "base_uri": "https://localhost:8080/",
- "height": 206
- },
- "id": "jUAZueb0luKN",
- "outputId": "b05799e5-665d-4dcf-fb30-7eb7df1ed03f"
- },
- "outputs": [
- {
- "data": {
- "text/html": [
- "\n",
- " \n",
- " "
- ],
- "text/plain": [
- ""
- ]
- },
- "metadata": {},
- "output_type": "display_data"
- },
- {
- "data": {
- "text/html": [
- "
"
],
- "source": [
- "# Group the DataFrame by 'topic' and aggregate the 'keyword' column into sets (which automatically removes duplicates)\n",
- "topic_keywords_dict = {topic: list(set(group['keyword'])) for topic, group in df.groupby('topic')}"
+ "text/plain": [
+ " keyword volume topic\n",
+ "0 managing remote teams 1000 0\n",
+ "1 remote teams 390 1\n",
+ "2 collaboration tools for remote teams 320 1\n",
+ "3 online games for remote teams 320 3\n",
+ "4 how to manage remote teams 260 0"
]
- },
+ },
+ "execution_count": 7,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "NUM_TOPICS = 4\n",
+ "kmeans = KMeans(n_clusters=NUM_TOPICS, random_state=21, n_init=\"auto\").fit(embeds)\n",
+ "df['topic'] = list(kmeans.labels_)\n",
+ "df.head()"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "fLUzQoJD0IdY"
+ },
+ "source": [
+ "### Generate Topic Names with Cohere Chat"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "We use the Chat to generate a topic name for that cluster."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 14,
+ "metadata": {},
+ "outputs": [
{
- "cell_type": "code",
- "execution_count": 22,
- "metadata": {},
- "outputs": [
- {
- "data": {
- "text/html": [
- "\n",
- " \n",
- " "
- ],
- "text/plain": [
- ""
- ]
- },
- "metadata": {},
- "output_type": "display_data"
- }
+ "data": {
+ "text/html": [
+ "\n",
+ " \n",
+ " "
],
- "source": [
- "# Function to generate a topic name based on keywords\n",
- "def generate_topic_name(keywords):\n",
- " # Construct the prompt\n",
- " prompt = f\"\"\"Generate a concise topic name that best represents these keywords.\\\n",
- "Provide just the topic name and not any additional details.\n",
- "\n",
- "Keywords: {', '.join(keywords)}\"\"\"\n",
- " \n",
- " # Call the Cohere API\n",
- " response = co.chat(\n",
- " model='command-r', # Choose the model size\n",
- " message=prompt,\n",
- " preamble=\"\")\n",
- " \n",
- " # Return the generated text\n",
- " return response.text"
+ "text/plain": [
+ ""
]
- },
+ },
+ "metadata": {},
+ "output_type": "display_data"
+ }
+ ],
+ "source": [
+ "# Group the DataFrame by 'topic' and aggregate the 'keyword' column into sets (which automatically removes duplicates)\n",
+ "topic_keywords_dict = {topic: list(set(group['keyword'])) for topic, group in df.groupby('topic')}"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 22,
+ "metadata": {},
+ "outputs": [
{
- "cell_type": "code",
- "execution_count": 23,
- "metadata": {},
- "outputs": [
- {
- "data": {
- "text/html": [
- "\n",
- " \n",
- " "
- ],
- "text/plain": [
- ""
- ]
- },
- "metadata": {},
- "output_type": "display_data"
- },
- {
- "data": {
- "text/html": [
- "
\n",
- "\n",
- "
\n",
- " \n",
- "
\n",
- "
\n",
- "
keyword
\n",
- "
volume
\n",
- "
topic
\n",
- "
topic_name
\n",
- "
\n",
- " \n",
- " \n",
- "
\n",
- "
0
\n",
- "
managing remote teams
\n",
- "
1000
\n",
- "
0
\n",
- "
Remote Team Management
\n",
- "
\n",
- "
\n",
- "
1
\n",
- "
remote teams
\n",
- "
390
\n",
- "
1
\n",
- "
Remote Team Tools and Tips
\n",
- "
\n",
- "
\n",
- "
2
\n",
- "
collaboration tools for remote teams
\n",
- "
320
\n",
- "
1
\n",
- "
Remote Team Tools and Tips
\n",
- "
\n",
- "
\n",
- "
3
\n",
- "
online games for remote teams
\n",
- "
320
\n",
- "
3
\n",
- "
Remote Team Fun
\n",
- "
\n",
- "
\n",
- "
4
\n",
- "
how to manage remote teams
\n",
- "
260
\n",
- "
0
\n",
- "
Remote Team Management
\n",
- "
\n",
- " \n",
- "
\n",
- "
"
- ],
- "text/plain": [
- " keyword volume topic \\\n",
- "0 managing remote teams 1000 0 \n",
- "1 remote teams 390 1 \n",
- "2 collaboration tools for remote teams 320 1 \n",
- "3 online games for remote teams 320 3 \n",
- "4 how to manage remote teams 260 0 \n",
- "\n",
- " topic_name \n",
- "0 Remote Team Management \n",
- "1 Remote Team Tools and Tips \n",
- "2 Remote Team Tools and Tips \n",
- "3 Remote Team Fun \n",
- "4 Remote Team Management "
- ]
- },
- "execution_count": 23,
- "metadata": {},
- "output_type": "execute_result"
- }
+ "data": {
+ "text/html": [
+ "\n",
+ " \n",
+ " "
],
- "source": [
- "# Generate topic names and create a mapping of topic number to topic name\n",
- "topic_name_mapping = {topic: generate_topic_name(keywords) for topic, keywords in topic_keywords_dict.items()}\n",
- "\n",
- "# Use the mapping to create a new column in the DataFrame\n",
- "df['topic_name'] = df['topic'].map(topic_name_mapping)\n",
- "\n",
- "# Display the first few rows to verify the new column\n",
- "df.head()"
+ "text/plain": [
+ ""
]
- },
+ },
+ "metadata": {},
+ "output_type": "display_data"
+ }
+ ],
+ "source": [
+ "# Function to generate a topic name based on keywords\n",
+ "def generate_topic_name(keywords):\n",
+ " # Construct the prompt\n",
+ " prompt = f\"\"\"Generate a concise topic name that best represents these keywords.\\\n",
+ "Provide just the topic name and not any additional details.\n",
+ "\n",
+ "Keywords: {', '.join(keywords)}\"\"\"\n",
+ " \n",
+ " # Call the Cohere API\n",
+ " response = co.chat(\n",
+ " model='command-r', # Choose the model size\n",
+ " message=prompt,\n",
+ " preamble=\"\")\n",
+ " \n",
+ " # Return the generated text\n",
+ " return response.text"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 23,
+ "metadata": {},
+ "outputs": [
{
- "cell_type": "code",
- "execution_count": 25,
- "metadata": {},
- "outputs": [
- {
- "data": {
- "text/html": [
- "\n",
- " \n",
- " "
- ],
- "text/plain": [
- ""
- ]
- },
- "metadata": {},
- "output_type": "display_data"
- },
- {
- "name": "stdout",
- "output_type": "stream",
- "text": [
- "Topic 0: Remote Team Management\n",
- "Topic 1: Remote Team Tools and Tips\n",
- "Topic 2: Remote Team Resources\n",
- "Topic 3: Remote Team Fun\n"
- ]
- }
+ "data": {
+ "text/html": [
+ "\n",
+ " \n",
+ " "
],
- "source": [
- "# View the list of topics\n",
- "for topic, name in topic_name_mapping.items():\n",
- " print(f\"Topic {topic}: {name}\")"
+ "text/plain": [
+ ""
]
+ },
+ "metadata": {},
+ "output_type": "display_data"
},
{
- "cell_type": "markdown",
- "metadata": {
- "id": "pMQzL5sL0YNN"
- },
- "source": [
- "# Step 3: Generate Blog Post Ideas for Each Topic"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "Now that we have the keywords nicely grouped into topics, we can proceed to generate the content ideas.\n"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {
- "id": "1tlIS2db0fWU"
- },
- "source": [
- "### Take the Top Keywords from Each Topic"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "Here we can implement a filter to take just the top N keywords from each topic, sorted by the search volume. In our case, we use 10."
- ]
- },
- {
- "cell_type": "code",
- "execution_count": 26,
- "metadata": {
- "colab": {
- "base_uri": "https://localhost:8080/",
- "height": 17
- },
- "id": "1bnYoTUCGQTv",
- "outputId": "cabdd711-641f-44e7-b7c5-19bf85f2512c"
- },
- "outputs": [
- {
- "data": {
- "text/html": [
- "\n",
- " \n",
- " "
- ],
- "text/plain": [
- ""
- ]
- },
- "metadata": {},
- "output_type": "display_data"
- }
+ "data": {
+ "text/html": [
+ "
\n",
+ "\n",
+ "
\n",
+ " \n",
+ "
\n",
+ "
\n",
+ "
keyword
\n",
+ "
volume
\n",
+ "
topic
\n",
+ "
topic_name
\n",
+ "
\n",
+ " \n",
+ " \n",
+ "
\n",
+ "
0
\n",
+ "
managing remote teams
\n",
+ "
1000
\n",
+ "
0
\n",
+ "
Remote Team Management
\n",
+ "
\n",
+ "
\n",
+ "
1
\n",
+ "
remote teams
\n",
+ "
390
\n",
+ "
1
\n",
+ "
Remote Team Tools and Tips
\n",
+ "
\n",
+ "
\n",
+ "
2
\n",
+ "
collaboration tools for remote teams
\n",
+ "
320
\n",
+ "
1
\n",
+ "
Remote Team Tools and Tips
\n",
+ "
\n",
+ "
\n",
+ "
3
\n",
+ "
online games for remote teams
\n",
+ "
320
\n",
+ "
3
\n",
+ "
Remote Team Fun
\n",
+ "
\n",
+ "
\n",
+ "
4
\n",
+ "
how to manage remote teams
\n",
+ "
260
\n",
+ "
0
\n",
+ "
Remote Team Management
\n",
+ "
\n",
+ " \n",
+ "
\n",
+ "
"
],
- "source": [
- "TOP_N = 10\n",
- "\n",
- "# Group the DataFrame by topic and select the top N keywords sorted by volume\n",
- "top_keywords = (df.groupby('topic')\n",
- " .apply(lambda x: x.nlargest(TOP_N, 'volume'))\n",
- " .reset_index(drop=True))\n",
- "\n",
- "\n",
- "# Convert the DataFrame to a nested dictionary\n",
- "content_by_topic = {}\n",
- "for topic, group in top_keywords.groupby('topic'):\n",
- " keywords = ', '.join(list(group['keyword']))\n",
- " topic2name = topic2name = dict(df.groupby('topic')['topic_name'].first())\n",
- " topic_name = topic2name[topic]\n",
- " content_by_topic[topic] = {'topic_name': topic_name, 'keywords': keywords}"
+ "text/plain": [
+ " keyword volume topic \\\n",
+ "0 managing remote teams 1000 0 \n",
+ "1 remote teams 390 1 \n",
+ "2 collaboration tools for remote teams 320 1 \n",
+ "3 online games for remote teams 320 3 \n",
+ "4 how to manage remote teams 260 0 \n",
+ "\n",
+ " topic_name \n",
+ "0 Remote Team Management \n",
+ "1 Remote Team Tools and Tips \n",
+ "2 Remote Team Tools and Tips \n",
+ "3 Remote Team Fun \n",
+ "4 Remote Team Management "
]
- },
+ },
+ "execution_count": 23,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "# Generate topic names and create a mapping of topic number to topic name\n",
+ "topic_name_mapping = {topic: generate_topic_name(keywords) for topic, keywords in topic_keywords_dict.items()}\n",
+ "\n",
+ "# Use the mapping to create a new column in the DataFrame\n",
+ "df['topic_name'] = df['topic'].map(topic_name_mapping)\n",
+ "\n",
+ "# Display the first few rows to verify the new column\n",
+ "df.head()"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 25,
+ "metadata": {},
+ "outputs": [
{
- "cell_type": "code",
- "execution_count": 27,
- "metadata": {
- "colab": {
- "base_uri": "https://localhost:8080/",
- "height": 285
- },
- "id": "erNJQT3a5kkD",
- "outputId": "f8b0c153-0bc9-46bd-b7fe-e5d0283dc3ab"
- },
- "outputs": [
- {
- "data": {
- "text/html": [
- "\n",
- " \n",
- " "
- ],
- "text/plain": [
- ""
- ]
- },
- "metadata": {},
- "output_type": "display_data"
- },
- {
- "data": {
- "text/plain": [
- "{0: {'topic_name': 'Remote Team Management',\n",
- " 'keywords': 'managing remote teams, how to manage remote teams, leading remote teams, managing remote teams best practices, remote teams best practices, best practices for managing remote teams, manage remote teams, building culture in remote teams, culture building for remote teams, managing remote teams training'},\n",
- " 1: {'topic_name': 'Remote Team Tools and Tips',\n",
- " 'keywords': 'remote teams, collaboration tools for remote teams, team building for remote teams, scrum remote teams, tools for remote teams, zapier remote teams, working agreements for remote teams, working with remote teams, free collaboration tools for remote teams, free retrospective tools for remote teams'},\n",
- " 2: {'topic_name': 'Remote Team Resources',\n",
- " 'keywords': 'best collaboration tools for remote teams, slack best practices for remote teams, best communication tools for remote teams, best tools for remote teams, always on video for remote teams, best apps for remote teams, best free collaboration tools for remote teams, best games for remote teams, best gifts for remote teams, best ice breaker questions for remote teams'},\n",
- " 3: {'topic_name': 'Remote Team Fun',\n",
- " 'keywords': 'online games for remote teams, team building activities for remote teams, games for remote teams, retrospective ideas for remote teams, team building ideas for remote teams, fun retrospective ideas for remote teams, retro ideas for remote teams, team building exercises for remote teams, trust building exercises for remote teams, activities for remote teams'}}"
- ]
- },
- "execution_count": 27,
- "metadata": {},
- "output_type": "execute_result"
- }
+ "data": {
+ "text/html": [
+ "\n",
+ " \n",
+ " "
],
- "source": [
- "# Print the topics and they top keywords\n",
- "content_by_topic"
+ "text/plain": [
+ ""
]
+ },
+ "metadata": {},
+ "output_type": "display_data"
},
{
- "cell_type": "markdown",
- "metadata": {
- "id": "VCXY5cNj0qAW"
- },
- "source": [
- "### Create a Prompt with These Keywords"
- ]
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "Topic 0: Remote Team Management\n",
+ "Topic 1: Remote Team Tools and Tips\n",
+ "Topic 2: Remote Team Resources\n",
+ "Topic 3: Remote Team Fun\n"
+ ]
+ }
+ ],
+ "source": [
+ "# View the list of topics\n",
+ "for topic, name in topic_name_mapping.items():\n",
+ " print(f\"Topic {topic}: {name}\")"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "pMQzL5sL0YNN"
+ },
+ "source": [
+ "# Step 3: Generate Blog Post Ideas for Each Topic"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Now that we have the keywords nicely grouped into topics, we can proceed to generate the content ideas.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "1tlIS2db0fWU"
+ },
+ "source": [
+ "### Take the Top Keywords from Each Topic"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Here we can implement a filter to take just the top N keywords from each topic, sorted by the search volume. In our case, we use 10."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 26,
+ "metadata": {
+ "colab": {
+ "base_uri": "https://localhost:8080/",
+ "height": 17
},
+ "id": "1bnYoTUCGQTv",
+ "outputId": "cabdd711-641f-44e7-b7c5-19bf85f2512c"
+ },
+ "outputs": [
{
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "Next, we use the Chat endpoint to produce the content ideas. The prompt we’ll use is as follows"
+ "data": {
+ "text/html": [
+ "\n",
+ " \n",
+ " "
+ ],
+ "text/plain": [
+ ""
]
+ },
+ "metadata": {},
+ "output_type": "display_data"
+ }
+ ],
+ "source": [
+ "TOP_N = 10\n",
+ "\n",
+ "# Group the DataFrame by topic and select the top N keywords sorted by volume\n",
+ "top_keywords = (df.groupby('topic')\n",
+ " .apply(lambda x: x.nlargest(TOP_N, 'volume'))\n",
+ " .reset_index(drop=True))\n",
+ "\n",
+ "\n",
+ "# Convert the DataFrame to a nested dictionary\n",
+ "content_by_topic = {}\n",
+ "for topic, group in top_keywords.groupby('topic'):\n",
+ " keywords = ', '.join(list(group['keyword']))\n",
+ " topic2name = topic2name = dict(df.groupby('topic')['topic_name'].first())\n",
+ " topic_name = topic2name[topic]\n",
+ " content_by_topic[topic] = {'topic_name': topic_name, 'keywords': keywords}"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 27,
+ "metadata": {
+ "colab": {
+ "base_uri": "https://localhost:8080/",
+ "height": 285
},
+ "id": "erNJQT3a5kkD",
+ "outputId": "f8b0c153-0bc9-46bd-b7fe-e5d0283dc3ab"
+ },
+ "outputs": [
{
- "cell_type": "code",
- "execution_count": 28,
- "metadata": {
- "colab": {
- "base_uri": "https://localhost:8080/",
- "height": 17
- },
- "id": "M-g_FiA6fX2w",
- "outputId": "d4545e7c-ca43-4b58-ac87-2f2c4227eadc"
- },
- "outputs": [
- {
- "data": {
- "text/html": [
- "\n",
- " \n",
- " "
- ],
- "text/plain": [
- ""
- ]
- },
- "metadata": {},
- "output_type": "display_data"
- }
+ "data": {
+ "text/html": [
+ "\n",
+ " \n",
+ " "
],
- "source": [
- "def generate_blog_ideas(keywords):\n",
- " prompt = f\"\"\"{keywords}\\n\\nThe above is a list of high-traffic keywords obtained from a keyword research tool. \n",
- "Suggest three blog post ideas that are highly relevant to these keywords. \n",
- "For each idea, write a one paragraph abstract about the topic. \n",
- "Use this format:\n",
- "Blog title: \n",
- "Abstract: \"\"\"\n",
- " \n",
- " response = co.chat(\n",
- " model='command-r',\n",
- " message = prompt)\n",
- " return response.text\n"
+ "text/plain": [
+ ""
]
+ },
+ "metadata": {},
+ "output_type": "display_data"
},
{
- "cell_type": "markdown",
- "metadata": {
- "id": "wTV6GG8i0tdK"
- },
- "source": [
- "### Generate Content Ideas"
+ "data": {
+ "text/plain": [
+ "{0: {'topic_name': 'Remote Team Management',\n",
+ " 'keywords': 'managing remote teams, how to manage remote teams, leading remote teams, managing remote teams best practices, remote teams best practices, best practices for managing remote teams, manage remote teams, building culture in remote teams, culture building for remote teams, managing remote teams training'},\n",
+ " 1: {'topic_name': 'Remote Team Tools and Tips',\n",
+ " 'keywords': 'remote teams, collaboration tools for remote teams, team building for remote teams, scrum remote teams, tools for remote teams, zapier remote teams, working agreements for remote teams, working with remote teams, free collaboration tools for remote teams, free retrospective tools for remote teams'},\n",
+ " 2: {'topic_name': 'Remote Team Resources',\n",
+ " 'keywords': 'best collaboration tools for remote teams, slack best practices for remote teams, best communication tools for remote teams, best tools for remote teams, always on video for remote teams, best apps for remote teams, best free collaboration tools for remote teams, best games for remote teams, best gifts for remote teams, best ice breaker questions for remote teams'},\n",
+ " 3: {'topic_name': 'Remote Team Fun',\n",
+ " 'keywords': 'online games for remote teams, team building activities for remote teams, games for remote teams, retrospective ideas for remote teams, team building ideas for remote teams, fun retrospective ideas for remote teams, retro ideas for remote teams, team building exercises for remote teams, trust building exercises for remote teams, activities for remote teams'}}"
]
+ },
+ "execution_count": 27,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "# Print the topics and they top keywords\n",
+ "content_by_topic"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "VCXY5cNj0qAW"
+ },
+ "source": [
+ "### Create a Prompt with These Keywords"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Next, we use the Chat endpoint to produce the content ideas. The prompt we’ll use is as follows"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 28,
+ "metadata": {
+ "colab": {
+ "base_uri": "https://localhost:8080/",
+ "height": 17
},
+ "id": "M-g_FiA6fX2w",
+ "outputId": "d4545e7c-ca43-4b58-ac87-2f2c4227eadc"
+ },
+ "outputs": [
{
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "Next, we generate the blog post ideas. It takes in a string of keywords, calls the Chat endpoint, and returns the generated text."
+ "data": {
+ "text/html": [
+ "\n",
+ " \n",
+ " "
+ ],
+ "text/plain": [
+ ""
]
+ },
+ "metadata": {},
+ "output_type": "display_data"
+ }
+ ],
+ "source": [
+ "def generate_blog_ideas(keywords):\n",
+ " prompt = f\"\"\"{keywords}\\n\\nThe above is a list of high-traffic keywords obtained from a keyword research tool. \n",
+ "Suggest three blog post ideas that are highly relevant to these keywords. \n",
+ "For each idea, write a one paragraph abstract about the topic. \n",
+ "Use this format:\n",
+ "Blog title: \n",
+ "Abstract: \"\"\"\n",
+ " \n",
+ " response = co.chat(\n",
+ " model='command-r',\n",
+ " message = prompt)\n",
+ " return response.text\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "wTV6GG8i0tdK"
+ },
+ "source": [
+ "### Generate Content Ideas"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Next, we generate the blog post ideas. It takes in a string of keywords, calls the Chat endpoint, and returns the generated text."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 30,
+ "metadata": {
+ "colab": {
+ "base_uri": "https://localhost:8080/",
+ "height": 1000
},
+ "id": "-qGuVkIfmJ_d",
+ "outputId": "5b7588cb-cfda-4590-80ca-532fceb6f125"
+ },
+ "outputs": [
{
- "cell_type": "code",
- "execution_count": 30,
- "metadata": {
- "colab": {
- "base_uri": "https://localhost:8080/",
- "height": 1000
- },
- "id": "-qGuVkIfmJ_d",
- "outputId": "5b7588cb-cfda-4590-80ca-532fceb6f125"
- },
- "outputs": [
- {
- "data": {
- "text/html": [
- "\n",
- " \n",
- " "
- ],
- "text/plain": [
- ""
- ]
- },
- "metadata": {},
- "output_type": "display_data"
- },
- {
- "name": "stdout",
- "output_type": "stream",
- "text": [
- "Topic Name: Remote Team Management\n",
- "\n",
- "Top Keywords: managing remote teams, how to manage remote teams, leading remote teams, managing remote teams best practices, remote teams best practices, best practices for managing remote teams, manage remote teams, building culture in remote teams, culture building for remote teams, managing remote teams training\n",
- "\n",
- "Blog Post Ideas: Here are three blog post ideas:\n",
- "\n",
- "1. Blog title: \"Leading Remote Teams: Strategies for Effective Management\"\n",
- " Abstract: Effective management of remote teams is crucial for success, but it comes with unique challenges. This blog will explore practical strategies for leading dispersed employees, focusing on building a cohesive and productive virtual workforce. It will cover topics such as establishing clear communication protocols, fostering a collaborative environment, and the importance of trusting and empowering your remote employees for enhanced performance.\n",
- "\n",
- "2. Blog title: \"Remote Teams' Best Practices: Creating a Vibrant and Engaging Culture\"\n",
- " Abstract: Building a rich culture in a remote team setting is essential for employee engagement and retention. The blog will delve into creative ways to foster a sense of community and connection among team members who may be scattered across the globe. It will offer practical tips on creating virtual rituals, fostering open communication, and harnessing the power of technology for cultural development, ensuring remote employees feel valued and engaged.\n",
- "\n",
- "3. Blog title: \"Managing Remote Teams: A Comprehensive Guide to Training and Development\"\n",
- " Abstract: Training and developing remote teams present specific challenges and opportunities. This comprehensive guide will arm managers with techniques to enhance their remote team's skills and knowledge. It will explore the latest tools and methodologies for remote training, including virtual workshops, e-learning platforms, and performance coaching. Additionally, the blog will discuss the significance of ongoing development and how to create an environment that encourages self-improvement and learning.\n",
- "\n",
- "Each of these topics explores a specific aspect of managing remote teams, providing valuable insights and practical guidance for leaders and managers in the evolving remote work landscape.\n",
- "\n",
- "--------------------------------------------------\n",
- "Topic Name: Remote Team Tools and Tips\n",
- "\n",
- "Top Keywords: remote teams, collaboration tools for remote teams, team building for remote teams, scrum remote teams, tools for remote teams, zapier remote teams, working agreements for remote teams, working with remote teams, free collaboration tools for remote teams, free retrospective tools for remote teams\n",
- "\n",
- "Blog Post Ideas: 1. Blog title: \"The Ultimate Guide to Building Effective Remote Teams\"\n",
- " Abstract: Building a cohesive and productive remote team can be challenging. This blog will serve as a comprehensive guide, offering practical tips and insights on how to create a united and successful virtual workforce. It will cover essential topics such as building a strong team culture, utilizing collaboration tools, and fostering effective communication strategies, ensuring remote teams can thrive and achieve their full potential.\n",
- "\n",
- "2. Blog title: \"The Best Collaboration Tools for Remote Teams: A Comprehensive Review\"\n",
- " Abstract: With the rapid rise of remote work, collaboration tools have become essential for teams' productivity and efficiency. This blog aims to review and compare the most popular collaboration tools, providing an in-depth analysis of their features, ease of use, and benefits. It will offer insights into choosing the right tools for remote collaboration, helping teams streamline their workflows and enhance their overall performance.\n",
- "\n",
- "3. Blog title: \"Remote Retrospective: A Guide to Reflect and Grow as a Remote Team\"\n",
- " Abstract: Conducting effective retrospectives is crucial for remote teams to reflect on their experiences, learn from the past, and chart a course for the future. This blog will focus on remote retrospectives, exploring different formats, techniques, and free tools that teams can use to foster continuous improvement. It will also provide tips on creating a safe and inclusive environment, encouraging honest feedback and productive discussions.\n",
- "\n",
- "--------------------------------------------------\n",
- "Topic Name: Remote Team Resources\n",
- "\n",
- "Top Keywords: best collaboration tools for remote teams, slack best practices for remote teams, best communication tools for remote teams, best tools for remote teams, always on video for remote teams, best apps for remote teams, best free collaboration tools for remote teams, best games for remote teams, best gifts for remote teams, best ice breaker questions for remote teams\n",
- "\n",
- "Blog Post Ideas: 1. Blog title: \"The Ultimate Guide to Remote Team Collaboration Tools\"\n",
- " Abstract: With the rise of remote work, choosing the right collaboration tools can be crucial to a team's success and productivity. This blog aims to be an comprehensive guide, outlining the various types of tools available, from communication platforms like Slack to project management software and online collaboration tools. It will offer best practices and guidelines for selecting and utilizing these tools, ensuring remote teams can work seamlessly together and maximize their output.\n",
- "\n",
- "2. Blog title: \"Remote Team Management: Tips for Leading a Successful Virtual Workforce\"\n",
- " Abstract: Managing a remote team comes with its own set of challenges. This blog will provide an in-depth exploration of best practices for leading and motivating virtual teams. Covering topics such as effective communication strategies, performance evaluation, and maintaining a cohesive team culture, it will offer practical tips for managers and leaders to ensure their remote teams are engaged, productive, and well-managed.\n",
- "\n",
- "3. Blog title: \"The Fun Side of Remote Work: Games, Icebreakers, and Team Building Activities\"\n",
- " Abstract: Remote work can be isolating, and this blog aims to provide some fun and creative solutions. It will offer a comprehensive guide to the best online games, icebreaker questions, and virtual team building activities that remote teams can use to connect and bond. From virtual escape rooms to interactive games and thought-provoking icebreakers, these ideas will help enhance team spirit, foster collaboration, and create a enjoyable remote work experience.\n",
- "\n",
- "--------------------------------------------------\n",
- "Topic Name: Remote Team Fun\n",
- "\n",
- "Top Keywords: online games for remote teams, team building activities for remote teams, games for remote teams, retrospective ideas for remote teams, team building ideas for remote teams, fun retrospective ideas for remote teams, retro ideas for remote teams, team building exercises for remote teams, trust building exercises for remote teams, activities for remote teams\n",
- "\n",
- "Blog Post Ideas: 1. Blog title: \"The Great Remote Retro: Fun Games and Activities for Your Team\"\n",
- " Abstract: Remote work can make team building challenging. This blog post will be a fun guide to hosting interactive retro games and activities that bring your remote team together. From online escape rooms to virtual scavenger hunts, we'll explore the best ways to engage and unite your team, fostering collaboration and camaraderie. Virtual icebreakers and retrospective ideas will also be included to make your remote meetings more interactive and enjoyable.\n",
- "\n",
- "2. Blog title: \"Trust Falls: Building Trust Among Remote Teams\"\n",
- " Abstract: Trust is the foundation of every successful team, but how do you build it when everyone is scattered across different locations? This blog will focus on trust-building exercises and activities designed specifically for remote teams. From virtual trust falls to transparent communication practices, we'll discover innovative ways to strengthen team bonds and foster a culture of trust. You'll learn how to create an environment where your remote team can thrive and collaborate effectively.\n",
- "\n",
- "3. Blog title: \"Game Night for Remote Teams: A Guide to Online Games and Activities\"\n",
- " Abstract: Miss the old office game nights? This blog will bring the fun back with a guide to hosting online game nights and activities that are perfect for remote teams. From trivia games to virtual board games and even remote-friendly outdoor adventures, we'll keep your team engaged and entertained. With tips on setting up online tournaments and ideas for encouraging participation, your virtual game nights will be the highlight of your team's week. Keep your remote team spirit high!\n",
- "\n",
- "--------------------------------------------------\n"
- ]
- }
+ "data": {
+ "text/html": [
+ "\n",
+ " \n",
+ " "
],
- "source": [
- "# Generate content ideas\n",
- "for key,value in content_by_topic.items():\n",
- " value['ideas'] = generate_blog_ideas(value['keywords'])\n",
- "\n",
- "\n",
- "# Print the results\n",
- "for key,value in content_by_topic.items():\n",
- " print(f\"Topic Name: {value['topic_name']}\\n\")\n",
- " print(f\"Top Keywords: {value['keywords']}\\n\")\n",
- " print(f\"Blog Post Ideas: {value['ideas']}\\n\")\n",
- " print(\"-\"*50)"
+ "text/plain": [
+ ""
]
+ },
+ "metadata": {},
+ "output_type": "display_data"
},
{
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "id": "Anr0Hu3hLxE3"
- },
- "outputs": [],
- "source": []
- }
- ],
- "metadata": {
- "colab": {
- "provenance": []
- },
- "kernelspec": {
- "display_name": "Python 3",
- "name": "python3"
- },
- "language_info": {
- "codemirror_mode": {
- "name": "ipython",
- "version": 3
- },
- "file_extension": ".py",
- "mimetype": "text/x-python",
- "name": "python",
- "nbconvert_exporter": "python",
- "pygments_lexer": "ipython3",
- "version": "3.11.4"
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "Topic Name: Remote Team Management\n",
+ "\n",
+ "Top Keywords: managing remote teams, how to manage remote teams, leading remote teams, managing remote teams best practices, remote teams best practices, best practices for managing remote teams, manage remote teams, building culture in remote teams, culture building for remote teams, managing remote teams training\n",
+ "\n",
+ "Blog Post Ideas: Here are three blog post ideas:\n",
+ "\n",
+ "1. Blog title: \"Leading Remote Teams: Strategies for Effective Management\"\n",
+ " Abstract: Effective management of remote teams is crucial for success, but it comes with unique challenges. This blog will explore practical strategies for leading dispersed employees, focusing on building a cohesive and productive virtual workforce. It will cover topics such as establishing clear communication protocols, fostering a collaborative environment, and the importance of trusting and empowering your remote employees for enhanced performance.\n",
+ "\n",
+ "2. Blog title: \"Remote Teams' Best Practices: Creating a Vibrant and Engaging Culture\"\n",
+ " Abstract: Building a rich culture in a remote team setting is essential for employee engagement and retention. The blog will delve into creative ways to foster a sense of community and connection among team members who may be scattered across the globe. It will offer practical tips on creating virtual rituals, fostering open communication, and harnessing the power of technology for cultural development, ensuring remote employees feel valued and engaged.\n",
+ "\n",
+ "3. Blog title: \"Managing Remote Teams: A Comprehensive Guide to Training and Development\"\n",
+ " Abstract: Training and developing remote teams present specific challenges and opportunities. This comprehensive guide will arm managers with techniques to enhance their remote team's skills and knowledge. It will explore the latest tools and methodologies for remote training, including virtual workshops, e-learning platforms, and performance coaching. Additionally, the blog will discuss the significance of ongoing development and how to create an environment that encourages self-improvement and learning.\n",
+ "\n",
+ "Each of these topics explores a specific aspect of managing remote teams, providing valuable insights and practical guidance for leaders and managers in the evolving remote work landscape.\n",
+ "\n",
+ "--------------------------------------------------\n",
+ "Topic Name: Remote Team Tools and Tips\n",
+ "\n",
+ "Top Keywords: remote teams, collaboration tools for remote teams, team building for remote teams, scrum remote teams, tools for remote teams, zapier remote teams, working agreements for remote teams, working with remote teams, free collaboration tools for remote teams, free retrospective tools for remote teams\n",
+ "\n",
+ "Blog Post Ideas: 1. Blog title: \"The Ultimate Guide to Building Effective Remote Teams\"\n",
+ " Abstract: Building a cohesive and productive remote team can be challenging. This blog will serve as a comprehensive guide, offering practical tips and insights on how to create a united and successful virtual workforce. It will cover essential topics such as building a strong team culture, utilizing collaboration tools, and fostering effective communication strategies, ensuring remote teams can thrive and achieve their full potential.\n",
+ "\n",
+ "2. Blog title: \"The Best Collaboration Tools for Remote Teams: A Comprehensive Review\"\n",
+ " Abstract: With the rapid rise of remote work, collaboration tools have become essential for teams' productivity and efficiency. This blog aims to review and compare the most popular collaboration tools, providing an in-depth analysis of their features, ease of use, and benefits. It will offer insights into choosing the right tools for remote collaboration, helping teams streamline their workflows and enhance their overall performance.\n",
+ "\n",
+ "3. Blog title: \"Remote Retrospective: A Guide to Reflect and Grow as a Remote Team\"\n",
+ " Abstract: Conducting effective retrospectives is crucial for remote teams to reflect on their experiences, learn from the past, and chart a course for the future. This blog will focus on remote retrospectives, exploring different formats, techniques, and free tools that teams can use to foster continuous improvement. It will also provide tips on creating a safe and inclusive environment, encouraging honest feedback and productive discussions.\n",
+ "\n",
+ "--------------------------------------------------\n",
+ "Topic Name: Remote Team Resources\n",
+ "\n",
+ "Top Keywords: best collaboration tools for remote teams, slack best practices for remote teams, best communication tools for remote teams, best tools for remote teams, always on video for remote teams, best apps for remote teams, best free collaboration tools for remote teams, best games for remote teams, best gifts for remote teams, best ice breaker questions for remote teams\n",
+ "\n",
+ "Blog Post Ideas: 1. Blog title: \"The Ultimate Guide to Remote Team Collaboration Tools\"\n",
+ " Abstract: With the rise of remote work, choosing the right collaboration tools can be crucial to a team's success and productivity. This blog aims to be an comprehensive guide, outlining the various types of tools available, from communication platforms like Slack to project management software and online collaboration tools. It will offer best practices and guidelines for selecting and utilizing these tools, ensuring remote teams can work seamlessly together and maximize their output.\n",
+ "\n",
+ "2. Blog title: \"Remote Team Management: Tips for Leading a Successful Virtual Workforce\"\n",
+ " Abstract: Managing a remote team comes with its own set of challenges. This blog will provide an in-depth exploration of best practices for leading and motivating virtual teams. Covering topics such as effective communication strategies, performance evaluation, and maintaining a cohesive team culture, it will offer practical tips for managers and leaders to ensure their remote teams are engaged, productive, and well-managed.\n",
+ "\n",
+ "3. Blog title: \"The Fun Side of Remote Work: Games, Icebreakers, and Team Building Activities\"\n",
+ " Abstract: Remote work can be isolating, and this blog aims to provide some fun and creative solutions. It will offer a comprehensive guide to the best online games, icebreaker questions, and virtual team building activities that remote teams can use to connect and bond. From virtual escape rooms to interactive games and thought-provoking icebreakers, these ideas will help enhance team spirit, foster collaboration, and create a enjoyable remote work experience.\n",
+ "\n",
+ "--------------------------------------------------\n",
+ "Topic Name: Remote Team Fun\n",
+ "\n",
+ "Top Keywords: online games for remote teams, team building activities for remote teams, games for remote teams, retrospective ideas for remote teams, team building ideas for remote teams, fun retrospective ideas for remote teams, retro ideas for remote teams, team building exercises for remote teams, trust building exercises for remote teams, activities for remote teams\n",
+ "\n",
+ "Blog Post Ideas: 1. Blog title: \"The Great Remote Retro: Fun Games and Activities for Your Team\"\n",
+ " Abstract: Remote work can make team building challenging. This blog post will be a fun guide to hosting interactive retro games and activities that bring your remote team together. From online escape rooms to virtual scavenger hunts, we'll explore the best ways to engage and unite your team, fostering collaboration and camaraderie. Virtual icebreakers and retrospective ideas will also be included to make your remote meetings more interactive and enjoyable.\n",
+ "\n",
+ "2. Blog title: \"Trust Falls: Building Trust Among Remote Teams\"\n",
+ " Abstract: Trust is the foundation of every successful team, but how do you build it when everyone is scattered across different locations? This blog will focus on trust-building exercises and activities designed specifically for remote teams. From virtual trust falls to transparent communication practices, we'll discover innovative ways to strengthen team bonds and foster a culture of trust. You'll learn how to create an environment where your remote team can thrive and collaborate effectively.\n",
+ "\n",
+ "3. Blog title: \"Game Night for Remote Teams: A Guide to Online Games and Activities\"\n",
+ " Abstract: Miss the old office game nights? This blog will bring the fun back with a guide to hosting online game nights and activities that are perfect for remote teams. From trivia games to virtual board games and even remote-friendly outdoor adventures, we'll keep your team engaged and entertained. With tips on setting up online tournaments and ideas for encouraging participation, your virtual game nights will be the highlight of your team's week. Keep your remote team spirit high!\n",
+ "\n",
+ "--------------------------------------------------\n"
+ ]
}
+ ],
+ "source": [
+ "# Generate content ideas\n",
+ "for key,value in content_by_topic.items():\n",
+ " value['ideas'] = generate_blog_ideas(value['keywords'])\n",
+ "\n",
+ "\n",
+ "# Print the results\n",
+ "for key,value in content_by_topic.items():\n",
+ " print(f\"Topic Name: {value['topic_name']}\\n\")\n",
+ " print(f\"Top Keywords: {value['keywords']}\\n\")\n",
+ " print(f\"Blog Post Ideas: {value['ideas']}\\n\")\n",
+ " print(\"-\"*50)"
+ ]
+ }
+ ],
+ "metadata": {
+ "colab": {
+ "provenance": []
+ },
+ "kernelspec": {
+ "display_name": "Python 3 (ipykernel)",
+ "language": "python",
+ "name": "python3"
},
- "nbformat": 4,
- "nbformat_minor": 0
+ "language_info": {
+ "codemirror_mode": {
+ "name": "ipython",
+ "version": 3
+ },
+ "file_extension": ".py",
+ "mimetype": "text/x-python",
+ "name": "python",
+ "nbconvert_exporter": "python",
+ "pygments_lexer": "ipython3",
+ "version": "3.8.16"
+ }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 4
}
diff --git a/notebooks/guides/Long_form_General_Strategies.ipynb b/notebooks/guides/Long_form_General_Strategies.ipynb
index 0112fdff..5d51d5dc 100644
--- a/notebooks/guides/Long_form_General_Strategies.ipynb
+++ b/notebooks/guides/Long_form_General_Strategies.ipynb
@@ -1,44 +1,32 @@
{
"cells": [
- {
- "cell_type": "markdown",
- "metadata": {
- "id": "1CmEDootEpNV"
- },
- "source": [
- ""
- ]
- },
{
"cell_type": "markdown",
"metadata": {
"id": "_q-7fEZxmIHm"
},
"source": [
+ "# Long form - general strategies \n",
+ "\n",
"Large Language Models (LLMs) are becoming increasingly capable of comprehending text, among others excelling in document analysis. The new Cohere model, [Command-R](https://huggingface.co/CohereForAI/c4ai-command-r-v01), boasts a context length of 128k, which makes it particularly effective for such tasks. Nevertheless, even with the extended context window, some documents might be too lengthy to accommodate in full.\n",
"\n",
"In this cookbook, we'll explore techniques to address cases when relevant information doesn't fit in the model context window.\n",
"\n",
"We'll show you three potential mitigation strategies: truncating the document, query-based retrieval, and a \"text rank\" approach we use internally at Cohere.\n",
"\n",
- "### Table of content:\n",
+ "## Table of contents:\n",
"1. [Getting started](#getting-started)\n",
- "2. [Approach 1: Truncate](#truncate)\n",
- "3. [Approach 2: Query Based Retrieval](#query-based-retrieval)\n",
- "4. [Approach 3: Text Rank](#text-rank)\n",
+ "2. [Approach 1: Truncate](#approach-1)\n",
+ "3. [Approach 2: Query Based Retrieval](#approach-2)\n",
+ "4. [Approach 3: Text Rank](#approach-3)\n",
"\n",
- "### Summary\n",
+ "## Summary\n",
"\n",
"| Approach | Description | Pros | Cons | When to use? |\n",
"|-----------------------|-------------------------------------------|-------------------------------------------|-------------------------------------------|-------------------------------------------|\n",
"| Truncation | Truncate the document to fit the context window. | - Simplicity of implementation (does not rely on extrenal infrastructure)| - Loses information at the end of the document | Utilize when all relevant information is contained at the beginning of the document. |\n",
"| Query Based Retrieval| Utilize semantic similarity to retrieve text chunks that are most relevant to the query. | - Focuses on sections directly relevant to the query | - Relies on a semantic similarity algorithm. - Might lose broader context | Employ when seeking specific information within the text. |\n",
- "| Text Rank | Apply graph theory to generate a cohesive set of chunks that effectively represent the document. | - Preserves the broader picture. | - Might lose detailed information. | Utilize in summaries and when the question requires broader context. |\n",
- "\n",
- "\n",
- "\n",
- "\n",
- "\n"
+ "| Text Rank | Apply graph theory to generate a cohesive set of chunks that effectively represent the document. | - Preserves the broader picture. | - Might lose detailed information. | Utilize in summaries and when the question requires broader context. |"
]
},
{
@@ -48,253 +36,38 @@
},
"source": [
"\n",
- "\n",
- "# Getting Started"
+ "## Getting Started"
]
},
{
"cell_type": "code",
- "execution_count": null,
- "metadata": {
- "colab": {
- "background_save": true,
- "base_uri": "https://localhost:8080/"
- },
- "id": "wuQ1PO8FadQf",
- "outputId": "c6cc36f7-6368-4cdc-ce57-4d95249e409f"
- },
- "outputs": [
- {
- "name": "stdout",
- "output_type": "stream",
- "text": [
- "Collecting cohere\n",
- " Downloading cohere-4.54-py3-none-any.whl (52 kB)\n",
- "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m52.8/52.8 kB\u001b[0m \u001b[31m1.7 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
- "\u001b[?25hRequirement already satisfied: aiohttp<4.0,>=3.0 in /usr/local/lib/python3.10/dist-packages (from cohere) (3.9.3)\n",
- "Collecting backoff<3.0,>=2.0 (from cohere)\n",
- " Downloading backoff-2.2.1-py3-none-any.whl (15 kB)\n",
- "Collecting fastavro<2.0,>=1.8 (from cohere)\n",
- " Downloading fastavro-1.9.4-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.1 MB)\n",
- "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m3.1/3.1 MB\u001b[0m \u001b[31m33.1 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
- "\u001b[?25hCollecting importlib_metadata<7.0,>=6.0 (from cohere)\n",
- " Downloading importlib_metadata-6.11.0-py3-none-any.whl (23 kB)\n",
- "Requirement already satisfied: requests<3.0.0,>=2.25.0 in /usr/local/lib/python3.10/dist-packages (from cohere) (2.31.0)\n",
- "Requirement already satisfied: urllib3<3,>=1.26 in /usr/local/lib/python3.10/dist-packages (from cohere) (2.0.7)\n",
- "Requirement already satisfied: aiosignal>=1.1.2 in /usr/local/lib/python3.10/dist-packages (from aiohttp<4.0,>=3.0->cohere) (1.3.1)\n",
- "Requirement already satisfied: attrs>=17.3.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp<4.0,>=3.0->cohere) (23.2.0)\n",
- "Requirement already satisfied: frozenlist>=1.1.1 in /usr/local/lib/python3.10/dist-packages (from aiohttp<4.0,>=3.0->cohere) (1.4.1)\n",
- "Requirement already satisfied: multidict<7.0,>=4.5 in /usr/local/lib/python3.10/dist-packages (from aiohttp<4.0,>=3.0->cohere) (6.0.5)\n",
- "Requirement already satisfied: yarl<2.0,>=1.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp<4.0,>=3.0->cohere) (1.9.4)\n",
- "Requirement already satisfied: async-timeout<5.0,>=4.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp<4.0,>=3.0->cohere) (4.0.3)\n",
- "Requirement already satisfied: zipp>=0.5 in /usr/local/lib/python3.10/dist-packages (from importlib_metadata<7.0,>=6.0->cohere) (3.17.0)\n",
- "Requirement already satisfied: charset-normalizer<4,>=2 in /usr/local/lib/python3.10/dist-packages (from requests<3.0.0,>=2.25.0->cohere) (3.3.2)\n",
- "Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.10/dist-packages (from requests<3.0.0,>=2.25.0->cohere) (3.6)\n",
- "Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.10/dist-packages (from requests<3.0.0,>=2.25.0->cohere) (2024.2.2)\n",
- "\u001b[31mERROR: Operation cancelled by user\u001b[0m\u001b[31m\n",
- "\u001b[0mTraceback (most recent call last):\n",
- " File \"/usr/local/lib/python3.10/dist-packages/pip/_internal/cli/base_command.py\", line 169, in exc_logging_wrapper\n",
- " status = run_func(*args)\n",
- " File \"/usr/local/lib/python3.10/dist-packages/pip/_internal/cli/req_command.py\", line 242, in wrapper\n",
- " return func(self, options, args)\n",
- " File \"/usr/local/lib/python3.10/dist-packages/pip/_internal/commands/install.py\", line 324, in run\n",
- " session = self.get_default_session(options)\n",
- " File \"/usr/local/lib/python3.10/dist-packages/pip/_internal/cli/req_command.py\", line 98, in get_default_session\n",
- " self._session = self.enter_context(self._build_session(options))\n",
- " File \"/usr/local/lib/python3.10/dist-packages/pip/_internal/cli/req_command.py\", line 125, in _build_session\n",
- " session = PipSession(\n",
- " File \"/usr/local/lib/python3.10/dist-packages/pip/_internal/network/session.py\", line 342, in __init__\n",
- " self.headers[\"User-Agent\"] = user_agent()\n",
- " File \"/usr/local/lib/python3.10/dist-packages/pip/_internal/network/session.py\", line 175, in user_agent\n",
- " setuptools_dist = get_default_environment().get_distribution(\"setuptools\")\n",
- " File \"/usr/local/lib/python3.10/dist-packages/pip/_internal/metadata/__init__.py\", line 75, in get_default_environment\n",
- " return select_backend().Environment.default()\n",
- " File \"/usr/local/lib/python3.10/dist-packages/pip/_internal/metadata/__init__.py\", line 63, in select_backend\n",
- " from . import pkg_resources\n",
- " File \"/usr/local/lib/python3.10/dist-packages/pip/_internal/metadata/pkg_resources.py\", line 8, in \n",
- " from pip._vendor import pkg_resources\n",
- " File \"/usr/local/lib/python3.10/dist-packages/pip/_vendor/pkg_resources/__init__.py\", line 3327, in \n",
- " def _initialize_master_working_set():\n",
- " File \"/usr/local/lib/python3.10/dist-packages/pip/_vendor/pkg_resources/__init__.py\", line 3301, in _call_aside\n",
- " f(*args, **kwargs)\n",
- " File \"/usr/local/lib/python3.10/dist-packages/pip/_vendor/pkg_resources/__init__.py\", line 3339, in _initialize_master_working_set\n",
- " working_set = WorkingSet._build_master()\n",
- " File \"/usr/local/lib/python3.10/dist-packages/pip/_vendor/pkg_resources/__init__.py\", line 620, in _build_master\n",
- " ws = cls()\n",
- " File \"/usr/local/lib/python3.10/dist-packages/pip/_vendor/pkg_resources/__init__.py\", line 613, in __init__\n",
- " self.add_entry(entry)\n",
- " File \"/usr/local/lib/python3.10/dist-packages/pip/_vendor/pkg_resources/__init__.py\", line 669, in add_entry\n",
- " for dist in find_distributions(entry, True):\n",
- " File \"/usr/local/lib/python3.10/dist-packages/pip/_vendor/pkg_resources/__init__.py\", line 2130, in find_on_path\n",
- " for entry in sorted(entries):\n",
- " File \"/usr/local/lib/python3.10/dist-packages/pip/_vendor/pkg_resources/__init__.py\", line 2127, in \n",
- " entries = (os.path.join(path_item, child) for child in safe_listdir(path_item))\n",
- " File \"/usr/lib/python3.10/posixpath.py\", line 71, in join\n",
- " def join(a, *p):\n",
- "KeyboardInterrupt\n",
- "\n",
- "During handling of the above exception, another exception occurred:\n",
- "\n",
- "Traceback (most recent call last):\n",
- " File \"/usr/local/bin/pip3\", line 8, in \n",
- " sys.exit(main())\n",
- " File \"/usr/local/lib/python3.10/dist-packages/pip/_internal/cli/main.py\", line 79, in main\n",
- " return command.main(cmd_args)\n",
- " File \"/usr/local/lib/python3.10/dist-packages/pip/_internal/cli/base_command.py\", line 101, in main\n",
- " return self._main(args)\n",
- " File \"/usr/local/lib/python3.10/dist-packages/pip/_internal/cli/base_command.py\", line 223, in _main\n",
- " return run(options, args)\n",
- " File \"/usr/local/lib/python3.10/dist-packages/pip/_internal/cli/base_command.py\", line 206, in exc_logging_wrapper\n",
- " logger.critical(\"Operation cancelled by user\")\n",
- " File \"/usr/lib/python3.10/logging/__init__.py\", line 1524, in critical\n",
- " self._log(CRITICAL, msg, args, **kwargs)\n",
- " File \"/usr/lib/python3.10/logging/__init__.py\", line 1624, in _log\n",
- " self.handle(record)\n",
- " File \"/usr/lib/python3.10/logging/__init__.py\", line 1634, in handle\n",
- " self.callHandlers(record)\n",
- " File \"/usr/lib/python3.10/logging/__init__.py\", line 1696, in callHandlers\n",
- " hdlr.handle(record)\n",
- " File \"/usr/lib/python3.10/logging/__init__.py\", line 968, in handle\n",
- " self.emit(record)\n",
- " File \"/usr/local/lib/python3.10/dist-packages/pip/_internal/utils/logging.py\", line 168, in emit\n",
- " message = self.format(record)\n",
- " File \"/usr/lib/python3.10/logging/__init__.py\", line 943, in format\n",
- " return fmt.format(record)\n",
- " File \"/usr/local/lib/python3.10/dist-packages/pip/_internal/utils/logging.py\", line 119, in format\n",
- " prefix += \" \" * get_indentation()\n",
- " File \"/usr/local/lib/python3.10/dist-packages/pip/_internal/utils/logging.py\", line 69, in get_indentation\n",
- " def get_indentation() -> int:\n",
- "KeyboardInterrupt\n",
- "^C\n",
- "Requirement already satisfied: tokenizers in /usr/local/lib/python3.10/dist-packages (0.15.2)\n",
- "Requirement already satisfied: huggingface_hub<1.0,>=0.16.4 in /usr/local/lib/python3.10/dist-packages (from tokenizers) (0.20.3)\n",
- "Requirement already satisfied: filelock in /usr/local/lib/python3.10/dist-packages (from huggingface_hub<1.0,>=0.16.4->tokenizers) (3.13.1)\n",
- "Requirement already satisfied: fsspec>=2023.5.0 in /usr/local/lib/python3.10/dist-packages (from huggingface_hub<1.0,>=0.16.4->tokenizers) (2023.6.0)\n",
- "Requirement already satisfied: requests in /usr/local/lib/python3.10/dist-packages (from huggingface_hub<1.0,>=0.16.4->tokenizers) (2.31.0)\n",
- "Requirement already satisfied: tqdm>=4.42.1 in /usr/local/lib/python3.10/dist-packages (from huggingface_hub<1.0,>=0.16.4->tokenizers) (4.66.2)\n",
- "Requirement already satisfied: pyyaml>=5.1 in /usr/local/lib/python3.10/dist-packages (from huggingface_hub<1.0,>=0.16.4->tokenizers) (6.0.1)\n",
- "Requirement already satisfied: typing-extensions>=3.7.4.3 in /usr/local/lib/python3.10/dist-packages (from huggingface_hub<1.0,>=0.16.4->tokenizers) (4.10.0)\n",
- "Requirement already satisfied: packaging>=20.9 in /usr/local/lib/python3.10/dist-packages (from huggingface_hub<1.0,>=0.16.4->tokenizers) (23.2)\n",
- "Requirement already satisfied: charset-normalizer<4,>=2 in /usr/local/lib/python3.10/dist-packages (from requests->huggingface_hub<1.0,>=0.16.4->tokenizers) (3.3.2)\n",
- "Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.10/dist-packages (from requests->huggingface_hub<1.0,>=0.16.4->tokenizers) (3.6)\n",
- "Requirement already satisfied: urllib3<3,>=1.21.1 in /usr/local/lib/python3.10/dist-packages (from requests->huggingface_hub<1.0,>=0.16.4->tokenizers) (2.0.7)\n",
- "Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.10/dist-packages (from requests->huggingface_hub<1.0,>=0.16.4->tokenizers) (2024.2.2)\n",
- "\u001b[31mERROR: Operation cancelled by user\u001b[0m\u001b[31m\n",
- "\u001b[0mCollecting langchain\n",
- " Downloading langchain-0.1.11-py3-none-any.whl (807 kB)\n",
- "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m807.5/807.5 kB\u001b[0m \u001b[31m5.4 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
- "\u001b[?25hRequirement already satisfied: PyYAML>=5.3 in /usr/local/lib/python3.10/dist-packages (from langchain) (6.0.1)\n",
- "Requirement already satisfied: SQLAlchemy<3,>=1.4 in /usr/local/lib/python3.10/dist-packages (from langchain) (2.0.28)\n",
- "Requirement already satisfied: aiohttp<4.0.0,>=3.8.3 in /usr/local/lib/python3.10/dist-packages (from langchain) (3.9.3)\n",
- "Requirement already satisfied: async-timeout<5.0.0,>=4.0.0 in /usr/local/lib/python3.10/dist-packages (from langchain) (4.0.3)\n",
- "Collecting dataclasses-json<0.7,>=0.5.7 (from langchain)\n",
- " Downloading dataclasses_json-0.6.4-py3-none-any.whl (28 kB)\n",
- "Collecting jsonpatch<2.0,>=1.33 (from langchain)\n",
- " Downloading jsonpatch-1.33-py2.py3-none-any.whl (12 kB)\n",
- "Collecting langchain-community<0.1,>=0.0.25 (from langchain)\n",
- " Downloading langchain_community-0.0.27-py3-none-any.whl (1.8 MB)\n",
- "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m1.8/1.8 MB\u001b[0m \u001b[31m7.7 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
- "\u001b[?25hCollecting langchain-core<0.2,>=0.1.29 (from langchain)\n",
- " Downloading langchain_core-0.1.30-py3-none-any.whl (256 kB)\n",
- "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m256.9/256.9 kB\u001b[0m \u001b[31m18.7 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
- "\u001b[?25hCollecting langchain-text-splitters<0.1,>=0.0.1 (from langchain)\n",
- " Downloading langchain_text_splitters-0.0.1-py3-none-any.whl (21 kB)\n",
- "Collecting langsmith<0.2.0,>=0.1.17 (from langchain)\n",
- " Downloading langsmith-0.1.23-py3-none-any.whl (66 kB)\n",
- "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m66.6/66.6 kB\u001b[0m \u001b[31m7.2 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
- "\u001b[?25hRequirement already satisfied: numpy<2,>=1 in /usr/local/lib/python3.10/dist-packages (from langchain) (1.25.2)\n",
- "Requirement already satisfied: pydantic<3,>=1 in /usr/local/lib/python3.10/dist-packages (from langchain) (2.6.3)\n",
- "Requirement already satisfied: requests<3,>=2 in /usr/local/lib/python3.10/dist-packages (from langchain) (2.31.0)\n",
- "Requirement already satisfied: tenacity<9.0.0,>=8.1.0 in /usr/local/lib/python3.10/dist-packages (from langchain) (8.2.3)\n",
- "Requirement already satisfied: aiosignal>=1.1.2 in /usr/local/lib/python3.10/dist-packages (from aiohttp<4.0.0,>=3.8.3->langchain) (1.3.1)\n",
- "Requirement already satisfied: attrs>=17.3.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp<4.0.0,>=3.8.3->langchain) (23.2.0)\n",
- "Requirement already satisfied: frozenlist>=1.1.1 in /usr/local/lib/python3.10/dist-packages (from aiohttp<4.0.0,>=3.8.3->langchain) (1.4.1)\n",
- "Requirement already satisfied: multidict<7.0,>=4.5 in /usr/local/lib/python3.10/dist-packages (from aiohttp<4.0.0,>=3.8.3->langchain) (6.0.5)\n",
- "Requirement already satisfied: yarl<2.0,>=1.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp<4.0.0,>=3.8.3->langchain) (1.9.4)\n",
- "Collecting marshmallow<4.0.0,>=3.18.0 (from dataclasses-json<0.7,>=0.5.7->langchain)\n",
- " Downloading marshmallow-3.21.1-py3-none-any.whl (49 kB)\n",
- "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m49.4/49.4 kB\u001b[0m \u001b[31m5.2 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
- "\u001b[?25hCollecting typing-inspect<1,>=0.4.0 (from dataclasses-json<0.7,>=0.5.7->langchain)\n",
- " Downloading typing_inspect-0.9.0-py3-none-any.whl (8.8 kB)\n",
- "Collecting jsonpointer>=1.9 (from jsonpatch<2.0,>=1.33->langchain)\n",
- " Downloading jsonpointer-2.4-py2.py3-none-any.whl (7.8 kB)\n",
- "Requirement already satisfied: anyio<5,>=3 in /usr/local/lib/python3.10/dist-packages (from langchain-core<0.2,>=0.1.29->langchain) (3.7.1)\n",
- "Requirement already satisfied: packaging<24.0,>=23.2 in /usr/local/lib/python3.10/dist-packages (from langchain-core<0.2,>=0.1.29->langchain) (23.2)\n",
- "Collecting orjson<4.0.0,>=3.9.14 (from langsmith<0.2.0,>=0.1.17->langchain)\n",
- " Downloading orjson-3.9.15-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (138 kB)\n",
- "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m138.5/138.5 kB\u001b[0m \u001b[31m15.4 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
- "\u001b[?25hRequirement already satisfied: annotated-types>=0.4.0 in /usr/local/lib/python3.10/dist-packages (from pydantic<3,>=1->langchain) (0.6.0)\n",
- "Requirement already satisfied: pydantic-core==2.16.3 in /usr/local/lib/python3.10/dist-packages (from pydantic<3,>=1->langchain) (2.16.3)\n",
- "Requirement already satisfied: typing-extensions>=4.6.1 in /usr/local/lib/python3.10/dist-packages (from pydantic<3,>=1->langchain) (4.10.0)\n",
- "Requirement already satisfied: charset-normalizer<4,>=2 in /usr/local/lib/python3.10/dist-packages (from requests<3,>=2->langchain) (3.3.2)\n",
- "Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.10/dist-packages (from requests<3,>=2->langchain) (3.6)\n",
- "Requirement already satisfied: urllib3<3,>=1.21.1 in /usr/local/lib/python3.10/dist-packages (from requests<3,>=2->langchain) (2.0.7)\n",
- "Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.10/dist-packages (from requests<3,>=2->langchain) (2024.2.2)\n",
- "Requirement already satisfied: greenlet!=0.4.17 in /usr/local/lib/python3.10/dist-packages (from SQLAlchemy<3,>=1.4->langchain) (3.0.3)\n",
- "Requirement already satisfied: sniffio>=1.1 in /usr/local/lib/python3.10/dist-packages (from anyio<5,>=3->langchain-core<0.2,>=0.1.29->langchain) (1.3.1)\n",
- "Requirement already satisfied: exceptiongroup in /usr/local/lib/python3.10/dist-packages (from anyio<5,>=3->langchain-core<0.2,>=0.1.29->langchain) (1.2.0)\n",
- "Collecting mypy-extensions>=0.3.0 (from typing-inspect<1,>=0.4.0->dataclasses-json<0.7,>=0.5.7->langchain)\n",
- " Downloading mypy_extensions-1.0.0-py3-none-any.whl (4.7 kB)\n",
- "Installing collected packages: orjson, mypy-extensions, marshmallow, jsonpointer, typing-inspect, jsonpatch, langsmith, dataclasses-json, langchain-core, langchain-text-splitters, langchain-community, langchain\n",
- "Successfully installed dataclasses-json-0.6.4 jsonpatch-1.33 jsonpointer-2.4 langchain-0.1.11 langchain-community-0.0.27 langchain-core-0.1.30 langchain-text-splitters-0.0.1 langsmith-0.1.23 marshmallow-3.21.1 mypy-extensions-1.0.0 orjson-3.9.15 typing-inspect-0.9.0\n",
- "Requirement already satisfied: nltk in /usr/local/lib/python3.10/dist-packages (3.8.1)\n",
- "Requirement already satisfied: click in /usr/local/lib/python3.10/dist-packages (from nltk) (8.1.7)\n",
- "Requirement already satisfied: joblib in /usr/local/lib/python3.10/dist-packages (from nltk) (1.3.2)\n",
- "Requirement already satisfied: regex>=2021.8.3 in /usr/local/lib/python3.10/dist-packages (from nltk) (2023.12.25)\n",
- "Requirement already satisfied: tqdm in /usr/local/lib/python3.10/dist-packages (from nltk) (4.66.2)\n",
- "Requirement already satisfied: networkx in /usr/local/lib/python3.10/dist-packages (3.2.1)\n",
- "Collecting pypdf2\n",
- " Downloading pypdf2-3.0.1-py3-none-any.whl (232 kB)\n",
- "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m232.6/232.6 kB\u001b[0m \u001b[31m4.1 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
- "\u001b[?25hInstalling collected packages: pypdf2\n",
- "Successfully installed pypdf2-3.0.1\n"
- ]
- }
- ],
+ "execution_count": 1,
+ "metadata": {},
+ "outputs": [],
"source": [
- "####################################################################################################\n",
- "#\n",
- "# Uncomment if you need to install the following packages\n",
- "#\n",
- "####################################################################################################\n",
- "# %%capture\n",
- "# !pip install cohere\n",
- "# !pip install python-dotenv\n",
- "# !pip install tokenizers\n",
- "# !pip install langchain\n",
- "# !pip install nltk\n",
- "# !pip install networkx\n",
- "# !pip install pypdf2"
+ "%%capture\n",
+ "!pip install cohere\n",
+ "!pip install python-dotenv\n",
+ "!pip install tokenizers\n",
+ "!pip install langchain\n",
+ "!pip install nltk\n",
+ "!pip install networkx\n",
+ "!pip install pypdf2"
]
},
{
"cell_type": "code",
- "execution_count": null,
- "metadata": {
- "colab": {
- "base_uri": "https://localhost:8080/"
- },
- "id": "UFi1QH3sZUEZ",
- "outputId": "3e72afe2-2c60-472e-b838-949676897e13"
- },
+ "execution_count": 3,
+ "metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
- "[nltk_data] Downloading package punkt to /root/nltk_data...\n",
+ "[nltk_data] Downloading package punkt to\n",
+ "[nltk_data] /home/anna_cohere_com/nltk_data...\n",
"[nltk_data] Package punkt is already up-to-date!\n"
]
- },
- {
- "data": {
- "text/plain": [
- "False"
- ]
- },
- "execution_count": 51,
- "metadata": {},
- "output_type": "execute_result"
}
],
"source": [
@@ -323,106 +96,7 @@
},
{
"cell_type": "code",
- "execution_count": null,
- "metadata": {
- "colab": {
- "base_uri": "https://localhost:8080/",
- "height": 17
- },
- "id": "6BtV3iJ_UQ2w",
- "outputId": "76303020-4c08-4f5e-aa5a-03762686455d"
- },
- "outputs": [
- {
- "data": {
- "text/html": [
- "\n",
- " \n",
- " "
- ],
- "text/plain": [
- ""
- ]
- },
- "metadata": {},
- "output_type": "display_data"
- },
- {
- "data": {
- "text/html": [
- "\n",
- " \n",
- " "
- ],
- "text/plain": [
- ""
- ]
- },
- "metadata": {},
- "output_type": "display_data"
- },
- {
- "data": {
- "text/html": [
- "\n",
- " \n",
- " "
- ],
- "text/plain": [
- ""
- ]
- },
- "metadata": {},
- "output_type": "display_data"
- },
- {
- "data": {
- "text/html": [
- "\n",
- " \n",
- " "
- ],
- "text/plain": [
- ""
- ]
- },
- "metadata": {},
- "output_type": "display_data"
- }
- ],
- "source": [
- "def set_css():\n",
- " display(HTML('''\n",
- " \n",
- " '''))\n",
- "get_ipython().events.register('pre_run_cell', set_css)\n",
- "\n",
- "set_css()"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
+ "execution_count": 4,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
@@ -430,15 +104,7 @@
"id": "K603hzyKda2f",
"outputId": "611374e9-1e51-4012-c99c-9ecb17a17e19"
},
- "outputs": [
- {
- "name": "stdout",
- "output_type": "stream",
- "text": [
- "Enter your Cohere API key: ··········\n"
- ]
- }
- ],
+ "outputs": [],
"source": [
"# Set up Cohere client\n",
"co_model = 'command-r'\n",
@@ -448,7 +114,7 @@
},
{
"cell_type": "code",
- "execution_count": null,
+ "execution_count": 5,
"metadata": {
"id": "kI0inogLmIHp"
},
@@ -501,7 +167,7 @@
},
{
"cell_type": "code",
- "execution_count": null,
+ "execution_count": 11,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
@@ -511,84 +177,12 @@
"outputId": "bc24d5a7-8bde-4d2a-e569-8a6941612629"
},
"outputs": [
- {
- "data": {
- "text/html": [
- "\n",
- " \n",
- " "
- ],
- "text/plain": [
- ""
- ]
- },
- "metadata": {},
- "output_type": "display_data"
- },
- {
- "data": {
- "text/html": [
- "\n",
- " \n",
- " "
- ],
- "text/plain": [
- ""
- ]
- },
- "metadata": {},
- "output_type": "display_data"
- },
- {
- "data": {
- "text/html": [
- "\n",
- " \n",
- " "
- ],
- "text/plain": [
- ""
- ]
- },
- "metadata": {},
- "output_type": "display_data"
- },
- {
- "data": {
- "text/html": [
- "\n",
- " \n",
- " "
- ],
- "text/plain": [
- ""
- ]
- },
- "metadata": {},
- "output_type": "display_data"
- },
{
"name": "stdout",
"output_type": "stream",
"text": [
"PDF saved successfully to 'example.pdf'\n",
- "Document length - #tokens: 128618\n"
+ "Document length - #tokens: 134184\n"
]
}
],
@@ -603,7 +197,7 @@
"long_text = long_text.replace('\\n', ' ')\n",
"\n",
"# Print the length of the document\n",
- "print(\"Document length - #tokens:\", len(co.tokenize(long_text)))"
+ "print(\"Document length - #tokens:\", len(co.tokenize(text=long_text, model=co_model).tokens))"
]
},
{
@@ -617,7 +211,7 @@
},
{
"cell_type": "code",
- "execution_count": null,
+ "execution_count": 12,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
@@ -626,80 +220,7 @@
"id": "PAa9xaU6-8ls",
"outputId": "7e60436a-d8a2-42ec-df85-6ae3402bc106"
},
- "outputs": [
- {
- "data": {
- "text/html": [
- "\n",
- " \n",
- " "
- ],
- "text/plain": [
- ""
- ]
- },
- "metadata": {},
- "output_type": "display_data"
- },
- {
- "data": {
- "text/html": [
- "\n",
- " \n",
- " "
- ],
- "text/plain": [
- ""
- ]
- },
- "metadata": {},
- "output_type": "display_data"
- },
- {
- "data": {
- "text/html": [
- "\n",
- " \n",
- " "
- ],
- "text/plain": [
- ""
- ]
- },
- "metadata": {},
- "output_type": "display_data"
- },
- {
- "data": {
- "text/html": [
- "\n",
- " \n",
- " "
- ],
- "text/plain": [
- ""
- ]
- },
- "metadata": {},
- "output_type": "display_data"
- }
- ],
+ "outputs": [],
"source": [
"def generate_response(message, max_tokens=300, temperature=0.2, k=0):\n",
" \"\"\"\n",
@@ -727,7 +248,7 @@
},
{
"cell_type": "code",
- "execution_count": null,
+ "execution_count": 13,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
@@ -736,80 +257,7 @@
"id": "80fgeyAhZDZD",
"outputId": "3a756564-c6d9-44d6-ac97-ffb8c33d09f6"
},
- "outputs": [
- {
- "data": {
- "text/html": [
- "\n",
- " \n",
- " "
- ],
- "text/plain": [
- ""
- ]
- },
- "metadata": {},
- "output_type": "display_data"
- },
- {
- "data": {
- "text/html": [
- "\n",
- " \n",
- " "
- ],
- "text/plain": [
- ""
- ]
- },
- "metadata": {},
- "output_type": "display_data"
- },
- {
- "data": {
- "text/html": [
- "\n",
- " \n",
- " "
- ],
- "text/plain": [
- ""
- ]
- },
- "metadata": {},
- "output_type": "display_data"
- },
- {
- "data": {
- "text/html": [
- "\n",
- " \n",
- " "
- ],
- "text/plain": [
- ""
- ]
- },
- "metadata": {},
- "output_type": "display_data"
- }
- ],
+ "outputs": [],
"source": [
"# Example summary prompt.\n",
"prompt_template = \"\"\"\n",
@@ -862,8 +310,8 @@
"id": "AH-tKADHmIHq"
},
"source": [
- "\n",
- "# Approach 1 - Truncate"
+ "\n",
+ "## Approach 1 - Truncate"
]
},
{
@@ -877,7 +325,7 @@
},
{
"cell_type": "code",
- "execution_count": null,
+ "execution_count": 16,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
@@ -886,80 +334,7 @@
"id": "JrvVls3smIHq",
"outputId": "6ec69e1b-0d6d-4b96-e8ae-650ebc049a7a"
},
- "outputs": [
- {
- "data": {
- "text/html": [
- "\n",
- " \n",
- " "
- ],
- "text/plain": [
- ""
- ]
- },
- "metadata": {},
- "output_type": "display_data"
- },
- {
- "data": {
- "text/html": [
- "\n",
- " \n",
- " "
- ],
- "text/plain": [
- ""
- ]
- },
- "metadata": {},
- "output_type": "display_data"
- },
- {
- "data": {
- "text/html": [
- "\n",
- " \n",
- " "
- ],
- "text/plain": [
- ""
- ]
- },
- "metadata": {},
- "output_type": "display_data"
- },
- {
- "data": {
- "text/html": [
- "\n",
- " \n",
- " "
- ],
- "text/plain": [
- ""
- ]
- },
- "metadata": {},
- "output_type": "display_data"
- }
- ],
+ "outputs": [],
"source": [
"# The new Cohere model has a context limit of 128k tokens. However, for the purpose of this exercise, we will assume a smaller context window.\n",
"# Employing a smaller context window also has the additional benefit of reducing the cost per request, especially if billed by the number of tokens.\n",
@@ -972,7 +347,7 @@
" This can break up sentences, passages, etc.\n",
" \"\"\"\n",
"\n",
- " tokenized = co.tokenize(long).token_strings\n",
+ " tokenized = co.tokenize(text=long, model=co_model).token_strings\n",
" truncated = tokenized[:max_tokens]\n",
" short = \"\".join(truncated)\n",
" return short"
@@ -980,7 +355,7 @@
},
{
"cell_type": "code",
- "execution_count": null,
+ "execution_count": 17,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
@@ -990,83 +365,13 @@
"outputId": "0273fe8b-814b-4d5b-8da5-be85bfa9ea45"
},
"outputs": [
- {
- "data": {
- "text/html": [
- "\n",
- " \n",
- " "
- ],
- "text/plain": [
- ""
- ]
- },
- "metadata": {},
- "output_type": "display_data"
- },
- {
- "data": {
- "text/html": [
- "\n",
- " \n",
- " "
- ],
- "text/plain": [
- ""
- ]
- },
- "metadata": {},
- "output_type": "display_data"
- },
- {
- "data": {
- "text/html": [
- "\n",
- " \n",
- " "
- ],
- "text/plain": [
- ""
- ]
- },
- "metadata": {},
- "output_type": "display_data"
- },
- {
- "data": {
- "text/html": [
- "\n",
- " \n",
- " "
- ],
- "text/plain": [
- ""
- ]
- },
- "metadata": {},
- "output_type": "display_data"
- },
{
"name": "stdout",
"output_type": "stream",
"text": [
- "The document outlines the European Union's proposed Regulation on Artificial Intelligence, aiming to establish harmonised rules for AI development and use while ensuring fundamental rights protection. It defines the scope, purposes, and risks addressed, excluding national security and military purposes. Prohibited AI practices, such as real-time biometric identification in public spaces, are detailed. The regulation proposes a risk-based approach, classifying high-risk AI systems and setting requirements for providers and deployers. It establishes governance structures and obligations for general-purpose AI models, including transparency and copyright compliance. The document also covers issues like conformity assessment, responsibilities along the AI value chain, and standardisation. The regulation aims to foster trustworthy AI while protecting public interests and fundamental rights.\n"
+ "The document discusses the impact of a specific protein, p53, on the process of angiogenesis, which is the growth of new blood vessels. Angiogenesis plays a critical role in various physiological processes, including wound healing and embryonic development. The presence of the p53 protein can inhibit angiogenesis by regulating the expression of certain genes and proteins. This inhibition can have significant implications for tumor growth, as angiogenesis is essential for tumor progression. Therefore, understanding the role of p53 in angiogenesis can contribute to our knowledge of tumor suppression and potential therapeutic interventions.\n",
+ "\n",
+ "Additionally, the document mentions that the regulation of angiogenesis by p53 occurs independently of the protein's role in cell cycle arrest and apoptosis, which are other key functions of p53 in tumor suppression. This suggests that p53 has a complex and multifaceted impact on cellular processes.\n"
]
}
],
@@ -1083,8 +388,8 @@
"id": "pC1l77odmIHr"
},
"source": [
- "\n",
- "# Approach 2: Query Based Retrieval\n",
+ "\n",
+ "## Approach 2: Query Based Retrieval\n",
"\n",
"In this section we present how we can leverage a query retriereval based approach to generate an answer to the following question: `Based on the document, are there any risks related to Elon Musk?`.\n",
"\n",
@@ -1116,7 +421,7 @@
},
{
"cell_type": "code",
- "execution_count": null,
+ "execution_count": 18,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
@@ -1125,86 +430,8 @@
"id": "XIbGWw2GmIHr",
"outputId": "9ca56d31-d7b4-4e48-e9d2-2279f9d2caaf"
},
- "outputs": [
- {
- "data": {
- "text/html": [
- "\n",
- " \n",
- " "
- ],
- "text/plain": [
- ""
- ]
- },
- "metadata": {},
- "output_type": "display_data"
- },
- {
- "data": {
- "text/html": [
- "\n",
- " \n",
- " "
- ],
- "text/plain": [
- ""
- ]
- },
- "metadata": {},
- "output_type": "display_data"
- },
- {
- "data": {
- "text/html": [
- "\n",
- " \n",
- " "
- ],
- "text/plain": [
- ""
- ]
- },
- "metadata": {},
- "output_type": "display_data"
- },
- {
- "data": {
- "text/html": [
- "\n",
- " \n",
- " "
- ],
- "text/plain": [
- ""
- ]
- },
- "metadata": {},
- "output_type": "display_data"
- }
- ],
+ "outputs": [],
"source": [
- "############################################################\n",
- "#\n",
- "# Utility functions for chunking\n",
- "#\n",
- "############################################################\n",
"def split_text_into_sentences(text) -> List[str]:\n",
" \"\"\"\n",
" Split the input text into a list of sentences.\n",
@@ -1237,7 +464,7 @@
},
{
"cell_type": "code",
- "execution_count": null,
+ "execution_count": 19,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
@@ -1247,85 +474,13 @@
"outputId": "ae67c468-b5d3-43a4-da36-83a2dc16956a"
},
"outputs": [
- {
- "data": {
- "text/html": [
- "\n",
- " \n",
- " "
- ],
- "text/plain": [
- ""
- ]
- },
- "metadata": {},
- "output_type": "display_data"
- },
- {
- "data": {
- "text/html": [
- "\n",
- " \n",
- " "
- ],
- "text/plain": [
- ""
- ]
- },
- "metadata": {},
- "output_type": "display_data"
- },
- {
- "data": {
- "text/html": [
- "\n",
- " \n",
- " "
- ],
- "text/plain": [
- ""
- ]
- },
- "metadata": {},
- "output_type": "display_data"
- },
- {
- "data": {
- "text/html": [
- "\n",
- " \n",
- " "
- ],
- "text/plain": [
- ""
- ]
- },
- "metadata": {},
- "output_type": "display_data"
- },
{
"name": "stdout",
"output_type": "stream",
"text": [
- "Example sentence: ['This is to ensure that the deployer is aware and takes them into acc ount when using the high - risk AI system.']\n",
+ "Example sentence: ['4.']\n",
"\n",
- "Example passage: ['Notified bodies shall have procedures for the performance of activities which take due account of the size of an undertaking, th e sector in which it operates, its structure, the degree of complexity of the AI system in question. 8. Notified bodies shall take out appropriate liability insurance for their conformity assessment activities, unless liability is assumed by the Member Sta te in which they are established in accordance with national law or that Member State is itself directly responsible for the conformity assessment. 9. Notified bodies shall be capable of carrying out all the tasks falling to them under this Regulation with the highest degree of professional integrity and the requisite competence in the specific field, whether those tasks are carried out by notified bodies themselves or on their behalf and under their responsibility. ']\n"
+ "Example passage: ['T echnical robustness and safety means that AI systems are developed and used in a way that allows robustness in case of problems and resilience against attempts to alter the use or performance of the AI system so as to allow unlawful use by third parties, a nd minimise unintended harm. Privacy and data governance means that AI systems are developed and used in compliance with existing privacy and data protection rules, while processing data that meets high standards in terms of quality and integrity. Transpar ency means that AI systems are developed and used in a way that allows appropriate traceability and explainability, while making humans aware that they communicate or interact with an AI system, as well as duly informing deployers of the capabilities and l imitations of that AI system and affected persons about their rights. Diversity, non - discrimination and fairness means that AI systems are developed and used in a way that includes diverse actors and promotes equal access, gender equality and cultural dive rsity, while avoiding discriminatory impacts and unfair biases that are prohibited by Union or national law. Social and environmental well - being means that AI systems are developed and used in a sustainable and environmentally friendly manner as well as in a way to benefit all human beings, while monitoring and assessing the long - term impacts on the individual, society and democracy. ']\n"
]
}
],
@@ -1339,7 +494,7 @@
},
{
"cell_type": "code",
- "execution_count": null,
+ "execution_count": 35,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
@@ -1348,80 +503,7 @@
"id": "w6pjBAA9mIHr",
"outputId": "ab3f3ae1-3b38-4999-f8f7-1259d070c88b"
},
- "outputs": [
- {
- "data": {
- "text/html": [
- "\n",
- " \n",
- " "
- ],
- "text/plain": [
- ""
- ]
- },
- "metadata": {},
- "output_type": "display_data"
- },
- {
- "data": {
- "text/html": [
- "\n",
- " \n",
- " "
- ],
- "text/plain": [
- ""
- ]
- },
- "metadata": {},
- "output_type": "display_data"
- },
- {
- "data": {
- "text/html": [
- "\n",
- " \n",
- " "
- ],
- "text/plain": [
- ""
- ]
- },
- "metadata": {},
- "output_type": "display_data"
- },
- {
- "data": {
- "text/html": [
- "\n",
- " \n",
- " "
- ],
- "text/plain": [
- ""
- ]
- },
- "metadata": {},
- "output_type": "display_data"
- }
- ],
+ "outputs": [],
"source": [
"def _add_chunks_by_priority(\n",
" chunks: List[str],\n",
@@ -1442,7 +524,7 @@
"\n",
" while num_tokens < max_tokens and len(idcs_queue) > 0:\n",
" next_idx = idcs_queue.popleft()\n",
- " num_tokens += co.tokenize(chunks[next_idx]).length\n",
+ " num_tokens += len(co.tokenize(text=chunks[next_idx], model=co_model).tokens)\n",
" # keep index and chunk, to reorder chronologically\n",
" selected.append((next_idx, chunks[next_idx]))\n",
" if num_tokens > max_tokens:\n",
@@ -1465,7 +547,7 @@
" # 2. Use co.rerank to rank chunks vs. query\n",
" chunks_reranked = co.rerank(query=query, documents=chunks, model=\"rerank-english-v3.0\")\n",
" idcs_sorted_by_relevance = [\n",
- " chunk.index for chunk in sorted(chunks_reranked, key=lambda c: c.relevance_score, reverse=True)\n",
+ " chunk.index for chunk in sorted(chunks_reranked.results, key=lambda c: c.relevance_score, reverse=True)\n",
" ]\n",
"\n",
" # 3. Add chunks back in order of relevance\n",
@@ -1479,7 +561,7 @@
},
{
"cell_type": "code",
- "execution_count": null,
+ "execution_count": 36,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
@@ -1488,80 +570,7 @@
"id": "SX04iD3emIHr",
"outputId": "dbe55fc4-bbc4-482a-cace-f5e2fc9318ba"
},
- "outputs": [
- {
- "data": {
- "text/html": [
- "\n",
- " \n",
- " "
- ],
- "text/plain": [
- ""
- ]
- },
- "metadata": {},
- "output_type": "display_data"
- },
- {
- "data": {
- "text/html": [
- "\n",
- " \n",
- " "
- ],
- "text/plain": [
- ""
- ]
- },
- "metadata": {},
- "output_type": "display_data"
- },
- {
- "data": {
- "text/html": [
- "\n",
- " \n",
- " "
- ],
- "text/plain": [
- ""
- ]
- },
- "metadata": {},
- "output_type": "display_data"
- },
- {
- "data": {
- "text/html": [
- "\n",
- " \n",
- " "
- ],
- "text/plain": [
- ""
- ]
- },
- "metadata": {},
- "output_type": "display_data"
- }
- ],
+ "outputs": [],
"source": [
"# Example prompt\n",
"prompt_template = \"\"\"\n",
@@ -1577,7 +586,7 @@
},
{
"cell_type": "code",
- "execution_count": null,
+ "execution_count": 37,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
@@ -1587,93 +596,13 @@
"outputId": "093c1167-d66d-42f7-e65d-ffe02b2d71ff"
},
"outputs": [
- {
- "data": {
- "text/html": [
- "\n",
- " \n",
- " "
- ],
- "text/plain": [
- ""
- ]
- },
- "metadata": {},
- "output_type": "display_data"
- },
- {
- "data": {
- "text/html": [
- "\n",
- " \n",
- " "
- ],
- "text/plain": [
- ""
- ]
- },
- "metadata": {},
- "output_type": "display_data"
- },
- {
- "data": {
- "text/html": [
- "\n",
- " \n",
- " "
- ],
- "text/plain": [
- ""
- ]
- },
- "metadata": {},
- "output_type": "display_data"
- },
- {
- "data": {
- "text/html": [
- "\n",
- " \n",
- " "
- ],
- "text/plain": [
- ""
- ]
- },
- "metadata": {},
- "output_type": "display_data"
- },
{
"name": "stdout",
"output_type": "stream",
"text": [
- "The report discusses the regulation of biometric identification, specifically the use of real-time systems in publicly accessible spaces for law enforcement purposes. The use of these systems is generally prohibited except in certain circumstances, such as searching for missing people or identifying perpetrators of serious criminal offences.\n",
- "\n",
- "The report outlines a number of exceptions and additional prohibitions related to biometric identification. For instance, biometric categorisation based on specific beliefs or characteristics is generally prohibited, along with untargeted scraping of facial images for creating facial recognition databases.\n",
+ "The report discusses the restrictions on the use of biometric identification by law enforcement in publicly accessible spaces. According to the document, real-time biometric identification is prohibited unless in exceptional cases where its use is strictly necessary and proportionate to achieving a substantial public interest. The use of post-remote biometric identification systems is also mentioned, noting the requirements for authorization and limitations on its use.\n",
"\n",
- "Additionally, the document mentions post-remote biometric identification, which is subject to additional safeguards. These include authorisation requirements and restrictions on their use for law enforcement purposes.\n",
- "\n",
- "The regulation also addresses the accuracy, robustness and cybersecurity of AI systems, including high-risk systems that involve biometric identification. Providers of such systems are responsible for ensuring compliance with these requirements.\n",
- "\n",
- "National competent authorities and market surveillance authorities are designated to oversee the implementation of these regulations. They have the power to enforce compliance, including the imposition of penalties for non-compliance.\n",
- "\n",
- "Overall, the report aims to establish a uniform legal framework for AI systems, especially those involving biometric identification, while ensuring the protection of fundamental rights and individual freedoms.\n"
+ "The report also highlights the classification of certain AI systems as high-risk, including biometric identification systems, emotion recognition systems, and biometric categorisation systems, with the exception of systems used for biometric verification. High-risk AI systems are subject to specific requirements and obligations.\n"
]
}
],
@@ -1690,8 +619,8 @@
"id": "zfpvo9WwbB6e"
},
"source": [
- "\n",
- "# Approach 3: Text rank"
+ "\n",
+ "## Approach 3: Text rank"
]
},
{
@@ -1705,7 +634,7 @@
"The solution presented in this document can be broken down into five functional steps:\n",
"\n",
"1. Break the document into chunks.\n",
- " - This mirrors the first step in [Approach 2](#query-based-retrieval).\n",
+ " - This mirrors the first step in [Approach 2](#approach-2).\n",
"\n",
"2. Embed each chunk using an embedding model and construct a similarity matrix.\n",
" - We utilize `co.embed` [documentation link](https://docs.cohere.com/reference/embed).\n",
@@ -1714,10 +643,10 @@
" - We employ a package called [`NetworkX`](https://networkx.org/documentation/networkx-1.10/overview.html). It constructs a graph where the chunks are nodes, and the similarity score between them serves as the weight of the edges. Then, we calculate the centrality of each chunk as the sum of the edge weights adjacent to the node representing that chunk.\n",
"\n",
"4. Retain the highest-centrality chunks until the context limit is reached.\n",
- " - This step follows a similar approach to [Approach 2](#query-based-retrieval).\n",
+ " - This step follows a similar approach to [Approach 2](#approach-2).\n",
"\n",
"5. Reassemble the shortened text by reordering chunks in their original order.\n",
- " - This step mirrors the last step in [Approach 2](#query-based-retrieval).\n",
+ " - This step mirrors the last step in [Approach 2](#approach-2).\n",
"\n",
"See `text_rank` as the starting point.\n"
]
@@ -1733,7 +662,7 @@
},
{
"cell_type": "code",
- "execution_count": null,
+ "execution_count": 38,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
@@ -1742,80 +671,7 @@
"id": "4fWR-HKpmIHs",
"outputId": "1ddc2bf4-90d1-44df-d4cd-6908546db506"
},
- "outputs": [
- {
- "data": {
- "text/html": [
- "\n",
- " \n",
- " "
- ],
- "text/plain": [
- ""
- ]
- },
- "metadata": {},
- "output_type": "display_data"
- },
- {
- "data": {
- "text/html": [
- "\n",
- " \n",
- " "
- ],
- "text/plain": [
- ""
- ]
- },
- "metadata": {},
- "output_type": "display_data"
- },
- {
- "data": {
- "text/html": [
- "\n",
- " \n",
- " "
- ],
- "text/plain": [
- ""
- ]
- },
- "metadata": {},
- "output_type": "display_data"
- },
- {
- "data": {
- "text/html": [
- "\n",
- " \n",
- " "
- ],
- "text/plain": [
- ""
- ]
- },
- "metadata": {},
- "output_type": "display_data"
- }
- ],
+ "outputs": [],
"source": [
"def text_rank(text: str, max_tokens: int, n_setences_per_passage: int) -> str:\n",
" \"\"\"\n",
@@ -1853,7 +709,7 @@
},
{
"cell_type": "code",
- "execution_count": null,
+ "execution_count": 39,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
@@ -1862,80 +718,7 @@
"id": "YVVMr-S9mIHs",
"outputId": "2fe871db-ac7a-4ac1-e7a1-e8c8292d95e4"
},
- "outputs": [
- {
- "data": {
- "text/html": [
- "\n",
- " \n",
- " "
- ],
- "text/plain": [
- ""
- ]
- },
- "metadata": {},
- "output_type": "display_data"
- },
- {
- "data": {
- "text/html": [
- "\n",
- " \n",
- " "
- ],
- "text/plain": [
- ""
- ]
- },
- "metadata": {},
- "output_type": "display_data"
- },
- {
- "data": {
- "text/html": [
- "\n",
- " \n",
- " "
- ],
- "text/plain": [
- ""
- ]
- },
- "metadata": {},
- "output_type": "display_data"
- },
- {
- "data": {
- "text/html": [
- "\n",
- " \n",
- " "
- ],
- "text/plain": [
- ""
- ]
- },
- "metadata": {},
- "output_type": "display_data"
- }
- ],
+ "outputs": [],
"source": [
"# Example summary prompt.\n",
"prompt_template = \"\"\"\n",
@@ -1951,7 +734,7 @@
},
{
"cell_type": "code",
- "execution_count": null,
+ "execution_count": 40,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
@@ -1961,83 +744,11 @@
"outputId": "0a12951a-b917-4e69-a5c4-951907e32a90"
},
"outputs": [
- {
- "data": {
- "text/html": [
- "\n",
- " \n",
- " "
- ],
- "text/plain": [
- ""
- ]
- },
- "metadata": {},
- "output_type": "display_data"
- },
- {
- "data": {
- "text/html": [
- "\n",
- " \n",
- " "
- ],
- "text/plain": [
- ""
- ]
- },
- "metadata": {},
- "output_type": "display_data"
- },
- {
- "data": {
- "text/html": [
- "\n",
- " \n",
- " "
- ],
- "text/plain": [
- ""
- ]
- },
- "metadata": {},
- "output_type": "display_data"
- },
- {
- "data": {
- "text/html": [
- "\n",
- " \n",
- " "
- ],
- "text/plain": [
- ""
- ]
- },
- "metadata": {},
- "output_type": "display_data"
- },
{
"name": "stdout",
"output_type": "stream",
"text": [
- "The document outlines the European Union's Artificial Intelligence Act, which aims to regulate AI systems and models while promoting innovation. It establishes rules for developing, using, and monitoring AI, especially high-risk systems, to protect public interests and fundamental rights. The Act mandates risk management systems, data governance requirements, and transparency measures for high-risk AI. It also sets obligations for providers and deployers, including registration, and empowers competent authorities to enforce compliance. Additionally, it encourages codes of conduct and provides for AI regulatory sandboxes to support innovation. The EU will establish a database for high-risk AI systems, and the Commission will coordinate enforcement and promote best practices. Fines and penalties are proposed for non-compliance. The Act seeks to balance AI development and oversight, ensuring trustworthy and responsible use while fostering the EU's AI ecosystem.\n"
+ "The document outlines the requirements and obligations for developing and deploying AI systems in the European Union. It aims to establish a regulatory framework to foster innovation while ensuring the protection of fundamental rights and public interests. The regulation applies to providers and deployers of AI systems, including those established outside the EU. High-risk AI systems are subject to specific requirements, such as risk management, data governance, and transparency. Providers must ensure compliance and keep records, and deployers must use AI systems responsibly. The regulation also establishes an AI Office, advisory bodies, and a database for high-risk AI systems. Additionally, it addresses issues like testing, codes of conduct, and cooperation with third countries. Fines and penalties are proposed for non-compliance.\n"
]
}
],
@@ -2055,16 +766,12 @@
"source": [
"## Summary\n",
"\n",
- "In this notebook we present three useful methods to over come the limitations of context window size. In the following [blog post](TODO:add link), we talk more about how these methods can be evaluated."
+ "In this notebook we present three useful methods to over come the limitations of context window size. In the following [blog post](https://docs.cohere.com/page/chunking-strategies), we talk more about how these methods can be evaluated."
]
},
{
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "id": "iUqiXtILroUY"
- },
- "outputs": [],
+ "cell_type": "markdown",
+ "metadata": {},
"source": []
}
],
@@ -2086,9 +793,9 @@
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
- "version": "3.12.2"
+ "version": "3.10.13"
}
},
"nbformat": 4,
"nbformat_minor": 0
-}
\ No newline at end of file
+}
diff --git a/notebooks/guides/Meeting_Summaries_General_&_LangChain.ipynb b/notebooks/guides/Meeting_Summaries_General_&_LangChain.ipynb
index 73117b24..2fd1876b 100644
--- a/notebooks/guides/Meeting_Summaries_General_&_LangChain.ipynb
+++ b/notebooks/guides/Meeting_Summaries_General_&_LangChain.ipynb
@@ -11,7 +11,15 @@
"Welcome to Cohere! In this notebook, you'll learn how to:\n",
"* Use Cohere's Command-R model to summarize meeting transcripts.\n",
"* Modify your prompt to include specific formatting instructions, especially if the model output is used in downstream applications.\n",
- "* Use Command-R with LangChain for summarization."
+ "* Use Command-R with LangChain for summarization.\n",
+ "\n",
+ "### Table of contents:\n",
+ "1. [Setup](#setup)\n",
+ "2. [Load test data](#load)\n",
+ "3. [Simple summarization](#simple)\n",
+ "4. [More advanced formatting](#more)\n",
+ "5. [LangChain summarization](#langchain)\n",
+ "6. [Wrap up](#wrap)"
]
},
{
@@ -20,6 +28,8 @@
"id": "6SfEhd3O74--"
},
"source": [
+ "\n",
+ "\n",
"## Setup\n",
"\n",
"You'll need a Cohere API key to run this notebook. If you don't have a key, head to https://cohere.com/ to generate your key."
@@ -27,20 +37,19 @@
},
{
"cell_type": "code",
- "execution_count": 1,
+ "execution_count": 7,
"metadata": {
"id": "H2TfKiPM3a6f"
},
"outputs": [],
"source": [
"%%capture\n",
- "# TODO: upgrade to \"cohere>5\"",
-"!pip install \"cohere<5\" datasets tokenizers langchain"
+ "!pip install cohere datasets tokenizers langchain langchain-cohere"
]
},
{
"cell_type": "code",
- "execution_count": 2,
+ "execution_count": 8,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
@@ -48,15 +57,7 @@
"id": "J1HzGqnY74bj",
"outputId": "24fba7c5-a2ce-4c86-bc14-4ca2fb9137b5"
},
- "outputs": [
- {
- "name": "stdout",
- "output_type": "stream",
- "text": [
- "Enter your Cohere API key: ··········\n"
- ]
- }
- ],
+ "outputs": [],
"source": [
"import re\n",
"from string import Template\n",
@@ -74,7 +75,7 @@
},
{
"cell_type": "code",
- "execution_count": 3,
+ "execution_count": 9,
"metadata": {
"id": "SbVCPxMSD3fs"
},
@@ -89,7 +90,7 @@
" if not s:\n",
" print()\n",
" else:\n",
- " print(\"\\n\".join(line.strip() for line in re.findall(rf\".{{1,{maxchars}}}(?:\\s+|$)\", s)))\n"
+ " print(\"\\n\".join(line.strip() for line in re.findall(rf\".{{1,{maxchars}}}(?:\\s+|$)\", s)))"
]
},
{
@@ -98,6 +99,8 @@
"id": "ABgVdI7I8Jmk"
},
"source": [
+ "\n",
+ "\n",
"## Load test data\n",
"\n",
"Let's load a meeting transcript to see Command in action!\n",
@@ -109,7 +112,7 @@
},
{
"cell_type": "code",
- "execution_count": 4,
+ "execution_count": 10,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
@@ -198,116 +201,6 @@
"outputId": "30c66020-c5b0-4984-fce6-9ae018b7abf0"
},
"outputs": [
- {
- "name": "stderr",
- "output_type": "stream",
- "text": [
- "/usr/local/lib/python3.10/dist-packages/huggingface_hub/utils/_token.py:88: UserWarning: \n",
- "The secret `HF_TOKEN` does not exist in your Colab secrets.\n",
- "To authenticate with the Hugging Face Hub, create a token in your settings tab (https://huggingface.co/settings/tokens), set it as secret in your Google Colab and restart your session.\n",
- "You will be able to reuse this secret in all of your notebooks.\n",
- "Please note that authentication is recommended but still optional to access public models or datasets.\n",
- " warnings.warn(\n"
- ]
- },
- {
- "data": {
- "application/vnd.jupyter.widget-view+json": {
- "model_id": "43225bc344ed4374931511b8630baae4",
- "version_major": 2,
- "version_minor": 0
- },
- "text/plain": [
- "Downloading readme: 0%| | 0.00/21.0 [00:00\n",
+ "\n",
"## Simple summarization\n",
"\n",
"First, we'll show an example of a simple summarization task. The prompt template we are using is shown below, and we can obtain the completion using a one line call to the Cohere API."
@@ -438,7 +295,7 @@
},
{
"cell_type": "code",
- "execution_count": 6,
+ "execution_count": 14,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
@@ -452,9 +309,9 @@
"output_type": "stream",
"text": [
"The team discusses the time-consuming process of identifying instances of speaker overlap in audio\n",
- "recordings. PhD C suggests using close-talking microphones and automated methods to establish ground\n",
- "truth data, which Grad G is working on. This data would help develop a more efficient system than\n",
- "manual marking.\n"
+ "data. PhD C suggests using close-talking microphones and automated methods to establish ground truth\n",
+ "data, which Grad G is working on. This data would help develop a more efficient system than manual\n",
+ "marking.\n"
]
}
],
@@ -484,7 +341,7 @@
"1. We include a preamble imbuing the model with a persona.\n",
"2. We use Markdown-style headers (i.e. with `##`) to delineate the preamble, the text-to-summarise, the instructions, and the output\n",
"\n",
- "For some other prompting best practices and tips/tricks, see our [prompting guide](https://docs.google.com/document/d/1k0KBiOUEPq65vqEuAQyg6Y5l1s2_Ybwv-lOLho6NHsw/edit)!"
+ "For some other prompting best practices and tips/tricks, see our [prompting guide](https://docs.cohere.com/docs/crafting-effective-prompts)!"
]
},
{
@@ -493,6 +350,8 @@
"id": "YT6nsN6DK_xt"
},
"source": [
+ "\n",
+ "\n",
"## More advanced formatting\n",
"\n",
"That worked well, but what if you want to have more control over the summary? What if you wanted a bulleted summary and wanted exactly 3 bullets in a json structure?\n",
@@ -502,7 +361,7 @@
},
{
"cell_type": "code",
- "execution_count": 7,
+ "execution_count": 15,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
@@ -518,9 +377,9 @@
"```json\n",
"{\n",
" \"summary_bullets\": [\n",
- " \"There's a need to identify instances of speaker overlap in recorded meetings, but it's a time-consuming task to mark these events manually.\",\n",
- " \"An alternative is to infer speaker overlap using various measurements of energy in the audio, such as close-talking microphones, LPC residuals, etc.\",\n",
- " \"A threshold-based program that detects speech based on volume can speed up the process, and further improvements can be explored once a robust system is in place.\"\n",
+ " \"There are many ways to study and mark speech overlap in a busy environment, but it's a time-consuming task.\",\n",
+ " \"A threshold-based program that uses close-talking mics to infer speech overlap could speed up the process. The program looks for runs after applying a median filter and appears to work.\",\n",
+ " \"Having a ground truth to compare the automated results against is important, and a human review of the data is still necessary.\"\n",
" ]\n",
"}\n",
"```\n"
@@ -563,7 +422,7 @@
},
{
"cell_type": "code",
- "execution_count": 8,
+ "execution_count": 16,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
@@ -576,8 +435,9 @@
"name": "stdout",
"output_type": "stream",
"text": [
- "The team discusses using various methods, including close-talking mics and threshold-based volume\n",
- "filters, to accurately identify instances of speaker overlap in recorded meetings.\n"
+ "The team discusses using various methods, including close-talking microphones and automated volume\n",
+ "thresholding, to accurately identify instances of speaker overlap in recorded meetings as part of a\n",
+ "research project.\n"
]
}
],
@@ -602,13 +462,15 @@
"id": "KOaKiT6Ztkbz"
},
"source": [
+ "\n",
+ "\n",
"## LangChain summarization\n",
"It's also very easy to use Command-R with LangChain for summarization."
]
},
{
"cell_type": "code",
- "execution_count": 9,
+ "execution_count": 17,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
@@ -621,7 +483,9 @@
"name": "stderr",
"output_type": "stream",
"text": [
- "/usr/local/lib/python3.10/dist-packages/langchain_core/_api/deprecation.py:117: LangChainDeprecationWarning: The function `run` was deprecated in LangChain 0.1.0 and will be removed in 0.2.0. Use invoke instead.\n",
+ "/Users/boyu/miniconda3/envs/test/lib/python3.11/site-packages/langchain_core/_api/deprecation.py:119: LangChainDeprecationWarning: The class `LLMChain` was deprecated in LangChain 0.1.17 and will be removed in 0.3.0. Use RunnableSequence, e.g., `prompt | llm` instead.\n",
+ " warn_deprecated(\n",
+ "/Users/boyu/miniconda3/envs/test/lib/python3.11/site-packages/langchain_core/_api/deprecation.py:119: LangChainDeprecationWarning: The method `Chain.run` was deprecated in langchain 0.1.0 and will be removed in 0.3.0. Use invoke instead.\n",
" warn_deprecated(\n"
]
},
@@ -635,18 +499,19 @@
"\n",
"\u001b[1m> Finished chain.\u001b[0m\n",
"The team discusses the time-consuming process of identifying instances of speaker overlap in audio\n",
- "recordings. PhD C suggests using close-talking microphones and automated methods to establish ground\n",
- "truth data, which Grad G is working on. This data would help develop a more efficient system than\n",
- "manual marking.\n"
+ "data. PhD C suggests using close-talking microphones and automated methods to establish ground truth\n",
+ "data, which Grad G is working on. This data would help develop a more efficient system than manual\n",
+ "marking.\n"
]
}
],
"source": [
"from langchain.chains.combine_documents.stuff import StuffDocumentsChain\n",
"from langchain.chains.llm import LLMChain\n",
- "from langchain.prompts import PromptTemplate\n",
+ "from langchain_cohere import ChatCohere\n",
"from langchain.docstore.document import Document\n",
- "from langchain_community.chat_models import ChatCohere\n",
+ "from langchain.prompts import PromptTemplate\n",
+ "\n",
"\n",
"llm = ChatCohere(\n",
" cohere_api_key=co_api_key,\n",
@@ -691,13 +556,15 @@
"id": "uj8kdy5auKKY"
},
"source": [
+ "\n",
+ "\n",
"## Wrap up\n",
"\n",
"Those were some simple examples of how you can use Cohere's Command-R model for summarization.\n",
"\n",
- "For more advanced summarization objectives and to see some other cool things you can do with the Command-R model, see [recipes for better meeting notes summaries](https://colab.research.google.com/drive/1XqRpJH7qRnRTbOEt6kthwqZG6gtEn4gN#scrollTo=LplM9PSe8djM).\n",
+ "For more advanced summarization objectives and to see some other cool things you can do with the Command-R model, see [recipes for better meeting notes summaries](https://github.com/cohere-ai/notebooks/blob/main/notebooks/guides/Recipes_for_better_meeting_notes_summary.ipynb).\n",
"\n",
- "In cases where you can't fit the input text into the context window, see our [cookbook for long document strategies](https://colab.research.google.com/drive/1zxSAbruOWwWJHNsj3N56uxZtUeiS7Evd#scrollTo=4JgYYTg7Shvq).\n",
+ "In cases where you can't fit the input text into the context window, see our [cookbook for long document strategies](https://github.com/cohere-ai/notebooks/blob/main/notebooks/guides/Long_form_General_Strategies.ipynb).\n",
"\n",
"For more information, you can reach out to us at summarize@cohere.com!"
]
@@ -721,7 +588,7 @@
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
- "version": "3.10.11"
+ "version": "3.11.5"
},
"widgets": {
"application/vnd.jupyter.widget-state+json": {
diff --git a/notebooks/guides/Migrating_Monolithic_Prompts_to_Command_R_with_RAG.ipynb b/notebooks/guides/Migrating_Monolithic_Prompts_to_Command_R_with_RAG.ipynb
index 6f60eb74..8b288d09 100644
--- a/notebooks/guides/Migrating_Monolithic_Prompts_to_Command_R_with_RAG.ipynb
+++ b/notebooks/guides/Migrating_Monolithic_Prompts_to_Command_R_with_RAG.ipynb
@@ -97,7 +97,7 @@
"id": "JfS7y2zD-ehv"
},
"source": [
- "This application scenario is a common LLM-as-assistant use case. Given some context, help the user to complete a task. In this case, the task is to write a concise autobiographical summary."
+ "This application scenario is a common LLM-as-assistant use case: given some context, help the user to complete a task. In this case, the task is to write a concise autobiographical summary."
]
},
{
@@ -535,7 +535,7 @@
"id": "cEdYRffp06d2"
},
"source": [
- "On March 21st, the DOJ announced that it is [suing apple](https://www.theverge.com/2024/3/21/24107659/apple-doj-lawsuit-antitrust-documents-suing) for anti-competitive practices. The [complaint](https://www.justice.gov/opa/media/1344546/dl) is 88 pages long and consists of about 230 paragraphs of text. To understand what the suit alleges, a common use case would be to ask for a summary. Because Command-R has a context window of 128K, even an 88-page legal complaint fits comfortably within the window."
+ "On March 21st, the DOJ announced that it is [suing Apple](https://www.theverge.com/2024/3/21/24107659/apple-doj-lawsuit-antitrust-documents-suing) for anti-competitive practices. The [complaint](https://www.justice.gov/opa/media/1344546/dl) is 88 pages long and consists of about 230 paragraphs of text. To understand what the suit alleges, a common use case would be to ask for a summary. Because Command-R has a context window of 128K, even an 88-page legal complaint fits comfortably within the window."
]
},
{
@@ -658,7 +658,7 @@
"id": "7a81dU5U3LIY"
},
"source": [
- "The summary seems clear enough. But I am interested in the specific allegations that the DOJ makes. For example, skimming the full complaint, it looks like the DOJ is alleging that Apple could encrypt text messages sent to Android phones if it wanted to do so. We can ammend the rendered prompt and ask:"
+ "The summary seems clear enough. But we are interested in the specific allegations that the DOJ makes. For example, skimming the full complaint, it looks like the DOJ is alleging that Apple could encrypt text messages sent to Android phones if it wanted to do so. We can amend the rendered prompt and ask:"
]
},
{
@@ -955,15 +955,6 @@
"\n",
"In this cookbook we have shown how one can easily take an existing monolithic prompt and migrate it to the RAG paradigm to get less hallucination, grounded information, and in-line citations. We also demonstrated Command-R's ability to re-write an instruction prompt in a single shot to make it more concise and potentially lead to higher quality completions."
]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "id": "0vTsJc-ZHleg"
- },
- "outputs": [],
- "source": []
}
],
"metadata": {
@@ -985,7 +976,7 @@
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
- "version": "3.10.13"
+ "version": "3.8.16"
}
},
"nbformat": 4,
diff --git a/notebooks/guides/Multilingual_Search_with_Cohere_and_Langchain.ipynb b/notebooks/guides/Multilingual_Search_with_Cohere_and_Langchain.ipynb
index 83eedfc6..253c5a3b 100644
--- a/notebooks/guides/Multilingual_Search_with_Cohere_and_Langchain.ipynb
+++ b/notebooks/guides/Multilingual_Search_with_Cohere_and_Langchain.ipynb
@@ -1,3687 +1,3658 @@
{
- "cells": [
- {
- "attachments": {},
- "cell_type": "markdown",
- "metadata": {
- "id": "5WNncDXelfhy"
- },
- "source": [
- "# Multilingual Search with Cohere and Langchain"
- ]
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "5WNncDXelfhy"
+ },
+ "source": [
+ "# Multilingual Search with Cohere and Langchain"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "PGbjnPUwH8tp"
+ },
+ "source": [
+ "***Read the accompanying [blog post here](https://txt.cohere.ai/search-cohere-langchain/).***"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "mjR6RAkgf6EM"
+ },
+ "source": [
+ "This notebook contains two examples for performing multilingual search using Cohere and Langchain. Langchain is a library that assists the development of applications built on top of large language models (LLMs), such as Cohere's models.\n",
+ "\n",
+ "In short, Cohere makes it easy for developers to leverage LLMs and Langchain makes it easy to build applications with these models.\n",
+ "\n",
+ "We'll go through the following examples:\n",
+ "- **Example 1 - Basic Multilingual Search**\n",
+ "\n",
+ " This is a simple example of multilingual search over a list of documents.\n",
+ "\n",
+ " The steps in summary:\n",
+ " - Import a list of documents\n",
+ " - Embed the documents and store them in an index\n",
+ " - Enter a query\n",
+ " - Return the document most similar to the query\n",
+ "- **Example 2 - Search-Based Question Answering**\n",
+ "\n",
+ " This example shows a more involved example where search is combined with text generation to answer questions about long-form documents.\n",
+ "\n",
+ " The steps in summary:\n",
+ " - Add an article and chunk it into smaller passages\n",
+ " - Embed the passages and store them in an index\n",
+ " - Enter a question\n",
+ " - Answer the question based on the most relevant documents"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "colab": {
+ "base_uri": "https://localhost:8080/"
},
+ "id": "LDkRgKDwEWWy",
+ "outputId": "6afa8b14-f88f-4984-f9cb-15a332a4eebf"
+ },
+ "outputs": [
{
- "attachments": {},
- "cell_type": "markdown",
- "metadata": {
- "id": "PGbjnPUwH8tp"
- },
- "source": [
- "***Read the accompanying [blog post here](https://txt.cohere.ai/search-cohere-langchain/).***"
+ "data": {
+ "text/plain": [
+ "True"
]
+ },
+ "execution_count": 3,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "from langchain.embeddings.cohere import CohereEmbeddings\n",
+ "from langchain.llms import Cohere\n",
+ "from langchain.prompts import PromptTemplate\n",
+ "from langchain.text_splitter import RecursiveCharacterTextSplitter\n",
+ "from langchain.chains.question_answering import load_qa_chain\n",
+ "from langchain.chains import RetrievalQA\n",
+ "from langchain.vectorstores import Qdrant\n",
+ "from langchain.document_loaders import TextLoader\n",
+ "import textwrap as tr\n",
+ "import random\n",
+ "import dotenv\n",
+ "import os\n",
+ "\n",
+ "dotenv.load_dotenv(\".env\") # Upload an '.env' file containing an environment variable named 'COHERE_API_KEY' using your Cohere API Key"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "oHUJHp6dZxG2"
+ },
+ "source": [
+ "# Example 1 - Basic Multilingual Search"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "U111MGZvhM7O"
+ },
+ "source": [
+ ""
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "PVTkZXLsHiVs"
+ },
+ "source": [
+ "### Import a list of documents"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "colab": {
+ "base_uri": "https://localhost:8080/",
+ "height": 187,
+ "referenced_widgets": [
+ "7c8e1e4eee714bc086c3f84eee5941fc",
+ "4334b4f6fc7048eab0d146b6ffc437d1",
+ "c8e13e321a9a4e0b88c717a1a8065619",
+ "c1f14f69e97f44598d9c4f8d789131fb",
+ "25a079a3d58644f4a5306ee2cb42dbde",
+ "be8b3f4e73fc44f390e9a147df786e53",
+ "3bda8233967642229fe778b4a8402f52",
+ "c37f294926f945898096874d324ee2ae",
+ "941ab80a14aa4a1bb3943dd77aca85d7",
+ "1f1331ab4e734cdaa66cf28d7278f980",
+ "d773c0f65a014ae08f533bf2348c1831",
+ "4cf6a31b3be945cbb82f8239e7e2642e",
+ "7fd82914b89343538aa08b36f4752f4b",
+ "35132c00838046bca253f31350a3190b",
+ "190a428569a74beb8b90aa278c629bd2",
+ "01545efb5b78418791ce0f8c763873ce",
+ "d426d4b5b86c4582a17bf21fe5f021d4",
+ "2d69c8378b194aa3a8634848da5ee79a",
+ "88ff8d07f6ce430db2772b8bf24294e8",
+ "f5e168c246714295a10a64fbb1ef2ee3",
+ "6d1ea14b22034d2d9ab16aa5a24566dc",
+ "6c0193f68df143e0a7907c6a577b07a6",
+ "7fa8f3f5c00c493bb7283bc90c33df5e",
+ "59f2ed0257d6422d84fa23898e5d9a10",
+ "9236baef198c44ae8607c7888e899bc8",
+ "3647d22aaf934b53bf925dc22e2ce69a",
+ "687cfefc80a54edd84eb22baf805c905",
+ "533f3a17f8d1442490a2d6167d89550b",
+ "ce82f835dd2b45a181a53407505b0cfc",
+ "02e46682b2434a0c872f35db3d7b367e",
+ "3da4f3751cb54cf2938aea924ab33aa0",
+ "89f757fbb1ef4fa8b557c04926871096",
+ "94ebb17c331649fc812a66231532ee03",
+ "df522159441841cda1f97d63aab5ad9e",
+ "a9f4707824454e35b2273412c4586b8d",
+ "d849ae343e51419c803c29c24a16dd01",
+ "65b7f23304a243878c43ea391885ee85",
+ "85f4ab947d774115baf64332f64248bd",
+ "efa03832003244a490cfcb34edf7ccbe",
+ "9e46610e715444cbbc2f978a0acd0ed0",
+ "3948a575b3ee4a199f1055e93d8270ee",
+ "2ee59cbc091346168c3b4511f64b018a",
+ "c5296022707240b393be4146e983083c",
+ "8653f9b6a25446cc8d7d34ffd4e48ca8",
+ "a61d6191c53e4dd2a1cbdb9826f5ae70",
+ "e868b829ec7a4795955034ebb8b109a6",
+ "1385286d855a46c8b19704a95e5992b4",
+ "b7bbc695e0f248a0ac1e79a1d546df8e",
+ "034072a591b94d3daf4284aadcff935f",
+ "d24e60c7e4664e0ba11cce85c647e526",
+ "23bbe3c6dfcb42c7a0a1c826bc35076d",
+ "5cf957caabe043139dc587e8651a28b1",
+ "bfb987eb24644819ba52b2fc7bc865de",
+ "8e7d772e693c427c9bcfc7535a524979",
+ "d646a9a4cc5a4876a7fa185bf9eb3ccf",
+ "09c17a96b5534b78b1d9bfc2f7bc83d5",
+ "735bfe1c1ff6402d826257313d68b21b",
+ "df1b4d3501ed457b86b60a5449d1e1da",
+ "771c7097b5be45ffac31e70117376903",
+ "996f65fcf7a04703868717652927e5e2",
+ "5edafe62c55449709c304c11dc847e82",
+ "5ed53479e65749a68e2ae1f70fff1db0",
+ "b04f406f1a2a46818d034a80d13f6664",
+ "fcc06cd646ff425b9172312808dd3e24",
+ "d9dd5b64f0ee40359a4c577f4358248f",
+ "baf658cc369745b7bad7f18302e385d9",
+ "b07ffeedc45a4fe9919f99d71a2ea3f2",
+ "211f7336a3b9455f8cd67bd0ee4113ae",
+ "2bd5fe14d3d44d85be32c5c78f5b7568",
+ "6850b87ff2fa49eb9fbfb3c3492efbb3",
+ "6def2ecb657845a3bae18bad85950991",
+ "68bf09957e83431b8d299f1ab15d5f7e",
+ "28adfb59f3e24851ba617e0087e5293f",
+ "10ca522adeca4ba4bb2d4f9373a363f4",
+ "5f9f737d68234b9a9cd069105ab20ea2",
+ "d298dc9202dc49aba09b66969b3d19f9",
+ "6e09dadf883440038a7a47f0d9723ae6",
+ "cd23755e6122427e9d95eb808b7900c2",
+ "62eb05321e2d4265b033544921f92053",
+ "3627640fb7d14ac09c9fdcbfb2159152",
+ "c05f2a082d5e4a0c8f3b9b6952eff208",
+ "bf6033cd5d0c4ef5ac934ed6e8adaeb6",
+ "463628ae5afc4c8f9a51f2c4ed038d12",
+ "1ac7e3656e43480c8af4038d2e7d5432",
+ "6e600a0c3f36439899b40cdc53ed7c8c",
+ "672ec8abe8b34b8dab6d4af20f0302c7",
+ "57cf77bba44141359639c0268ced8934",
+ "6a7d1421176f4b62af003a4b2f18815b"
+ ]
},
+ "id": "WN-sQquOb4VQ",
+ "outputId": "f547e445-3b89-4b87-f9a3-7f7fa08053f4"
+ },
+ "outputs": [
{
- "attachments": {},
- "cell_type": "markdown",
- "metadata": {
- "id": "mjR6RAkgf6EM"
- },
- "source": [
- "This notebook contains two examples for performing multilingual search using Cohere and Langchain. Langchain is a library that assists the development of applications built on top of large language models (LLMs), such as Cohere's models.\n",
- "\n",
- "In short, Cohere makes it easy for developers to leverage LLMs and Langchain makes it easy to build applications with these models.\n",
- "\n",
- "We'll go through the following examples:\n",
- "- **Example 1 - Basic Multilingual Search**\n",
- "\n",
- " This is a simple example of multilingual search over a list of documents.\n",
- "\n",
- " The steps in summary:\n",
- " - Import a list of documents\n",
- " - Embed the documents and store them in an index\n",
- " - Enter a query\n",
- " - Return the document most similar to the query\n",
- "- **Example 2 - Search-Based Question Answering**\n",
- "\n",
- " This example shows a more involved example where search is combined with text generation to answer questions about long-form documents.\n",
- "\n",
- " The steps in summary:\n",
- " - Add an article and chunk it into smaller passages\n",
- " - Embed the passages and store them in an index\n",
- " - Enter a question\n",
- " - Answer the question based on the most relevant documents"
- ]
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "Downloading and preparing dataset 350.79 KiB (download: 350.79 KiB, generated: 636.90 KiB, total: 987.69 KiB) to /root/tensorflow_datasets/trec/1.0.0...\n"
+ ]
},
{
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "id": "sDhfWxSaEOpS"
+ "data": {
+ "application/vnd.jupyter.widget-view+json": {
+ "model_id": "7c8e1e4eee714bc086c3f84eee5941fc",
+ "version_major": 2,
+ "version_minor": 0
},
- "outputs": [],
- "source": [
- "# TODO: upgrade to \"cohere>5\"",
-"! pip install \"cohere<5\" langchain qdrant-client tfds-nightly python-dotenv > /dev/null"
+ "text/plain": [
+ "Dl Completed...: 0 url [00:00, ? url/s]"
]
+ },
+ "metadata": {},
+ "output_type": "display_data"
},
{
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "colab": {
- "base_uri": "https://localhost:8080/"
- },
- "id": "LDkRgKDwEWWy",
- "outputId": "6afa8b14-f88f-4984-f9cb-15a332a4eebf"
+ "data": {
+ "application/vnd.jupyter.widget-view+json": {
+ "model_id": "4cf6a31b3be945cbb82f8239e7e2642e",
+ "version_major": 2,
+ "version_minor": 0
},
- "outputs": [
- {
- "data": {
- "text/plain": [
- "True"
- ]
- },
- "execution_count": 3,
- "metadata": {},
- "output_type": "execute_result"
- }
- ],
- "source": [
- "from langchain.embeddings.cohere import CohereEmbeddings\n",
- "from langchain.llms import Cohere\n",
- "from langchain.prompts import PromptTemplate\n",
- "from langchain.text_splitter import RecursiveCharacterTextSplitter\n",
- "from langchain.chains.question_answering import load_qa_chain\n",
- "from langchain.chains import RetrievalQA\n",
- "from langchain.vectorstores import Qdrant\n",
- "from langchain.document_loaders import TextLoader\n",
- "import textwrap as tr\n",
- "import random\n",
- "import dotenv\n",
- "import os\n",
- "\n",
- "dotenv.load_dotenv(\".env\") # Upload an '.env' file containing an environment variable named 'COHERE_API_KEY' using your Cohere API Key"
+ "text/plain": [
+ "Dl Size...: 0 MiB [00:00, ? MiB/s]"
]
+ },
+ "metadata": {},
+ "output_type": "display_data"
},
{
- "attachments": {},
- "cell_type": "markdown",
- "metadata": {
- "id": "oHUJHp6dZxG2"
+ "data": {
+ "application/vnd.jupyter.widget-view+json": {
+ "model_id": "7fa8f3f5c00c493bb7283bc90c33df5e",
+ "version_major": 2,
+ "version_minor": 0
},
- "source": [
- "# Example 1 - Basic Multilingual Search"
+ "text/plain": [
+ "Extraction completed...: 0 file [00:00, ? file/s]"
]
+ },
+ "metadata": {},
+ "output_type": "display_data"
},
{
- "attachments": {},
- "cell_type": "markdown",
- "metadata": {
- "id": "U111MGZvhM7O"
+ "data": {
+ "application/vnd.jupyter.widget-view+json": {
+ "model_id": "df522159441841cda1f97d63aab5ad9e",
+ "version_major": 2,
+ "version_minor": 0
},
- "source": [
- ""
+ "text/plain": [
+ "Generating splits...: 0%| | 0/2 [00:00] 11.71K --.-KB/s in 0s \n",
+ "\n",
+ "2023-06-08 06:11:20 (115 MB/s) - ‘steve-jobs-commencement.txt’ saved [11993/11993]\n",
+ "\n"
+ ]
+ }
+ ],
+ "source": [
+ "# We'll use Steve Jobs' Stanford University commencement address as the example. Link: https://news.stanford.edu/2005/06/12/youve-got-find-love-jobs-says/\n",
+ "\n",
+ "!wget 'https://docs.google.com/uc?export=download&id=1f1INWOfJrHTFmbyF_0be5b4u_moz3a4F' -O steve-jobs-commencement.txt"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "obv6NbUcNfB_"
+ },
+ "outputs": [],
+ "source": [
+ "loader = TextLoader(\"steve-jobs-commencement.txt\")\n",
+ "documents = loader.load()\n",
+ "text_splitter = RecursiveCharacterTextSplitter(chunk_size=500, chunk_overlap=0)\n",
+ "texts = text_splitter.split_documents(documents)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "DHvoF9CB9apc"
+ },
+ "source": [
+ "## Embed the passages and store them in an index"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "11Bx0k2LGIH-"
+ },
+ "outputs": [],
+ "source": [
+ "embeddings = CohereEmbeddings(model = \"multilingual-22-12\")\n",
+ "db = Qdrant.from_documents(texts, embeddings, location=\":memory:\", collection_name=\"my_documents\", distance_func=\"Dot\")"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "xQsrDUQo9poh"
+ },
+ "source": [
+ "## Enter a question"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "SW90c8kOqdow"
+ },
+ "outputs": [],
+ "source": [
+ "questions = [\n",
+ " \"What did the author liken The Whole Earth Catalog to?\",\n",
+ " \"What was Reed College great at?\",\n",
+ " \"What was the author diagnosed with?\",\n",
+ " \"What is the key lesson from this article?\",\n",
+ " \"What did the article say about Michael Jackson?\",\n",
+ " ]"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "HXhkK22JvNa1"
+ },
+ "source": [
+ "## Answer the question based on the most relevant documents\n"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "-dgBvF5HI9Wd"
+ },
+ "outputs": [],
+ "source": [
+ "# Create our own prompt template\n",
+ "\n",
+ "prompt_template = \"\"\"Text: {context}\n",
+ "\n",
+ "Question: {question}\n",
+ "\n",
+ "Answer the question based on the text provided. If the text doesn't contain the answer, reply that the answer is not available.\"\"\"\n",
+ "\n",
+ "PROMPT = PromptTemplate(\n",
+ " template=prompt_template, input_variables=[\"context\", \"question\"]\n",
+ ")"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "colab": {
+ "base_uri": "https://localhost:8080/"
},
+ "id": "8wa_YjzpcEAE",
+ "outputId": "31120ad3-034a-42f1-96a3-7454cd68ade5"
+ },
+ "outputs": [
{
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "id": "d6qxf0N0H_Cl"
- },
- "outputs": [],
- "source": [
- "# Return document most similar to the query\n",
- "answers = []\n",
- "for query in queries:\n",
- " docs = db.similarity_search(query)\n",
- " answers.append(docs[0].page_content)"
- ]
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "------------------------------------------------------------------------------------------------------------------------------------------------------ \n",
+ "\n",
+ "Question: What did the author liken The Whole Earth Catalog to?\n",
+ "Answer: It was sort of like Google in paperback form, 35 years before Google came along\n",
+ "\n",
+ "Sources:\n",
+ "1: When I was young, there was an amazing publication called The Whole Earth Catalog, which was one of the bibles of my generation. It was created by a\n",
+ "fellow named Stewart Brand not far from here in Menlo Park, and he brought it to life with his poetic touch. This was in the late 1960s, before\n",
+ "personal computers and desktop publishing, so it was all made with typewriters, scissors and Polaroid cameras. It was sort of like Google in paperback\n",
+ "form, 35 years before Google came along: It was\n",
+ "2: Stewart and his team put out several issues of The Whole Earth Catalog, and then when it had run its course, they put out a final issue. It was the\n",
+ "mid-1970s, and I was your age. On the back cover of their final issue was a photograph of an early morning country road, the kind you might find\n",
+ "yourself hitchhiking on if you were so adventurous. Beneath it were the words: “Stay Hungry. Stay Foolish.” It was their farewell message as they\n",
+ "signed off. Stay Hungry. Stay Foolish. And I have always\n",
+ "3: idealistic, and overflowing with neat tools and great notions.\n",
+ "4: beautiful, historical, artistically subtle in a way that science can’t capture, and I found it fascinating.\n",
+ "------------------------------------------------------------------------------------------------------------------------------------------------------ \n",
+ "\n",
+ "Question: What was Reed College great at?\n",
+ "Answer: Reed College was great at calligraphy instruction.\n",
+ "\n",
+ "Sources:\n",
+ "1: Reed College at that time offered perhaps the best calligraphy instruction in the country. Throughout the campus every poster, every label on every\n",
+ "drawer, was beautifully hand calligraphed. Because I had dropped out and didn’t have to take the normal classes, I decided to take a calligraphy class\n",
+ "to learn how to do this. I learned about serif and sans serif typefaces, about varying the amount of space between different letter combinations,\n",
+ "about what makes great typography great. It was\n",
+ "2: I dropped out of Reed College after the first 6 months, but then stayed around as a drop-in for another 18 months or so before I really quit. So why\n",
+ "did I drop out?\n",
+ "3: never dropped out, I would have never dropped in on this calligraphy class, and personal computers might not have the wonderful typography that they\n",
+ "do. Of course it was impossible to connect the dots looking forward when I was in college. But it was very, very clear looking backward 10 years\n",
+ "later.\n",
+ "4: OK. It was pretty scary at the time, but looking back it was one of the best decisions I ever made. The minute I dropped out I could stop taking the\n",
+ "required classes that didn’t interest me, and begin dropping in on the ones that looked interesting.\n",
+ "------------------------------------------------------------------------------------------------------------------------------------------------------ \n",
+ "\n",
+ "Question: What was the author diagnosed with?\n",
+ "Answer: The author was diagnosed with cancer.\n",
+ "\n",
+ "Sources:\n",
+ "1: I lived with that diagnosis all day. Later that evening I had a biopsy, where they stuck an endoscope down my throat, through my stomach and into my\n",
+ "intestines, put a needle into my pancreas and got a few cells from the tumor. I was sedated, but my wife, who was there, told me that when they viewed\n",
+ "the cells under a microscope the doctors started crying because it turned out to be a very rare form of pancreatic cancer that is curable with\n",
+ "surgery. I had the surgery and I’m fine now.\n",
+ "2: About a year ago I was diagnosed with cancer. I had a scan at 7:30 in the morning, and it clearly showed a tumor on my pancreas. I didn’t even know\n",
+ "what a pancreas was. The doctors told me this was almost certainly a type of cancer that is incurable, and that I should expect to live no longer than\n",
+ "three to six months. My doctor advised me to go home and get my affairs in order, which is doctor’s code for prepare to die. It means to try to tell\n",
+ "your kids everything you thought you’d have the\n",
+ "3: Stewart and his team put out several issues of The Whole Earth Catalog, and then when it had run its course, they put out a final issue. It was the\n",
+ "mid-1970s, and I was your age. On the back cover of their final issue was a photograph of an early morning country road, the kind you might find\n",
+ "yourself hitchhiking on if you were so adventurous. Beneath it were the words: “Stay Hungry. Stay Foolish.” It was their farewell message as they\n",
+ "signed off. Stay Hungry. Stay Foolish. And I have always\n",
+ "4: beautiful, historical, artistically subtle in a way that science can’t capture, and I found it fascinating.\n",
+ "------------------------------------------------------------------------------------------------------------------------------------------------------ \n",
+ "\n",
+ "Question: What is the key lesson from this article?\n",
+ "Answer: The key lesson from this article is that you have to trust that the dots will somehow connect in your future. You have to trust in something -- your gut, destiny, life, karma, whatever. This approach has never let me down, and it has made all the difference in my life.\n",
+ "\n",
+ "Sources:\n",
+ "1: Again, you can’t connect the dots looking forward; you can only connect them looking backward. So you have to trust that the dots will somehow connect\n",
+ "in your future. You have to trust in something — your gut, destiny, life, karma, whatever. This approach has never let me down, and it has made all\n",
+ "the difference in my life. My second story is about love and loss.\n",
+ "2: Remembering that I’ll be dead soon is the most important tool I’ve ever encountered to help me make the big choices in life. Because almost everything\n",
+ "— all external expectations, all pride, all fear of embarrassment or failure — these things just fall away in the face of death, leaving only what is\n",
+ "truly important. Remembering that you are going to die is the best way I know to avoid the trap of thinking you have something to lose. You are\n",
+ "already naked. There is no reason not to follow your\n",
+ "3: Your time is limited, so don’t waste it living someone else’s life. Don’t be trapped by dogma — which is living with the results of other people’s\n",
+ "thinking. Don’t let the noise of others’ opinions drown out your own inner voice. And most important, have the courage to follow your heart and\n",
+ "intuition. They somehow already know what you truly want to become. Everything else is secondary.\n",
+ "4: I really didn’t know what to do for a few months. I felt that I had let the previous generation of entrepreneurs down — that I had dropped the baton\n",
+ "as it was being passed to me. I met with David Packard and Bob Noyce and tried to apologize for screwing up so badly. I was a very public failure, and\n",
+ "I even thought about running away from the valley. But something slowly began to dawn on me — I still loved what I did. The turn of events at Apple\n",
+ "had not changed that one bit. I had been rejected,\n",
+ "------------------------------------------------------------------------------------------------------------------------------------------------------ \n",
+ "\n",
+ "Question: What did the article say about Michael Jackson?\n",
+ "Answer: The text did not provide information about Michael Jackson.\n",
+ "\n",
+ "Sources:\n",
+ "1: baby boy; do you want him?” They said: “Of course.” My biological mother later found out that my mother had never graduated from college and that my\n",
+ "father had never graduated from high school. She refused to sign the final adoption papers. She only relented a few months later when my parents\n",
+ "promised that I would someday go to college.\n",
+ "2: beautiful, historical, artistically subtle in a way that science can’t capture, and I found it fascinating.\n",
+ "3: I really didn’t know what to do for a few months. I felt that I had let the previous generation of entrepreneurs down — that I had dropped the baton\n",
+ "as it was being passed to me. I met with David Packard and Bob Noyce and tried to apologize for screwing up so badly. I was a very public failure, and\n",
+ "I even thought about running away from the valley. But something slowly began to dawn on me — I still loved what I did. The turn of events at Apple\n",
+ "had not changed that one bit. I had been rejected,\n",
+ "4: This was the closest I’ve been to facing death, and I hope it’s the closest I get for a few more decades. Having lived through it, I can now say this\n",
+ "to you with a bit more certainty than when death was a useful but purely intellectual concept:\n"
+ ]
+ }
+ ],
+ "source": [
+ "chain_type_kwargs = {\"prompt\": PROMPT}\n",
+ "\n",
+ "qa = RetrievalQA.from_chain_type(llm=Cohere(model=\"command\", temperature=0), \n",
+ " chain_type=\"stuff\", \n",
+ " retriever=db.as_retriever(), \n",
+ " chain_type_kwargs=chain_type_kwargs, \n",
+ " return_source_documents=True)\n",
+ "\n",
+ "for question in questions:\n",
+ " answer = qa({\"query\": question})\n",
+ " result = answer[\"result\"].replace(\"\\n\",\"\").replace(\"Answer:\",\"\")\n",
+ " sources = answer['source_documents']\n",
+ " print(\"-\"*150,\"\\n\")\n",
+ " print(f\"Question: {question}\")\n",
+ " print(f\"Answer: {result}\")\n",
+ "\n",
+ " ### COMMENT OUT THE 4 LINES BELOW TO HIDE THE SOURCES\n",
+ " print(f\"\\nSources:\")\n",
+ " for idx, source in enumerate(sources):\n",
+ " source_wrapped = tr.fill(str(source.page_content), width=150)\n",
+ " print(f\"{idx+1}: {source_wrapped}\")"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "bX77dCbuCxCu"
+ },
+ "source": [
+ "## Questions in French"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "rg29qxkPCdJL"
+ },
+ "outputs": [],
+ "source": [
+ "questions_fr = [\n",
+ " \"À quoi se compare The Whole Earth Catalog ?\",\n",
+ " \"Dans quoi Reed College était-il excellent ?\",\n",
+ " \"De quoi l'auteur a-t-il été diagnostiqué ?\",\n",
+ " \"Quelle est la leçon clé de cet article ?\",\n",
+ " \"Que disait l'article sur Michael Jackson ?\",\n",
+ " ]"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "EFuLtdgy5aZw"
+ },
+ "outputs": [],
+ "source": [
+ "# import langchain\n",
+ "# langchain.debug = False"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "colab": {
+ "base_uri": "https://localhost:8080/"
},
+ "id": "La7K_N7UTRdb",
+ "outputId": "8931bf1c-afe6-4b80-8979-841456e79a01"
+ },
+ "outputs": [
{
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "colab": {
- "base_uri": "https://localhost:8080/"
- },
- "id": "Tp_PyQ_FbG4z",
- "outputId": "b7dc4c49-989e-4a25-ee73-e123f8941aa1"
- },
- "outputs": [
- {
- "name": "stdout",
- "output_type": "stream",
- "text": [
- "Query language: English\n",
- "Query: How to get in touch with Bill Gates\n",
- "Most similar existing question: What is Bill Gates of Microsoft E-mail address ?\n",
- "-------------------- \n",
- "\n",
- "Query language: French\n",
- "Query: Comment entrer en contact avec Bill Gates\n",
- "Most similar existing question: What is Bill Gates of Microsoft E-mail address ?\n",
- "-------------------- \n",
- "\n",
- "Query language: Indonesian\n",
- "Query: Cara menghubungi Bill Gates\n",
- "Most similar existing question: What is Bill Gates of Microsoft E-mail address ?\n",
- "-------------------- \n",
- "\n"
- ]
- }
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "-------------------- \n",
+ "\n",
+ "Question: À quoi se compare The Whole Earth Catalog ?\n",
+ "Answer: The Whole Earth Catalog was like Google in paperback form, 35 years before Google came along.\n",
+ "-------------------- \n",
+ "\n",
+ "Question: Dans quoi Reed College était-il excellent ?\n",
+ "Answer: Reed College offered the best calligraphy instruction in the country.\n",
+ "-------------------- \n",
+ "\n",
+ "Question: De quoi l'auteur a-t-il été diagnostiqué ?\n",
+ "Answer: The author was diagnosed with a very rare form of pancreatic cancer that is curable with surgery.\n",
+ "-------------------- \n",
+ "\n",
+ "Question: Quelle est la leçon clé de cet article ?\n",
+ "Answer: The key lesson of this article is that remembering that you will die soon is the most important tool to help one make the big choices in life.\n",
+ "-------------------- \n",
+ "\n",
+ "Question: Que disait l'article sur Michael Jackson ?\n",
+ "Answer: The text does not contain the answer to the question.\n"
+ ]
+ }
+ ],
+ "source": [
+ "# Generate the answer given the context\n",
+ "\n",
+ "chain_type_kwargs = {\"prompt\": PROMPT}\n",
+ "\n",
+ "qa = RetrievalQA.from_chain_type(llm=Cohere(model=\"command\", temperature=0), \n",
+ " chain_type=\"stuff\", \n",
+ " retriever=db.as_retriever(), \n",
+ " chain_type_kwargs=chain_type_kwargs, \n",
+ " return_source_documents=True)\n",
+ "\n",
+ "for question in questions_fr:\n",
+ " answer = qa({\"query\": question})\n",
+ " result = answer[\"result\"].replace(\"\\n\",\"\").replace(\"Answer:\",\"\")\n",
+ " sources = answer['source_documents']\n",
+ " print(\"-\"*20,\"\\n\")\n",
+ " print(f\"Question: {question}\")\n",
+ " print(f\"Answer: {result}\")"
+ ]
+ }
+ ],
+ "metadata": {
+ "accelerator": "GPU",
+ "colab": {
+ "machine_shape": "hm",
+ "provenance": []
+ },
+ "gpuClass": "standard",
+ "kernelspec": {
+ "display_name": "Python 3 (ipykernel)",
+ "language": "python",
+ "name": "python3"
+ },
+ "language_info": {
+ "codemirror_mode": {
+ "name": "ipython",
+ "version": 3
+ },
+ "file_extension": ".py",
+ "mimetype": "text/x-python",
+ "name": "python",
+ "nbconvert_exporter": "python",
+ "pygments_lexer": "ipython3",
+ "version": "3.8.16"
+ },
+ "vscode": {
+ "interpreter": {
+ "hash": "3a0ab37a1f07e7d320af811f0819b193749e9675a96eea7a4830e92d810d141d"
+ }
+ },
+ "widgets": {
+ "application/vnd.jupyter.widget-state+json": {
+ "01545efb5b78418791ce0f8c763873ce": {
+ "model_module": "@jupyter-widgets/base",
+ "model_module_version": "1.2.0",
+ "model_name": "LayoutModel",
+ "state": {
+ "_model_module": "@jupyter-widgets/base",
+ "_model_module_version": "1.2.0",
+ "_model_name": "LayoutModel",
+ "_view_count": null,
+ "_view_module": "@jupyter-widgets/base",
+ "_view_module_version": "1.2.0",
+ "_view_name": "LayoutView",
+ "align_content": null,
+ "align_items": null,
+ "align_self": null,
+ "border": null,
+ "bottom": null,
+ "display": null,
+ "flex": null,
+ "flex_flow": null,
+ "grid_area": null,
+ "grid_auto_columns": null,
+ "grid_auto_flow": null,
+ "grid_auto_rows": null,
+ "grid_column": null,
+ "grid_gap": null,
+ "grid_row": null,
+ "grid_template_areas": null,
+ "grid_template_columns": null,
+ "grid_template_rows": null,
+ "height": null,
+ "justify_content": null,
+ "justify_items": null,
+ "left": null,
+ "margin": null,
+ "max_height": null,
+ "max_width": null,
+ "min_height": null,
+ "min_width": null,
+ "object_fit": null,
+ "object_position": null,
+ "order": null,
+ "overflow": null,
+ "overflow_x": null,
+ "overflow_y": null,
+ "padding": null,
+ "right": null,
+ "top": null,
+ "visibility": null,
+ "width": null
+ }
+ },
+ "02e46682b2434a0c872f35db3d7b367e": {
+ "model_module": "@jupyter-widgets/base",
+ "model_module_version": "1.2.0",
+ "model_name": "LayoutModel",
+ "state": {
+ "_model_module": "@jupyter-widgets/base",
+ "_model_module_version": "1.2.0",
+ "_model_name": "LayoutModel",
+ "_view_count": null,
+ "_view_module": "@jupyter-widgets/base",
+ "_view_module_version": "1.2.0",
+ "_view_name": "LayoutView",
+ "align_content": null,
+ "align_items": null,
+ "align_self": null,
+ "border": null,
+ "bottom": null,
+ "display": null,
+ "flex": null,
+ "flex_flow": null,
+ "grid_area": null,
+ "grid_auto_columns": null,
+ "grid_auto_flow": null,
+ "grid_auto_rows": null,
+ "grid_column": null,
+ "grid_gap": null,
+ "grid_row": null,
+ "grid_template_areas": null,
+ "grid_template_columns": null,
+ "grid_template_rows": null,
+ "height": null,
+ "justify_content": null,
+ "justify_items": null,
+ "left": null,
+ "margin": null,
+ "max_height": null,
+ "max_width": null,
+ "min_height": null,
+ "min_width": null,
+ "object_fit": null,
+ "object_position": null,
+ "order": null,
+ "overflow": null,
+ "overflow_x": null,
+ "overflow_y": null,
+ "padding": null,
+ "right": null,
+ "top": null,
+ "visibility": null,
+ "width": "20px"
+ }
+ },
+ "034072a591b94d3daf4284aadcff935f": {
+ "model_module": "@jupyter-widgets/base",
+ "model_module_version": "1.2.0",
+ "model_name": "LayoutModel",
+ "state": {
+ "_model_module": "@jupyter-widgets/base",
+ "_model_module_version": "1.2.0",
+ "_model_name": "LayoutModel",
+ "_view_count": null,
+ "_view_module": "@jupyter-widgets/base",
+ "_view_module_version": "1.2.0",
+ "_view_name": "LayoutView",
+ "align_content": null,
+ "align_items": null,
+ "align_self": null,
+ "border": null,
+ "bottom": null,
+ "display": null,
+ "flex": null,
+ "flex_flow": null,
+ "grid_area": null,
+ "grid_auto_columns": null,
+ "grid_auto_flow": null,
+ "grid_auto_rows": null,
+ "grid_column": null,
+ "grid_gap": null,
+ "grid_row": null,
+ "grid_template_areas": null,
+ "grid_template_columns": null,
+ "grid_template_rows": null,
+ "height": null,
+ "justify_content": null,
+ "justify_items": null,
+ "left": null,
+ "margin": null,
+ "max_height": null,
+ "max_width": null,
+ "min_height": null,
+ "min_width": null,
+ "object_fit": null,
+ "object_position": null,
+ "order": null,
+ "overflow": null,
+ "overflow_x": null,
+ "overflow_y": null,
+ "padding": null,
+ "right": null,
+ "top": null,
+ "visibility": "hidden",
+ "width": null
+ }
+ },
+ "09c17a96b5534b78b1d9bfc2f7bc83d5": {
+ "model_module": "@jupyter-widgets/controls",
+ "model_module_version": "1.5.0",
+ "model_name": "HBoxModel",
+ "state": {
+ "_dom_classes": [],
+ "_model_module": "@jupyter-widgets/controls",
+ "_model_module_version": "1.5.0",
+ "_model_name": "HBoxModel",
+ "_view_count": null,
+ "_view_module": "@jupyter-widgets/controls",
+ "_view_module_version": "1.5.0",
+ "_view_name": "HBoxView",
+ "box_style": "",
+ "children": [
+ "IPY_MODEL_735bfe1c1ff6402d826257313d68b21b",
+ "IPY_MODEL_df1b4d3501ed457b86b60a5449d1e1da",
+ "IPY_MODEL_771c7097b5be45ffac31e70117376903"
],
- "source": [
- "# Print the top document match for each query\n",
- "for idx,query in enumerate(queries):\n",
- " print(f\"Query language: {queries_lang[idx]}\")\n",
- " print(f\"Query: {query}\")\n",
- " print(f\"Most similar existing question: {answers[idx]}\")\n",
- " print(\"-\"*20,\"\\n\")"
- ]
+ "layout": "IPY_MODEL_996f65fcf7a04703868717652927e5e2"
+ }
},
- {
- "attachments": {},
- "cell_type": "markdown",
- "metadata": {
- "id": "aRpRAfLB9JhY"
- },
- "source": [
- "# Example 2 - Search-Based Question Answering"
- ]
+ "10ca522adeca4ba4bb2d4f9373a363f4": {
+ "model_module": "@jupyter-widgets/base",
+ "model_module_version": "1.2.0",
+ "model_name": "LayoutModel",
+ "state": {
+ "_model_module": "@jupyter-widgets/base",
+ "_model_module_version": "1.2.0",
+ "_model_name": "LayoutModel",
+ "_view_count": null,
+ "_view_module": "@jupyter-widgets/base",
+ "_view_module_version": "1.2.0",
+ "_view_name": "LayoutView",
+ "align_content": null,
+ "align_items": null,
+ "align_self": null,
+ "border": null,
+ "bottom": null,
+ "display": null,
+ "flex": null,
+ "flex_flow": null,
+ "grid_area": null,
+ "grid_auto_columns": null,
+ "grid_auto_flow": null,
+ "grid_auto_rows": null,
+ "grid_column": null,
+ "grid_gap": null,
+ "grid_row": null,
+ "grid_template_areas": null,
+ "grid_template_columns": null,
+ "grid_template_rows": null,
+ "height": null,
+ "justify_content": null,
+ "justify_items": null,
+ "left": null,
+ "margin": null,
+ "max_height": null,
+ "max_width": null,
+ "min_height": null,
+ "min_width": null,
+ "object_fit": null,
+ "object_position": null,
+ "order": null,
+ "overflow": null,
+ "overflow_x": null,
+ "overflow_y": null,
+ "padding": null,
+ "right": null,
+ "top": null,
+ "visibility": null,
+ "width": null
+ }
},
- {
- "attachments": {},
- "cell_type": "markdown",
- "metadata": {
- "id": "-f2kF2vShj9S"
- },
- "source": [
- ""
- ]
+ "1385286d855a46c8b19704a95e5992b4": {
+ "model_module": "@jupyter-widgets/controls",
+ "model_module_version": "1.5.0",
+ "model_name": "FloatProgressModel",
+ "state": {
+ "_dom_classes": [],
+ "_model_module": "@jupyter-widgets/controls",
+ "_model_module_version": "1.5.0",
+ "_model_name": "FloatProgressModel",
+ "_view_count": null,
+ "_view_module": "@jupyter-widgets/controls",
+ "_view_module_version": "1.5.0",
+ "_view_name": "ProgressView",
+ "bar_style": "",
+ "description": "",
+ "description_tooltip": null,
+ "layout": "IPY_MODEL_5cf957caabe043139dc587e8651a28b1",
+ "max": 5452,
+ "min": 0,
+ "orientation": "horizontal",
+ "style": "IPY_MODEL_bfb987eb24644819ba52b2fc7bc865de",
+ "value": 5452
+ }
},
- {
- "attachments": {},
- "cell_type": "markdown",
- "metadata": {
- "id": "nV0A1Z0Z9VTc"
- },
- "source": [
- "## Add an article and chunk it into smaller passages"
- ]
+ "190a428569a74beb8b90aa278c629bd2": {
+ "model_module": "@jupyter-widgets/controls",
+ "model_module_version": "1.5.0",
+ "model_name": "HTMLModel",
+ "state": {
+ "_dom_classes": [],
+ "_model_module": "@jupyter-widgets/controls",
+ "_model_module_version": "1.5.0",
+ "_model_name": "HTMLModel",
+ "_view_count": null,
+ "_view_module": "@jupyter-widgets/controls",
+ "_view_module_version": "1.5.0",
+ "_view_name": "HTMLView",
+ "description": "",
+ "description_tooltip": null,
+ "layout": "IPY_MODEL_6d1ea14b22034d2d9ab16aa5a24566dc",
+ "placeholder": "",
+ "style": "IPY_MODEL_6c0193f68df143e0a7907c6a577b07a6",
+ "value": " 0/0 [00:04<?, ? MiB/s]"
+ }
},
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "colab": {
- "base_uri": "https://localhost:8080/"
- },
- "id": "s12ZE7vcHRJI",
- "outputId": "7ebb06e3-d39d-4c65-92cb-73be8b032d1a"
- },
- "outputs": [
- {
- "name": "stdout",
- "output_type": "stream",
- "text": [
- "--2023-06-08 06:11:19-- https://docs.google.com/uc?export=download&id=1f1INWOfJrHTFmbyF_0be5b4u_moz3a4F\n",
- "Resolving docs.google.com (docs.google.com)... 74.125.200.101, 74.125.200.138, 74.125.200.102, ...\n",
- "Connecting to docs.google.com (docs.google.com)|74.125.200.101|:443... connected.\n",
- "HTTP request sent, awaiting response... 303 See Other\n",
- "Location: https://doc-0g-84-docs.googleusercontent.com/docs/securesc/ha0ro937gcuc7l7deffksulhg5h7mbp1/84t4moii9dmg08hmrh6rfpp8ecrjh6jq/1686204675000/12721472133292131824/*/1f1INWOfJrHTFmbyF_0be5b4u_moz3a4F?e=download&uuid=a26288c7-ad0c-4707-ae0b-72cb94c224dc [following]\n",
- "Warning: wildcards not supported in HTTP.\n",
- "--2023-06-08 06:11:19-- https://doc-0g-84-docs.googleusercontent.com/docs/securesc/ha0ro937gcuc7l7deffksulhg5h7mbp1/84t4moii9dmg08hmrh6rfpp8ecrjh6jq/1686204675000/12721472133292131824/*/1f1INWOfJrHTFmbyF_0be5b4u_moz3a4F?e=download&uuid=a26288c7-ad0c-4707-ae0b-72cb94c224dc\n",
- "Resolving doc-0g-84-docs.googleusercontent.com (doc-0g-84-docs.googleusercontent.com)... 74.125.130.132, 2404:6800:4003:c01::84\n",
- "Connecting to doc-0g-84-docs.googleusercontent.com (doc-0g-84-docs.googleusercontent.com)|74.125.130.132|:443... connected.\n",
- "HTTP request sent, awaiting response... 200 OK\n",
- "Length: 11993 (12K) [text/plain]\n",
- "Saving to: ‘steve-jobs-commencement.txt’\n",
- "\n",
- "steve-jobs-commence 100%[===================>] 11.71K --.-KB/s in 0s \n",
- "\n",
- "2023-06-08 06:11:20 (115 MB/s) - ‘steve-jobs-commencement.txt’ saved [11993/11993]\n",
- "\n"
- ]
- }
+ "1ac7e3656e43480c8af4038d2e7d5432": {
+ "model_module": "@jupyter-widgets/controls",
+ "model_module_version": "1.5.0",
+ "model_name": "DescriptionStyleModel",
+ "state": {
+ "_model_module": "@jupyter-widgets/controls",
+ "_model_module_version": "1.5.0",
+ "_model_name": "DescriptionStyleModel",
+ "_view_count": null,
+ "_view_module": "@jupyter-widgets/base",
+ "_view_module_version": "1.2.0",
+ "_view_name": "StyleView",
+ "description_width": ""
+ }
+ },
+ "1f1331ab4e734cdaa66cf28d7278f980": {
+ "model_module": "@jupyter-widgets/base",
+ "model_module_version": "1.2.0",
+ "model_name": "LayoutModel",
+ "state": {
+ "_model_module": "@jupyter-widgets/base",
+ "_model_module_version": "1.2.0",
+ "_model_name": "LayoutModel",
+ "_view_count": null,
+ "_view_module": "@jupyter-widgets/base",
+ "_view_module_version": "1.2.0",
+ "_view_name": "LayoutView",
+ "align_content": null,
+ "align_items": null,
+ "align_self": null,
+ "border": null,
+ "bottom": null,
+ "display": null,
+ "flex": null,
+ "flex_flow": null,
+ "grid_area": null,
+ "grid_auto_columns": null,
+ "grid_auto_flow": null,
+ "grid_auto_rows": null,
+ "grid_column": null,
+ "grid_gap": null,
+ "grid_row": null,
+ "grid_template_areas": null,
+ "grid_template_columns": null,
+ "grid_template_rows": null,
+ "height": null,
+ "justify_content": null,
+ "justify_items": null,
+ "left": null,
+ "margin": null,
+ "max_height": null,
+ "max_width": null,
+ "min_height": null,
+ "min_width": null,
+ "object_fit": null,
+ "object_position": null,
+ "order": null,
+ "overflow": null,
+ "overflow_x": null,
+ "overflow_y": null,
+ "padding": null,
+ "right": null,
+ "top": null,
+ "visibility": null,
+ "width": null
+ }
+ },
+ "211f7336a3b9455f8cd67bd0ee4113ae": {
+ "model_module": "@jupyter-widgets/controls",
+ "model_module_version": "1.5.0",
+ "model_name": "HTMLModel",
+ "state": {
+ "_dom_classes": [],
+ "_model_module": "@jupyter-widgets/controls",
+ "_model_module_version": "1.5.0",
+ "_model_name": "HTMLModel",
+ "_view_count": null,
+ "_view_module": "@jupyter-widgets/controls",
+ "_view_module_version": "1.5.0",
+ "_view_name": "HTMLView",
+ "description": "",
+ "description_tooltip": null,
+ "layout": "IPY_MODEL_68bf09957e83431b8d299f1ab15d5f7e",
+ "placeholder": "",
+ "style": "IPY_MODEL_28adfb59f3e24851ba617e0087e5293f",
+ "value": "Generating test examples...: 0%"
+ }
+ },
+ "23bbe3c6dfcb42c7a0a1c826bc35076d": {
+ "model_module": "@jupyter-widgets/controls",
+ "model_module_version": "1.5.0",
+ "model_name": "DescriptionStyleModel",
+ "state": {
+ "_model_module": "@jupyter-widgets/controls",
+ "_model_module_version": "1.5.0",
+ "_model_name": "DescriptionStyleModel",
+ "_view_count": null,
+ "_view_module": "@jupyter-widgets/base",
+ "_view_module_version": "1.2.0",
+ "_view_name": "StyleView",
+ "description_width": ""
+ }
+ },
+ "25a079a3d58644f4a5306ee2cb42dbde": {
+ "model_module": "@jupyter-widgets/base",
+ "model_module_version": "1.2.0",
+ "model_name": "LayoutModel",
+ "state": {
+ "_model_module": "@jupyter-widgets/base",
+ "_model_module_version": "1.2.0",
+ "_model_name": "LayoutModel",
+ "_view_count": null,
+ "_view_module": "@jupyter-widgets/base",
+ "_view_module_version": "1.2.0",
+ "_view_name": "LayoutView",
+ "align_content": null,
+ "align_items": null,
+ "align_self": null,
+ "border": null,
+ "bottom": null,
+ "display": null,
+ "flex": null,
+ "flex_flow": null,
+ "grid_area": null,
+ "grid_auto_columns": null,
+ "grid_auto_flow": null,
+ "grid_auto_rows": null,
+ "grid_column": null,
+ "grid_gap": null,
+ "grid_row": null,
+ "grid_template_areas": null,
+ "grid_template_columns": null,
+ "grid_template_rows": null,
+ "height": null,
+ "justify_content": null,
+ "justify_items": null,
+ "left": null,
+ "margin": null,
+ "max_height": null,
+ "max_width": null,
+ "min_height": null,
+ "min_width": null,
+ "object_fit": null,
+ "object_position": null,
+ "order": null,
+ "overflow": null,
+ "overflow_x": null,
+ "overflow_y": null,
+ "padding": null,
+ "right": null,
+ "top": null,
+ "visibility": null,
+ "width": null
+ }
+ },
+ "28adfb59f3e24851ba617e0087e5293f": {
+ "model_module": "@jupyter-widgets/controls",
+ "model_module_version": "1.5.0",
+ "model_name": "DescriptionStyleModel",
+ "state": {
+ "_model_module": "@jupyter-widgets/controls",
+ "_model_module_version": "1.5.0",
+ "_model_name": "DescriptionStyleModel",
+ "_view_count": null,
+ "_view_module": "@jupyter-widgets/base",
+ "_view_module_version": "1.2.0",
+ "_view_name": "StyleView",
+ "description_width": ""
+ }
+ },
+ "2bd5fe14d3d44d85be32c5c78f5b7568": {
+ "model_module": "@jupyter-widgets/controls",
+ "model_module_version": "1.5.0",
+ "model_name": "FloatProgressModel",
+ "state": {
+ "_dom_classes": [],
+ "_model_module": "@jupyter-widgets/controls",
+ "_model_module_version": "1.5.0",
+ "_model_name": "FloatProgressModel",
+ "_view_count": null,
+ "_view_module": "@jupyter-widgets/controls",
+ "_view_module_version": "1.5.0",
+ "_view_name": "ProgressView",
+ "bar_style": "",
+ "description": "",
+ "description_tooltip": null,
+ "layout": "IPY_MODEL_10ca522adeca4ba4bb2d4f9373a363f4",
+ "max": 500,
+ "min": 0,
+ "orientation": "horizontal",
+ "style": "IPY_MODEL_5f9f737d68234b9a9cd069105ab20ea2",
+ "value": 500
+ }
+ },
+ "2d69c8378b194aa3a8634848da5ee79a": {
+ "model_module": "@jupyter-widgets/controls",
+ "model_module_version": "1.5.0",
+ "model_name": "DescriptionStyleModel",
+ "state": {
+ "_model_module": "@jupyter-widgets/controls",
+ "_model_module_version": "1.5.0",
+ "_model_name": "DescriptionStyleModel",
+ "_view_count": null,
+ "_view_module": "@jupyter-widgets/base",
+ "_view_module_version": "1.2.0",
+ "_view_name": "StyleView",
+ "description_width": ""
+ }
+ },
+ "2ee59cbc091346168c3b4511f64b018a": {
+ "model_module": "@jupyter-widgets/controls",
+ "model_module_version": "1.5.0",
+ "model_name": "ProgressStyleModel",
+ "state": {
+ "_model_module": "@jupyter-widgets/controls",
+ "_model_module_version": "1.5.0",
+ "_model_name": "ProgressStyleModel",
+ "_view_count": null,
+ "_view_module": "@jupyter-widgets/base",
+ "_view_module_version": "1.2.0",
+ "_view_name": "StyleView",
+ "bar_color": null,
+ "description_width": ""
+ }
+ },
+ "35132c00838046bca253f31350a3190b": {
+ "model_module": "@jupyter-widgets/controls",
+ "model_module_version": "1.5.0",
+ "model_name": "FloatProgressModel",
+ "state": {
+ "_dom_classes": [],
+ "_model_module": "@jupyter-widgets/controls",
+ "_model_module_version": "1.5.0",
+ "_model_name": "FloatProgressModel",
+ "_view_count": null,
+ "_view_module": "@jupyter-widgets/controls",
+ "_view_module_version": "1.5.0",
+ "_view_name": "ProgressView",
+ "bar_style": "success",
+ "description": "",
+ "description_tooltip": null,
+ "layout": "IPY_MODEL_88ff8d07f6ce430db2772b8bf24294e8",
+ "max": 1,
+ "min": 0,
+ "orientation": "horizontal",
+ "style": "IPY_MODEL_f5e168c246714295a10a64fbb1ef2ee3",
+ "value": 0
+ }
+ },
+ "3627640fb7d14ac09c9fdcbfb2159152": {
+ "model_module": "@jupyter-widgets/controls",
+ "model_module_version": "1.5.0",
+ "model_name": "FloatProgressModel",
+ "state": {
+ "_dom_classes": [],
+ "_model_module": "@jupyter-widgets/controls",
+ "_model_module_version": "1.5.0",
+ "_model_name": "FloatProgressModel",
+ "_view_count": null,
+ "_view_module": "@jupyter-widgets/controls",
+ "_view_module_version": "1.5.0",
+ "_view_name": "ProgressView",
+ "bar_style": "",
+ "description": "",
+ "description_tooltip": null,
+ "layout": "IPY_MODEL_6e600a0c3f36439899b40cdc53ed7c8c",
+ "max": 500,
+ "min": 0,
+ "orientation": "horizontal",
+ "style": "IPY_MODEL_672ec8abe8b34b8dab6d4af20f0302c7",
+ "value": 500
+ }
+ },
+ "3647d22aaf934b53bf925dc22e2ce69a": {
+ "model_module": "@jupyter-widgets/controls",
+ "model_module_version": "1.5.0",
+ "model_name": "HTMLModel",
+ "state": {
+ "_dom_classes": [],
+ "_model_module": "@jupyter-widgets/controls",
+ "_model_module_version": "1.5.0",
+ "_model_name": "HTMLModel",
+ "_view_count": null,
+ "_view_module": "@jupyter-widgets/controls",
+ "_view_module_version": "1.5.0",
+ "_view_name": "HTMLView",
+ "description": "",
+ "description_tooltip": null,
+ "layout": "IPY_MODEL_89f757fbb1ef4fa8b557c04926871096",
+ "placeholder": "",
+ "style": "IPY_MODEL_94ebb17c331649fc812a66231532ee03",
+ "value": " 0/0 [00:04<?, ? file/s]"
+ }
+ },
+ "3948a575b3ee4a199f1055e93d8270ee": {
+ "model_module": "@jupyter-widgets/base",
+ "model_module_version": "1.2.0",
+ "model_name": "LayoutModel",
+ "state": {
+ "_model_module": "@jupyter-widgets/base",
+ "_model_module_version": "1.2.0",
+ "_model_name": "LayoutModel",
+ "_view_count": null,
+ "_view_module": "@jupyter-widgets/base",
+ "_view_module_version": "1.2.0",
+ "_view_name": "LayoutView",
+ "align_content": null,
+ "align_items": null,
+ "align_self": null,
+ "border": null,
+ "bottom": null,
+ "display": null,
+ "flex": null,
+ "flex_flow": null,
+ "grid_area": null,
+ "grid_auto_columns": null,
+ "grid_auto_flow": null,
+ "grid_auto_rows": null,
+ "grid_column": null,
+ "grid_gap": null,
+ "grid_row": null,
+ "grid_template_areas": null,
+ "grid_template_columns": null,
+ "grid_template_rows": null,
+ "height": null,
+ "justify_content": null,
+ "justify_items": null,
+ "left": null,
+ "margin": null,
+ "max_height": null,
+ "max_width": null,
+ "min_height": null,
+ "min_width": null,
+ "object_fit": null,
+ "object_position": null,
+ "order": null,
+ "overflow": null,
+ "overflow_x": null,
+ "overflow_y": null,
+ "padding": null,
+ "right": null,
+ "top": null,
+ "visibility": null,
+ "width": null
+ }
+ },
+ "3bda8233967642229fe778b4a8402f52": {
+ "model_module": "@jupyter-widgets/controls",
+ "model_module_version": "1.5.0",
+ "model_name": "DescriptionStyleModel",
+ "state": {
+ "_model_module": "@jupyter-widgets/controls",
+ "_model_module_version": "1.5.0",
+ "_model_name": "DescriptionStyleModel",
+ "_view_count": null,
+ "_view_module": "@jupyter-widgets/base",
+ "_view_module_version": "1.2.0",
+ "_view_name": "StyleView",
+ "description_width": ""
+ }
+ },
+ "3da4f3751cb54cf2938aea924ab33aa0": {
+ "model_module": "@jupyter-widgets/controls",
+ "model_module_version": "1.5.0",
+ "model_name": "ProgressStyleModel",
+ "state": {
+ "_model_module": "@jupyter-widgets/controls",
+ "_model_module_version": "1.5.0",
+ "_model_name": "ProgressStyleModel",
+ "_view_count": null,
+ "_view_module": "@jupyter-widgets/base",
+ "_view_module_version": "1.2.0",
+ "_view_name": "StyleView",
+ "bar_color": null,
+ "description_width": ""
+ }
+ },
+ "4334b4f6fc7048eab0d146b6ffc437d1": {
+ "model_module": "@jupyter-widgets/controls",
+ "model_module_version": "1.5.0",
+ "model_name": "HTMLModel",
+ "state": {
+ "_dom_classes": [],
+ "_model_module": "@jupyter-widgets/controls",
+ "_model_module_version": "1.5.0",
+ "_model_name": "HTMLModel",
+ "_view_count": null,
+ "_view_module": "@jupyter-widgets/controls",
+ "_view_module_version": "1.5.0",
+ "_view_name": "HTMLView",
+ "description": "",
+ "description_tooltip": null,
+ "layout": "IPY_MODEL_be8b3f4e73fc44f390e9a147df786e53",
+ "placeholder": "",
+ "style": "IPY_MODEL_3bda8233967642229fe778b4a8402f52",
+ "value": "Dl Completed...: 100%"
+ }
+ },
+ "463628ae5afc4c8f9a51f2c4ed038d12": {
+ "model_module": "@jupyter-widgets/base",
+ "model_module_version": "1.2.0",
+ "model_name": "LayoutModel",
+ "state": {
+ "_model_module": "@jupyter-widgets/base",
+ "_model_module_version": "1.2.0",
+ "_model_name": "LayoutModel",
+ "_view_count": null,
+ "_view_module": "@jupyter-widgets/base",
+ "_view_module_version": "1.2.0",
+ "_view_name": "LayoutView",
+ "align_content": null,
+ "align_items": null,
+ "align_self": null,
+ "border": null,
+ "bottom": null,
+ "display": null,
+ "flex": null,
+ "flex_flow": null,
+ "grid_area": null,
+ "grid_auto_columns": null,
+ "grid_auto_flow": null,
+ "grid_auto_rows": null,
+ "grid_column": null,
+ "grid_gap": null,
+ "grid_row": null,
+ "grid_template_areas": null,
+ "grid_template_columns": null,
+ "grid_template_rows": null,
+ "height": null,
+ "justify_content": null,
+ "justify_items": null,
+ "left": null,
+ "margin": null,
+ "max_height": null,
+ "max_width": null,
+ "min_height": null,
+ "min_width": null,
+ "object_fit": null,
+ "object_position": null,
+ "order": null,
+ "overflow": null,
+ "overflow_x": null,
+ "overflow_y": null,
+ "padding": null,
+ "right": null,
+ "top": null,
+ "visibility": null,
+ "width": null
+ }
+ },
+ "4cf6a31b3be945cbb82f8239e7e2642e": {
+ "model_module": "@jupyter-widgets/controls",
+ "model_module_version": "1.5.0",
+ "model_name": "HBoxModel",
+ "state": {
+ "_dom_classes": [],
+ "_model_module": "@jupyter-widgets/controls",
+ "_model_module_version": "1.5.0",
+ "_model_name": "HBoxModel",
+ "_view_count": null,
+ "_view_module": "@jupyter-widgets/controls",
+ "_view_module_version": "1.5.0",
+ "_view_name": "HBoxView",
+ "box_style": "",
+ "children": [
+ "IPY_MODEL_7fd82914b89343538aa08b36f4752f4b",
+ "IPY_MODEL_35132c00838046bca253f31350a3190b",
+ "IPY_MODEL_190a428569a74beb8b90aa278c629bd2"
],
- "source": [
- "# We'll use Steve Jobs' Stanford University commencement address as the example. Link: https://news.stanford.edu/2005/06/12/youve-got-find-love-jobs-says/\n",
- "\n",
- "!wget 'https://docs.google.com/uc?export=download&id=1f1INWOfJrHTFmbyF_0be5b4u_moz3a4F' -O steve-jobs-commencement.txt"
- ]
+ "layout": "IPY_MODEL_01545efb5b78418791ce0f8c763873ce"
+ }
},
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "id": "obv6NbUcNfB_"
- },
- "outputs": [],
- "source": [
- "loader = TextLoader(\"steve-jobs-commencement.txt\")\n",
- "documents = loader.load()\n",
- "text_splitter = RecursiveCharacterTextSplitter(chunk_size=500, chunk_overlap=0)\n",
- "texts = text_splitter.split_documents(documents)"
- ]
+ "533f3a17f8d1442490a2d6167d89550b": {
+ "model_module": "@jupyter-widgets/base",
+ "model_module_version": "1.2.0",
+ "model_name": "LayoutModel",
+ "state": {
+ "_model_module": "@jupyter-widgets/base",
+ "_model_module_version": "1.2.0",
+ "_model_name": "LayoutModel",
+ "_view_count": null,
+ "_view_module": "@jupyter-widgets/base",
+ "_view_module_version": "1.2.0",
+ "_view_name": "LayoutView",
+ "align_content": null,
+ "align_items": null,
+ "align_self": null,
+ "border": null,
+ "bottom": null,
+ "display": null,
+ "flex": null,
+ "flex_flow": null,
+ "grid_area": null,
+ "grid_auto_columns": null,
+ "grid_auto_flow": null,
+ "grid_auto_rows": null,
+ "grid_column": null,
+ "grid_gap": null,
+ "grid_row": null,
+ "grid_template_areas": null,
+ "grid_template_columns": null,
+ "grid_template_rows": null,
+ "height": null,
+ "justify_content": null,
+ "justify_items": null,
+ "left": null,
+ "margin": null,
+ "max_height": null,
+ "max_width": null,
+ "min_height": null,
+ "min_width": null,
+ "object_fit": null,
+ "object_position": null,
+ "order": null,
+ "overflow": null,
+ "overflow_x": null,
+ "overflow_y": null,
+ "padding": null,
+ "right": null,
+ "top": null,
+ "visibility": null,
+ "width": null
+ }
},
- {
- "attachments": {},
- "cell_type": "markdown",
- "metadata": {
- "id": "DHvoF9CB9apc"
- },
- "source": [
- "## Embed the passages and store them in an index"
- ]
+ "57cf77bba44141359639c0268ced8934": {
+ "model_module": "@jupyter-widgets/base",
+ "model_module_version": "1.2.0",
+ "model_name": "LayoutModel",
+ "state": {
+ "_model_module": "@jupyter-widgets/base",
+ "_model_module_version": "1.2.0",
+ "_model_name": "LayoutModel",
+ "_view_count": null,
+ "_view_module": "@jupyter-widgets/base",
+ "_view_module_version": "1.2.0",
+ "_view_name": "LayoutView",
+ "align_content": null,
+ "align_items": null,
+ "align_self": null,
+ "border": null,
+ "bottom": null,
+ "display": null,
+ "flex": null,
+ "flex_flow": null,
+ "grid_area": null,
+ "grid_auto_columns": null,
+ "grid_auto_flow": null,
+ "grid_auto_rows": null,
+ "grid_column": null,
+ "grid_gap": null,
+ "grid_row": null,
+ "grid_template_areas": null,
+ "grid_template_columns": null,
+ "grid_template_rows": null,
+ "height": null,
+ "justify_content": null,
+ "justify_items": null,
+ "left": null,
+ "margin": null,
+ "max_height": null,
+ "max_width": null,
+ "min_height": null,
+ "min_width": null,
+ "object_fit": null,
+ "object_position": null,
+ "order": null,
+ "overflow": null,
+ "overflow_x": null,
+ "overflow_y": null,
+ "padding": null,
+ "right": null,
+ "top": null,
+ "visibility": null,
+ "width": null
+ }
},
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "id": "11Bx0k2LGIH-"
- },
- "outputs": [],
- "source": [
- "embeddings = CohereEmbeddings(model = \"multilingual-22-12\")\n",
- "db = Qdrant.from_documents(texts, embeddings, location=\":memory:\", collection_name=\"my_documents\", distance_func=\"Dot\")"
- ]
+ "59f2ed0257d6422d84fa23898e5d9a10": {
+ "model_module": "@jupyter-widgets/controls",
+ "model_module_version": "1.5.0",
+ "model_name": "HTMLModel",
+ "state": {
+ "_dom_classes": [],
+ "_model_module": "@jupyter-widgets/controls",
+ "_model_module_version": "1.5.0",
+ "_model_name": "HTMLModel",
+ "_view_count": null,
+ "_view_module": "@jupyter-widgets/controls",
+ "_view_module_version": "1.5.0",
+ "_view_name": "HTMLView",
+ "description": "",
+ "description_tooltip": null,
+ "layout": "IPY_MODEL_533f3a17f8d1442490a2d6167d89550b",
+ "placeholder": "",
+ "style": "IPY_MODEL_ce82f835dd2b45a181a53407505b0cfc",
+ "value": "Extraction completed...: "
+ }
},
- {
- "attachments": {},
- "cell_type": "markdown",
- "metadata": {
- "id": "xQsrDUQo9poh"
- },
- "source": [
- "## Enter a question"
- ]
+ "5cf957caabe043139dc587e8651a28b1": {
+ "model_module": "@jupyter-widgets/base",
+ "model_module_version": "1.2.0",
+ "model_name": "LayoutModel",
+ "state": {
+ "_model_module": "@jupyter-widgets/base",
+ "_model_module_version": "1.2.0",
+ "_model_name": "LayoutModel",
+ "_view_count": null,
+ "_view_module": "@jupyter-widgets/base",
+ "_view_module_version": "1.2.0",
+ "_view_name": "LayoutView",
+ "align_content": null,
+ "align_items": null,
+ "align_self": null,
+ "border": null,
+ "bottom": null,
+ "display": null,
+ "flex": null,
+ "flex_flow": null,
+ "grid_area": null,
+ "grid_auto_columns": null,
+ "grid_auto_flow": null,
+ "grid_auto_rows": null,
+ "grid_column": null,
+ "grid_gap": null,
+ "grid_row": null,
+ "grid_template_areas": null,
+ "grid_template_columns": null,
+ "grid_template_rows": null,
+ "height": null,
+ "justify_content": null,
+ "justify_items": null,
+ "left": null,
+ "margin": null,
+ "max_height": null,
+ "max_width": null,
+ "min_height": null,
+ "min_width": null,
+ "object_fit": null,
+ "object_position": null,
+ "order": null,
+ "overflow": null,
+ "overflow_x": null,
+ "overflow_y": null,
+ "padding": null,
+ "right": null,
+ "top": null,
+ "visibility": null,
+ "width": null
+ }
},
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "id": "SW90c8kOqdow"
- },
- "outputs": [],
- "source": [
- "questions = [\n",
- " \"What did the author liken The Whole Earth Catalog to?\",\n",
- " \"What was Reed College great at?\",\n",
- " \"What was the author diagnosed with?\",\n",
- " \"What is the key lesson from this article?\",\n",
- " \"What did the article say about Michael Jackson?\",\n",
- " ]"
- ]
+ "5ed53479e65749a68e2ae1f70fff1db0": {
+ "model_module": "@jupyter-widgets/controls",
+ "model_module_version": "1.5.0",
+ "model_name": "DescriptionStyleModel",
+ "state": {
+ "_model_module": "@jupyter-widgets/controls",
+ "_model_module_version": "1.5.0",
+ "_model_name": "DescriptionStyleModel",
+ "_view_count": null,
+ "_view_module": "@jupyter-widgets/base",
+ "_view_module_version": "1.2.0",
+ "_view_name": "StyleView",
+ "description_width": ""
+ }
},
- {
- "attachments": {},
- "cell_type": "markdown",
- "metadata": {
- "id": "HXhkK22JvNa1"
- },
- "source": [
- "## Answer the question based on the most relevant documents\n"
- ]
+ "5edafe62c55449709c304c11dc847e82": {
+ "model_module": "@jupyter-widgets/base",
+ "model_module_version": "1.2.0",
+ "model_name": "LayoutModel",
+ "state": {
+ "_model_module": "@jupyter-widgets/base",
+ "_model_module_version": "1.2.0",
+ "_model_name": "LayoutModel",
+ "_view_count": null,
+ "_view_module": "@jupyter-widgets/base",
+ "_view_module_version": "1.2.0",
+ "_view_name": "LayoutView",
+ "align_content": null,
+ "align_items": null,
+ "align_self": null,
+ "border": null,
+ "bottom": null,
+ "display": null,
+ "flex": null,
+ "flex_flow": null,
+ "grid_area": null,
+ "grid_auto_columns": null,
+ "grid_auto_flow": null,
+ "grid_auto_rows": null,
+ "grid_column": null,
+ "grid_gap": null,
+ "grid_row": null,
+ "grid_template_areas": null,
+ "grid_template_columns": null,
+ "grid_template_rows": null,
+ "height": null,
+ "justify_content": null,
+ "justify_items": null,
+ "left": null,
+ "margin": null,
+ "max_height": null,
+ "max_width": null,
+ "min_height": null,
+ "min_width": null,
+ "object_fit": null,
+ "object_position": null,
+ "order": null,
+ "overflow": null,
+ "overflow_x": null,
+ "overflow_y": null,
+ "padding": null,
+ "right": null,
+ "top": null,
+ "visibility": null,
+ "width": null
+ }
},
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "id": "-dgBvF5HI9Wd"
- },
- "outputs": [],
- "source": [
- "# Create our own prompt template\n",
- "\n",
- "prompt_template = \"\"\"Text: {context}\n",
- "\n",
- "Question: {question}\n",
- "\n",
- "Answer the question based on the text provided. If the text doesn't contain the answer, reply that the answer is not available.\"\"\"\n",
- "\n",
- "PROMPT = PromptTemplate(\n",
- " template=prompt_template, input_variables=[\"context\", \"question\"]\n",
- ")"
- ]
+ "5f9f737d68234b9a9cd069105ab20ea2": {
+ "model_module": "@jupyter-widgets/controls",
+ "model_module_version": "1.5.0",
+ "model_name": "ProgressStyleModel",
+ "state": {
+ "_model_module": "@jupyter-widgets/controls",
+ "_model_module_version": "1.5.0",
+ "_model_name": "ProgressStyleModel",
+ "_view_count": null,
+ "_view_module": "@jupyter-widgets/base",
+ "_view_module_version": "1.2.0",
+ "_view_name": "StyleView",
+ "bar_color": null,
+ "description_width": ""
+ }
},
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "colab": {
- "base_uri": "https://localhost:8080/"
- },
- "id": "8wa_YjzpcEAE",
- "outputId": "31120ad3-034a-42f1-96a3-7454cd68ade5"
- },
- "outputs": [
- {
- "name": "stdout",
- "output_type": "stream",
- "text": [
- "------------------------------------------------------------------------------------------------------------------------------------------------------ \n",
- "\n",
- "Question: What did the author liken The Whole Earth Catalog to?\n",
- "Answer: It was sort of like Google in paperback form, 35 years before Google came along\n",
- "\n",
- "Sources:\n",
- "1: When I was young, there was an amazing publication called The Whole Earth Catalog, which was one of the bibles of my generation. It was created by a\n",
- "fellow named Stewart Brand not far from here in Menlo Park, and he brought it to life with his poetic touch. This was in the late 1960s, before\n",
- "personal computers and desktop publishing, so it was all made with typewriters, scissors and Polaroid cameras. It was sort of like Google in paperback\n",
- "form, 35 years before Google came along: It was\n",
- "2: Stewart and his team put out several issues of The Whole Earth Catalog, and then when it had run its course, they put out a final issue. It was the\n",
- "mid-1970s, and I was your age. On the back cover of their final issue was a photograph of an early morning country road, the kind you might find\n",
- "yourself hitchhiking on if you were so adventurous. Beneath it were the words: “Stay Hungry. Stay Foolish.” It was their farewell message as they\n",
- "signed off. Stay Hungry. Stay Foolish. And I have always\n",
- "3: idealistic, and overflowing with neat tools and great notions.\n",
- "4: beautiful, historical, artistically subtle in a way that science can’t capture, and I found it fascinating.\n",
- "------------------------------------------------------------------------------------------------------------------------------------------------------ \n",
- "\n",
- "Question: What was Reed College great at?\n",
- "Answer: Reed College was great at calligraphy instruction.\n",
- "\n",
- "Sources:\n",
- "1: Reed College at that time offered perhaps the best calligraphy instruction in the country. Throughout the campus every poster, every label on every\n",
- "drawer, was beautifully hand calligraphed. Because I had dropped out and didn’t have to take the normal classes, I decided to take a calligraphy class\n",
- "to learn how to do this. I learned about serif and sans serif typefaces, about varying the amount of space between different letter combinations,\n",
- "about what makes great typography great. It was\n",
- "2: I dropped out of Reed College after the first 6 months, but then stayed around as a drop-in for another 18 months or so before I really quit. So why\n",
- "did I drop out?\n",
- "3: never dropped out, I would have never dropped in on this calligraphy class, and personal computers might not have the wonderful typography that they\n",
- "do. Of course it was impossible to connect the dots looking forward when I was in college. But it was very, very clear looking backward 10 years\n",
- "later.\n",
- "4: OK. It was pretty scary at the time, but looking back it was one of the best decisions I ever made. The minute I dropped out I could stop taking the\n",
- "required classes that didn’t interest me, and begin dropping in on the ones that looked interesting.\n",
- "------------------------------------------------------------------------------------------------------------------------------------------------------ \n",
- "\n",
- "Question: What was the author diagnosed with?\n",
- "Answer: The author was diagnosed with cancer.\n",
- "\n",
- "Sources:\n",
- "1: I lived with that diagnosis all day. Later that evening I had a biopsy, where they stuck an endoscope down my throat, through my stomach and into my\n",
- "intestines, put a needle into my pancreas and got a few cells from the tumor. I was sedated, but my wife, who was there, told me that when they viewed\n",
- "the cells under a microscope the doctors started crying because it turned out to be a very rare form of pancreatic cancer that is curable with\n",
- "surgery. I had the surgery and I’m fine now.\n",
- "2: About a year ago I was diagnosed with cancer. I had a scan at 7:30 in the morning, and it clearly showed a tumor on my pancreas. I didn’t even know\n",
- "what a pancreas was. The doctors told me this was almost certainly a type of cancer that is incurable, and that I should expect to live no longer than\n",
- "three to six months. My doctor advised me to go home and get my affairs in order, which is doctor’s code for prepare to die. It means to try to tell\n",
- "your kids everything you thought you’d have the\n",
- "3: Stewart and his team put out several issues of The Whole Earth Catalog, and then when it had run its course, they put out a final issue. It was the\n",
- "mid-1970s, and I was your age. On the back cover of their final issue was a photograph of an early morning country road, the kind you might find\n",
- "yourself hitchhiking on if you were so adventurous. Beneath it were the words: “Stay Hungry. Stay Foolish.” It was their farewell message as they\n",
- "signed off. Stay Hungry. Stay Foolish. And I have always\n",
- "4: beautiful, historical, artistically subtle in a way that science can’t capture, and I found it fascinating.\n",
- "------------------------------------------------------------------------------------------------------------------------------------------------------ \n",
- "\n",
- "Question: What is the key lesson from this article?\n",
- "Answer: The key lesson from this article is that you have to trust that the dots will somehow connect in your future. You have to trust in something -- your gut, destiny, life, karma, whatever. This approach has never let me down, and it has made all the difference in my life.\n",
- "\n",
- "Sources:\n",
- "1: Again, you can’t connect the dots looking forward; you can only connect them looking backward. So you have to trust that the dots will somehow connect\n",
- "in your future. You have to trust in something — your gut, destiny, life, karma, whatever. This approach has never let me down, and it has made all\n",
- "the difference in my life. My second story is about love and loss.\n",
- "2: Remembering that I’ll be dead soon is the most important tool I’ve ever encountered to help me make the big choices in life. Because almost everything\n",
- "— all external expectations, all pride, all fear of embarrassment or failure — these things just fall away in the face of death, leaving only what is\n",
- "truly important. Remembering that you are going to die is the best way I know to avoid the trap of thinking you have something to lose. You are\n",
- "already naked. There is no reason not to follow your\n",
- "3: Your time is limited, so don’t waste it living someone else’s life. Don’t be trapped by dogma — which is living with the results of other people’s\n",
- "thinking. Don’t let the noise of others’ opinions drown out your own inner voice. And most important, have the courage to follow your heart and\n",
- "intuition. They somehow already know what you truly want to become. Everything else is secondary.\n",
- "4: I really didn’t know what to do for a few months. I felt that I had let the previous generation of entrepreneurs down — that I had dropped the baton\n",
- "as it was being passed to me. I met with David Packard and Bob Noyce and tried to apologize for screwing up so badly. I was a very public failure, and\n",
- "I even thought about running away from the valley. But something slowly began to dawn on me — I still loved what I did. The turn of events at Apple\n",
- "had not changed that one bit. I had been rejected,\n",
- "------------------------------------------------------------------------------------------------------------------------------------------------------ \n",
- "\n",
- "Question: What did the article say about Michael Jackson?\n",
- "Answer: The text did not provide information about Michael Jackson.\n",
- "\n",
- "Sources:\n",
- "1: baby boy; do you want him?” They said: “Of course.” My biological mother later found out that my mother had never graduated from college and that my\n",
- "father had never graduated from high school. She refused to sign the final adoption papers. She only relented a few months later when my parents\n",
- "promised that I would someday go to college.\n",
- "2: beautiful, historical, artistically subtle in a way that science can’t capture, and I found it fascinating.\n",
- "3: I really didn’t know what to do for a few months. I felt that I had let the previous generation of entrepreneurs down — that I had dropped the baton\n",
- "as it was being passed to me. I met with David Packard and Bob Noyce and tried to apologize for screwing up so badly. I was a very public failure, and\n",
- "I even thought about running away from the valley. But something slowly began to dawn on me — I still loved what I did. The turn of events at Apple\n",
- "had not changed that one bit. I had been rejected,\n",
- "4: This was the closest I’ve been to facing death, and I hope it’s the closest I get for a few more decades. Having lived through it, I can now say this\n",
- "to you with a bit more certainty than when death was a useful but purely intellectual concept:\n"
- ]
- }
+ "62eb05321e2d4265b033544921f92053": {
+ "model_module": "@jupyter-widgets/controls",
+ "model_module_version": "1.5.0",
+ "model_name": "HTMLModel",
+ "state": {
+ "_dom_classes": [],
+ "_model_module": "@jupyter-widgets/controls",
+ "_model_module_version": "1.5.0",
+ "_model_name": "HTMLModel",
+ "_view_count": null,
+ "_view_module": "@jupyter-widgets/controls",
+ "_view_module_version": "1.5.0",
+ "_view_name": "HTMLView",
+ "description": "",
+ "description_tooltip": null,
+ "layout": "IPY_MODEL_463628ae5afc4c8f9a51f2c4ed038d12",
+ "placeholder": "",
+ "style": "IPY_MODEL_1ac7e3656e43480c8af4038d2e7d5432",
+ "value": "Shuffling /root/tensorflow_datasets/trec/1.0.0.incompleteWOR5EP/trec-test.tfrecord*...: 0%"
+ }
+ },
+ "65b7f23304a243878c43ea391885ee85": {
+ "model_module": "@jupyter-widgets/controls",
+ "model_module_version": "1.5.0",
+ "model_name": "HTMLModel",
+ "state": {
+ "_dom_classes": [],
+ "_model_module": "@jupyter-widgets/controls",
+ "_model_module_version": "1.5.0",
+ "_model_name": "HTMLModel",
+ "_view_count": null,
+ "_view_module": "@jupyter-widgets/controls",
+ "_view_module_version": "1.5.0",
+ "_view_name": "HTMLView",
+ "description": "",
+ "description_tooltip": null,
+ "layout": "IPY_MODEL_c5296022707240b393be4146e983083c",
+ "placeholder": "",
+ "style": "IPY_MODEL_8653f9b6a25446cc8d7d34ffd4e48ca8",
+ "value": " 1/2 [00:00<00:00, 2.06 splits/s]"
+ }
+ },
+ "672ec8abe8b34b8dab6d4af20f0302c7": {
+ "model_module": "@jupyter-widgets/controls",
+ "model_module_version": "1.5.0",
+ "model_name": "ProgressStyleModel",
+ "state": {
+ "_model_module": "@jupyter-widgets/controls",
+ "_model_module_version": "1.5.0",
+ "_model_name": "ProgressStyleModel",
+ "_view_count": null,
+ "_view_module": "@jupyter-widgets/base",
+ "_view_module_version": "1.2.0",
+ "_view_name": "StyleView",
+ "bar_color": null,
+ "description_width": ""
+ }
+ },
+ "6850b87ff2fa49eb9fbfb3c3492efbb3": {
+ "model_module": "@jupyter-widgets/controls",
+ "model_module_version": "1.5.0",
+ "model_name": "HTMLModel",
+ "state": {
+ "_dom_classes": [],
+ "_model_module": "@jupyter-widgets/controls",
+ "_model_module_version": "1.5.0",
+ "_model_name": "HTMLModel",
+ "_view_count": null,
+ "_view_module": "@jupyter-widgets/controls",
+ "_view_module_version": "1.5.0",
+ "_view_name": "HTMLView",
+ "description": "",
+ "description_tooltip": null,
+ "layout": "IPY_MODEL_d298dc9202dc49aba09b66969b3d19f9",
+ "placeholder": "",
+ "style": "IPY_MODEL_6e09dadf883440038a7a47f0d9723ae6",
+ "value": " 0/500 [00:00<?, ? examples/s]"
+ }
+ },
+ "687cfefc80a54edd84eb22baf805c905": {
+ "model_module": "@jupyter-widgets/base",
+ "model_module_version": "1.2.0",
+ "model_name": "LayoutModel",
+ "state": {
+ "_model_module": "@jupyter-widgets/base",
+ "_model_module_version": "1.2.0",
+ "_model_name": "LayoutModel",
+ "_view_count": null,
+ "_view_module": "@jupyter-widgets/base",
+ "_view_module_version": "1.2.0",
+ "_view_name": "LayoutView",
+ "align_content": null,
+ "align_items": null,
+ "align_self": null,
+ "border": null,
+ "bottom": null,
+ "display": null,
+ "flex": null,
+ "flex_flow": null,
+ "grid_area": null,
+ "grid_auto_columns": null,
+ "grid_auto_flow": null,
+ "grid_auto_rows": null,
+ "grid_column": null,
+ "grid_gap": null,
+ "grid_row": null,
+ "grid_template_areas": null,
+ "grid_template_columns": null,
+ "grid_template_rows": null,
+ "height": null,
+ "justify_content": null,
+ "justify_items": null,
+ "left": null,
+ "margin": null,
+ "max_height": null,
+ "max_width": null,
+ "min_height": null,
+ "min_width": null,
+ "object_fit": null,
+ "object_position": null,
+ "order": null,
+ "overflow": null,
+ "overflow_x": null,
+ "overflow_y": null,
+ "padding": null,
+ "right": null,
+ "top": null,
+ "visibility": null,
+ "width": null
+ }
+ },
+ "68bf09957e83431b8d299f1ab15d5f7e": {
+ "model_module": "@jupyter-widgets/base",
+ "model_module_version": "1.2.0",
+ "model_name": "LayoutModel",
+ "state": {
+ "_model_module": "@jupyter-widgets/base",
+ "_model_module_version": "1.2.0",
+ "_model_name": "LayoutModel",
+ "_view_count": null,
+ "_view_module": "@jupyter-widgets/base",
+ "_view_module_version": "1.2.0",
+ "_view_name": "LayoutView",
+ "align_content": null,
+ "align_items": null,
+ "align_self": null,
+ "border": null,
+ "bottom": null,
+ "display": null,
+ "flex": null,
+ "flex_flow": null,
+ "grid_area": null,
+ "grid_auto_columns": null,
+ "grid_auto_flow": null,
+ "grid_auto_rows": null,
+ "grid_column": null,
+ "grid_gap": null,
+ "grid_row": null,
+ "grid_template_areas": null,
+ "grid_template_columns": null,
+ "grid_template_rows": null,
+ "height": null,
+ "justify_content": null,
+ "justify_items": null,
+ "left": null,
+ "margin": null,
+ "max_height": null,
+ "max_width": null,
+ "min_height": null,
+ "min_width": null,
+ "object_fit": null,
+ "object_position": null,
+ "order": null,
+ "overflow": null,
+ "overflow_x": null,
+ "overflow_y": null,
+ "padding": null,
+ "right": null,
+ "top": null,
+ "visibility": null,
+ "width": null
+ }
+ },
+ "6a7d1421176f4b62af003a4b2f18815b": {
+ "model_module": "@jupyter-widgets/controls",
+ "model_module_version": "1.5.0",
+ "model_name": "DescriptionStyleModel",
+ "state": {
+ "_model_module": "@jupyter-widgets/controls",
+ "_model_module_version": "1.5.0",
+ "_model_name": "DescriptionStyleModel",
+ "_view_count": null,
+ "_view_module": "@jupyter-widgets/base",
+ "_view_module_version": "1.2.0",
+ "_view_name": "StyleView",
+ "description_width": ""
+ }
+ },
+ "6c0193f68df143e0a7907c6a577b07a6": {
+ "model_module": "@jupyter-widgets/controls",
+ "model_module_version": "1.5.0",
+ "model_name": "DescriptionStyleModel",
+ "state": {
+ "_model_module": "@jupyter-widgets/controls",
+ "_model_module_version": "1.5.0",
+ "_model_name": "DescriptionStyleModel",
+ "_view_count": null,
+ "_view_module": "@jupyter-widgets/base",
+ "_view_module_version": "1.2.0",
+ "_view_name": "StyleView",
+ "description_width": ""
+ }
+ },
+ "6d1ea14b22034d2d9ab16aa5a24566dc": {
+ "model_module": "@jupyter-widgets/base",
+ "model_module_version": "1.2.0",
+ "model_name": "LayoutModel",
+ "state": {
+ "_model_module": "@jupyter-widgets/base",
+ "_model_module_version": "1.2.0",
+ "_model_name": "LayoutModel",
+ "_view_count": null,
+ "_view_module": "@jupyter-widgets/base",
+ "_view_module_version": "1.2.0",
+ "_view_name": "LayoutView",
+ "align_content": null,
+ "align_items": null,
+ "align_self": null,
+ "border": null,
+ "bottom": null,
+ "display": null,
+ "flex": null,
+ "flex_flow": null,
+ "grid_area": null,
+ "grid_auto_columns": null,
+ "grid_auto_flow": null,
+ "grid_auto_rows": null,
+ "grid_column": null,
+ "grid_gap": null,
+ "grid_row": null,
+ "grid_template_areas": null,
+ "grid_template_columns": null,
+ "grid_template_rows": null,
+ "height": null,
+ "justify_content": null,
+ "justify_items": null,
+ "left": null,
+ "margin": null,
+ "max_height": null,
+ "max_width": null,
+ "min_height": null,
+ "min_width": null,
+ "object_fit": null,
+ "object_position": null,
+ "order": null,
+ "overflow": null,
+ "overflow_x": null,
+ "overflow_y": null,
+ "padding": null,
+ "right": null,
+ "top": null,
+ "visibility": null,
+ "width": null
+ }
+ },
+ "6def2ecb657845a3bae18bad85950991": {
+ "model_module": "@jupyter-widgets/base",
+ "model_module_version": "1.2.0",
+ "model_name": "LayoutModel",
+ "state": {
+ "_model_module": "@jupyter-widgets/base",
+ "_model_module_version": "1.2.0",
+ "_model_name": "LayoutModel",
+ "_view_count": null,
+ "_view_module": "@jupyter-widgets/base",
+ "_view_module_version": "1.2.0",
+ "_view_name": "LayoutView",
+ "align_content": null,
+ "align_items": null,
+ "align_self": null,
+ "border": null,
+ "bottom": null,
+ "display": null,
+ "flex": null,
+ "flex_flow": null,
+ "grid_area": null,
+ "grid_auto_columns": null,
+ "grid_auto_flow": null,
+ "grid_auto_rows": null,
+ "grid_column": null,
+ "grid_gap": null,
+ "grid_row": null,
+ "grid_template_areas": null,
+ "grid_template_columns": null,
+ "grid_template_rows": null,
+ "height": null,
+ "justify_content": null,
+ "justify_items": null,
+ "left": null,
+ "margin": null,
+ "max_height": null,
+ "max_width": null,
+ "min_height": null,
+ "min_width": null,
+ "object_fit": null,
+ "object_position": null,
+ "order": null,
+ "overflow": null,
+ "overflow_x": null,
+ "overflow_y": null,
+ "padding": null,
+ "right": null,
+ "top": null,
+ "visibility": "hidden",
+ "width": null
+ }
+ },
+ "6e09dadf883440038a7a47f0d9723ae6": {
+ "model_module": "@jupyter-widgets/controls",
+ "model_module_version": "1.5.0",
+ "model_name": "DescriptionStyleModel",
+ "state": {
+ "_model_module": "@jupyter-widgets/controls",
+ "_model_module_version": "1.5.0",
+ "_model_name": "DescriptionStyleModel",
+ "_view_count": null,
+ "_view_module": "@jupyter-widgets/base",
+ "_view_module_version": "1.2.0",
+ "_view_name": "StyleView",
+ "description_width": ""
+ }
+ },
+ "6e600a0c3f36439899b40cdc53ed7c8c": {
+ "model_module": "@jupyter-widgets/base",
+ "model_module_version": "1.2.0",
+ "model_name": "LayoutModel",
+ "state": {
+ "_model_module": "@jupyter-widgets/base",
+ "_model_module_version": "1.2.0",
+ "_model_name": "LayoutModel",
+ "_view_count": null,
+ "_view_module": "@jupyter-widgets/base",
+ "_view_module_version": "1.2.0",
+ "_view_name": "LayoutView",
+ "align_content": null,
+ "align_items": null,
+ "align_self": null,
+ "border": null,
+ "bottom": null,
+ "display": null,
+ "flex": null,
+ "flex_flow": null,
+ "grid_area": null,
+ "grid_auto_columns": null,
+ "grid_auto_flow": null,
+ "grid_auto_rows": null,
+ "grid_column": null,
+ "grid_gap": null,
+ "grid_row": null,
+ "grid_template_areas": null,
+ "grid_template_columns": null,
+ "grid_template_rows": null,
+ "height": null,
+ "justify_content": null,
+ "justify_items": null,
+ "left": null,
+ "margin": null,
+ "max_height": null,
+ "max_width": null,
+ "min_height": null,
+ "min_width": null,
+ "object_fit": null,
+ "object_position": null,
+ "order": null,
+ "overflow": null,
+ "overflow_x": null,
+ "overflow_y": null,
+ "padding": null,
+ "right": null,
+ "top": null,
+ "visibility": null,
+ "width": null
+ }
+ },
+ "735bfe1c1ff6402d826257313d68b21b": {
+ "model_module": "@jupyter-widgets/controls",
+ "model_module_version": "1.5.0",
+ "model_name": "HTMLModel",
+ "state": {
+ "_dom_classes": [],
+ "_model_module": "@jupyter-widgets/controls",
+ "_model_module_version": "1.5.0",
+ "_model_name": "HTMLModel",
+ "_view_count": null,
+ "_view_module": "@jupyter-widgets/controls",
+ "_view_module_version": "1.5.0",
+ "_view_name": "HTMLView",
+ "description": "",
+ "description_tooltip": null,
+ "layout": "IPY_MODEL_5edafe62c55449709c304c11dc847e82",
+ "placeholder": "",
+ "style": "IPY_MODEL_5ed53479e65749a68e2ae1f70fff1db0",
+ "value": "Shuffling /root/tensorflow_datasets/trec/1.0.0.incompleteWOR5EP/trec-train.tfrecord*...: 0%"
+ }
+ },
+ "771c7097b5be45ffac31e70117376903": {
+ "model_module": "@jupyter-widgets/controls",
+ "model_module_version": "1.5.0",
+ "model_name": "HTMLModel",
+ "state": {
+ "_dom_classes": [],
+ "_model_module": "@jupyter-widgets/controls",
+ "_model_module_version": "1.5.0",
+ "_model_name": "HTMLModel",
+ "_view_count": null,
+ "_view_module": "@jupyter-widgets/controls",
+ "_view_module_version": "1.5.0",
+ "_view_name": "HTMLView",
+ "description": "",
+ "description_tooltip": null,
+ "layout": "IPY_MODEL_d9dd5b64f0ee40359a4c577f4358248f",
+ "placeholder": "",
+ "style": "IPY_MODEL_baf658cc369745b7bad7f18302e385d9",
+ "value": " 0/5452 [00:00<?, ? examples/s]"
+ }
+ },
+ "7c8e1e4eee714bc086c3f84eee5941fc": {
+ "model_module": "@jupyter-widgets/controls",
+ "model_module_version": "1.5.0",
+ "model_name": "HBoxModel",
+ "state": {
+ "_dom_classes": [],
+ "_model_module": "@jupyter-widgets/controls",
+ "_model_module_version": "1.5.0",
+ "_model_name": "HBoxModel",
+ "_view_count": null,
+ "_view_module": "@jupyter-widgets/controls",
+ "_view_module_version": "1.5.0",
+ "_view_name": "HBoxView",
+ "box_style": "",
+ "children": [
+ "IPY_MODEL_4334b4f6fc7048eab0d146b6ffc437d1",
+ "IPY_MODEL_c8e13e321a9a4e0b88c717a1a8065619",
+ "IPY_MODEL_c1f14f69e97f44598d9c4f8d789131fb"
],
- "source": [
- "chain_type_kwargs = {\"prompt\": PROMPT}\n",
- "\n",
- "qa = RetrievalQA.from_chain_type(llm=Cohere(model=\"command\", temperature=0), \n",
- " chain_type=\"stuff\", \n",
- " retriever=db.as_retriever(), \n",
- " chain_type_kwargs=chain_type_kwargs, \n",
- " return_source_documents=True)\n",
- "\n",
- "for question in questions:\n",
- " answer = qa({\"query\": question})\n",
- " result = answer[\"result\"].replace(\"\\n\",\"\").replace(\"Answer:\",\"\")\n",
- " sources = answer['source_documents']\n",
- " print(\"-\"*150,\"\\n\")\n",
- " print(f\"Question: {question}\")\n",
- " print(f\"Answer: {result}\")\n",
- "\n",
- " ### COMMENT OUT THE 4 LINES BELOW TO HIDE THE SOURCES\n",
- " print(f\"\\nSources:\")\n",
- " for idx, source in enumerate(sources):\n",
- " source_wrapped = tr.fill(str(source.page_content), width=150)\n",
- " print(f\"{idx+1}: {source_wrapped}\")"
- ]
+ "layout": "IPY_MODEL_25a079a3d58644f4a5306ee2cb42dbde"
+ }
},
- {
- "attachments": {},
- "cell_type": "markdown",
- "metadata": {
- "id": "bX77dCbuCxCu"
- },
- "source": [
- "## Questions in French"
- ]
+ "7fa8f3f5c00c493bb7283bc90c33df5e": {
+ "model_module": "@jupyter-widgets/controls",
+ "model_module_version": "1.5.0",
+ "model_name": "HBoxModel",
+ "state": {
+ "_dom_classes": [],
+ "_model_module": "@jupyter-widgets/controls",
+ "_model_module_version": "1.5.0",
+ "_model_name": "HBoxModel",
+ "_view_count": null,
+ "_view_module": "@jupyter-widgets/controls",
+ "_view_module_version": "1.5.0",
+ "_view_name": "HBoxView",
+ "box_style": "",
+ "children": [
+ "IPY_MODEL_59f2ed0257d6422d84fa23898e5d9a10",
+ "IPY_MODEL_9236baef198c44ae8607c7888e899bc8",
+ "IPY_MODEL_3647d22aaf934b53bf925dc22e2ce69a"
+ ],
+ "layout": "IPY_MODEL_687cfefc80a54edd84eb22baf805c905"
+ }
},
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "id": "rg29qxkPCdJL"
- },
- "outputs": [],
- "source": [
- "questions_fr = [\n",
- " \"À quoi se compare The Whole Earth Catalog ?\",\n",
- " \"Dans quoi Reed College était-il excellent ?\",\n",
- " \"De quoi l'auteur a-t-il été diagnostiqué ?\",\n",
- " \"Quelle est la leçon clé de cet article ?\",\n",
- " \"Que disait l'article sur Michael Jackson ?\",\n",
- " ]"
- ]
+ "7fd82914b89343538aa08b36f4752f4b": {
+ "model_module": "@jupyter-widgets/controls",
+ "model_module_version": "1.5.0",
+ "model_name": "HTMLModel",
+ "state": {
+ "_dom_classes": [],
+ "_model_module": "@jupyter-widgets/controls",
+ "_model_module_version": "1.5.0",
+ "_model_name": "HTMLModel",
+ "_view_count": null,
+ "_view_module": "@jupyter-widgets/controls",
+ "_view_module_version": "1.5.0",
+ "_view_name": "HTMLView",
+ "description": "",
+ "description_tooltip": null,
+ "layout": "IPY_MODEL_d426d4b5b86c4582a17bf21fe5f021d4",
+ "placeholder": "",
+ "style": "IPY_MODEL_2d69c8378b194aa3a8634848da5ee79a",
+ "value": "Dl Size...: "
+ }
},
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "id": "EFuLtdgy5aZw"
- },
- "outputs": [],
- "source": [
- "# import langchain\n",
- "# langchain.debug = False"
- ]
+ "85f4ab947d774115baf64332f64248bd": {
+ "model_module": "@jupyter-widgets/base",
+ "model_module_version": "1.2.0",
+ "model_name": "LayoutModel",
+ "state": {
+ "_model_module": "@jupyter-widgets/base",
+ "_model_module_version": "1.2.0",
+ "_model_name": "LayoutModel",
+ "_view_count": null,
+ "_view_module": "@jupyter-widgets/base",
+ "_view_module_version": "1.2.0",
+ "_view_name": "LayoutView",
+ "align_content": null,
+ "align_items": null,
+ "align_self": null,
+ "border": null,
+ "bottom": null,
+ "display": null,
+ "flex": null,
+ "flex_flow": null,
+ "grid_area": null,
+ "grid_auto_columns": null,
+ "grid_auto_flow": null,
+ "grid_auto_rows": null,
+ "grid_column": null,
+ "grid_gap": null,
+ "grid_row": null,
+ "grid_template_areas": null,
+ "grid_template_columns": null,
+ "grid_template_rows": null,
+ "height": null,
+ "justify_content": null,
+ "justify_items": null,
+ "left": null,
+ "margin": null,
+ "max_height": null,
+ "max_width": null,
+ "min_height": null,
+ "min_width": null,
+ "object_fit": null,
+ "object_position": null,
+ "order": null,
+ "overflow": null,
+ "overflow_x": null,
+ "overflow_y": null,
+ "padding": null,
+ "right": null,
+ "top": null,
+ "visibility": "hidden",
+ "width": null
+ }
},
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "colab": {
- "base_uri": "https://localhost:8080/"
- },
- "id": "La7K_N7UTRdb",
- "outputId": "8931bf1c-afe6-4b80-8979-841456e79a01"
- },
- "outputs": [
- {
- "name": "stdout",
- "output_type": "stream",
- "text": [
- "-------------------- \n",
- "\n",
- "Question: À quoi se compare The Whole Earth Catalog ?\n",
- "Answer: The Whole Earth Catalog was like Google in paperback form, 35 years before Google came along.\n",
- "-------------------- \n",
- "\n",
- "Question: Dans quoi Reed College était-il excellent ?\n",
- "Answer: Reed College offered the best calligraphy instruction in the country.\n",
- "-------------------- \n",
- "\n",
- "Question: De quoi l'auteur a-t-il été diagnostiqué ?\n",
- "Answer: The author was diagnosed with a very rare form of pancreatic cancer that is curable with surgery.\n",
- "-------------------- \n",
- "\n",
- "Question: Quelle est la leçon clé de cet article ?\n",
- "Answer: The key lesson of this article is that remembering that you will die soon is the most important tool to help one make the big choices in life.\n",
- "-------------------- \n",
- "\n",
- "Question: Que disait l'article sur Michael Jackson ?\n",
- "Answer: The text does not contain the answer to the question.\n"
- ]
- }
+ "8653f9b6a25446cc8d7d34ffd4e48ca8": {
+ "model_module": "@jupyter-widgets/controls",
+ "model_module_version": "1.5.0",
+ "model_name": "DescriptionStyleModel",
+ "state": {
+ "_model_module": "@jupyter-widgets/controls",
+ "_model_module_version": "1.5.0",
+ "_model_name": "DescriptionStyleModel",
+ "_view_count": null,
+ "_view_module": "@jupyter-widgets/base",
+ "_view_module_version": "1.2.0",
+ "_view_name": "StyleView",
+ "description_width": ""
+ }
+ },
+ "88ff8d07f6ce430db2772b8bf24294e8": {
+ "model_module": "@jupyter-widgets/base",
+ "model_module_version": "1.2.0",
+ "model_name": "LayoutModel",
+ "state": {
+ "_model_module": "@jupyter-widgets/base",
+ "_model_module_version": "1.2.0",
+ "_model_name": "LayoutModel",
+ "_view_count": null,
+ "_view_module": "@jupyter-widgets/base",
+ "_view_module_version": "1.2.0",
+ "_view_name": "LayoutView",
+ "align_content": null,
+ "align_items": null,
+ "align_self": null,
+ "border": null,
+ "bottom": null,
+ "display": null,
+ "flex": null,
+ "flex_flow": null,
+ "grid_area": null,
+ "grid_auto_columns": null,
+ "grid_auto_flow": null,
+ "grid_auto_rows": null,
+ "grid_column": null,
+ "grid_gap": null,
+ "grid_row": null,
+ "grid_template_areas": null,
+ "grid_template_columns": null,
+ "grid_template_rows": null,
+ "height": null,
+ "justify_content": null,
+ "justify_items": null,
+ "left": null,
+ "margin": null,
+ "max_height": null,
+ "max_width": null,
+ "min_height": null,
+ "min_width": null,
+ "object_fit": null,
+ "object_position": null,
+ "order": null,
+ "overflow": null,
+ "overflow_x": null,
+ "overflow_y": null,
+ "padding": null,
+ "right": null,
+ "top": null,
+ "visibility": null,
+ "width": "20px"
+ }
+ },
+ "89f757fbb1ef4fa8b557c04926871096": {
+ "model_module": "@jupyter-widgets/base",
+ "model_module_version": "1.2.0",
+ "model_name": "LayoutModel",
+ "state": {
+ "_model_module": "@jupyter-widgets/base",
+ "_model_module_version": "1.2.0",
+ "_model_name": "LayoutModel",
+ "_view_count": null,
+ "_view_module": "@jupyter-widgets/base",
+ "_view_module_version": "1.2.0",
+ "_view_name": "LayoutView",
+ "align_content": null,
+ "align_items": null,
+ "align_self": null,
+ "border": null,
+ "bottom": null,
+ "display": null,
+ "flex": null,
+ "flex_flow": null,
+ "grid_area": null,
+ "grid_auto_columns": null,
+ "grid_auto_flow": null,
+ "grid_auto_rows": null,
+ "grid_column": null,
+ "grid_gap": null,
+ "grid_row": null,
+ "grid_template_areas": null,
+ "grid_template_columns": null,
+ "grid_template_rows": null,
+ "height": null,
+ "justify_content": null,
+ "justify_items": null,
+ "left": null,
+ "margin": null,
+ "max_height": null,
+ "max_width": null,
+ "min_height": null,
+ "min_width": null,
+ "object_fit": null,
+ "object_position": null,
+ "order": null,
+ "overflow": null,
+ "overflow_x": null,
+ "overflow_y": null,
+ "padding": null,
+ "right": null,
+ "top": null,
+ "visibility": null,
+ "width": null
+ }
+ },
+ "8e7d772e693c427c9bcfc7535a524979": {
+ "model_module": "@jupyter-widgets/base",
+ "model_module_version": "1.2.0",
+ "model_name": "LayoutModel",
+ "state": {
+ "_model_module": "@jupyter-widgets/base",
+ "_model_module_version": "1.2.0",
+ "_model_name": "LayoutModel",
+ "_view_count": null,
+ "_view_module": "@jupyter-widgets/base",
+ "_view_module_version": "1.2.0",
+ "_view_name": "LayoutView",
+ "align_content": null,
+ "align_items": null,
+ "align_self": null,
+ "border": null,
+ "bottom": null,
+ "display": null,
+ "flex": null,
+ "flex_flow": null,
+ "grid_area": null,
+ "grid_auto_columns": null,
+ "grid_auto_flow": null,
+ "grid_auto_rows": null,
+ "grid_column": null,
+ "grid_gap": null,
+ "grid_row": null,
+ "grid_template_areas": null,
+ "grid_template_columns": null,
+ "grid_template_rows": null,
+ "height": null,
+ "justify_content": null,
+ "justify_items": null,
+ "left": null,
+ "margin": null,
+ "max_height": null,
+ "max_width": null,
+ "min_height": null,
+ "min_width": null,
+ "object_fit": null,
+ "object_position": null,
+ "order": null,
+ "overflow": null,
+ "overflow_x": null,
+ "overflow_y": null,
+ "padding": null,
+ "right": null,
+ "top": null,
+ "visibility": null,
+ "width": null
+ }
+ },
+ "9236baef198c44ae8607c7888e899bc8": {
+ "model_module": "@jupyter-widgets/controls",
+ "model_module_version": "1.5.0",
+ "model_name": "FloatProgressModel",
+ "state": {
+ "_dom_classes": [],
+ "_model_module": "@jupyter-widgets/controls",
+ "_model_module_version": "1.5.0",
+ "_model_name": "FloatProgressModel",
+ "_view_count": null,
+ "_view_module": "@jupyter-widgets/controls",
+ "_view_module_version": "1.5.0",
+ "_view_name": "ProgressView",
+ "bar_style": "success",
+ "description": "",
+ "description_tooltip": null,
+ "layout": "IPY_MODEL_02e46682b2434a0c872f35db3d7b367e",
+ "max": 1,
+ "min": 0,
+ "orientation": "horizontal",
+ "style": "IPY_MODEL_3da4f3751cb54cf2938aea924ab33aa0",
+ "value": 0
+ }
+ },
+ "941ab80a14aa4a1bb3943dd77aca85d7": {
+ "model_module": "@jupyter-widgets/controls",
+ "model_module_version": "1.5.0",
+ "model_name": "ProgressStyleModel",
+ "state": {
+ "_model_module": "@jupyter-widgets/controls",
+ "_model_module_version": "1.5.0",
+ "_model_name": "ProgressStyleModel",
+ "_view_count": null,
+ "_view_module": "@jupyter-widgets/base",
+ "_view_module_version": "1.2.0",
+ "_view_name": "StyleView",
+ "bar_color": null,
+ "description_width": ""
+ }
+ },
+ "94ebb17c331649fc812a66231532ee03": {
+ "model_module": "@jupyter-widgets/controls",
+ "model_module_version": "1.5.0",
+ "model_name": "DescriptionStyleModel",
+ "state": {
+ "_model_module": "@jupyter-widgets/controls",
+ "_model_module_version": "1.5.0",
+ "_model_name": "DescriptionStyleModel",
+ "_view_count": null,
+ "_view_module": "@jupyter-widgets/base",
+ "_view_module_version": "1.2.0",
+ "_view_name": "StyleView",
+ "description_width": ""
+ }
+ },
+ "996f65fcf7a04703868717652927e5e2": {
+ "model_module": "@jupyter-widgets/base",
+ "model_module_version": "1.2.0",
+ "model_name": "LayoutModel",
+ "state": {
+ "_model_module": "@jupyter-widgets/base",
+ "_model_module_version": "1.2.0",
+ "_model_name": "LayoutModel",
+ "_view_count": null,
+ "_view_module": "@jupyter-widgets/base",
+ "_view_module_version": "1.2.0",
+ "_view_name": "LayoutView",
+ "align_content": null,
+ "align_items": null,
+ "align_self": null,
+ "border": null,
+ "bottom": null,
+ "display": null,
+ "flex": null,
+ "flex_flow": null,
+ "grid_area": null,
+ "grid_auto_columns": null,
+ "grid_auto_flow": null,
+ "grid_auto_rows": null,
+ "grid_column": null,
+ "grid_gap": null,
+ "grid_row": null,
+ "grid_template_areas": null,
+ "grid_template_columns": null,
+ "grid_template_rows": null,
+ "height": null,
+ "justify_content": null,
+ "justify_items": null,
+ "left": null,
+ "margin": null,
+ "max_height": null,
+ "max_width": null,
+ "min_height": null,
+ "min_width": null,
+ "object_fit": null,
+ "object_position": null,
+ "order": null,
+ "overflow": null,
+ "overflow_x": null,
+ "overflow_y": null,
+ "padding": null,
+ "right": null,
+ "top": null,
+ "visibility": "hidden",
+ "width": null
+ }
+ },
+ "9e46610e715444cbbc2f978a0acd0ed0": {
+ "model_module": "@jupyter-widgets/controls",
+ "model_module_version": "1.5.0",
+ "model_name": "DescriptionStyleModel",
+ "state": {
+ "_model_module": "@jupyter-widgets/controls",
+ "_model_module_version": "1.5.0",
+ "_model_name": "DescriptionStyleModel",
+ "_view_count": null,
+ "_view_module": "@jupyter-widgets/base",
+ "_view_module_version": "1.2.0",
+ "_view_name": "StyleView",
+ "description_width": ""
+ }
+ },
+ "a61d6191c53e4dd2a1cbdb9826f5ae70": {
+ "model_module": "@jupyter-widgets/controls",
+ "model_module_version": "1.5.0",
+ "model_name": "HBoxModel",
+ "state": {
+ "_dom_classes": [],
+ "_model_module": "@jupyter-widgets/controls",
+ "_model_module_version": "1.5.0",
+ "_model_name": "HBoxModel",
+ "_view_count": null,
+ "_view_module": "@jupyter-widgets/controls",
+ "_view_module_version": "1.5.0",
+ "_view_name": "HBoxView",
+ "box_style": "",
+ "children": [
+ "IPY_MODEL_e868b829ec7a4795955034ebb8b109a6",
+ "IPY_MODEL_1385286d855a46c8b19704a95e5992b4",
+ "IPY_MODEL_b7bbc695e0f248a0ac1e79a1d546df8e"
],
- "source": [
- "# Generate the answer given the context\n",
- "\n",
- "chain_type_kwargs = {\"prompt\": PROMPT}\n",
- "\n",
- "qa = RetrievalQA.from_chain_type(llm=Cohere(model=\"command\", temperature=0), \n",
- " chain_type=\"stuff\", \n",
- " retriever=db.as_retriever(), \n",
- " chain_type_kwargs=chain_type_kwargs, \n",
- " return_source_documents=True)\n",
- "\n",
- "for question in questions_fr:\n",
- " answer = qa({\"query\": question})\n",
- " result = answer[\"result\"].replace(\"\\n\",\"\").replace(\"Answer:\",\"\")\n",
- " sources = answer['source_documents']\n",
- " print(\"-\"*20,\"\\n\")\n",
- " print(f\"Question: {question}\")\n",
- " print(f\"Answer: {result}\")"
- ]
+ "layout": "IPY_MODEL_034072a591b94d3daf4284aadcff935f"
+ }
},
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "id": "06UhYMFdoi0A"
- },
- "outputs": [],
- "source": []
- }
- ],
- "metadata": {
- "accelerator": "GPU",
- "colab": {
- "machine_shape": "hm",
- "provenance": []
- },
- "gpuClass": "standard",
- "kernelspec": {
- "display_name": "Python 3.10.5 64-bit ('3.10.5')",
- "language": "python",
- "name": "python3"
- },
- "language_info": {
- "name": "python",
- "version": "3.10.5"
- },
- "vscode": {
- "interpreter": {
- "hash": "3a0ab37a1f07e7d320af811f0819b193749e9675a96eea7a4830e92d810d141d"
- }
- },
- "widgets": {
- "application/vnd.jupyter.widget-state+json": {
- "01545efb5b78418791ce0f8c763873ce": {
- "model_module": "@jupyter-widgets/base",
- "model_module_version": "1.2.0",
- "model_name": "LayoutModel",
- "state": {
- "_model_module": "@jupyter-widgets/base",
- "_model_module_version": "1.2.0",
- "_model_name": "LayoutModel",
- "_view_count": null,
- "_view_module": "@jupyter-widgets/base",
- "_view_module_version": "1.2.0",
- "_view_name": "LayoutView",
- "align_content": null,
- "align_items": null,
- "align_self": null,
- "border": null,
- "bottom": null,
- "display": null,
- "flex": null,
- "flex_flow": null,
- "grid_area": null,
- "grid_auto_columns": null,
- "grid_auto_flow": null,
- "grid_auto_rows": null,
- "grid_column": null,
- "grid_gap": null,
- "grid_row": null,
- "grid_template_areas": null,
- "grid_template_columns": null,
- "grid_template_rows": null,
- "height": null,
- "justify_content": null,
- "justify_items": null,
- "left": null,
- "margin": null,
- "max_height": null,
- "max_width": null,
- "min_height": null,
- "min_width": null,
- "object_fit": null,
- "object_position": null,
- "order": null,
- "overflow": null,
- "overflow_x": null,
- "overflow_y": null,
- "padding": null,
- "right": null,
- "top": null,
- "visibility": null,
- "width": null
- }
- },
- "02e46682b2434a0c872f35db3d7b367e": {
- "model_module": "@jupyter-widgets/base",
- "model_module_version": "1.2.0",
- "model_name": "LayoutModel",
- "state": {
- "_model_module": "@jupyter-widgets/base",
- "_model_module_version": "1.2.0",
- "_model_name": "LayoutModel",
- "_view_count": null,
- "_view_module": "@jupyter-widgets/base",
- "_view_module_version": "1.2.0",
- "_view_name": "LayoutView",
- "align_content": null,
- "align_items": null,
- "align_self": null,
- "border": null,
- "bottom": null,
- "display": null,
- "flex": null,
- "flex_flow": null,
- "grid_area": null,
- "grid_auto_columns": null,
- "grid_auto_flow": null,
- "grid_auto_rows": null,
- "grid_column": null,
- "grid_gap": null,
- "grid_row": null,
- "grid_template_areas": null,
- "grid_template_columns": null,
- "grid_template_rows": null,
- "height": null,
- "justify_content": null,
- "justify_items": null,
- "left": null,
- "margin": null,
- "max_height": null,
- "max_width": null,
- "min_height": null,
- "min_width": null,
- "object_fit": null,
- "object_position": null,
- "order": null,
- "overflow": null,
- "overflow_x": null,
- "overflow_y": null,
- "padding": null,
- "right": null,
- "top": null,
- "visibility": null,
- "width": "20px"
- }
- },
- "034072a591b94d3daf4284aadcff935f": {
- "model_module": "@jupyter-widgets/base",
- "model_module_version": "1.2.0",
- "model_name": "LayoutModel",
- "state": {
- "_model_module": "@jupyter-widgets/base",
- "_model_module_version": "1.2.0",
- "_model_name": "LayoutModel",
- "_view_count": null,
- "_view_module": "@jupyter-widgets/base",
- "_view_module_version": "1.2.0",
- "_view_name": "LayoutView",
- "align_content": null,
- "align_items": null,
- "align_self": null,
- "border": null,
- "bottom": null,
- "display": null,
- "flex": null,
- "flex_flow": null,
- "grid_area": null,
- "grid_auto_columns": null,
- "grid_auto_flow": null,
- "grid_auto_rows": null,
- "grid_column": null,
- "grid_gap": null,
- "grid_row": null,
- "grid_template_areas": null,
- "grid_template_columns": null,
- "grid_template_rows": null,
- "height": null,
- "justify_content": null,
- "justify_items": null,
- "left": null,
- "margin": null,
- "max_height": null,
- "max_width": null,
- "min_height": null,
- "min_width": null,
- "object_fit": null,
- "object_position": null,
- "order": null,
- "overflow": null,
- "overflow_x": null,
- "overflow_y": null,
- "padding": null,
- "right": null,
- "top": null,
- "visibility": "hidden",
- "width": null
- }
- },
- "09c17a96b5534b78b1d9bfc2f7bc83d5": {
- "model_module": "@jupyter-widgets/controls",
- "model_module_version": "1.5.0",
- "model_name": "HBoxModel",
- "state": {
- "_dom_classes": [],
- "_model_module": "@jupyter-widgets/controls",
- "_model_module_version": "1.5.0",
- "_model_name": "HBoxModel",
- "_view_count": null,
- "_view_module": "@jupyter-widgets/controls",
- "_view_module_version": "1.5.0",
- "_view_name": "HBoxView",
- "box_style": "",
- "children": [
- "IPY_MODEL_735bfe1c1ff6402d826257313d68b21b",
- "IPY_MODEL_df1b4d3501ed457b86b60a5449d1e1da",
- "IPY_MODEL_771c7097b5be45ffac31e70117376903"
- ],
- "layout": "IPY_MODEL_996f65fcf7a04703868717652927e5e2"
- }
- },
- "10ca522adeca4ba4bb2d4f9373a363f4": {
- "model_module": "@jupyter-widgets/base",
- "model_module_version": "1.2.0",
- "model_name": "LayoutModel",
- "state": {
- "_model_module": "@jupyter-widgets/base",
- "_model_module_version": "1.2.0",
- "_model_name": "LayoutModel",
- "_view_count": null,
- "_view_module": "@jupyter-widgets/base",
- "_view_module_version": "1.2.0",
- "_view_name": "LayoutView",
- "align_content": null,
- "align_items": null,
- "align_self": null,
- "border": null,
- "bottom": null,
- "display": null,
- "flex": null,
- "flex_flow": null,
- "grid_area": null,
- "grid_auto_columns": null,
- "grid_auto_flow": null,
- "grid_auto_rows": null,
- "grid_column": null,
- "grid_gap": null,
- "grid_row": null,
- "grid_template_areas": null,
- "grid_template_columns": null,
- "grid_template_rows": null,
- "height": null,
- "justify_content": null,
- "justify_items": null,
- "left": null,
- "margin": null,
- "max_height": null,
- "max_width": null,
- "min_height": null,
- "min_width": null,
- "object_fit": null,
- "object_position": null,
- "order": null,
- "overflow": null,
- "overflow_x": null,
- "overflow_y": null,
- "padding": null,
- "right": null,
- "top": null,
- "visibility": null,
- "width": null
- }
- },
- "1385286d855a46c8b19704a95e5992b4": {
- "model_module": "@jupyter-widgets/controls",
- "model_module_version": "1.5.0",
- "model_name": "FloatProgressModel",
- "state": {
- "_dom_classes": [],
- "_model_module": "@jupyter-widgets/controls",
- "_model_module_version": "1.5.0",
- "_model_name": "FloatProgressModel",
- "_view_count": null,
- "_view_module": "@jupyter-widgets/controls",
- "_view_module_version": "1.5.0",
- "_view_name": "ProgressView",
- "bar_style": "",
- "description": "",
- "description_tooltip": null,
- "layout": "IPY_MODEL_5cf957caabe043139dc587e8651a28b1",
- "max": 5452,
- "min": 0,
- "orientation": "horizontal",
- "style": "IPY_MODEL_bfb987eb24644819ba52b2fc7bc865de",
- "value": 5452
- }
- },
- "190a428569a74beb8b90aa278c629bd2": {
- "model_module": "@jupyter-widgets/controls",
- "model_module_version": "1.5.0",
- "model_name": "HTMLModel",
- "state": {
- "_dom_classes": [],
- "_model_module": "@jupyter-widgets/controls",
- "_model_module_version": "1.5.0",
- "_model_name": "HTMLModel",
- "_view_count": null,
- "_view_module": "@jupyter-widgets/controls",
- "_view_module_version": "1.5.0",
- "_view_name": "HTMLView",
- "description": "",
- "description_tooltip": null,
- "layout": "IPY_MODEL_6d1ea14b22034d2d9ab16aa5a24566dc",
- "placeholder": "",
- "style": "IPY_MODEL_6c0193f68df143e0a7907c6a577b07a6",
- "value": " 0/0 [00:04<?, ? MiB/s]"
- }
- },
- "1ac7e3656e43480c8af4038d2e7d5432": {
- "model_module": "@jupyter-widgets/controls",
- "model_module_version": "1.5.0",
- "model_name": "DescriptionStyleModel",
- "state": {
- "_model_module": "@jupyter-widgets/controls",
- "_model_module_version": "1.5.0",
- "_model_name": "DescriptionStyleModel",
- "_view_count": null,
- "_view_module": "@jupyter-widgets/base",
- "_view_module_version": "1.2.0",
- "_view_name": "StyleView",
- "description_width": ""
- }
- },
- "1f1331ab4e734cdaa66cf28d7278f980": {
- "model_module": "@jupyter-widgets/base",
- "model_module_version": "1.2.0",
- "model_name": "LayoutModel",
- "state": {
- "_model_module": "@jupyter-widgets/base",
- "_model_module_version": "1.2.0",
- "_model_name": "LayoutModel",
- "_view_count": null,
- "_view_module": "@jupyter-widgets/base",
- "_view_module_version": "1.2.0",
- "_view_name": "LayoutView",
- "align_content": null,
- "align_items": null,
- "align_self": null,
- "border": null,
- "bottom": null,
- "display": null,
- "flex": null,
- "flex_flow": null,
- "grid_area": null,
- "grid_auto_columns": null,
- "grid_auto_flow": null,
- "grid_auto_rows": null,
- "grid_column": null,
- "grid_gap": null,
- "grid_row": null,
- "grid_template_areas": null,
- "grid_template_columns": null,
- "grid_template_rows": null,
- "height": null,
- "justify_content": null,
- "justify_items": null,
- "left": null,
- "margin": null,
- "max_height": null,
- "max_width": null,
- "min_height": null,
- "min_width": null,
- "object_fit": null,
- "object_position": null,
- "order": null,
- "overflow": null,
- "overflow_x": null,
- "overflow_y": null,
- "padding": null,
- "right": null,
- "top": null,
- "visibility": null,
- "width": null
- }
- },
- "211f7336a3b9455f8cd67bd0ee4113ae": {
- "model_module": "@jupyter-widgets/controls",
- "model_module_version": "1.5.0",
- "model_name": "HTMLModel",
- "state": {
- "_dom_classes": [],
- "_model_module": "@jupyter-widgets/controls",
- "_model_module_version": "1.5.0",
- "_model_name": "HTMLModel",
- "_view_count": null,
- "_view_module": "@jupyter-widgets/controls",
- "_view_module_version": "1.5.0",
- "_view_name": "HTMLView",
- "description": "",
- "description_tooltip": null,
- "layout": "IPY_MODEL_68bf09957e83431b8d299f1ab15d5f7e",
- "placeholder": "",
- "style": "IPY_MODEL_28adfb59f3e24851ba617e0087e5293f",
- "value": "Generating test examples...: 0%"
- }
- },
- "23bbe3c6dfcb42c7a0a1c826bc35076d": {
- "model_module": "@jupyter-widgets/controls",
- "model_module_version": "1.5.0",
- "model_name": "DescriptionStyleModel",
- "state": {
- "_model_module": "@jupyter-widgets/controls",
- "_model_module_version": "1.5.0",
- "_model_name": "DescriptionStyleModel",
- "_view_count": null,
- "_view_module": "@jupyter-widgets/base",
- "_view_module_version": "1.2.0",
- "_view_name": "StyleView",
- "description_width": ""
- }
- },
- "25a079a3d58644f4a5306ee2cb42dbde": {
- "model_module": "@jupyter-widgets/base",
- "model_module_version": "1.2.0",
- "model_name": "LayoutModel",
- "state": {
- "_model_module": "@jupyter-widgets/base",
- "_model_module_version": "1.2.0",
- "_model_name": "LayoutModel",
- "_view_count": null,
- "_view_module": "@jupyter-widgets/base",
- "_view_module_version": "1.2.0",
- "_view_name": "LayoutView",
- "align_content": null,
- "align_items": null,
- "align_self": null,
- "border": null,
- "bottom": null,
- "display": null,
- "flex": null,
- "flex_flow": null,
- "grid_area": null,
- "grid_auto_columns": null,
- "grid_auto_flow": null,
- "grid_auto_rows": null,
- "grid_column": null,
- "grid_gap": null,
- "grid_row": null,
- "grid_template_areas": null,
- "grid_template_columns": null,
- "grid_template_rows": null,
- "height": null,
- "justify_content": null,
- "justify_items": null,
- "left": null,
- "margin": null,
- "max_height": null,
- "max_width": null,
- "min_height": null,
- "min_width": null,
- "object_fit": null,
- "object_position": null,
- "order": null,
- "overflow": null,
- "overflow_x": null,
- "overflow_y": null,
- "padding": null,
- "right": null,
- "top": null,
- "visibility": null,
- "width": null
- }
- },
- "28adfb59f3e24851ba617e0087e5293f": {
- "model_module": "@jupyter-widgets/controls",
- "model_module_version": "1.5.0",
- "model_name": "DescriptionStyleModel",
- "state": {
- "_model_module": "@jupyter-widgets/controls",
- "_model_module_version": "1.5.0",
- "_model_name": "DescriptionStyleModel",
- "_view_count": null,
- "_view_module": "@jupyter-widgets/base",
- "_view_module_version": "1.2.0",
- "_view_name": "StyleView",
- "description_width": ""
- }
- },
- "2bd5fe14d3d44d85be32c5c78f5b7568": {
- "model_module": "@jupyter-widgets/controls",
- "model_module_version": "1.5.0",
- "model_name": "FloatProgressModel",
- "state": {
- "_dom_classes": [],
- "_model_module": "@jupyter-widgets/controls",
- "_model_module_version": "1.5.0",
- "_model_name": "FloatProgressModel",
- "_view_count": null,
- "_view_module": "@jupyter-widgets/controls",
- "_view_module_version": "1.5.0",
- "_view_name": "ProgressView",
- "bar_style": "",
- "description": "",
- "description_tooltip": null,
- "layout": "IPY_MODEL_10ca522adeca4ba4bb2d4f9373a363f4",
- "max": 500,
- "min": 0,
- "orientation": "horizontal",
- "style": "IPY_MODEL_5f9f737d68234b9a9cd069105ab20ea2",
- "value": 500
- }
- },
- "2d69c8378b194aa3a8634848da5ee79a": {
- "model_module": "@jupyter-widgets/controls",
- "model_module_version": "1.5.0",
- "model_name": "DescriptionStyleModel",
- "state": {
- "_model_module": "@jupyter-widgets/controls",
- "_model_module_version": "1.5.0",
- "_model_name": "DescriptionStyleModel",
- "_view_count": null,
- "_view_module": "@jupyter-widgets/base",
- "_view_module_version": "1.2.0",
- "_view_name": "StyleView",
- "description_width": ""
- }
- },
- "2ee59cbc091346168c3b4511f64b018a": {
- "model_module": "@jupyter-widgets/controls",
- "model_module_version": "1.5.0",
- "model_name": "ProgressStyleModel",
- "state": {
- "_model_module": "@jupyter-widgets/controls",
- "_model_module_version": "1.5.0",
- "_model_name": "ProgressStyleModel",
- "_view_count": null,
- "_view_module": "@jupyter-widgets/base",
- "_view_module_version": "1.2.0",
- "_view_name": "StyleView",
- "bar_color": null,
- "description_width": ""
- }
- },
- "35132c00838046bca253f31350a3190b": {
- "model_module": "@jupyter-widgets/controls",
- "model_module_version": "1.5.0",
- "model_name": "FloatProgressModel",
- "state": {
- "_dom_classes": [],
- "_model_module": "@jupyter-widgets/controls",
- "_model_module_version": "1.5.0",
- "_model_name": "FloatProgressModel",
- "_view_count": null,
- "_view_module": "@jupyter-widgets/controls",
- "_view_module_version": "1.5.0",
- "_view_name": "ProgressView",
- "bar_style": "success",
- "description": "",
- "description_tooltip": null,
- "layout": "IPY_MODEL_88ff8d07f6ce430db2772b8bf24294e8",
- "max": 1,
- "min": 0,
- "orientation": "horizontal",
- "style": "IPY_MODEL_f5e168c246714295a10a64fbb1ef2ee3",
- "value": 0
- }
- },
- "3627640fb7d14ac09c9fdcbfb2159152": {
- "model_module": "@jupyter-widgets/controls",
- "model_module_version": "1.5.0",
- "model_name": "FloatProgressModel",
- "state": {
- "_dom_classes": [],
- "_model_module": "@jupyter-widgets/controls",
- "_model_module_version": "1.5.0",
- "_model_name": "FloatProgressModel",
- "_view_count": null,
- "_view_module": "@jupyter-widgets/controls",
- "_view_module_version": "1.5.0",
- "_view_name": "ProgressView",
- "bar_style": "",
- "description": "",
- "description_tooltip": null,
- "layout": "IPY_MODEL_6e600a0c3f36439899b40cdc53ed7c8c",
- "max": 500,
- "min": 0,
- "orientation": "horizontal",
- "style": "IPY_MODEL_672ec8abe8b34b8dab6d4af20f0302c7",
- "value": 500
- }
- },
- "3647d22aaf934b53bf925dc22e2ce69a": {
- "model_module": "@jupyter-widgets/controls",
- "model_module_version": "1.5.0",
- "model_name": "HTMLModel",
- "state": {
- "_dom_classes": [],
- "_model_module": "@jupyter-widgets/controls",
- "_model_module_version": "1.5.0",
- "_model_name": "HTMLModel",
- "_view_count": null,
- "_view_module": "@jupyter-widgets/controls",
- "_view_module_version": "1.5.0",
- "_view_name": "HTMLView",
- "description": "",
- "description_tooltip": null,
- "layout": "IPY_MODEL_89f757fbb1ef4fa8b557c04926871096",
- "placeholder": "",
- "style": "IPY_MODEL_94ebb17c331649fc812a66231532ee03",
- "value": " 0/0 [00:04<?, ? file/s]"
- }
- },
- "3948a575b3ee4a199f1055e93d8270ee": {
- "model_module": "@jupyter-widgets/base",
- "model_module_version": "1.2.0",
- "model_name": "LayoutModel",
- "state": {
- "_model_module": "@jupyter-widgets/base",
- "_model_module_version": "1.2.0",
- "_model_name": "LayoutModel",
- "_view_count": null,
- "_view_module": "@jupyter-widgets/base",
- "_view_module_version": "1.2.0",
- "_view_name": "LayoutView",
- "align_content": null,
- "align_items": null,
- "align_self": null,
- "border": null,
- "bottom": null,
- "display": null,
- "flex": null,
- "flex_flow": null,
- "grid_area": null,
- "grid_auto_columns": null,
- "grid_auto_flow": null,
- "grid_auto_rows": null,
- "grid_column": null,
- "grid_gap": null,
- "grid_row": null,
- "grid_template_areas": null,
- "grid_template_columns": null,
- "grid_template_rows": null,
- "height": null,
- "justify_content": null,
- "justify_items": null,
- "left": null,
- "margin": null,
- "max_height": null,
- "max_width": null,
- "min_height": null,
- "min_width": null,
- "object_fit": null,
- "object_position": null,
- "order": null,
- "overflow": null,
- "overflow_x": null,
- "overflow_y": null,
- "padding": null,
- "right": null,
- "top": null,
- "visibility": null,
- "width": null
- }
- },
- "3bda8233967642229fe778b4a8402f52": {
- "model_module": "@jupyter-widgets/controls",
- "model_module_version": "1.5.0",
- "model_name": "DescriptionStyleModel",
- "state": {
- "_model_module": "@jupyter-widgets/controls",
- "_model_module_version": "1.5.0",
- "_model_name": "DescriptionStyleModel",
- "_view_count": null,
- "_view_module": "@jupyter-widgets/base",
- "_view_module_version": "1.2.0",
- "_view_name": "StyleView",
- "description_width": ""
- }
- },
- "3da4f3751cb54cf2938aea924ab33aa0": {
- "model_module": "@jupyter-widgets/controls",
- "model_module_version": "1.5.0",
- "model_name": "ProgressStyleModel",
- "state": {
- "_model_module": "@jupyter-widgets/controls",
- "_model_module_version": "1.5.0",
- "_model_name": "ProgressStyleModel",
- "_view_count": null,
- "_view_module": "@jupyter-widgets/base",
- "_view_module_version": "1.2.0",
- "_view_name": "StyleView",
- "bar_color": null,
- "description_width": ""
- }
- },
- "4334b4f6fc7048eab0d146b6ffc437d1": {
- "model_module": "@jupyter-widgets/controls",
- "model_module_version": "1.5.0",
- "model_name": "HTMLModel",
- "state": {
- "_dom_classes": [],
- "_model_module": "@jupyter-widgets/controls",
- "_model_module_version": "1.5.0",
- "_model_name": "HTMLModel",
- "_view_count": null,
- "_view_module": "@jupyter-widgets/controls",
- "_view_module_version": "1.5.0",
- "_view_name": "HTMLView",
- "description": "",
- "description_tooltip": null,
- "layout": "IPY_MODEL_be8b3f4e73fc44f390e9a147df786e53",
- "placeholder": "",
- "style": "IPY_MODEL_3bda8233967642229fe778b4a8402f52",
- "value": "Dl Completed...: 100%"
- }
- },
- "463628ae5afc4c8f9a51f2c4ed038d12": {
- "model_module": "@jupyter-widgets/base",
- "model_module_version": "1.2.0",
- "model_name": "LayoutModel",
- "state": {
- "_model_module": "@jupyter-widgets/base",
- "_model_module_version": "1.2.0",
- "_model_name": "LayoutModel",
- "_view_count": null,
- "_view_module": "@jupyter-widgets/base",
- "_view_module_version": "1.2.0",
- "_view_name": "LayoutView",
- "align_content": null,
- "align_items": null,
- "align_self": null,
- "border": null,
- "bottom": null,
- "display": null,
- "flex": null,
- "flex_flow": null,
- "grid_area": null,
- "grid_auto_columns": null,
- "grid_auto_flow": null,
- "grid_auto_rows": null,
- "grid_column": null,
- "grid_gap": null,
- "grid_row": null,
- "grid_template_areas": null,
- "grid_template_columns": null,
- "grid_template_rows": null,
- "height": null,
- "justify_content": null,
- "justify_items": null,
- "left": null,
- "margin": null,
- "max_height": null,
- "max_width": null,
- "min_height": null,
- "min_width": null,
- "object_fit": null,
- "object_position": null,
- "order": null,
- "overflow": null,
- "overflow_x": null,
- "overflow_y": null,
- "padding": null,
- "right": null,
- "top": null,
- "visibility": null,
- "width": null
- }
- },
- "4cf6a31b3be945cbb82f8239e7e2642e": {
- "model_module": "@jupyter-widgets/controls",
- "model_module_version": "1.5.0",
- "model_name": "HBoxModel",
- "state": {
- "_dom_classes": [],
- "_model_module": "@jupyter-widgets/controls",
- "_model_module_version": "1.5.0",
- "_model_name": "HBoxModel",
- "_view_count": null,
- "_view_module": "@jupyter-widgets/controls",
- "_view_module_version": "1.5.0",
- "_view_name": "HBoxView",
- "box_style": "",
- "children": [
- "IPY_MODEL_7fd82914b89343538aa08b36f4752f4b",
- "IPY_MODEL_35132c00838046bca253f31350a3190b",
- "IPY_MODEL_190a428569a74beb8b90aa278c629bd2"
- ],
- "layout": "IPY_MODEL_01545efb5b78418791ce0f8c763873ce"
- }
- },
- "533f3a17f8d1442490a2d6167d89550b": {
- "model_module": "@jupyter-widgets/base",
- "model_module_version": "1.2.0",
- "model_name": "LayoutModel",
- "state": {
- "_model_module": "@jupyter-widgets/base",
- "_model_module_version": "1.2.0",
- "_model_name": "LayoutModel",
- "_view_count": null,
- "_view_module": "@jupyter-widgets/base",
- "_view_module_version": "1.2.0",
- "_view_name": "LayoutView",
- "align_content": null,
- "align_items": null,
- "align_self": null,
- "border": null,
- "bottom": null,
- "display": null,
- "flex": null,
- "flex_flow": null,
- "grid_area": null,
- "grid_auto_columns": null,
- "grid_auto_flow": null,
- "grid_auto_rows": null,
- "grid_column": null,
- "grid_gap": null,
- "grid_row": null,
- "grid_template_areas": null,
- "grid_template_columns": null,
- "grid_template_rows": null,
- "height": null,
- "justify_content": null,
- "justify_items": null,
- "left": null,
- "margin": null,
- "max_height": null,
- "max_width": null,
- "min_height": null,
- "min_width": null,
- "object_fit": null,
- "object_position": null,
- "order": null,
- "overflow": null,
- "overflow_x": null,
- "overflow_y": null,
- "padding": null,
- "right": null,
- "top": null,
- "visibility": null,
- "width": null
- }
- },
- "57cf77bba44141359639c0268ced8934": {
- "model_module": "@jupyter-widgets/base",
- "model_module_version": "1.2.0",
- "model_name": "LayoutModel",
- "state": {
- "_model_module": "@jupyter-widgets/base",
- "_model_module_version": "1.2.0",
- "_model_name": "LayoutModel",
- "_view_count": null,
- "_view_module": "@jupyter-widgets/base",
- "_view_module_version": "1.2.0",
- "_view_name": "LayoutView",
- "align_content": null,
- "align_items": null,
- "align_self": null,
- "border": null,
- "bottom": null,
- "display": null,
- "flex": null,
- "flex_flow": null,
- "grid_area": null,
- "grid_auto_columns": null,
- "grid_auto_flow": null,
- "grid_auto_rows": null,
- "grid_column": null,
- "grid_gap": null,
- "grid_row": null,
- "grid_template_areas": null,
- "grid_template_columns": null,
- "grid_template_rows": null,
- "height": null,
- "justify_content": null,
- "justify_items": null,
- "left": null,
- "margin": null,
- "max_height": null,
- "max_width": null,
- "min_height": null,
- "min_width": null,
- "object_fit": null,
- "object_position": null,
- "order": null,
- "overflow": null,
- "overflow_x": null,
- "overflow_y": null,
- "padding": null,
- "right": null,
- "top": null,
- "visibility": null,
- "width": null
- }
- },
- "59f2ed0257d6422d84fa23898e5d9a10": {
- "model_module": "@jupyter-widgets/controls",
- "model_module_version": "1.5.0",
- "model_name": "HTMLModel",
- "state": {
- "_dom_classes": [],
- "_model_module": "@jupyter-widgets/controls",
- "_model_module_version": "1.5.0",
- "_model_name": "HTMLModel",
- "_view_count": null,
- "_view_module": "@jupyter-widgets/controls",
- "_view_module_version": "1.5.0",
- "_view_name": "HTMLView",
- "description": "",
- "description_tooltip": null,
- "layout": "IPY_MODEL_533f3a17f8d1442490a2d6167d89550b",
- "placeholder": "",
- "style": "IPY_MODEL_ce82f835dd2b45a181a53407505b0cfc",
- "value": "Extraction completed...: "
- }
- },
- "5cf957caabe043139dc587e8651a28b1": {
- "model_module": "@jupyter-widgets/base",
- "model_module_version": "1.2.0",
- "model_name": "LayoutModel",
- "state": {
- "_model_module": "@jupyter-widgets/base",
- "_model_module_version": "1.2.0",
- "_model_name": "LayoutModel",
- "_view_count": null,
- "_view_module": "@jupyter-widgets/base",
- "_view_module_version": "1.2.0",
- "_view_name": "LayoutView",
- "align_content": null,
- "align_items": null,
- "align_self": null,
- "border": null,
- "bottom": null,
- "display": null,
- "flex": null,
- "flex_flow": null,
- "grid_area": null,
- "grid_auto_columns": null,
- "grid_auto_flow": null,
- "grid_auto_rows": null,
- "grid_column": null,
- "grid_gap": null,
- "grid_row": null,
- "grid_template_areas": null,
- "grid_template_columns": null,
- "grid_template_rows": null,
- "height": null,
- "justify_content": null,
- "justify_items": null,
- "left": null,
- "margin": null,
- "max_height": null,
- "max_width": null,
- "min_height": null,
- "min_width": null,
- "object_fit": null,
- "object_position": null,
- "order": null,
- "overflow": null,
- "overflow_x": null,
- "overflow_y": null,
- "padding": null,
- "right": null,
- "top": null,
- "visibility": null,
- "width": null
- }
- },
- "5ed53479e65749a68e2ae1f70fff1db0": {
- "model_module": "@jupyter-widgets/controls",
- "model_module_version": "1.5.0",
- "model_name": "DescriptionStyleModel",
- "state": {
- "_model_module": "@jupyter-widgets/controls",
- "_model_module_version": "1.5.0",
- "_model_name": "DescriptionStyleModel",
- "_view_count": null,
- "_view_module": "@jupyter-widgets/base",
- "_view_module_version": "1.2.0",
- "_view_name": "StyleView",
- "description_width": ""
- }
- },
- "5edafe62c55449709c304c11dc847e82": {
- "model_module": "@jupyter-widgets/base",
- "model_module_version": "1.2.0",
- "model_name": "LayoutModel",
- "state": {
- "_model_module": "@jupyter-widgets/base",
- "_model_module_version": "1.2.0",
- "_model_name": "LayoutModel",
- "_view_count": null,
- "_view_module": "@jupyter-widgets/base",
- "_view_module_version": "1.2.0",
- "_view_name": "LayoutView",
- "align_content": null,
- "align_items": null,
- "align_self": null,
- "border": null,
- "bottom": null,
- "display": null,
- "flex": null,
- "flex_flow": null,
- "grid_area": null,
- "grid_auto_columns": null,
- "grid_auto_flow": null,
- "grid_auto_rows": null,
- "grid_column": null,
- "grid_gap": null,
- "grid_row": null,
- "grid_template_areas": null,
- "grid_template_columns": null,
- "grid_template_rows": null,
- "height": null,
- "justify_content": null,
- "justify_items": null,
- "left": null,
- "margin": null,
- "max_height": null,
- "max_width": null,
- "min_height": null,
- "min_width": null,
- "object_fit": null,
- "object_position": null,
- "order": null,
- "overflow": null,
- "overflow_x": null,
- "overflow_y": null,
- "padding": null,
- "right": null,
- "top": null,
- "visibility": null,
- "width": null
- }
- },
- "5f9f737d68234b9a9cd069105ab20ea2": {
- "model_module": "@jupyter-widgets/controls",
- "model_module_version": "1.5.0",
- "model_name": "ProgressStyleModel",
- "state": {
- "_model_module": "@jupyter-widgets/controls",
- "_model_module_version": "1.5.0",
- "_model_name": "ProgressStyleModel",
- "_view_count": null,
- "_view_module": "@jupyter-widgets/base",
- "_view_module_version": "1.2.0",
- "_view_name": "StyleView",
- "bar_color": null,
- "description_width": ""
- }
- },
- "62eb05321e2d4265b033544921f92053": {
- "model_module": "@jupyter-widgets/controls",
- "model_module_version": "1.5.0",
- "model_name": "HTMLModel",
- "state": {
- "_dom_classes": [],
- "_model_module": "@jupyter-widgets/controls",
- "_model_module_version": "1.5.0",
- "_model_name": "HTMLModel",
- "_view_count": null,
- "_view_module": "@jupyter-widgets/controls",
- "_view_module_version": "1.5.0",
- "_view_name": "HTMLView",
- "description": "",
- "description_tooltip": null,
- "layout": "IPY_MODEL_463628ae5afc4c8f9a51f2c4ed038d12",
- "placeholder": "",
- "style": "IPY_MODEL_1ac7e3656e43480c8af4038d2e7d5432",
- "value": "Shuffling /root/tensorflow_datasets/trec/1.0.0.incompleteWOR5EP/trec-test.tfrecord*...: 0%"
- }
- },
- "65b7f23304a243878c43ea391885ee85": {
- "model_module": "@jupyter-widgets/controls",
- "model_module_version": "1.5.0",
- "model_name": "HTMLModel",
- "state": {
- "_dom_classes": [],
- "_model_module": "@jupyter-widgets/controls",
- "_model_module_version": "1.5.0",
- "_model_name": "HTMLModel",
- "_view_count": null,
- "_view_module": "@jupyter-widgets/controls",
- "_view_module_version": "1.5.0",
- "_view_name": "HTMLView",
- "description": "",
- "description_tooltip": null,
- "layout": "IPY_MODEL_c5296022707240b393be4146e983083c",
- "placeholder": "",
- "style": "IPY_MODEL_8653f9b6a25446cc8d7d34ffd4e48ca8",
- "value": " 1/2 [00:00<00:00, 2.06 splits/s]"
- }
- },
- "672ec8abe8b34b8dab6d4af20f0302c7": {
- "model_module": "@jupyter-widgets/controls",
- "model_module_version": "1.5.0",
- "model_name": "ProgressStyleModel",
- "state": {
- "_model_module": "@jupyter-widgets/controls",
- "_model_module_version": "1.5.0",
- "_model_name": "ProgressStyleModel",
- "_view_count": null,
- "_view_module": "@jupyter-widgets/base",
- "_view_module_version": "1.2.0",
- "_view_name": "StyleView",
- "bar_color": null,
- "description_width": ""
- }
- },
- "6850b87ff2fa49eb9fbfb3c3492efbb3": {
- "model_module": "@jupyter-widgets/controls",
- "model_module_version": "1.5.0",
- "model_name": "HTMLModel",
- "state": {
- "_dom_classes": [],
- "_model_module": "@jupyter-widgets/controls",
- "_model_module_version": "1.5.0",
- "_model_name": "HTMLModel",
- "_view_count": null,
- "_view_module": "@jupyter-widgets/controls",
- "_view_module_version": "1.5.0",
- "_view_name": "HTMLView",
- "description": "",
- "description_tooltip": null,
- "layout": "IPY_MODEL_d298dc9202dc49aba09b66969b3d19f9",
- "placeholder": "",
- "style": "IPY_MODEL_6e09dadf883440038a7a47f0d9723ae6",
- "value": " 0/500 [00:00<?, ? examples/s]"
- }
- },
- "687cfefc80a54edd84eb22baf805c905": {
- "model_module": "@jupyter-widgets/base",
- "model_module_version": "1.2.0",
- "model_name": "LayoutModel",
- "state": {
- "_model_module": "@jupyter-widgets/base",
- "_model_module_version": "1.2.0",
- "_model_name": "LayoutModel",
- "_view_count": null,
- "_view_module": "@jupyter-widgets/base",
- "_view_module_version": "1.2.0",
- "_view_name": "LayoutView",
- "align_content": null,
- "align_items": null,
- "align_self": null,
- "border": null,
- "bottom": null,
- "display": null,
- "flex": null,
- "flex_flow": null,
- "grid_area": null,
- "grid_auto_columns": null,
- "grid_auto_flow": null,
- "grid_auto_rows": null,
- "grid_column": null,
- "grid_gap": null,
- "grid_row": null,
- "grid_template_areas": null,
- "grid_template_columns": null,
- "grid_template_rows": null,
- "height": null,
- "justify_content": null,
- "justify_items": null,
- "left": null,
- "margin": null,
- "max_height": null,
- "max_width": null,
- "min_height": null,
- "min_width": null,
- "object_fit": null,
- "object_position": null,
- "order": null,
- "overflow": null,
- "overflow_x": null,
- "overflow_y": null,
- "padding": null,
- "right": null,
- "top": null,
- "visibility": null,
- "width": null
- }
- },
- "68bf09957e83431b8d299f1ab15d5f7e": {
- "model_module": "@jupyter-widgets/base",
- "model_module_version": "1.2.0",
- "model_name": "LayoutModel",
- "state": {
- "_model_module": "@jupyter-widgets/base",
- "_model_module_version": "1.2.0",
- "_model_name": "LayoutModel",
- "_view_count": null,
- "_view_module": "@jupyter-widgets/base",
- "_view_module_version": "1.2.0",
- "_view_name": "LayoutView",
- "align_content": null,
- "align_items": null,
- "align_self": null,
- "border": null,
- "bottom": null,
- "display": null,
- "flex": null,
- "flex_flow": null,
- "grid_area": null,
- "grid_auto_columns": null,
- "grid_auto_flow": null,
- "grid_auto_rows": null,
- "grid_column": null,
- "grid_gap": null,
- "grid_row": null,
- "grid_template_areas": null,
- "grid_template_columns": null,
- "grid_template_rows": null,
- "height": null,
- "justify_content": null,
- "justify_items": null,
- "left": null,
- "margin": null,
- "max_height": null,
- "max_width": null,
- "min_height": null,
- "min_width": null,
- "object_fit": null,
- "object_position": null,
- "order": null,
- "overflow": null,
- "overflow_x": null,
- "overflow_y": null,
- "padding": null,
- "right": null,
- "top": null,
- "visibility": null,
- "width": null
- }
- },
- "6a7d1421176f4b62af003a4b2f18815b": {
- "model_module": "@jupyter-widgets/controls",
- "model_module_version": "1.5.0",
- "model_name": "DescriptionStyleModel",
- "state": {
- "_model_module": "@jupyter-widgets/controls",
- "_model_module_version": "1.5.0",
- "_model_name": "DescriptionStyleModel",
- "_view_count": null,
- "_view_module": "@jupyter-widgets/base",
- "_view_module_version": "1.2.0",
- "_view_name": "StyleView",
- "description_width": ""
- }
- },
- "6c0193f68df143e0a7907c6a577b07a6": {
- "model_module": "@jupyter-widgets/controls",
- "model_module_version": "1.5.0",
- "model_name": "DescriptionStyleModel",
- "state": {
- "_model_module": "@jupyter-widgets/controls",
- "_model_module_version": "1.5.0",
- "_model_name": "DescriptionStyleModel",
- "_view_count": null,
- "_view_module": "@jupyter-widgets/base",
- "_view_module_version": "1.2.0",
- "_view_name": "StyleView",
- "description_width": ""
- }
- },
- "6d1ea14b22034d2d9ab16aa5a24566dc": {
- "model_module": "@jupyter-widgets/base",
- "model_module_version": "1.2.0",
- "model_name": "LayoutModel",
- "state": {
- "_model_module": "@jupyter-widgets/base",
- "_model_module_version": "1.2.0",
- "_model_name": "LayoutModel",
- "_view_count": null,
- "_view_module": "@jupyter-widgets/base",
- "_view_module_version": "1.2.0",
- "_view_name": "LayoutView",
- "align_content": null,
- "align_items": null,
- "align_self": null,
- "border": null,
- "bottom": null,
- "display": null,
- "flex": null,
- "flex_flow": null,
- "grid_area": null,
- "grid_auto_columns": null,
- "grid_auto_flow": null,
- "grid_auto_rows": null,
- "grid_column": null,
- "grid_gap": null,
- "grid_row": null,
- "grid_template_areas": null,
- "grid_template_columns": null,
- "grid_template_rows": null,
- "height": null,
- "justify_content": null,
- "justify_items": null,
- "left": null,
- "margin": null,
- "max_height": null,
- "max_width": null,
- "min_height": null,
- "min_width": null,
- "object_fit": null,
- "object_position": null,
- "order": null,
- "overflow": null,
- "overflow_x": null,
- "overflow_y": null,
- "padding": null,
- "right": null,
- "top": null,
- "visibility": null,
- "width": null
- }
- },
- "6def2ecb657845a3bae18bad85950991": {
- "model_module": "@jupyter-widgets/base",
- "model_module_version": "1.2.0",
- "model_name": "LayoutModel",
- "state": {
- "_model_module": "@jupyter-widgets/base",
- "_model_module_version": "1.2.0",
- "_model_name": "LayoutModel",
- "_view_count": null,
- "_view_module": "@jupyter-widgets/base",
- "_view_module_version": "1.2.0",
- "_view_name": "LayoutView",
- "align_content": null,
- "align_items": null,
- "align_self": null,
- "border": null,
- "bottom": null,
- "display": null,
- "flex": null,
- "flex_flow": null,
- "grid_area": null,
- "grid_auto_columns": null,
- "grid_auto_flow": null,
- "grid_auto_rows": null,
- "grid_column": null,
- "grid_gap": null,
- "grid_row": null,
- "grid_template_areas": null,
- "grid_template_columns": null,
- "grid_template_rows": null,
- "height": null,
- "justify_content": null,
- "justify_items": null,
- "left": null,
- "margin": null,
- "max_height": null,
- "max_width": null,
- "min_height": null,
- "min_width": null,
- "object_fit": null,
- "object_position": null,
- "order": null,
- "overflow": null,
- "overflow_x": null,
- "overflow_y": null,
- "padding": null,
- "right": null,
- "top": null,
- "visibility": "hidden",
- "width": null
- }
- },
- "6e09dadf883440038a7a47f0d9723ae6": {
- "model_module": "@jupyter-widgets/controls",
- "model_module_version": "1.5.0",
- "model_name": "DescriptionStyleModel",
- "state": {
- "_model_module": "@jupyter-widgets/controls",
- "_model_module_version": "1.5.0",
- "_model_name": "DescriptionStyleModel",
- "_view_count": null,
- "_view_module": "@jupyter-widgets/base",
- "_view_module_version": "1.2.0",
- "_view_name": "StyleView",
- "description_width": ""
- }
- },
- "6e600a0c3f36439899b40cdc53ed7c8c": {
- "model_module": "@jupyter-widgets/base",
- "model_module_version": "1.2.0",
- "model_name": "LayoutModel",
- "state": {
- "_model_module": "@jupyter-widgets/base",
- "_model_module_version": "1.2.0",
- "_model_name": "LayoutModel",
- "_view_count": null,
- "_view_module": "@jupyter-widgets/base",
- "_view_module_version": "1.2.0",
- "_view_name": "LayoutView",
- "align_content": null,
- "align_items": null,
- "align_self": null,
- "border": null,
- "bottom": null,
- "display": null,
- "flex": null,
- "flex_flow": null,
- "grid_area": null,
- "grid_auto_columns": null,
- "grid_auto_flow": null,
- "grid_auto_rows": null,
- "grid_column": null,
- "grid_gap": null,
- "grid_row": null,
- "grid_template_areas": null,
- "grid_template_columns": null,
- "grid_template_rows": null,
- "height": null,
- "justify_content": null,
- "justify_items": null,
- "left": null,
- "margin": null,
- "max_height": null,
- "max_width": null,
- "min_height": null,
- "min_width": null,
- "object_fit": null,
- "object_position": null,
- "order": null,
- "overflow": null,
- "overflow_x": null,
- "overflow_y": null,
- "padding": null,
- "right": null,
- "top": null,
- "visibility": null,
- "width": null
- }
- },
- "735bfe1c1ff6402d826257313d68b21b": {
- "model_module": "@jupyter-widgets/controls",
- "model_module_version": "1.5.0",
- "model_name": "HTMLModel",
- "state": {
- "_dom_classes": [],
- "_model_module": "@jupyter-widgets/controls",
- "_model_module_version": "1.5.0",
- "_model_name": "HTMLModel",
- "_view_count": null,
- "_view_module": "@jupyter-widgets/controls",
- "_view_module_version": "1.5.0",
- "_view_name": "HTMLView",
- "description": "",
- "description_tooltip": null,
- "layout": "IPY_MODEL_5edafe62c55449709c304c11dc847e82",
- "placeholder": "",
- "style": "IPY_MODEL_5ed53479e65749a68e2ae1f70fff1db0",
- "value": "Shuffling /root/tensorflow_datasets/trec/1.0.0.incompleteWOR5EP/trec-train.tfrecord*...: 0%"
- }
- },
- "771c7097b5be45ffac31e70117376903": {
- "model_module": "@jupyter-widgets/controls",
- "model_module_version": "1.5.0",
- "model_name": "HTMLModel",
- "state": {
- "_dom_classes": [],
- "_model_module": "@jupyter-widgets/controls",
- "_model_module_version": "1.5.0",
- "_model_name": "HTMLModel",
- "_view_count": null,
- "_view_module": "@jupyter-widgets/controls",
- "_view_module_version": "1.5.0",
- "_view_name": "HTMLView",
- "description": "",
- "description_tooltip": null,
- "layout": "IPY_MODEL_d9dd5b64f0ee40359a4c577f4358248f",
- "placeholder": "",
- "style": "IPY_MODEL_baf658cc369745b7bad7f18302e385d9",
- "value": " 0/5452 [00:00<?, ? examples/s]"
- }
- },
- "7c8e1e4eee714bc086c3f84eee5941fc": {
- "model_module": "@jupyter-widgets/controls",
- "model_module_version": "1.5.0",
- "model_name": "HBoxModel",
- "state": {
- "_dom_classes": [],
- "_model_module": "@jupyter-widgets/controls",
- "_model_module_version": "1.5.0",
- "_model_name": "HBoxModel",
- "_view_count": null,
- "_view_module": "@jupyter-widgets/controls",
- "_view_module_version": "1.5.0",
- "_view_name": "HBoxView",
- "box_style": "",
- "children": [
- "IPY_MODEL_4334b4f6fc7048eab0d146b6ffc437d1",
- "IPY_MODEL_c8e13e321a9a4e0b88c717a1a8065619",
- "IPY_MODEL_c1f14f69e97f44598d9c4f8d789131fb"
- ],
- "layout": "IPY_MODEL_25a079a3d58644f4a5306ee2cb42dbde"
- }
- },
- "7fa8f3f5c00c493bb7283bc90c33df5e": {
- "model_module": "@jupyter-widgets/controls",
- "model_module_version": "1.5.0",
- "model_name": "HBoxModel",
- "state": {
- "_dom_classes": [],
- "_model_module": "@jupyter-widgets/controls",
- "_model_module_version": "1.5.0",
- "_model_name": "HBoxModel",
- "_view_count": null,
- "_view_module": "@jupyter-widgets/controls",
- "_view_module_version": "1.5.0",
- "_view_name": "HBoxView",
- "box_style": "",
- "children": [
- "IPY_MODEL_59f2ed0257d6422d84fa23898e5d9a10",
- "IPY_MODEL_9236baef198c44ae8607c7888e899bc8",
- "IPY_MODEL_3647d22aaf934b53bf925dc22e2ce69a"
- ],
- "layout": "IPY_MODEL_687cfefc80a54edd84eb22baf805c905"
- }
- },
- "7fd82914b89343538aa08b36f4752f4b": {
- "model_module": "@jupyter-widgets/controls",
- "model_module_version": "1.5.0",
- "model_name": "HTMLModel",
- "state": {
- "_dom_classes": [],
- "_model_module": "@jupyter-widgets/controls",
- "_model_module_version": "1.5.0",
- "_model_name": "HTMLModel",
- "_view_count": null,
- "_view_module": "@jupyter-widgets/controls",
- "_view_module_version": "1.5.0",
- "_view_name": "HTMLView",
- "description": "",
- "description_tooltip": null,
- "layout": "IPY_MODEL_d426d4b5b86c4582a17bf21fe5f021d4",
- "placeholder": "",
- "style": "IPY_MODEL_2d69c8378b194aa3a8634848da5ee79a",
- "value": "Dl Size...: "
- }
- },
- "85f4ab947d774115baf64332f64248bd": {
- "model_module": "@jupyter-widgets/base",
- "model_module_version": "1.2.0",
- "model_name": "LayoutModel",
- "state": {
- "_model_module": "@jupyter-widgets/base",
- "_model_module_version": "1.2.0",
- "_model_name": "LayoutModel",
- "_view_count": null,
- "_view_module": "@jupyter-widgets/base",
- "_view_module_version": "1.2.0",
- "_view_name": "LayoutView",
- "align_content": null,
- "align_items": null,
- "align_self": null,
- "border": null,
- "bottom": null,
- "display": null,
- "flex": null,
- "flex_flow": null,
- "grid_area": null,
- "grid_auto_columns": null,
- "grid_auto_flow": null,
- "grid_auto_rows": null,
- "grid_column": null,
- "grid_gap": null,
- "grid_row": null,
- "grid_template_areas": null,
- "grid_template_columns": null,
- "grid_template_rows": null,
- "height": null,
- "justify_content": null,
- "justify_items": null,
- "left": null,
- "margin": null,
- "max_height": null,
- "max_width": null,
- "min_height": null,
- "min_width": null,
- "object_fit": null,
- "object_position": null,
- "order": null,
- "overflow": null,
- "overflow_x": null,
- "overflow_y": null,
- "padding": null,
- "right": null,
- "top": null,
- "visibility": "hidden",
- "width": null
- }
- },
- "8653f9b6a25446cc8d7d34ffd4e48ca8": {
- "model_module": "@jupyter-widgets/controls",
- "model_module_version": "1.5.0",
- "model_name": "DescriptionStyleModel",
- "state": {
- "_model_module": "@jupyter-widgets/controls",
- "_model_module_version": "1.5.0",
- "_model_name": "DescriptionStyleModel",
- "_view_count": null,
- "_view_module": "@jupyter-widgets/base",
- "_view_module_version": "1.2.0",
- "_view_name": "StyleView",
- "description_width": ""
- }
- },
- "88ff8d07f6ce430db2772b8bf24294e8": {
- "model_module": "@jupyter-widgets/base",
- "model_module_version": "1.2.0",
- "model_name": "LayoutModel",
- "state": {
- "_model_module": "@jupyter-widgets/base",
- "_model_module_version": "1.2.0",
- "_model_name": "LayoutModel",
- "_view_count": null,
- "_view_module": "@jupyter-widgets/base",
- "_view_module_version": "1.2.0",
- "_view_name": "LayoutView",
- "align_content": null,
- "align_items": null,
- "align_self": null,
- "border": null,
- "bottom": null,
- "display": null,
- "flex": null,
- "flex_flow": null,
- "grid_area": null,
- "grid_auto_columns": null,
- "grid_auto_flow": null,
- "grid_auto_rows": null,
- "grid_column": null,
- "grid_gap": null,
- "grid_row": null,
- "grid_template_areas": null,
- "grid_template_columns": null,
- "grid_template_rows": null,
- "height": null,
- "justify_content": null,
- "justify_items": null,
- "left": null,
- "margin": null,
- "max_height": null,
- "max_width": null,
- "min_height": null,
- "min_width": null,
- "object_fit": null,
- "object_position": null,
- "order": null,
- "overflow": null,
- "overflow_x": null,
- "overflow_y": null,
- "padding": null,
- "right": null,
- "top": null,
- "visibility": null,
- "width": "20px"
- }
- },
- "89f757fbb1ef4fa8b557c04926871096": {
- "model_module": "@jupyter-widgets/base",
- "model_module_version": "1.2.0",
- "model_name": "LayoutModel",
- "state": {
- "_model_module": "@jupyter-widgets/base",
- "_model_module_version": "1.2.0",
- "_model_name": "LayoutModel",
- "_view_count": null,
- "_view_module": "@jupyter-widgets/base",
- "_view_module_version": "1.2.0",
- "_view_name": "LayoutView",
- "align_content": null,
- "align_items": null,
- "align_self": null,
- "border": null,
- "bottom": null,
- "display": null,
- "flex": null,
- "flex_flow": null,
- "grid_area": null,
- "grid_auto_columns": null,
- "grid_auto_flow": null,
- "grid_auto_rows": null,
- "grid_column": null,
- "grid_gap": null,
- "grid_row": null,
- "grid_template_areas": null,
- "grid_template_columns": null,
- "grid_template_rows": null,
- "height": null,
- "justify_content": null,
- "justify_items": null,
- "left": null,
- "margin": null,
- "max_height": null,
- "max_width": null,
- "min_height": null,
- "min_width": null,
- "object_fit": null,
- "object_position": null,
- "order": null,
- "overflow": null,
- "overflow_x": null,
- "overflow_y": null,
- "padding": null,
- "right": null,
- "top": null,
- "visibility": null,
- "width": null
- }
- },
- "8e7d772e693c427c9bcfc7535a524979": {
- "model_module": "@jupyter-widgets/base",
- "model_module_version": "1.2.0",
- "model_name": "LayoutModel",
- "state": {
- "_model_module": "@jupyter-widgets/base",
- "_model_module_version": "1.2.0",
- "_model_name": "LayoutModel",
- "_view_count": null,
- "_view_module": "@jupyter-widgets/base",
- "_view_module_version": "1.2.0",
- "_view_name": "LayoutView",
- "align_content": null,
- "align_items": null,
- "align_self": null,
- "border": null,
- "bottom": null,
- "display": null,
- "flex": null,
- "flex_flow": null,
- "grid_area": null,
- "grid_auto_columns": null,
- "grid_auto_flow": null,
- "grid_auto_rows": null,
- "grid_column": null,
- "grid_gap": null,
- "grid_row": null,
- "grid_template_areas": null,
- "grid_template_columns": null,
- "grid_template_rows": null,
- "height": null,
- "justify_content": null,
- "justify_items": null,
- "left": null,
- "margin": null,
- "max_height": null,
- "max_width": null,
- "min_height": null,
- "min_width": null,
- "object_fit": null,
- "object_position": null,
- "order": null,
- "overflow": null,
- "overflow_x": null,
- "overflow_y": null,
- "padding": null,
- "right": null,
- "top": null,
- "visibility": null,
- "width": null
- }
- },
- "9236baef198c44ae8607c7888e899bc8": {
- "model_module": "@jupyter-widgets/controls",
- "model_module_version": "1.5.0",
- "model_name": "FloatProgressModel",
- "state": {
- "_dom_classes": [],
- "_model_module": "@jupyter-widgets/controls",
- "_model_module_version": "1.5.0",
- "_model_name": "FloatProgressModel",
- "_view_count": null,
- "_view_module": "@jupyter-widgets/controls",
- "_view_module_version": "1.5.0",
- "_view_name": "ProgressView",
- "bar_style": "success",
- "description": "",
- "description_tooltip": null,
- "layout": "IPY_MODEL_02e46682b2434a0c872f35db3d7b367e",
- "max": 1,
- "min": 0,
- "orientation": "horizontal",
- "style": "IPY_MODEL_3da4f3751cb54cf2938aea924ab33aa0",
- "value": 0
- }
- },
- "941ab80a14aa4a1bb3943dd77aca85d7": {
- "model_module": "@jupyter-widgets/controls",
- "model_module_version": "1.5.0",
- "model_name": "ProgressStyleModel",
- "state": {
- "_model_module": "@jupyter-widgets/controls",
- "_model_module_version": "1.5.0",
- "_model_name": "ProgressStyleModel",
- "_view_count": null,
- "_view_module": "@jupyter-widgets/base",
- "_view_module_version": "1.2.0",
- "_view_name": "StyleView",
- "bar_color": null,
- "description_width": ""
- }
- },
- "94ebb17c331649fc812a66231532ee03": {
- "model_module": "@jupyter-widgets/controls",
- "model_module_version": "1.5.0",
- "model_name": "DescriptionStyleModel",
- "state": {
- "_model_module": "@jupyter-widgets/controls",
- "_model_module_version": "1.5.0",
- "_model_name": "DescriptionStyleModel",
- "_view_count": null,
- "_view_module": "@jupyter-widgets/base",
- "_view_module_version": "1.2.0",
- "_view_name": "StyleView",
- "description_width": ""
- }
- },
- "996f65fcf7a04703868717652927e5e2": {
- "model_module": "@jupyter-widgets/base",
- "model_module_version": "1.2.0",
- "model_name": "LayoutModel",
- "state": {
- "_model_module": "@jupyter-widgets/base",
- "_model_module_version": "1.2.0",
- "_model_name": "LayoutModel",
- "_view_count": null,
- "_view_module": "@jupyter-widgets/base",
- "_view_module_version": "1.2.0",
- "_view_name": "LayoutView",
- "align_content": null,
- "align_items": null,
- "align_self": null,
- "border": null,
- "bottom": null,
- "display": null,
- "flex": null,
- "flex_flow": null,
- "grid_area": null,
- "grid_auto_columns": null,
- "grid_auto_flow": null,
- "grid_auto_rows": null,
- "grid_column": null,
- "grid_gap": null,
- "grid_row": null,
- "grid_template_areas": null,
- "grid_template_columns": null,
- "grid_template_rows": null,
- "height": null,
- "justify_content": null,
- "justify_items": null,
- "left": null,
- "margin": null,
- "max_height": null,
- "max_width": null,
- "min_height": null,
- "min_width": null,
- "object_fit": null,
- "object_position": null,
- "order": null,
- "overflow": null,
- "overflow_x": null,
- "overflow_y": null,
- "padding": null,
- "right": null,
- "top": null,
- "visibility": "hidden",
- "width": null
- }
- },
- "9e46610e715444cbbc2f978a0acd0ed0": {
- "model_module": "@jupyter-widgets/controls",
- "model_module_version": "1.5.0",
- "model_name": "DescriptionStyleModel",
- "state": {
- "_model_module": "@jupyter-widgets/controls",
- "_model_module_version": "1.5.0",
- "_model_name": "DescriptionStyleModel",
- "_view_count": null,
- "_view_module": "@jupyter-widgets/base",
- "_view_module_version": "1.2.0",
- "_view_name": "StyleView",
- "description_width": ""
- }
- },
- "a61d6191c53e4dd2a1cbdb9826f5ae70": {
- "model_module": "@jupyter-widgets/controls",
- "model_module_version": "1.5.0",
- "model_name": "HBoxModel",
- "state": {
- "_dom_classes": [],
- "_model_module": "@jupyter-widgets/controls",
- "_model_module_version": "1.5.0",
- "_model_name": "HBoxModel",
- "_view_count": null,
- "_view_module": "@jupyter-widgets/controls",
- "_view_module_version": "1.5.0",
- "_view_name": "HBoxView",
- "box_style": "",
- "children": [
- "IPY_MODEL_e868b829ec7a4795955034ebb8b109a6",
- "IPY_MODEL_1385286d855a46c8b19704a95e5992b4",
- "IPY_MODEL_b7bbc695e0f248a0ac1e79a1d546df8e"
- ],
- "layout": "IPY_MODEL_034072a591b94d3daf4284aadcff935f"
- }
- },
- "a9f4707824454e35b2273412c4586b8d": {
- "model_module": "@jupyter-widgets/controls",
- "model_module_version": "1.5.0",
- "model_name": "HTMLModel",
- "state": {
- "_dom_classes": [],
- "_model_module": "@jupyter-widgets/controls",
- "_model_module_version": "1.5.0",
- "_model_name": "HTMLModel",
- "_view_count": null,
- "_view_module": "@jupyter-widgets/controls",
- "_view_module_version": "1.5.0",
- "_view_name": "HTMLView",
- "description": "",
- "description_tooltip": null,
- "layout": "IPY_MODEL_efa03832003244a490cfcb34edf7ccbe",
- "placeholder": "",
- "style": "IPY_MODEL_9e46610e715444cbbc2f978a0acd0ed0",
- "value": "Generating splits...: 50%"
- }
- },
- "b04f406f1a2a46818d034a80d13f6664": {
- "model_module": "@jupyter-widgets/base",
- "model_module_version": "1.2.0",
- "model_name": "LayoutModel",
- "state": {
- "_model_module": "@jupyter-widgets/base",
- "_model_module_version": "1.2.0",
- "_model_name": "LayoutModel",
- "_view_count": null,
- "_view_module": "@jupyter-widgets/base",
- "_view_module_version": "1.2.0",
- "_view_name": "LayoutView",
- "align_content": null,
- "align_items": null,
- "align_self": null,
- "border": null,
- "bottom": null,
- "display": null,
- "flex": null,
- "flex_flow": null,
- "grid_area": null,
- "grid_auto_columns": null,
- "grid_auto_flow": null,
- "grid_auto_rows": null,
- "grid_column": null,
- "grid_gap": null,
- "grid_row": null,
- "grid_template_areas": null,
- "grid_template_columns": null,
- "grid_template_rows": null,
- "height": null,
- "justify_content": null,
- "justify_items": null,
- "left": null,
- "margin": null,
- "max_height": null,
- "max_width": null,
- "min_height": null,
- "min_width": null,
- "object_fit": null,
- "object_position": null,
- "order": null,
- "overflow": null,
- "overflow_x": null,
- "overflow_y": null,
- "padding": null,
- "right": null,
- "top": null,
- "visibility": null,
- "width": null
- }
- },
- "b07ffeedc45a4fe9919f99d71a2ea3f2": {
- "model_module": "@jupyter-widgets/controls",
- "model_module_version": "1.5.0",
- "model_name": "HBoxModel",
- "state": {
- "_dom_classes": [],
- "_model_module": "@jupyter-widgets/controls",
- "_model_module_version": "1.5.0",
- "_model_name": "HBoxModel",
- "_view_count": null,
- "_view_module": "@jupyter-widgets/controls",
- "_view_module_version": "1.5.0",
- "_view_name": "HBoxView",
- "box_style": "",
- "children": [
- "IPY_MODEL_211f7336a3b9455f8cd67bd0ee4113ae",
- "IPY_MODEL_2bd5fe14d3d44d85be32c5c78f5b7568",
- "IPY_MODEL_6850b87ff2fa49eb9fbfb3c3492efbb3"
- ],
- "layout": "IPY_MODEL_6def2ecb657845a3bae18bad85950991"
- }
- },
- "b7bbc695e0f248a0ac1e79a1d546df8e": {
- "model_module": "@jupyter-widgets/controls",
- "model_module_version": "1.5.0",
- "model_name": "HTMLModel",
- "state": {
- "_dom_classes": [],
- "_model_module": "@jupyter-widgets/controls",
- "_model_module_version": "1.5.0",
- "_model_name": "HTMLModel",
- "_view_count": null,
- "_view_module": "@jupyter-widgets/controls",
- "_view_module_version": "1.5.0",
- "_view_name": "HTMLView",
- "description": "",
- "description_tooltip": null,
- "layout": "IPY_MODEL_8e7d772e693c427c9bcfc7535a524979",
- "placeholder": "",
- "style": "IPY_MODEL_d646a9a4cc5a4876a7fa185bf9eb3ccf",
- "value": " 0/5452 [00:00<?, ? examples/s]"
- }
- },
- "baf658cc369745b7bad7f18302e385d9": {
- "model_module": "@jupyter-widgets/controls",
- "model_module_version": "1.5.0",
- "model_name": "DescriptionStyleModel",
- "state": {
- "_model_module": "@jupyter-widgets/controls",
- "_model_module_version": "1.5.0",
- "_model_name": "DescriptionStyleModel",
- "_view_count": null,
- "_view_module": "@jupyter-widgets/base",
- "_view_module_version": "1.2.0",
- "_view_name": "StyleView",
- "description_width": ""
- }
- },
- "be8b3f4e73fc44f390e9a147df786e53": {
- "model_module": "@jupyter-widgets/base",
- "model_module_version": "1.2.0",
- "model_name": "LayoutModel",
- "state": {
- "_model_module": "@jupyter-widgets/base",
- "_model_module_version": "1.2.0",
- "_model_name": "LayoutModel",
- "_view_count": null,
- "_view_module": "@jupyter-widgets/base",
- "_view_module_version": "1.2.0",
- "_view_name": "LayoutView",
- "align_content": null,
- "align_items": null,
- "align_self": null,
- "border": null,
- "bottom": null,
- "display": null,
- "flex": null,
- "flex_flow": null,
- "grid_area": null,
- "grid_auto_columns": null,
- "grid_auto_flow": null,
- "grid_auto_rows": null,
- "grid_column": null,
- "grid_gap": null,
- "grid_row": null,
- "grid_template_areas": null,
- "grid_template_columns": null,
- "grid_template_rows": null,
- "height": null,
- "justify_content": null,
- "justify_items": null,
- "left": null,
- "margin": null,
- "max_height": null,
- "max_width": null,
- "min_height": null,
- "min_width": null,
- "object_fit": null,
- "object_position": null,
- "order": null,
- "overflow": null,
- "overflow_x": null,
- "overflow_y": null,
- "padding": null,
- "right": null,
- "top": null,
- "visibility": null,
- "width": null
- }
- },
- "bf6033cd5d0c4ef5ac934ed6e8adaeb6": {
- "model_module": "@jupyter-widgets/base",
- "model_module_version": "1.2.0",
- "model_name": "LayoutModel",
- "state": {
- "_model_module": "@jupyter-widgets/base",
- "_model_module_version": "1.2.0",
- "_model_name": "LayoutModel",
- "_view_count": null,
- "_view_module": "@jupyter-widgets/base",
- "_view_module_version": "1.2.0",
- "_view_name": "LayoutView",
- "align_content": null,
- "align_items": null,
- "align_self": null,
- "border": null,
- "bottom": null,
- "display": null,
- "flex": null,
- "flex_flow": null,
- "grid_area": null,
- "grid_auto_columns": null,
- "grid_auto_flow": null,
- "grid_auto_rows": null,
- "grid_column": null,
- "grid_gap": null,
- "grid_row": null,
- "grid_template_areas": null,
- "grid_template_columns": null,
- "grid_template_rows": null,
- "height": null,
- "justify_content": null,
- "justify_items": null,
- "left": null,
- "margin": null,
- "max_height": null,
- "max_width": null,
- "min_height": null,
- "min_width": null,
- "object_fit": null,
- "object_position": null,
- "order": null,
- "overflow": null,
- "overflow_x": null,
- "overflow_y": null,
- "padding": null,
- "right": null,
- "top": null,
- "visibility": "hidden",
- "width": null
- }
- },
- "bfb987eb24644819ba52b2fc7bc865de": {
- "model_module": "@jupyter-widgets/controls",
- "model_module_version": "1.5.0",
- "model_name": "ProgressStyleModel",
- "state": {
- "_model_module": "@jupyter-widgets/controls",
- "_model_module_version": "1.5.0",
- "_model_name": "ProgressStyleModel",
- "_view_count": null,
- "_view_module": "@jupyter-widgets/base",
- "_view_module_version": "1.2.0",
- "_view_name": "StyleView",
- "bar_color": null,
- "description_width": ""
- }
- },
- "c05f2a082d5e4a0c8f3b9b6952eff208": {
- "model_module": "@jupyter-widgets/controls",
- "model_module_version": "1.5.0",
- "model_name": "HTMLModel",
- "state": {
- "_dom_classes": [],
- "_model_module": "@jupyter-widgets/controls",
- "_model_module_version": "1.5.0",
- "_model_name": "HTMLModel",
- "_view_count": null,
- "_view_module": "@jupyter-widgets/controls",
- "_view_module_version": "1.5.0",
- "_view_name": "HTMLView",
- "description": "",
- "description_tooltip": null,
- "layout": "IPY_MODEL_57cf77bba44141359639c0268ced8934",
- "placeholder": "",
- "style": "IPY_MODEL_6a7d1421176f4b62af003a4b2f18815b",
- "value": " 0/500 [00:00<?, ? examples/s]"
- }
- },
- "c1f14f69e97f44598d9c4f8d789131fb": {
- "model_module": "@jupyter-widgets/controls",
- "model_module_version": "1.5.0",
- "model_name": "HTMLModel",
- "state": {
- "_dom_classes": [],
- "_model_module": "@jupyter-widgets/controls",
- "_model_module_version": "1.5.0",
- "_model_name": "HTMLModel",
- "_view_count": null,
- "_view_module": "@jupyter-widgets/controls",
- "_view_module_version": "1.5.0",
- "_view_name": "HTMLView",
- "description": "",
- "description_tooltip": null,
- "layout": "IPY_MODEL_1f1331ab4e734cdaa66cf28d7278f980",
- "placeholder": "",
- "style": "IPY_MODEL_d773c0f65a014ae08f533bf2348c1831",
- "value": " 2/2 [00:04<00:00, 2.35s/ url]"
- }
- },
- "c37f294926f945898096874d324ee2ae": {
- "model_module": "@jupyter-widgets/base",
- "model_module_version": "1.2.0",
- "model_name": "LayoutModel",
- "state": {
- "_model_module": "@jupyter-widgets/base",
- "_model_module_version": "1.2.0",
- "_model_name": "LayoutModel",
- "_view_count": null,
- "_view_module": "@jupyter-widgets/base",
- "_view_module_version": "1.2.0",
- "_view_name": "LayoutView",
- "align_content": null,
- "align_items": null,
- "align_self": null,
- "border": null,
- "bottom": null,
- "display": null,
- "flex": null,
- "flex_flow": null,
- "grid_area": null,
- "grid_auto_columns": null,
- "grid_auto_flow": null,
- "grid_auto_rows": null,
- "grid_column": null,
- "grid_gap": null,
- "grid_row": null,
- "grid_template_areas": null,
- "grid_template_columns": null,
- "grid_template_rows": null,
- "height": null,
- "justify_content": null,
- "justify_items": null,
- "left": null,
- "margin": null,
- "max_height": null,
- "max_width": null,
- "min_height": null,
- "min_width": null,
- "object_fit": null,
- "object_position": null,
- "order": null,
- "overflow": null,
- "overflow_x": null,
- "overflow_y": null,
- "padding": null,
- "right": null,
- "top": null,
- "visibility": null,
- "width": "20px"
- }
- },
- "c5296022707240b393be4146e983083c": {
- "model_module": "@jupyter-widgets/base",
- "model_module_version": "1.2.0",
- "model_name": "LayoutModel",
- "state": {
- "_model_module": "@jupyter-widgets/base",
- "_model_module_version": "1.2.0",
- "_model_name": "LayoutModel",
- "_view_count": null,
- "_view_module": "@jupyter-widgets/base",
- "_view_module_version": "1.2.0",
- "_view_name": "LayoutView",
- "align_content": null,
- "align_items": null,
- "align_self": null,
- "border": null,
- "bottom": null,
- "display": null,
- "flex": null,
- "flex_flow": null,
- "grid_area": null,
- "grid_auto_columns": null,
- "grid_auto_flow": null,
- "grid_auto_rows": null,
- "grid_column": null,
- "grid_gap": null,
- "grid_row": null,
- "grid_template_areas": null,
- "grid_template_columns": null,
- "grid_template_rows": null,
- "height": null,
- "justify_content": null,
- "justify_items": null,
- "left": null,
- "margin": null,
- "max_height": null,
- "max_width": null,
- "min_height": null,
- "min_width": null,
- "object_fit": null,
- "object_position": null,
- "order": null,
- "overflow": null,
- "overflow_x": null,
- "overflow_y": null,
- "padding": null,
- "right": null,
- "top": null,
- "visibility": null,
- "width": null
- }
- },
- "c8e13e321a9a4e0b88c717a1a8065619": {
- "model_module": "@jupyter-widgets/controls",
- "model_module_version": "1.5.0",
- "model_name": "FloatProgressModel",
- "state": {
- "_dom_classes": [],
- "_model_module": "@jupyter-widgets/controls",
- "_model_module_version": "1.5.0",
- "_model_name": "FloatProgressModel",
- "_view_count": null,
- "_view_module": "@jupyter-widgets/controls",
- "_view_module_version": "1.5.0",
- "_view_name": "ProgressView",
- "bar_style": "success",
- "description": "",
- "description_tooltip": null,
- "layout": "IPY_MODEL_c37f294926f945898096874d324ee2ae",
- "max": 1,
- "min": 0,
- "orientation": "horizontal",
- "style": "IPY_MODEL_941ab80a14aa4a1bb3943dd77aca85d7",
- "value": 1
- }
- },
- "cd23755e6122427e9d95eb808b7900c2": {
- "model_module": "@jupyter-widgets/controls",
- "model_module_version": "1.5.0",
- "model_name": "HBoxModel",
- "state": {
- "_dom_classes": [],
- "_model_module": "@jupyter-widgets/controls",
- "_model_module_version": "1.5.0",
- "_model_name": "HBoxModel",
- "_view_count": null,
- "_view_module": "@jupyter-widgets/controls",
- "_view_module_version": "1.5.0",
- "_view_name": "HBoxView",
- "box_style": "",
- "children": [
- "IPY_MODEL_62eb05321e2d4265b033544921f92053",
- "IPY_MODEL_3627640fb7d14ac09c9fdcbfb2159152",
- "IPY_MODEL_c05f2a082d5e4a0c8f3b9b6952eff208"
- ],
- "layout": "IPY_MODEL_bf6033cd5d0c4ef5ac934ed6e8adaeb6"
- }
- },
- "ce82f835dd2b45a181a53407505b0cfc": {
- "model_module": "@jupyter-widgets/controls",
- "model_module_version": "1.5.0",
- "model_name": "DescriptionStyleModel",
- "state": {
- "_model_module": "@jupyter-widgets/controls",
- "_model_module_version": "1.5.0",
- "_model_name": "DescriptionStyleModel",
- "_view_count": null,
- "_view_module": "@jupyter-widgets/base",
- "_view_module_version": "1.2.0",
- "_view_name": "StyleView",
- "description_width": ""
- }
- },
- "d24e60c7e4664e0ba11cce85c647e526": {
- "model_module": "@jupyter-widgets/base",
- "model_module_version": "1.2.0",
- "model_name": "LayoutModel",
- "state": {
- "_model_module": "@jupyter-widgets/base",
- "_model_module_version": "1.2.0",
- "_model_name": "LayoutModel",
- "_view_count": null,
- "_view_module": "@jupyter-widgets/base",
- "_view_module_version": "1.2.0",
- "_view_name": "LayoutView",
- "align_content": null,
- "align_items": null,
- "align_self": null,
- "border": null,
- "bottom": null,
- "display": null,
- "flex": null,
- "flex_flow": null,
- "grid_area": null,
- "grid_auto_columns": null,
- "grid_auto_flow": null,
- "grid_auto_rows": null,
- "grid_column": null,
- "grid_gap": null,
- "grid_row": null,
- "grid_template_areas": null,
- "grid_template_columns": null,
- "grid_template_rows": null,
- "height": null,
- "justify_content": null,
- "justify_items": null,
- "left": null,
- "margin": null,
- "max_height": null,
- "max_width": null,
- "min_height": null,
- "min_width": null,
- "object_fit": null,
- "object_position": null,
- "order": null,
- "overflow": null,
- "overflow_x": null,
- "overflow_y": null,
- "padding": null,
- "right": null,
- "top": null,
- "visibility": null,
- "width": null
- }
- },
- "d298dc9202dc49aba09b66969b3d19f9": {
- "model_module": "@jupyter-widgets/base",
- "model_module_version": "1.2.0",
- "model_name": "LayoutModel",
- "state": {
- "_model_module": "@jupyter-widgets/base",
- "_model_module_version": "1.2.0",
- "_model_name": "LayoutModel",
- "_view_count": null,
- "_view_module": "@jupyter-widgets/base",
- "_view_module_version": "1.2.0",
- "_view_name": "LayoutView",
- "align_content": null,
- "align_items": null,
- "align_self": null,
- "border": null,
- "bottom": null,
- "display": null,
- "flex": null,
- "flex_flow": null,
- "grid_area": null,
- "grid_auto_columns": null,
- "grid_auto_flow": null,
- "grid_auto_rows": null,
- "grid_column": null,
- "grid_gap": null,
- "grid_row": null,
- "grid_template_areas": null,
- "grid_template_columns": null,
- "grid_template_rows": null,
- "height": null,
- "justify_content": null,
- "justify_items": null,
- "left": null,
- "margin": null,
- "max_height": null,
- "max_width": null,
- "min_height": null,
- "min_width": null,
- "object_fit": null,
- "object_position": null,
- "order": null,
- "overflow": null,
- "overflow_x": null,
- "overflow_y": null,
- "padding": null,
- "right": null,
- "top": null,
- "visibility": null,
- "width": null
- }
- },
- "d426d4b5b86c4582a17bf21fe5f021d4": {
- "model_module": "@jupyter-widgets/base",
- "model_module_version": "1.2.0",
- "model_name": "LayoutModel",
- "state": {
- "_model_module": "@jupyter-widgets/base",
- "_model_module_version": "1.2.0",
- "_model_name": "LayoutModel",
- "_view_count": null,
- "_view_module": "@jupyter-widgets/base",
- "_view_module_version": "1.2.0",
- "_view_name": "LayoutView",
- "align_content": null,
- "align_items": null,
- "align_self": null,
- "border": null,
- "bottom": null,
- "display": null,
- "flex": null,
- "flex_flow": null,
- "grid_area": null,
- "grid_auto_columns": null,
- "grid_auto_flow": null,
- "grid_auto_rows": null,
- "grid_column": null,
- "grid_gap": null,
- "grid_row": null,
- "grid_template_areas": null,
- "grid_template_columns": null,
- "grid_template_rows": null,
- "height": null,
- "justify_content": null,
- "justify_items": null,
- "left": null,
- "margin": null,
- "max_height": null,
- "max_width": null,
- "min_height": null,
- "min_width": null,
- "object_fit": null,
- "object_position": null,
- "order": null,
- "overflow": null,
- "overflow_x": null,
- "overflow_y": null,
- "padding": null,
- "right": null,
- "top": null,
- "visibility": null,
- "width": null
- }
- },
- "d646a9a4cc5a4876a7fa185bf9eb3ccf": {
- "model_module": "@jupyter-widgets/controls",
- "model_module_version": "1.5.0",
- "model_name": "DescriptionStyleModel",
- "state": {
- "_model_module": "@jupyter-widgets/controls",
- "_model_module_version": "1.5.0",
- "_model_name": "DescriptionStyleModel",
- "_view_count": null,
- "_view_module": "@jupyter-widgets/base",
- "_view_module_version": "1.2.0",
- "_view_name": "StyleView",
- "description_width": ""
- }
- },
- "d773c0f65a014ae08f533bf2348c1831": {
- "model_module": "@jupyter-widgets/controls",
- "model_module_version": "1.5.0",
- "model_name": "DescriptionStyleModel",
- "state": {
- "_model_module": "@jupyter-widgets/controls",
- "_model_module_version": "1.5.0",
- "_model_name": "DescriptionStyleModel",
- "_view_count": null,
- "_view_module": "@jupyter-widgets/base",
- "_view_module_version": "1.2.0",
- "_view_name": "StyleView",
- "description_width": ""
- }
- },
- "d849ae343e51419c803c29c24a16dd01": {
- "model_module": "@jupyter-widgets/controls",
- "model_module_version": "1.5.0",
- "model_name": "FloatProgressModel",
- "state": {
- "_dom_classes": [],
- "_model_module": "@jupyter-widgets/controls",
- "_model_module_version": "1.5.0",
- "_model_name": "FloatProgressModel",
- "_view_count": null,
- "_view_module": "@jupyter-widgets/controls",
- "_view_module_version": "1.5.0",
- "_view_name": "ProgressView",
- "bar_style": "",
- "description": "",
- "description_tooltip": null,
- "layout": "IPY_MODEL_3948a575b3ee4a199f1055e93d8270ee",
- "max": 2,
- "min": 0,
- "orientation": "horizontal",
- "style": "IPY_MODEL_2ee59cbc091346168c3b4511f64b018a",
- "value": 2
- }
- },
- "d9dd5b64f0ee40359a4c577f4358248f": {
- "model_module": "@jupyter-widgets/base",
- "model_module_version": "1.2.0",
- "model_name": "LayoutModel",
- "state": {
- "_model_module": "@jupyter-widgets/base",
- "_model_module_version": "1.2.0",
- "_model_name": "LayoutModel",
- "_view_count": null,
- "_view_module": "@jupyter-widgets/base",
- "_view_module_version": "1.2.0",
- "_view_name": "LayoutView",
- "align_content": null,
- "align_items": null,
- "align_self": null,
- "border": null,
- "bottom": null,
- "display": null,
- "flex": null,
- "flex_flow": null,
- "grid_area": null,
- "grid_auto_columns": null,
- "grid_auto_flow": null,
- "grid_auto_rows": null,
- "grid_column": null,
- "grid_gap": null,
- "grid_row": null,
- "grid_template_areas": null,
- "grid_template_columns": null,
- "grid_template_rows": null,
- "height": null,
- "justify_content": null,
- "justify_items": null,
- "left": null,
- "margin": null,
- "max_height": null,
- "max_width": null,
- "min_height": null,
- "min_width": null,
- "object_fit": null,
- "object_position": null,
- "order": null,
- "overflow": null,
- "overflow_x": null,
- "overflow_y": null,
- "padding": null,
- "right": null,
- "top": null,
- "visibility": null,
- "width": null
- }
- },
- "df1b4d3501ed457b86b60a5449d1e1da": {
- "model_module": "@jupyter-widgets/controls",
- "model_module_version": "1.5.0",
- "model_name": "FloatProgressModel",
- "state": {
- "_dom_classes": [],
- "_model_module": "@jupyter-widgets/controls",
- "_model_module_version": "1.5.0",
- "_model_name": "FloatProgressModel",
- "_view_count": null,
- "_view_module": "@jupyter-widgets/controls",
- "_view_module_version": "1.5.0",
- "_view_name": "ProgressView",
- "bar_style": "",
- "description": "",
- "description_tooltip": null,
- "layout": "IPY_MODEL_b04f406f1a2a46818d034a80d13f6664",
- "max": 5452,
- "min": 0,
- "orientation": "horizontal",
- "style": "IPY_MODEL_fcc06cd646ff425b9172312808dd3e24",
- "value": 5452
- }
- },
- "df522159441841cda1f97d63aab5ad9e": {
- "model_module": "@jupyter-widgets/controls",
- "model_module_version": "1.5.0",
- "model_name": "HBoxModel",
- "state": {
- "_dom_classes": [],
- "_model_module": "@jupyter-widgets/controls",
- "_model_module_version": "1.5.0",
- "_model_name": "HBoxModel",
- "_view_count": null,
- "_view_module": "@jupyter-widgets/controls",
- "_view_module_version": "1.5.0",
- "_view_name": "HBoxView",
- "box_style": "",
- "children": [
- "IPY_MODEL_a9f4707824454e35b2273412c4586b8d",
- "IPY_MODEL_d849ae343e51419c803c29c24a16dd01",
- "IPY_MODEL_65b7f23304a243878c43ea391885ee85"
- ],
- "layout": "IPY_MODEL_85f4ab947d774115baf64332f64248bd"
- }
- },
- "e868b829ec7a4795955034ebb8b109a6": {
- "model_module": "@jupyter-widgets/controls",
- "model_module_version": "1.5.0",
- "model_name": "HTMLModel",
- "state": {
- "_dom_classes": [],
- "_model_module": "@jupyter-widgets/controls",
- "_model_module_version": "1.5.0",
- "_model_name": "HTMLModel",
- "_view_count": null,
- "_view_module": "@jupyter-widgets/controls",
- "_view_module_version": "1.5.0",
- "_view_name": "HTMLView",
- "description": "",
- "description_tooltip": null,
- "layout": "IPY_MODEL_d24e60c7e4664e0ba11cce85c647e526",
- "placeholder": "",
- "style": "IPY_MODEL_23bbe3c6dfcb42c7a0a1c826bc35076d",
- "value": "Generating train examples...: 0%"
- }
- },
- "efa03832003244a490cfcb34edf7ccbe": {
- "model_module": "@jupyter-widgets/base",
- "model_module_version": "1.2.0",
- "model_name": "LayoutModel",
- "state": {
- "_model_module": "@jupyter-widgets/base",
- "_model_module_version": "1.2.0",
- "_model_name": "LayoutModel",
- "_view_count": null,
- "_view_module": "@jupyter-widgets/base",
- "_view_module_version": "1.2.0",
- "_view_name": "LayoutView",
- "align_content": null,
- "align_items": null,
- "align_self": null,
- "border": null,
- "bottom": null,
- "display": null,
- "flex": null,
- "flex_flow": null,
- "grid_area": null,
- "grid_auto_columns": null,
- "grid_auto_flow": null,
- "grid_auto_rows": null,
- "grid_column": null,
- "grid_gap": null,
- "grid_row": null,
- "grid_template_areas": null,
- "grid_template_columns": null,
- "grid_template_rows": null,
- "height": null,
- "justify_content": null,
- "justify_items": null,
- "left": null,
- "margin": null,
- "max_height": null,
- "max_width": null,
- "min_height": null,
- "min_width": null,
- "object_fit": null,
- "object_position": null,
- "order": null,
- "overflow": null,
- "overflow_x": null,
- "overflow_y": null,
- "padding": null,
- "right": null,
- "top": null,
- "visibility": null,
- "width": null
- }
- },
- "f5e168c246714295a10a64fbb1ef2ee3": {
- "model_module": "@jupyter-widgets/controls",
- "model_module_version": "1.5.0",
- "model_name": "ProgressStyleModel",
- "state": {
- "_model_module": "@jupyter-widgets/controls",
- "_model_module_version": "1.5.0",
- "_model_name": "ProgressStyleModel",
- "_view_count": null,
- "_view_module": "@jupyter-widgets/base",
- "_view_module_version": "1.2.0",
- "_view_name": "StyleView",
- "bar_color": null,
- "description_width": ""
- }
- },
- "fcc06cd646ff425b9172312808dd3e24": {
- "model_module": "@jupyter-widgets/controls",
- "model_module_version": "1.5.0",
- "model_name": "ProgressStyleModel",
- "state": {
- "_model_module": "@jupyter-widgets/controls",
- "_model_module_version": "1.5.0",
- "_model_name": "ProgressStyleModel",
- "_view_count": null,
- "_view_module": "@jupyter-widgets/base",
- "_view_module_version": "1.2.0",
- "_view_name": "StyleView",
- "bar_color": null,
- "description_width": ""
- }
- }
- }
+ "a9f4707824454e35b2273412c4586b8d": {
+ "model_module": "@jupyter-widgets/controls",
+ "model_module_version": "1.5.0",
+ "model_name": "HTMLModel",
+ "state": {
+ "_dom_classes": [],
+ "_model_module": "@jupyter-widgets/controls",
+ "_model_module_version": "1.5.0",
+ "_model_name": "HTMLModel",
+ "_view_count": null,
+ "_view_module": "@jupyter-widgets/controls",
+ "_view_module_version": "1.5.0",
+ "_view_name": "HTMLView",
+ "description": "",
+ "description_tooltip": null,
+ "layout": "IPY_MODEL_efa03832003244a490cfcb34edf7ccbe",
+ "placeholder": "",
+ "style": "IPY_MODEL_9e46610e715444cbbc2f978a0acd0ed0",
+ "value": "Generating splits...: 50%"
+ }
+ },
+ "b04f406f1a2a46818d034a80d13f6664": {
+ "model_module": "@jupyter-widgets/base",
+ "model_module_version": "1.2.0",
+ "model_name": "LayoutModel",
+ "state": {
+ "_model_module": "@jupyter-widgets/base",
+ "_model_module_version": "1.2.0",
+ "_model_name": "LayoutModel",
+ "_view_count": null,
+ "_view_module": "@jupyter-widgets/base",
+ "_view_module_version": "1.2.0",
+ "_view_name": "LayoutView",
+ "align_content": null,
+ "align_items": null,
+ "align_self": null,
+ "border": null,
+ "bottom": null,
+ "display": null,
+ "flex": null,
+ "flex_flow": null,
+ "grid_area": null,
+ "grid_auto_columns": null,
+ "grid_auto_flow": null,
+ "grid_auto_rows": null,
+ "grid_column": null,
+ "grid_gap": null,
+ "grid_row": null,
+ "grid_template_areas": null,
+ "grid_template_columns": null,
+ "grid_template_rows": null,
+ "height": null,
+ "justify_content": null,
+ "justify_items": null,
+ "left": null,
+ "margin": null,
+ "max_height": null,
+ "max_width": null,
+ "min_height": null,
+ "min_width": null,
+ "object_fit": null,
+ "object_position": null,
+ "order": null,
+ "overflow": null,
+ "overflow_x": null,
+ "overflow_y": null,
+ "padding": null,
+ "right": null,
+ "top": null,
+ "visibility": null,
+ "width": null
+ }
+ },
+ "b07ffeedc45a4fe9919f99d71a2ea3f2": {
+ "model_module": "@jupyter-widgets/controls",
+ "model_module_version": "1.5.0",
+ "model_name": "HBoxModel",
+ "state": {
+ "_dom_classes": [],
+ "_model_module": "@jupyter-widgets/controls",
+ "_model_module_version": "1.5.0",
+ "_model_name": "HBoxModel",
+ "_view_count": null,
+ "_view_module": "@jupyter-widgets/controls",
+ "_view_module_version": "1.5.0",
+ "_view_name": "HBoxView",
+ "box_style": "",
+ "children": [
+ "IPY_MODEL_211f7336a3b9455f8cd67bd0ee4113ae",
+ "IPY_MODEL_2bd5fe14d3d44d85be32c5c78f5b7568",
+ "IPY_MODEL_6850b87ff2fa49eb9fbfb3c3492efbb3"
+ ],
+ "layout": "IPY_MODEL_6def2ecb657845a3bae18bad85950991"
+ }
+ },
+ "b7bbc695e0f248a0ac1e79a1d546df8e": {
+ "model_module": "@jupyter-widgets/controls",
+ "model_module_version": "1.5.0",
+ "model_name": "HTMLModel",
+ "state": {
+ "_dom_classes": [],
+ "_model_module": "@jupyter-widgets/controls",
+ "_model_module_version": "1.5.0",
+ "_model_name": "HTMLModel",
+ "_view_count": null,
+ "_view_module": "@jupyter-widgets/controls",
+ "_view_module_version": "1.5.0",
+ "_view_name": "HTMLView",
+ "description": "",
+ "description_tooltip": null,
+ "layout": "IPY_MODEL_8e7d772e693c427c9bcfc7535a524979",
+ "placeholder": "",
+ "style": "IPY_MODEL_d646a9a4cc5a4876a7fa185bf9eb3ccf",
+ "value": " 0/5452 [00:00<?, ? examples/s]"
+ }
+ },
+ "baf658cc369745b7bad7f18302e385d9": {
+ "model_module": "@jupyter-widgets/controls",
+ "model_module_version": "1.5.0",
+ "model_name": "DescriptionStyleModel",
+ "state": {
+ "_model_module": "@jupyter-widgets/controls",
+ "_model_module_version": "1.5.0",
+ "_model_name": "DescriptionStyleModel",
+ "_view_count": null,
+ "_view_module": "@jupyter-widgets/base",
+ "_view_module_version": "1.2.0",
+ "_view_name": "StyleView",
+ "description_width": ""
+ }
+ },
+ "be8b3f4e73fc44f390e9a147df786e53": {
+ "model_module": "@jupyter-widgets/base",
+ "model_module_version": "1.2.0",
+ "model_name": "LayoutModel",
+ "state": {
+ "_model_module": "@jupyter-widgets/base",
+ "_model_module_version": "1.2.0",
+ "_model_name": "LayoutModel",
+ "_view_count": null,
+ "_view_module": "@jupyter-widgets/base",
+ "_view_module_version": "1.2.0",
+ "_view_name": "LayoutView",
+ "align_content": null,
+ "align_items": null,
+ "align_self": null,
+ "border": null,
+ "bottom": null,
+ "display": null,
+ "flex": null,
+ "flex_flow": null,
+ "grid_area": null,
+ "grid_auto_columns": null,
+ "grid_auto_flow": null,
+ "grid_auto_rows": null,
+ "grid_column": null,
+ "grid_gap": null,
+ "grid_row": null,
+ "grid_template_areas": null,
+ "grid_template_columns": null,
+ "grid_template_rows": null,
+ "height": null,
+ "justify_content": null,
+ "justify_items": null,
+ "left": null,
+ "margin": null,
+ "max_height": null,
+ "max_width": null,
+ "min_height": null,
+ "min_width": null,
+ "object_fit": null,
+ "object_position": null,
+ "order": null,
+ "overflow": null,
+ "overflow_x": null,
+ "overflow_y": null,
+ "padding": null,
+ "right": null,
+ "top": null,
+ "visibility": null,
+ "width": null
+ }
+ },
+ "bf6033cd5d0c4ef5ac934ed6e8adaeb6": {
+ "model_module": "@jupyter-widgets/base",
+ "model_module_version": "1.2.0",
+ "model_name": "LayoutModel",
+ "state": {
+ "_model_module": "@jupyter-widgets/base",
+ "_model_module_version": "1.2.0",
+ "_model_name": "LayoutModel",
+ "_view_count": null,
+ "_view_module": "@jupyter-widgets/base",
+ "_view_module_version": "1.2.0",
+ "_view_name": "LayoutView",
+ "align_content": null,
+ "align_items": null,
+ "align_self": null,
+ "border": null,
+ "bottom": null,
+ "display": null,
+ "flex": null,
+ "flex_flow": null,
+ "grid_area": null,
+ "grid_auto_columns": null,
+ "grid_auto_flow": null,
+ "grid_auto_rows": null,
+ "grid_column": null,
+ "grid_gap": null,
+ "grid_row": null,
+ "grid_template_areas": null,
+ "grid_template_columns": null,
+ "grid_template_rows": null,
+ "height": null,
+ "justify_content": null,
+ "justify_items": null,
+ "left": null,
+ "margin": null,
+ "max_height": null,
+ "max_width": null,
+ "min_height": null,
+ "min_width": null,
+ "object_fit": null,
+ "object_position": null,
+ "order": null,
+ "overflow": null,
+ "overflow_x": null,
+ "overflow_y": null,
+ "padding": null,
+ "right": null,
+ "top": null,
+ "visibility": "hidden",
+ "width": null
+ }
+ },
+ "bfb987eb24644819ba52b2fc7bc865de": {
+ "model_module": "@jupyter-widgets/controls",
+ "model_module_version": "1.5.0",
+ "model_name": "ProgressStyleModel",
+ "state": {
+ "_model_module": "@jupyter-widgets/controls",
+ "_model_module_version": "1.5.0",
+ "_model_name": "ProgressStyleModel",
+ "_view_count": null,
+ "_view_module": "@jupyter-widgets/base",
+ "_view_module_version": "1.2.0",
+ "_view_name": "StyleView",
+ "bar_color": null,
+ "description_width": ""
+ }
+ },
+ "c05f2a082d5e4a0c8f3b9b6952eff208": {
+ "model_module": "@jupyter-widgets/controls",
+ "model_module_version": "1.5.0",
+ "model_name": "HTMLModel",
+ "state": {
+ "_dom_classes": [],
+ "_model_module": "@jupyter-widgets/controls",
+ "_model_module_version": "1.5.0",
+ "_model_name": "HTMLModel",
+ "_view_count": null,
+ "_view_module": "@jupyter-widgets/controls",
+ "_view_module_version": "1.5.0",
+ "_view_name": "HTMLView",
+ "description": "",
+ "description_tooltip": null,
+ "layout": "IPY_MODEL_57cf77bba44141359639c0268ced8934",
+ "placeholder": "",
+ "style": "IPY_MODEL_6a7d1421176f4b62af003a4b2f18815b",
+ "value": " 0/500 [00:00<?, ? examples/s]"
+ }
+ },
+ "c1f14f69e97f44598d9c4f8d789131fb": {
+ "model_module": "@jupyter-widgets/controls",
+ "model_module_version": "1.5.0",
+ "model_name": "HTMLModel",
+ "state": {
+ "_dom_classes": [],
+ "_model_module": "@jupyter-widgets/controls",
+ "_model_module_version": "1.5.0",
+ "_model_name": "HTMLModel",
+ "_view_count": null,
+ "_view_module": "@jupyter-widgets/controls",
+ "_view_module_version": "1.5.0",
+ "_view_name": "HTMLView",
+ "description": "",
+ "description_tooltip": null,
+ "layout": "IPY_MODEL_1f1331ab4e734cdaa66cf28d7278f980",
+ "placeholder": "",
+ "style": "IPY_MODEL_d773c0f65a014ae08f533bf2348c1831",
+ "value": " 2/2 [00:04<00:00, 2.35s/ url]"
+ }
+ },
+ "c37f294926f945898096874d324ee2ae": {
+ "model_module": "@jupyter-widgets/base",
+ "model_module_version": "1.2.0",
+ "model_name": "LayoutModel",
+ "state": {
+ "_model_module": "@jupyter-widgets/base",
+ "_model_module_version": "1.2.0",
+ "_model_name": "LayoutModel",
+ "_view_count": null,
+ "_view_module": "@jupyter-widgets/base",
+ "_view_module_version": "1.2.0",
+ "_view_name": "LayoutView",
+ "align_content": null,
+ "align_items": null,
+ "align_self": null,
+ "border": null,
+ "bottom": null,
+ "display": null,
+ "flex": null,
+ "flex_flow": null,
+ "grid_area": null,
+ "grid_auto_columns": null,
+ "grid_auto_flow": null,
+ "grid_auto_rows": null,
+ "grid_column": null,
+ "grid_gap": null,
+ "grid_row": null,
+ "grid_template_areas": null,
+ "grid_template_columns": null,
+ "grid_template_rows": null,
+ "height": null,
+ "justify_content": null,
+ "justify_items": null,
+ "left": null,
+ "margin": null,
+ "max_height": null,
+ "max_width": null,
+ "min_height": null,
+ "min_width": null,
+ "object_fit": null,
+ "object_position": null,
+ "order": null,
+ "overflow": null,
+ "overflow_x": null,
+ "overflow_y": null,
+ "padding": null,
+ "right": null,
+ "top": null,
+ "visibility": null,
+ "width": "20px"
+ }
+ },
+ "c5296022707240b393be4146e983083c": {
+ "model_module": "@jupyter-widgets/base",
+ "model_module_version": "1.2.0",
+ "model_name": "LayoutModel",
+ "state": {
+ "_model_module": "@jupyter-widgets/base",
+ "_model_module_version": "1.2.0",
+ "_model_name": "LayoutModel",
+ "_view_count": null,
+ "_view_module": "@jupyter-widgets/base",
+ "_view_module_version": "1.2.0",
+ "_view_name": "LayoutView",
+ "align_content": null,
+ "align_items": null,
+ "align_self": null,
+ "border": null,
+ "bottom": null,
+ "display": null,
+ "flex": null,
+ "flex_flow": null,
+ "grid_area": null,
+ "grid_auto_columns": null,
+ "grid_auto_flow": null,
+ "grid_auto_rows": null,
+ "grid_column": null,
+ "grid_gap": null,
+ "grid_row": null,
+ "grid_template_areas": null,
+ "grid_template_columns": null,
+ "grid_template_rows": null,
+ "height": null,
+ "justify_content": null,
+ "justify_items": null,
+ "left": null,
+ "margin": null,
+ "max_height": null,
+ "max_width": null,
+ "min_height": null,
+ "min_width": null,
+ "object_fit": null,
+ "object_position": null,
+ "order": null,
+ "overflow": null,
+ "overflow_x": null,
+ "overflow_y": null,
+ "padding": null,
+ "right": null,
+ "top": null,
+ "visibility": null,
+ "width": null
+ }
+ },
+ "c8e13e321a9a4e0b88c717a1a8065619": {
+ "model_module": "@jupyter-widgets/controls",
+ "model_module_version": "1.5.0",
+ "model_name": "FloatProgressModel",
+ "state": {
+ "_dom_classes": [],
+ "_model_module": "@jupyter-widgets/controls",
+ "_model_module_version": "1.5.0",
+ "_model_name": "FloatProgressModel",
+ "_view_count": null,
+ "_view_module": "@jupyter-widgets/controls",
+ "_view_module_version": "1.5.0",
+ "_view_name": "ProgressView",
+ "bar_style": "success",
+ "description": "",
+ "description_tooltip": null,
+ "layout": "IPY_MODEL_c37f294926f945898096874d324ee2ae",
+ "max": 1,
+ "min": 0,
+ "orientation": "horizontal",
+ "style": "IPY_MODEL_941ab80a14aa4a1bb3943dd77aca85d7",
+ "value": 1
+ }
+ },
+ "cd23755e6122427e9d95eb808b7900c2": {
+ "model_module": "@jupyter-widgets/controls",
+ "model_module_version": "1.5.0",
+ "model_name": "HBoxModel",
+ "state": {
+ "_dom_classes": [],
+ "_model_module": "@jupyter-widgets/controls",
+ "_model_module_version": "1.5.0",
+ "_model_name": "HBoxModel",
+ "_view_count": null,
+ "_view_module": "@jupyter-widgets/controls",
+ "_view_module_version": "1.5.0",
+ "_view_name": "HBoxView",
+ "box_style": "",
+ "children": [
+ "IPY_MODEL_62eb05321e2d4265b033544921f92053",
+ "IPY_MODEL_3627640fb7d14ac09c9fdcbfb2159152",
+ "IPY_MODEL_c05f2a082d5e4a0c8f3b9b6952eff208"
+ ],
+ "layout": "IPY_MODEL_bf6033cd5d0c4ef5ac934ed6e8adaeb6"
+ }
+ },
+ "ce82f835dd2b45a181a53407505b0cfc": {
+ "model_module": "@jupyter-widgets/controls",
+ "model_module_version": "1.5.0",
+ "model_name": "DescriptionStyleModel",
+ "state": {
+ "_model_module": "@jupyter-widgets/controls",
+ "_model_module_version": "1.5.0",
+ "_model_name": "DescriptionStyleModel",
+ "_view_count": null,
+ "_view_module": "@jupyter-widgets/base",
+ "_view_module_version": "1.2.0",
+ "_view_name": "StyleView",
+ "description_width": ""
+ }
+ },
+ "d24e60c7e4664e0ba11cce85c647e526": {
+ "model_module": "@jupyter-widgets/base",
+ "model_module_version": "1.2.0",
+ "model_name": "LayoutModel",
+ "state": {
+ "_model_module": "@jupyter-widgets/base",
+ "_model_module_version": "1.2.0",
+ "_model_name": "LayoutModel",
+ "_view_count": null,
+ "_view_module": "@jupyter-widgets/base",
+ "_view_module_version": "1.2.0",
+ "_view_name": "LayoutView",
+ "align_content": null,
+ "align_items": null,
+ "align_self": null,
+ "border": null,
+ "bottom": null,
+ "display": null,
+ "flex": null,
+ "flex_flow": null,
+ "grid_area": null,
+ "grid_auto_columns": null,
+ "grid_auto_flow": null,
+ "grid_auto_rows": null,
+ "grid_column": null,
+ "grid_gap": null,
+ "grid_row": null,
+ "grid_template_areas": null,
+ "grid_template_columns": null,
+ "grid_template_rows": null,
+ "height": null,
+ "justify_content": null,
+ "justify_items": null,
+ "left": null,
+ "margin": null,
+ "max_height": null,
+ "max_width": null,
+ "min_height": null,
+ "min_width": null,
+ "object_fit": null,
+ "object_position": null,
+ "order": null,
+ "overflow": null,
+ "overflow_x": null,
+ "overflow_y": null,
+ "padding": null,
+ "right": null,
+ "top": null,
+ "visibility": null,
+ "width": null
+ }
+ },
+ "d298dc9202dc49aba09b66969b3d19f9": {
+ "model_module": "@jupyter-widgets/base",
+ "model_module_version": "1.2.0",
+ "model_name": "LayoutModel",
+ "state": {
+ "_model_module": "@jupyter-widgets/base",
+ "_model_module_version": "1.2.0",
+ "_model_name": "LayoutModel",
+ "_view_count": null,
+ "_view_module": "@jupyter-widgets/base",
+ "_view_module_version": "1.2.0",
+ "_view_name": "LayoutView",
+ "align_content": null,
+ "align_items": null,
+ "align_self": null,
+ "border": null,
+ "bottom": null,
+ "display": null,
+ "flex": null,
+ "flex_flow": null,
+ "grid_area": null,
+ "grid_auto_columns": null,
+ "grid_auto_flow": null,
+ "grid_auto_rows": null,
+ "grid_column": null,
+ "grid_gap": null,
+ "grid_row": null,
+ "grid_template_areas": null,
+ "grid_template_columns": null,
+ "grid_template_rows": null,
+ "height": null,
+ "justify_content": null,
+ "justify_items": null,
+ "left": null,
+ "margin": null,
+ "max_height": null,
+ "max_width": null,
+ "min_height": null,
+ "min_width": null,
+ "object_fit": null,
+ "object_position": null,
+ "order": null,
+ "overflow": null,
+ "overflow_x": null,
+ "overflow_y": null,
+ "padding": null,
+ "right": null,
+ "top": null,
+ "visibility": null,
+ "width": null
+ }
+ },
+ "d426d4b5b86c4582a17bf21fe5f021d4": {
+ "model_module": "@jupyter-widgets/base",
+ "model_module_version": "1.2.0",
+ "model_name": "LayoutModel",
+ "state": {
+ "_model_module": "@jupyter-widgets/base",
+ "_model_module_version": "1.2.0",
+ "_model_name": "LayoutModel",
+ "_view_count": null,
+ "_view_module": "@jupyter-widgets/base",
+ "_view_module_version": "1.2.0",
+ "_view_name": "LayoutView",
+ "align_content": null,
+ "align_items": null,
+ "align_self": null,
+ "border": null,
+ "bottom": null,
+ "display": null,
+ "flex": null,
+ "flex_flow": null,
+ "grid_area": null,
+ "grid_auto_columns": null,
+ "grid_auto_flow": null,
+ "grid_auto_rows": null,
+ "grid_column": null,
+ "grid_gap": null,
+ "grid_row": null,
+ "grid_template_areas": null,
+ "grid_template_columns": null,
+ "grid_template_rows": null,
+ "height": null,
+ "justify_content": null,
+ "justify_items": null,
+ "left": null,
+ "margin": null,
+ "max_height": null,
+ "max_width": null,
+ "min_height": null,
+ "min_width": null,
+ "object_fit": null,
+ "object_position": null,
+ "order": null,
+ "overflow": null,
+ "overflow_x": null,
+ "overflow_y": null,
+ "padding": null,
+ "right": null,
+ "top": null,
+ "visibility": null,
+ "width": null
+ }
+ },
+ "d646a9a4cc5a4876a7fa185bf9eb3ccf": {
+ "model_module": "@jupyter-widgets/controls",
+ "model_module_version": "1.5.0",
+ "model_name": "DescriptionStyleModel",
+ "state": {
+ "_model_module": "@jupyter-widgets/controls",
+ "_model_module_version": "1.5.0",
+ "_model_name": "DescriptionStyleModel",
+ "_view_count": null,
+ "_view_module": "@jupyter-widgets/base",
+ "_view_module_version": "1.2.0",
+ "_view_name": "StyleView",
+ "description_width": ""
+ }
+ },
+ "d773c0f65a014ae08f533bf2348c1831": {
+ "model_module": "@jupyter-widgets/controls",
+ "model_module_version": "1.5.0",
+ "model_name": "DescriptionStyleModel",
+ "state": {
+ "_model_module": "@jupyter-widgets/controls",
+ "_model_module_version": "1.5.0",
+ "_model_name": "DescriptionStyleModel",
+ "_view_count": null,
+ "_view_module": "@jupyter-widgets/base",
+ "_view_module_version": "1.2.0",
+ "_view_name": "StyleView",
+ "description_width": ""
+ }
+ },
+ "d849ae343e51419c803c29c24a16dd01": {
+ "model_module": "@jupyter-widgets/controls",
+ "model_module_version": "1.5.0",
+ "model_name": "FloatProgressModel",
+ "state": {
+ "_dom_classes": [],
+ "_model_module": "@jupyter-widgets/controls",
+ "_model_module_version": "1.5.0",
+ "_model_name": "FloatProgressModel",
+ "_view_count": null,
+ "_view_module": "@jupyter-widgets/controls",
+ "_view_module_version": "1.5.0",
+ "_view_name": "ProgressView",
+ "bar_style": "",
+ "description": "",
+ "description_tooltip": null,
+ "layout": "IPY_MODEL_3948a575b3ee4a199f1055e93d8270ee",
+ "max": 2,
+ "min": 0,
+ "orientation": "horizontal",
+ "style": "IPY_MODEL_2ee59cbc091346168c3b4511f64b018a",
+ "value": 2
+ }
+ },
+ "d9dd5b64f0ee40359a4c577f4358248f": {
+ "model_module": "@jupyter-widgets/base",
+ "model_module_version": "1.2.0",
+ "model_name": "LayoutModel",
+ "state": {
+ "_model_module": "@jupyter-widgets/base",
+ "_model_module_version": "1.2.0",
+ "_model_name": "LayoutModel",
+ "_view_count": null,
+ "_view_module": "@jupyter-widgets/base",
+ "_view_module_version": "1.2.0",
+ "_view_name": "LayoutView",
+ "align_content": null,
+ "align_items": null,
+ "align_self": null,
+ "border": null,
+ "bottom": null,
+ "display": null,
+ "flex": null,
+ "flex_flow": null,
+ "grid_area": null,
+ "grid_auto_columns": null,
+ "grid_auto_flow": null,
+ "grid_auto_rows": null,
+ "grid_column": null,
+ "grid_gap": null,
+ "grid_row": null,
+ "grid_template_areas": null,
+ "grid_template_columns": null,
+ "grid_template_rows": null,
+ "height": null,
+ "justify_content": null,
+ "justify_items": null,
+ "left": null,
+ "margin": null,
+ "max_height": null,
+ "max_width": null,
+ "min_height": null,
+ "min_width": null,
+ "object_fit": null,
+ "object_position": null,
+ "order": null,
+ "overflow": null,
+ "overflow_x": null,
+ "overflow_y": null,
+ "padding": null,
+ "right": null,
+ "top": null,
+ "visibility": null,
+ "width": null
+ }
+ },
+ "df1b4d3501ed457b86b60a5449d1e1da": {
+ "model_module": "@jupyter-widgets/controls",
+ "model_module_version": "1.5.0",
+ "model_name": "FloatProgressModel",
+ "state": {
+ "_dom_classes": [],
+ "_model_module": "@jupyter-widgets/controls",
+ "_model_module_version": "1.5.0",
+ "_model_name": "FloatProgressModel",
+ "_view_count": null,
+ "_view_module": "@jupyter-widgets/controls",
+ "_view_module_version": "1.5.0",
+ "_view_name": "ProgressView",
+ "bar_style": "",
+ "description": "",
+ "description_tooltip": null,
+ "layout": "IPY_MODEL_b04f406f1a2a46818d034a80d13f6664",
+ "max": 5452,
+ "min": 0,
+ "orientation": "horizontal",
+ "style": "IPY_MODEL_fcc06cd646ff425b9172312808dd3e24",
+ "value": 5452
+ }
+ },
+ "df522159441841cda1f97d63aab5ad9e": {
+ "model_module": "@jupyter-widgets/controls",
+ "model_module_version": "1.5.0",
+ "model_name": "HBoxModel",
+ "state": {
+ "_dom_classes": [],
+ "_model_module": "@jupyter-widgets/controls",
+ "_model_module_version": "1.5.0",
+ "_model_name": "HBoxModel",
+ "_view_count": null,
+ "_view_module": "@jupyter-widgets/controls",
+ "_view_module_version": "1.5.0",
+ "_view_name": "HBoxView",
+ "box_style": "",
+ "children": [
+ "IPY_MODEL_a9f4707824454e35b2273412c4586b8d",
+ "IPY_MODEL_d849ae343e51419c803c29c24a16dd01",
+ "IPY_MODEL_65b7f23304a243878c43ea391885ee85"
+ ],
+ "layout": "IPY_MODEL_85f4ab947d774115baf64332f64248bd"
+ }
+ },
+ "e868b829ec7a4795955034ebb8b109a6": {
+ "model_module": "@jupyter-widgets/controls",
+ "model_module_version": "1.5.0",
+ "model_name": "HTMLModel",
+ "state": {
+ "_dom_classes": [],
+ "_model_module": "@jupyter-widgets/controls",
+ "_model_module_version": "1.5.0",
+ "_model_name": "HTMLModel",
+ "_view_count": null,
+ "_view_module": "@jupyter-widgets/controls",
+ "_view_module_version": "1.5.0",
+ "_view_name": "HTMLView",
+ "description": "",
+ "description_tooltip": null,
+ "layout": "IPY_MODEL_d24e60c7e4664e0ba11cce85c647e526",
+ "placeholder": "",
+ "style": "IPY_MODEL_23bbe3c6dfcb42c7a0a1c826bc35076d",
+ "value": "Generating train examples...: 0%"
+ }
+ },
+ "efa03832003244a490cfcb34edf7ccbe": {
+ "model_module": "@jupyter-widgets/base",
+ "model_module_version": "1.2.0",
+ "model_name": "LayoutModel",
+ "state": {
+ "_model_module": "@jupyter-widgets/base",
+ "_model_module_version": "1.2.0",
+ "_model_name": "LayoutModel",
+ "_view_count": null,
+ "_view_module": "@jupyter-widgets/base",
+ "_view_module_version": "1.2.0",
+ "_view_name": "LayoutView",
+ "align_content": null,
+ "align_items": null,
+ "align_self": null,
+ "border": null,
+ "bottom": null,
+ "display": null,
+ "flex": null,
+ "flex_flow": null,
+ "grid_area": null,
+ "grid_auto_columns": null,
+ "grid_auto_flow": null,
+ "grid_auto_rows": null,
+ "grid_column": null,
+ "grid_gap": null,
+ "grid_row": null,
+ "grid_template_areas": null,
+ "grid_template_columns": null,
+ "grid_template_rows": null,
+ "height": null,
+ "justify_content": null,
+ "justify_items": null,
+ "left": null,
+ "margin": null,
+ "max_height": null,
+ "max_width": null,
+ "min_height": null,
+ "min_width": null,
+ "object_fit": null,
+ "object_position": null,
+ "order": null,
+ "overflow": null,
+ "overflow_x": null,
+ "overflow_y": null,
+ "padding": null,
+ "right": null,
+ "top": null,
+ "visibility": null,
+ "width": null
+ }
+ },
+ "f5e168c246714295a10a64fbb1ef2ee3": {
+ "model_module": "@jupyter-widgets/controls",
+ "model_module_version": "1.5.0",
+ "model_name": "ProgressStyleModel",
+ "state": {
+ "_model_module": "@jupyter-widgets/controls",
+ "_model_module_version": "1.5.0",
+ "_model_name": "ProgressStyleModel",
+ "_view_count": null,
+ "_view_module": "@jupyter-widgets/base",
+ "_view_module_version": "1.2.0",
+ "_view_name": "StyleView",
+ "bar_color": null,
+ "description_width": ""
+ }
+ },
+ "fcc06cd646ff425b9172312808dd3e24": {
+ "model_module": "@jupyter-widgets/controls",
+ "model_module_version": "1.5.0",
+ "model_name": "ProgressStyleModel",
+ "state": {
+ "_model_module": "@jupyter-widgets/controls",
+ "_model_module_version": "1.5.0",
+ "_model_name": "ProgressStyleModel",
+ "_view_count": null,
+ "_view_module": "@jupyter-widgets/base",
+ "_view_module_version": "1.2.0",
+ "_view_name": "StyleView",
+ "bar_color": null,
+ "description_width": ""
+ }
}
- },
- "nbformat": 4,
- "nbformat_minor": 0
+ }
+ }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 4
}
diff --git a/notebooks/guides/QuestionAnwering_Cohere_SagemakerJumpstart.ipynb b/notebooks/guides/QuestionAnwering_Cohere_SagemakerJumpstart.ipynb
index d5caa6c4..ddc32d16 100644
--- a/notebooks/guides/QuestionAnwering_Cohere_SagemakerJumpstart.ipynb
+++ b/notebooks/guides/QuestionAnwering_Cohere_SagemakerJumpstart.ipynb
@@ -1,7 +1,6 @@
{
"cells": [
{
- "attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
@@ -9,7 +8,6 @@
]
},
{
- "attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
@@ -17,7 +15,6 @@
]
},
{
- "attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
@@ -31,7 +28,6 @@
]
},
{
- "attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
@@ -46,7 +42,6 @@
]
},
{
- "attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
@@ -62,7 +57,6 @@
]
},
{
- "attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
@@ -80,6 +74,7 @@
"cell_type": "code",
"execution_count": null,
"metadata": {
+ "collapsed": true,
"jupyter": {
"outputs_hidden": true
},
@@ -125,7 +120,6 @@
]
},
{
- "attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
@@ -194,7 +188,6 @@
]
},
{
- "attachments": {},
"cell_type": "markdown",
"metadata": {
"tags": []
@@ -263,7 +256,6 @@
]
},
{
- "attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
@@ -333,7 +325,6 @@
]
},
{
- "attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
@@ -341,7 +332,6 @@
]
},
{
- "attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
@@ -389,7 +379,6 @@
]
},
{
- "attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
@@ -399,7 +388,6 @@
]
},
{
- "attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
@@ -430,7 +418,6 @@
]
},
{
- "attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
@@ -495,7 +482,6 @@
]
},
{
- "attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
@@ -547,7 +533,6 @@
]
},
{
- "attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
@@ -571,7 +556,6 @@
]
},
{
- "attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
@@ -600,7 +584,6 @@
]
},
{
- "attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
@@ -660,7 +643,6 @@
]
},
{
- "attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
@@ -693,19 +675,17 @@
]
},
{
- "attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
- "**Now, we can build an QA application. LangChain makes it extremly simple with following few lines of code.**"
+ "**Now, we can build an QA application. LangChain makes it extremely simple with following few lines of code.**"
]
},
{
- "attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
- "Based on the question below, we can achieven the points in Step 4 with just a few lines of code as shown below."
+ "Based on the question below, we can achieve the points in Step 4 with just a few lines of code as shown below."
]
},
{
@@ -757,7 +737,6 @@
]
},
{
- "attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
@@ -767,7 +746,6 @@
]
},
{
- "attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
@@ -797,7 +775,6 @@
]
},
{
- "attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
@@ -816,7 +793,6 @@
]
},
{
- "attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
@@ -835,7 +811,6 @@
]
},
{
- "attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
@@ -869,7 +844,6 @@
]
},
{
- "attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
@@ -890,7 +864,6 @@
]
},
{
- "attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
@@ -909,7 +882,6 @@
]
},
{
- "attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
@@ -950,7 +922,6 @@
]
},
{
- "attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": []
@@ -1534,9 +1505,9 @@
],
"instance_type": "ml.t3.medium",
"kernelspec": {
- "display_name": "Python 3 (PyTorch 1.13 Python 3.9 CPU Optimized)",
+ "display_name": "Python 3 (ipykernel)",
"language": "python",
- "name": "python3__SAGEMAKER_INTERNAL__arn:aws:sagemaker:us-east-1:081325390199:image/pytorch-1.13-cpu-py39"
+ "name": "python3"
},
"language_info": {
"codemirror_mode": {
@@ -1548,7 +1519,7 @@
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
- "version": "3.9.16"
+ "version": "3.8.16"
}
},
"nbformat": 4,
diff --git a/notebooks/guides/Recipes_for_better_meeting_notes_summary.ipynb b/notebooks/guides/Recipes_for_better_meeting_notes_summary.ipynb
index a2defd0e..bf0dc430 100644
--- a/notebooks/guides/Recipes_for_better_meeting_notes_summary.ipynb
+++ b/notebooks/guides/Recipes_for_better_meeting_notes_summary.ipynb
@@ -1,15 +1,5 @@
{
"cells": [
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "Note: we are in the process of updating the links in this notebook. If a link doesn't work, please open an issue and we'll rectify it ASAP. Thanks for your understanding!\n",
- "\n",
- "Links to add:\n",
- "* Cell 1: full system for auto-meeting notes"
- ]
- },
{
"cell_type": "markdown",
"metadata": {
@@ -28,7 +18,7 @@
"\n",
"Finally, we'll show that prompting the model for 1.-3. can be combined with the formatting instructions covered from our previous guide.\n",
"\n",
- "We're constantly improving our summarisation capabilities across domains. For more information, you can reach out to us at summarize@cohere.com.\n",
+ "We're constantly improving our summarisation capabilities across domains. For more information, you can reach out to us [using this link](https://cohere.com/contact-sales).\n",
"\n"
]
},
@@ -56,8 +46,8 @@
"outputs": [],
"source": [
"%%capture\n",
- "# TODO: upgrade to \"cohere>5\"",
-"!pip install \"cohere<5\" datasets tokenizers\n",
+ "# TODO: upgrade to \"cohere>5\"\n",
+ "!pip install \"cohere<5\" datasets tokenizers\n",
"\n",
"import cohere\n",
"from getpass import getpass\n",
diff --git a/notebooks/guides/Summarization_Evals.ipynb b/notebooks/guides/Summarization_Evals.ipynb
index 9c7c74b0..0d529664 100644
--- a/notebooks/guides/Summarization_Evals.ipynb
+++ b/notebooks/guides/Summarization_Evals.ipynb
@@ -1,21 +1,12 @@
{
"cells": [
- {
- "cell_type": "markdown",
- "metadata": {
- "id": "WClxySHIzi9G"
- },
- "source": [
- ""
- ]
- },
{
"cell_type": "markdown",
"metadata": {
"id": "LplM9PSe8djM"
},
"source": [
- "# Evaluating summarization\n",
+ "# Summarization Evals\n",
"\n",
"In this cookbook, we will be demonstrating an approach we use for evaluating summarization tasks using LLM evaluation.\n",
"\n",
@@ -34,26 +25,26 @@
"source": [
"\n",
"\n",
- "# Get Started\n",
+ "## Get Started\n",
"\n",
"You'll need a Cohere API key to run this notebook. If you don't have a key, head to https://cohere.com/ to generate your key."
]
},
{
"cell_type": "code",
- "execution_count": null,
+ "execution_count": 1,
"metadata": {
"id": "H2TfKiPM3a6f"
},
"outputs": [],
"source": [
"# %%capture\n",
- "!pip install \"cohere<5\" datasets --quiet"
+ "!pip install cohere datasets --quiet"
]
},
{
"cell_type": "code",
- "execution_count": null,
+ "execution_count": 2,
"metadata": {
"id": "J1HzGqnY74bj"
},
@@ -86,11 +77,54 @@
},
{
"cell_type": "code",
- "execution_count": null,
+ "execution_count": 3,
"metadata": {
"id": "1JfU70r113wo"
},
- "outputs": [],
+ "outputs": [
+ {
+ "data": {
+ "application/vnd.jupyter.widget-view+json": {
+ "model_id": "97a7a77756614778a3d9af18eb0a7d5e",
+ "version_major": 2,
+ "version_minor": 0
+ },
+ "text/plain": [
+ "Generating train split: 0%| | 0/1095 [00:00\n",
"\n",
- "# Construct the evaluation dataset\n",
+ "## Construct the evaluation dataset\n",
"\n",
"We are interested in evaluating summarization in real-world, enterprise use cases, which typically have two distinguishing features as compared to academic summarization benchmarks:\n",
"- Enterprise use cases often focus on specific summarization objectives, e.g. \"summarize action items\".\n",
@@ -120,7 +154,7 @@
},
{
"cell_type": "code",
- "execution_count": null,
+ "execution_count": 5,
"metadata": {
"id": "2Bj2I0If13wp"
},
@@ -144,7 +178,7 @@
},
{
"cell_type": "code",
- "execution_count": null,
+ "execution_count": 6,
"metadata": {
"id": "4uMI5VXm13wp"
},
@@ -212,7 +246,7 @@
},
{
"cell_type": "code",
- "execution_count": null,
+ "execution_count": 7,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
@@ -276,7 +310,7 @@
},
{
"cell_type": "code",
- "execution_count": null,
+ "execution_count": 8,
"metadata": {
"id": "LYTMTFAm13wr"
},
@@ -296,7 +330,7 @@
},
{
"cell_type": "code",
- "execution_count": null,
+ "execution_count": 9,
"metadata": {
"id": "igP0-oHN13wr"
},
@@ -308,7 +342,7 @@
},
{
"cell_type": "code",
- "execution_count": null,
+ "execution_count": 10,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
@@ -382,7 +416,7 @@
"source": [
"\n",
"\n",
- "# Build the evaluation framework\n",
+ "## Build the evaluation framework\n",
"\n",
"We now setup the tools we will use for evaluation.\n",
"\n",
@@ -396,7 +430,7 @@
},
{
"cell_type": "code",
- "execution_count": null,
+ "execution_count": 12,
"metadata": {
"id": "KGQQp2Yd13wr"
},
@@ -467,7 +501,7 @@
},
{
"cell_type": "code",
- "execution_count": null,
+ "execution_count": 13,
"metadata": {
"id": "i1niSoI-13wr"
},
@@ -561,7 +595,7 @@
"source": [
"\n",
"\n",
- "# Run evaluations\n",
+ "## Run evaluations\n",
"\n",
"Now that we have our evaluation dataset and defined our evaluation functions, let's run evaluations!\n",
"\n",
@@ -570,7 +604,7 @@
},
{
"cell_type": "code",
- "execution_count": null,
+ "execution_count": 14,
"metadata": {
"id": "HzbOQmWE13ws"
},
@@ -586,7 +620,7 @@
},
{
"cell_type": "code",
- "execution_count": null,
+ "execution_count": 15,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
@@ -599,7 +633,7 @@
"name": "stdout",
"output_type": "stream",
"text": [
- "PhD student D presented their method for detecting and counting acoustic events in recorded speech sessions, including speech overlaps and silence. The team sought clarification on D's definitions and counted events, discussing the challenges of distinguishing between different types of events.\n"
+ "PhD D is transcribing recorded sessions to locate overlapping speech zones and categorizing them as acoustic events. The team discusses the parameters PhD D should use and how to define these events, considering the number of speakers and silence.\n"
]
}
],
@@ -619,7 +653,7 @@
},
{
"cell_type": "code",
- "execution_count": null,
+ "execution_count": 16,
"metadata": {
"id": "6XesZHCN13ws"
},
@@ -659,7 +693,7 @@
},
{
"cell_type": "code",
- "execution_count": null,
+ "execution_count": 25,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
@@ -671,15 +705,8 @@
"outputs": [
{
"data": {
- "application/vnd.google.colaboratory.intrinsic+json": {
- "summary": "{\n \"name\": \"data\",\n \"rows\": 6,\n \"fields\": [\n {\n \"column\": \"instruction\",\n \"properties\": {\n \"dtype\": \"string\",\n \"num_unique_values\": 6,\n \"samples\": [\n \"Summarize the meeting based on the transcript. In paragraph form, output your response. Use at least 10 words and at most 50 words in total.\",\n \"Summarize the meeting based on the transcript. Return the answer in the form of paragraphs. Make sure your answer is between 50 and 200 words long.\",\n \"What are the follow-up items based on the meeting transcript? In bullets, output your response. Make sure to use exactly 2 bullets. Make sure each bullet is between 20 and 80 words long.\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"eval_metadata\",\n \"properties\": {\n \"dtype\": \"object\",\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"objective\",\n \"properties\": {\n \"dtype\": \"category\",\n \"num_unique_values\": 2,\n \"samples\": [\n \"action_items\",\n \"general_summarization\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"transcript\",\n \"properties\": {\n \"dtype\": \"string\",\n \"num_unique_values\": 6,\n \"samples\": [\n \"PhD F: As opposed to the rest of us \\nPhD D: Well comment OK I I remind that me my first objective eh in the project is to to study difference parameters to to find a a good solution to detect eh the overlapping zone in eh speech recorded But eh tsk comment ehhh comment In that way comment I I I begin to to study and to analyze the ehn the recorded speech eh the different session to to find and to locate and to mark eh the the different overlapping zone And eh so eh I was eh I am transcribing the the first session and I I have found eh eh one thousand acoustic events eh besides the overlapping zones eh I I I mean the eh breaths eh aspiration eh eh talk eh eh clap eh comment I do not know what is the different names eh you use to to name the the pause n speech \\nGrad G: Oh I do not think we ve been doing it at that level of detail So \\nPhD D: Eh I I I do I do not need to to to mmm to m to label the the different acoustic but I prefer because eh I would like to to study if eh I I will find eh eh a good eh parameters eh to detect overlapping I would like to to to test these parameters eh with the another eh eh acoustic events to nnn to eh to find what is the ehm the false eh the false eh hypothesis eh nnn which eh are produced when we use the the ehm this eh parameter eh I mean pitch eh eh difference eh feature \\nPhD A: You know I think some of these that are the nonspeech overlapping events may be difficult even for humans to tell that there s two there I mean if it s a tapping sound you would not necessarily or you know something like that it would be it might be hard to know that it was two separate events \\nGrad G: Well You were not talking about just overlaps were you ? You were just talking about acoustic events \\nPhD D: I I I I t I t I talk eh about eh acoustic events in general but eh my my objective eh will be eh to study eh overlapping zone Eh ? comment n Eh in twelve minutes I found eh eh one thousand acoustic events \\nProfessor E: How many overlaps were there in it ? No no how many of them were the overlaps of speech though ? \\nPhD D: How many ? Eh almost eh three hundred eh in one session in five eh in forty five minutes Alm Three hundred overlapping zone With the overlapping zone overlapping speech speech what eh different duration \\nPostdoc B: Does this ? So if you had an overlap involving three people how many times was that counted ? \\nPhD D: three people two people Eh I would like to consider eh one people with difference noise eh in the background be \\nProfessor E: No no but I think what she s asking is pause if at some particular for some particular stretch you had three people talking instead of two did you call that one event ? \\nPhD D: Oh Oh I consider one event eh for th for that eh for all the zone This th I I I con I consider I consider eh an acoustic event the overlapping zone the period where three speaker or eh are talking together \\nGrad G: So let s say me and Jane are talking at the same time and then Liz starts talking also over all of us How many events would that be ? \\nPhD D: So I do not understand \\nGrad G: So two people are talking comment and then a third person starts talking Is there an event right here ? \\nPhD D: Eh no No no For me is the overlapping zone because because you you have s you have more one eh more one voice eh eh produced in a in in a moment \\nGrad G: So i if two or more people are talking \\nProfessor E: OK So I think We just wanted to understand how you are defining it So then in the region between since there there is some continuous region in between regions where there is only one person speaking And one contiguous region like that you are calling an event Is it Are you calling the beginning or the end of it the event or are you calling the entire length of it the event ? \\nPhD D: I consider the the nnn the nnn nnn eh the entirety eh eh all all the time there were the voice has overlapped This is the idea But eh I I do not distinguish between the the numbers of eh speaker I m not considering eh the the ehm eh the fact of eh eh for example what did you say ? Eh at first eh eh two talkers are eh speaking and eh eh a third person eh join to to that For me it s eh it s eh all overlap zone with eh several numbers of speakers is eh eh the same acoustic event Wi but without any mark between the zone of the overlapping zone with two speakers eh speaking together and the zone with the three speakers \\nPostdoc B: That would j just be one \\nPhD D: Eh with eh a beginning mark and the ending mark Because eh for me is the is the zone with eh some kind of eh distortion the spectral I do not mind By the moment by the moment \\nGrad G: Well but But you could imagine that three people talking has a different spectral characteristic than two \\nPhD D: I I do not but eh but eh I have to study comment What will happen in a general way \\nGrad G: So You had to start somewhere \\nPhD C: So there s a lot of overlap \\nPhD D: I I do not know what eh will will happen with the \\nGrad G: That s a lot of overlap \\nProfessor E: So again that s that s three three hundred in forty five minutes that are that are speakers just speakers \\nPostdoc B: But a a a th \\nProfessor E: So that s about eight per minute \\nPostdoc B: But a thousand events in twelve minutes that s \\nPhD C: But that can include taps \\nPostdoc B: Well but a thousand taps in eight minutes is a l in twelve minutes is a lot \\nPhD D: I I con I consider I consider acoustic events eh the silent too \\nGrad G: Silence starting or silence ending \\nPhD D: silent ground to bec to detect eh because I consider acoustic event all the things are not eh speech In ge in in in a general point of view \\nProfessor E: OK so how many of those thousand were silence ? \\nPhD F: Not speech not speech or too much speech \\nProfessor E: Right So how many of those thousand were silence silent sections ? \\nPhD D: silent I I I I do not I I have not the eh I I would like to to do a stylistic study\",\n \"Lynne Neagle AM: Thank you very much And the next questions then are from Suzy Davies \\nSuzy Davies AM: Thank you Chair I just wanted to have a quick answer from probably the Minister I think about the primary legislation and the regulations that followed about which childrens rights impact assessments have been done Have any been done and can they be shared with the committee if they have ? Sorry Deputy Minister\\u2014my mistake \\nJulie Morgan AM: Well it is been a very difficult time as you appreciate in terms of having to make legislation very quickly and it has not been possible to do the impact assessments that we would normally do However I am very pleased to say that we are actually launching a survey of children We are going to be launching it next week And this is to try to get from children their views of what is happened what we have been doing and their views on the whole COVID19 situation So we are doing this in conjunction with the childrens commissioner and with Young Wales and with the Youth Parliament So this is an online survey that we hope will be going out to thousands of children and we will get their response in terms of what are the important issues that have arisen for them what they feel about what is happened during this period what they feel about the way that we have dealt with the schools the way that they have had to cope in not going school and being at home for so long And so we are trying to get feedback from young people So I am very pleased that we are doing that but in terms of an impact assessment it has been very difficult as I am sure you can imagine to be able to do those at these times I think that Albert wants to come in on that \\nSuzy Davies AM: Yes because I will pursue that in a sec \\nAlbert Heaney: Thank you Thank you Chair and I think Nicola indicated before me so apologies Nicola Just to say for the committee really importantly that we have not introduced any easements in relation to childrens services legislation I think that is really quite crucial So from a Welsh context the standards that are in place do remain so therefore there would not have been a necessity for us to do a childrens rights impact assessment in relation to the primary legislation I think that is particularly a strong point to us in Wales both in terms of safeguarding arrangements but also ensuring that childrens rights are protected at a crucial time \\nNicola Edwards: Thanks In terms of childcare and education we are obviously looking at the provisions under the coronavirus Act to allow us to maybe ease some of the statutory requirements and we are going to be undertaking a full suite of impact assessments on those Obviously the coronavirus Act itself was UK Government legislation and they ran their own impact assessments but in terms of how we implement it in the childcare and education space\\u2014and I think Albert was just saying the same thing\\u2014we definitely will be looking at those impacts in terms of going forward \\nSuzy Davies AM: Well just to come back on that then are you saying to me that as a result of the various coronavirus regulations that we have had no assessments for childrens needs have been postponed cancelled or done very quickly online rather than in person ? \\nJulie Morgan AM: Well I think as Albert said that there was no relaxation of regulation for childrens social care You know that is\\u2014there have not been any in Wales \\nSuzy Davies AM: No but that is what\\u2014 There is no relaxation but what is happening in practice ? We are down on staff across all our councils and in our third sectors\\u2014who is doing the childrens needs assessments particularly for young carers ? \\nJulie Morgan AM: Well I\\u2014 Albert can you answer that ? \\nAlbert Heaney: I think the first thing to say to the committee is that going back we took a very strong line at the beginning that we were not going to introduce easements in requirements to childrens social services Of course through the way that practitioners and social work practitioners have to operate they are having to operate through a different time So assessments are still taking place for child protection and safeguarding concerns assessments are still taking place and especially in relation to\\u2014as you mentioned\\u2014young carers to support their needs So arrangements\\u2014Inaudible But they are having to be slightly differently done\\u2014so some of the technology and keeping in contact and keeping those visits So we have used for example the St Davids Day fund to make sure that care leavers are well supported in terms of having contact and are accessible and able to engage as well So we are having to be a little bit more\\u2014and social services departments are having to be a little bit more\\u2014innovative in the use of technology in the way that they have engaged as well But personal visits are taking place and visits especially as the Minister mentioned earlier on\\u2014they actually individually assess each case to determine the frequency of visits to make sure that those contacts are maintained with children at a critical time \\nSuzy Davies AM: Thank you I do not want to take this much further but personal visits and social distancing could be slightly problematic I just want to finish with this one question if I may We have had recommendations from the Carers Trust or Carers Trust Wales Have they been accepted by Government and is it those that are driving the agenda of the task and finish group that you announced the other day Deputy Minister ? \\nJulie Morgan AM: Well those will certainly be considered by the task and finish group I have had a letter from the Carers Trust about those issues and we are setting up this group as you know and we will be looking at those issues in the group \\nSuzy Davies AM: Thank you Any steal on when that might report ? \\nJulie Morgan AM: I do not have that at the moment \\nLynne Neagle AM: Maybe we could have a note on that Deputy Minister Can I just say we are running short of time ? We did start late so if the Ministers are happy we will carry on until 210 pm\\u2014310 pm\\u2014if that is And the next questions are from Si\\u00e2n Gwenllian Hold on a sec Si\\u00e2n we have lost translation again Can we just see what can be done to get the translation back ? Sorry Si\\u00e2n Is there anyone who can help with the translation ? There you go Si\\u00e2n Thank you \\nSian Gwenllian AM: You will know Deputy Minister\\u2014because we have discussed this in private session\\u2014my major concerns with regard to the childcare sector and what kind of childcare sector we will have at the end of this crisis as families start to return to the workplace There are still some childcare providers who are falling between the cracks and are not receiving financial support Do you agree\\u2014are there people who are still not being supported and why is not the Welsh Government able to provide that support for everyone in the childcare sector ? \\nJulie Morgan AM: Thank you Si\\u00e2n for that question And I know that we have had a discussion about this before Basically we are aware that there are some sectors in the childcare sector that do fall through some of the loops We have guaranteed that we will pay the money for the childcare offer for three months So that is guaranteed to them and they are able to take advantage of the Governments job retainer scheme but that does mean that there is a problem as I think we have discussed before of the double funding issue and that is something that we have been trying to resolve and there have been discussions with the Treasury in Whitehall about ways forward on this I am going to ask Nicola to come in in a minute because she is much more up to date with the discussions about that but so far I do not think very much progress has been made on that But we are looking to see if there are any other ways that we can get help to the childcare sector and I am actually following this meeting with a meeting with the Deputy Minister for equality and chief whip who is responsible for the voluntary sector because obviously many of the groups that we are talking about would come under the voluntary sector because they have voluntary committees but they fall between many stools because they rent premises rather than own premises and they do not have high turnovers that would qualify them for some of these grants So perhaps I could ask Nicola to come in to expand on that\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"prompt\",\n \"properties\": {\n \"dtype\": \"string\",\n \"num_unique_values\": 6,\n \"samples\": [\n \"## meeting transcript\\nPhD F: As opposed to the rest of us \\nPhD D: Well comment OK I I remind that me my first objective eh in the project is to to study difference parameters to to find a a good solution to detect eh the overlapping zone in eh speech recorded But eh tsk comment ehhh comment In that way comment I I I begin to to study and to analyze the ehn the recorded speech eh the different session to to find and to locate and to mark eh the the different overlapping zone And eh so eh I was eh I am transcribing the the first session and I I have found eh eh one thousand acoustic events eh besides the overlapping zones eh I I I mean the eh breaths eh aspiration eh eh talk eh eh clap eh comment I do not know what is the different names eh you use to to name the the pause n speech \\nGrad G: Oh I do not think we ve been doing it at that level of detail So \\nPhD D: Eh I I I do I do not need to to to mmm to m to label the the different acoustic but I prefer because eh I would like to to study if eh I I will find eh eh a good eh parameters eh to detect overlapping I would like to to to test these parameters eh with the another eh eh acoustic events to nnn to eh to find what is the ehm the false eh the false eh hypothesis eh nnn which eh are produced when we use the the ehm this eh parameter eh I mean pitch eh eh difference eh feature \\nPhD A: You know I think some of these that are the nonspeech overlapping events may be difficult even for humans to tell that there s two there I mean if it s a tapping sound you would not necessarily or you know something like that it would be it might be hard to know that it was two separate events \\nGrad G: Well You were not talking about just overlaps were you ? You were just talking about acoustic events \\nPhD D: I I I I t I t I talk eh about eh acoustic events in general but eh my my objective eh will be eh to study eh overlapping zone Eh ? comment n Eh in twelve minutes I found eh eh one thousand acoustic events \\nProfessor E: How many overlaps were there in it ? No no how many of them were the overlaps of speech though ? \\nPhD D: How many ? Eh almost eh three hundred eh in one session in five eh in forty five minutes Alm Three hundred overlapping zone With the overlapping zone overlapping speech speech what eh different duration \\nPostdoc B: Does this ? So if you had an overlap involving three people how many times was that counted ? \\nPhD D: three people two people Eh I would like to consider eh one people with difference noise eh in the background be \\nProfessor E: No no but I think what she s asking is pause if at some particular for some particular stretch you had three people talking instead of two did you call that one event ? \\nPhD D: Oh Oh I consider one event eh for th for that eh for all the zone This th I I I con I consider I consider eh an acoustic event the overlapping zone the period where three speaker or eh are talking together \\nGrad G: So let s say me and Jane are talking at the same time and then Liz starts talking also over all of us How many events would that be ? \\nPhD D: So I do not understand \\nGrad G: So two people are talking comment and then a third person starts talking Is there an event right here ? \\nPhD D: Eh no No no For me is the overlapping zone because because you you have s you have more one eh more one voice eh eh produced in a in in a moment \\nGrad G: So i if two or more people are talking \\nProfessor E: OK So I think We just wanted to understand how you are defining it So then in the region between since there there is some continuous region in between regions where there is only one person speaking And one contiguous region like that you are calling an event Is it Are you calling the beginning or the end of it the event or are you calling the entire length of it the event ? \\nPhD D: I consider the the nnn the nnn nnn eh the entirety eh eh all all the time there were the voice has overlapped This is the idea But eh I I do not distinguish between the the numbers of eh speaker I m not considering eh the the ehm eh the fact of eh eh for example what did you say ? Eh at first eh eh two talkers are eh speaking and eh eh a third person eh join to to that For me it s eh it s eh all overlap zone with eh several numbers of speakers is eh eh the same acoustic event Wi but without any mark between the zone of the overlapping zone with two speakers eh speaking together and the zone with the three speakers \\nPostdoc B: That would j just be one \\nPhD D: Eh with eh a beginning mark and the ending mark Because eh for me is the is the zone with eh some kind of eh distortion the spectral I do not mind By the moment by the moment \\nGrad G: Well but But you could imagine that three people talking has a different spectral characteristic than two \\nPhD D: I I do not but eh but eh I have to study comment What will happen in a general way \\nGrad G: So You had to start somewhere \\nPhD C: So there s a lot of overlap \\nPhD D: I I do not know what eh will will happen with the \\nGrad G: That s a lot of overlap \\nProfessor E: So again that s that s three three hundred in forty five minutes that are that are speakers just speakers \\nPostdoc B: But a a a th \\nProfessor E: So that s about eight per minute \\nPostdoc B: But a thousand events in twelve minutes that s \\nPhD C: But that can include taps \\nPostdoc B: Well but a thousand taps in eight minutes is a l in twelve minutes is a lot \\nPhD D: I I con I consider I consider acoustic events eh the silent too \\nGrad G: Silence starting or silence ending \\nPhD D: silent ground to bec to detect eh because I consider acoustic event all the things are not eh speech In ge in in in a general point of view \\nProfessor E: OK so how many of those thousand were silence ? \\nPhD F: Not speech not speech or too much speech \\nProfessor E: Right So how many of those thousand were silence silent sections ? \\nPhD D: silent I I I I do not I I have not the eh I I would like to to do a stylistic study\\n\\n## instructions\\nSummarize the meeting based on the transcript. In paragraph form, output your response. Use at least 10 words and at most 50 words in total.\",\n \"## meeting transcript\\nLynne Neagle AM: Thank you very much And the next questions then are from Suzy Davies \\nSuzy Davies AM: Thank you Chair I just wanted to have a quick answer from probably the Minister I think about the primary legislation and the regulations that followed about which childrens rights impact assessments have been done Have any been done and can they be shared with the committee if they have ? Sorry Deputy Minister\\u2014my mistake \\nJulie Morgan AM: Well it is been a very difficult time as you appreciate in terms of having to make legislation very quickly and it has not been possible to do the impact assessments that we would normally do However I am very pleased to say that we are actually launching a survey of children We are going to be launching it next week And this is to try to get from children their views of what is happened what we have been doing and their views on the whole COVID19 situation So we are doing this in conjunction with the childrens commissioner and with Young Wales and with the Youth Parliament So this is an online survey that we hope will be going out to thousands of children and we will get their response in terms of what are the important issues that have arisen for them what they feel about what is happened during this period what they feel about the way that we have dealt with the schools the way that they have had to cope in not going school and being at home for so long And so we are trying to get feedback from young people So I am very pleased that we are doing that but in terms of an impact assessment it has been very difficult as I am sure you can imagine to be able to do those at these times I think that Albert wants to come in on that \\nSuzy Davies AM: Yes because I will pursue that in a sec \\nAlbert Heaney: Thank you Thank you Chair and I think Nicola indicated before me so apologies Nicola Just to say for the committee really importantly that we have not introduced any easements in relation to childrens services legislation I think that is really quite crucial So from a Welsh context the standards that are in place do remain so therefore there would not have been a necessity for us to do a childrens rights impact assessment in relation to the primary legislation I think that is particularly a strong point to us in Wales both in terms of safeguarding arrangements but also ensuring that childrens rights are protected at a crucial time \\nNicola Edwards: Thanks In terms of childcare and education we are obviously looking at the provisions under the coronavirus Act to allow us to maybe ease some of the statutory requirements and we are going to be undertaking a full suite of impact assessments on those Obviously the coronavirus Act itself was UK Government legislation and they ran their own impact assessments but in terms of how we implement it in the childcare and education space\\u2014and I think Albert was just saying the same thing\\u2014we definitely will be looking at those impacts in terms of going forward \\nSuzy Davies AM: Well just to come back on that then are you saying to me that as a result of the various coronavirus regulations that we have had no assessments for childrens needs have been postponed cancelled or done very quickly online rather than in person ? \\nJulie Morgan AM: Well I think as Albert said that there was no relaxation of regulation for childrens social care You know that is\\u2014there have not been any in Wales \\nSuzy Davies AM: No but that is what\\u2014 There is no relaxation but what is happening in practice ? We are down on staff across all our councils and in our third sectors\\u2014who is doing the childrens needs assessments particularly for young carers ? \\nJulie Morgan AM: Well I\\u2014 Albert can you answer that ? \\nAlbert Heaney: I think the first thing to say to the committee is that going back we took a very strong line at the beginning that we were not going to introduce easements in requirements to childrens social services Of course through the way that practitioners and social work practitioners have to operate they are having to operate through a different time So assessments are still taking place for child protection and safeguarding concerns assessments are still taking place and especially in relation to\\u2014as you mentioned\\u2014young carers to support their needs So arrangements\\u2014Inaudible But they are having to be slightly differently done\\u2014so some of the technology and keeping in contact and keeping those visits So we have used for example the St Davids Day fund to make sure that care leavers are well supported in terms of having contact and are accessible and able to engage as well So we are having to be a little bit more\\u2014and social services departments are having to be a little bit more\\u2014innovative in the use of technology in the way that they have engaged as well But personal visits are taking place and visits especially as the Minister mentioned earlier on\\u2014they actually individually assess each case to determine the frequency of visits to make sure that those contacts are maintained with children at a critical time \\nSuzy Davies AM: Thank you I do not want to take this much further but personal visits and social distancing could be slightly problematic I just want to finish with this one question if I may We have had recommendations from the Carers Trust or Carers Trust Wales Have they been accepted by Government and is it those that are driving the agenda of the task and finish group that you announced the other day Deputy Minister ? \\nJulie Morgan AM: Well those will certainly be considered by the task and finish group I have had a letter from the Carers Trust about those issues and we are setting up this group as you know and we will be looking at those issues in the group \\nSuzy Davies AM: Thank you Any steal on when that might report ? \\nJulie Morgan AM: I do not have that at the moment \\nLynne Neagle AM: Maybe we could have a note on that Deputy Minister Can I just say we are running short of time ? We did start late so if the Ministers are happy we will carry on until 210 pm\\u2014310 pm\\u2014if that is And the next questions are from Si\\u00e2n Gwenllian Hold on a sec Si\\u00e2n we have lost translation again Can we just see what can be done to get the translation back ? Sorry Si\\u00e2n Is there anyone who can help with the translation ? There you go Si\\u00e2n Thank you \\nSian Gwenllian AM: You will know Deputy Minister\\u2014because we have discussed this in private session\\u2014my major concerns with regard to the childcare sector and what kind of childcare sector we will have at the end of this crisis as families start to return to the workplace There are still some childcare providers who are falling between the cracks and are not receiving financial support Do you agree\\u2014are there people who are still not being supported and why is not the Welsh Government able to provide that support for everyone in the childcare sector ? \\nJulie Morgan AM: Thank you Si\\u00e2n for that question And I know that we have had a discussion about this before Basically we are aware that there are some sectors in the childcare sector that do fall through some of the loops We have guaranteed that we will pay the money for the childcare offer for three months So that is guaranteed to them and they are able to take advantage of the Governments job retainer scheme but that does mean that there is a problem as I think we have discussed before of the double funding issue and that is something that we have been trying to resolve and there have been discussions with the Treasury in Whitehall about ways forward on this I am going to ask Nicola to come in in a minute because she is much more up to date with the discussions about that but so far I do not think very much progress has been made on that But we are looking to see if there are any other ways that we can get help to the childcare sector and I am actually following this meeting with a meeting with the Deputy Minister for equality and chief whip who is responsible for the voluntary sector because obviously many of the groups that we are talking about would come under the voluntary sector because they have voluntary committees but they fall between many stools because they rent premises rather than own premises and they do not have high turnovers that would qualify them for some of these grants So perhaps I could ask Nicola to come in to expand on that\\n\\n## instructions\\nSummarize the meeting based on the transcript. Return the answer in the form of paragraphs. Make sure your answer is between 50 and 200 words long.\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"transcript_token_len\",\n \"properties\": {\n \"dtype\": \"number\",\n \"std\": 568,\n \"min\": 1100,\n \"max\": 2618,\n \"num_unique_values\": 6,\n \"samples\": [\n 1378,\n 1649\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"completion\",\n \"properties\": {\n \"dtype\": \"string\",\n \"num_unique_values\": 6,\n \"samples\": [\n \"PhD student D presented their method for detecting and counting acoustic events in recorded speech sessions, including speech overlaps and silence. The team sought clarification on D's definitions and counted events, discussing the challenges of distinguishing between different types of events.\",\n \"The meeting between Welsh politicians and officials centered around the impact of the coronavirus legislation on children's rights and the childcare sector in Wales. Julie Morgan, the AM, explained that a survey would be launched to gather the views of children on the pandemic situation, as it had been challenging to conduct children's rights impact assessments during these fast-paced times. \\n\\nAlbert Heaney emphasized that there had been no relaxation of children's social care regulations in Wales, but that the methods of assessment had to be adapted due to the pandemic. In response to Sian Gwenllian's concerns, Julie Morgan acknowledged that some childcare providers were struggling financially due to the pandemic. She assured that the government was trying to support them, but the complexity of the issue had hindered progress, especially regarding the double funding problem.\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"format_score\",\n \"properties\": {\n \"dtype\": \"number\",\n \"std\": 0,\n \"min\": 1,\n \"max\": 1,\n \"num_unique_values\": 1,\n \"samples\": [\n 1\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"length_score\",\n \"properties\": {\n \"dtype\": \"number\",\n \"std\": 0,\n \"min\": 0,\n \"max\": 1,\n \"num_unique_values\": 2,\n \"samples\": [\n 0\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"completeness_score\",\n \"properties\": {\n \"dtype\": \"number\",\n \"std\": 1.2161883888976234e-16,\n \"min\": 0.8,\n \"max\": 0.8,\n \"num_unique_values\": 1,\n \"samples\": [\n 0.8\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"correctness_score\",\n \"properties\": {\n \"dtype\": \"number\",\n \"std\": 0.0,\n \"min\": 1.0,\n \"max\": 1.0,\n \"num_unique_values\": 1,\n \"samples\": [\n 1.0\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"conciseness_score\",\n \"properties\": {\n \"dtype\": \"number\",\n \"std\": 1.2161883888976234e-16,\n \"min\": 0.8,\n \"max\": 0.8,\n \"num_unique_values\": 1,\n \"samples\": [\n 0.8\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n }\n ]\n}",
- "type": "dataframe",
- "variable_name": "data"
- },
"text/html": [
- "\n",
- "
![\"Open](\"https://colab.research.google.com/assets/colab-badge.svg\"){kind=icon}
\n",
+ "![](\"https://github.com/cohere-ai/notebooks/raw/main/notebooks/images/simple-classifier-embeddings.png\"\n",)
\n",
@@ -31,8 +31,7 @@
"outputs": [],
"source": [
"# Let's first install Cohere's python SDK\n",
- "# TODO: upgrade to \"cohere>5\"",
-"!pip install \"cohere<5\" scikit-learn"
+ "# TODO: upgrade to \"cohere>5\"!pip install \"cohere<5\" scikit-learn"
]
},
{
@@ -276,7 +275,7 @@
"source": [
"We now have two sets of embeddings, `embeddings_train` contains the embeddings of the training sentences while `embeddings_test` contains the embeddings of the testing sentences.\n",
"\n",
- "Curious what an embedding looks like? we can print it:"
+ "Curious what an embedding looks like? We can print it:"
]
},
{
@@ -311,7 +310,7 @@
},
"source": [
"## 3. Train a classifier using the training set\n",
- "Now that we have the embedding we can train our classifier. We'll use an SVM from sklearn."
+ "Now that we have the embedding, we can train our classifier. We'll use an SVM from sklearn."
]
},
{
@@ -428,7 +427,7 @@
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
- "version": "3.9.7"
+ "version": "3.8.16"
}
},
"nbformat": 4,
diff --git a/notebooks/guides/Wikipedia_search_demo_cohere_weaviate.ipynb b/notebooks/guides/Wikipedia_search_demo_cohere_weaviate.ipynb
index f7526361..47a89e02 100644
--- a/notebooks/guides/Wikipedia_search_demo_cohere_weaviate.ipynb
+++ b/notebooks/guides/Wikipedia_search_demo_cohere_weaviate.ipynb
@@ -6,18 +6,7 @@
"metadata": {},
"source": [
"# Wikipedia Semantic Search with Cohere + Weaviate\n",
- "This is starter code that you can use to search 10 million vectors from wikipedia embedded with Cohere's multilingual model and hosted as a Weaviate public dataset. This dataset contains 1M vectors in each of the Wikipedia sites in these languages: English, German, French, Spanish, Italian, Japanese, Arabic, Chinese (Simplified), Korean, Hindi \\[respective language codes: `en, de, fr, es, it, ja, ar, zh, ko, hi`\\]"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "id": "43debcc4-eb17-4bc7-8210-96a0905eec1a",
- "metadata": {},
- "outputs": [],
- "source": [
- "# TODO: upgrade to \"cohere>5\"",
-"!pip install \"cohere<5\" weaviate-client"
+ "This is starter code that you can use to search 10 million vectors from Wikipedia embedded with Cohere's multilingual model and hosted as a Weaviate public dataset. This dataset contains 1M vectors in each of the Wikipedia sites in these languages: English, German, French, Spanish, Italian, Japanese, Arabic, Chinese (Simplified), Korean, Hindi \\[respective language codes: `en, de, fr, es, it, ja, ar, zh, ko, hi`\\]"
]
},
{
@@ -139,7 +128,7 @@
"id": "7b0cb81c-8e56-48cf-a32c-f050afe646e4",
"metadata": {},
"source": [
- "We can now query the databse with any query we want. In the background, Weaviate uses your Cohere API key to embed the query, then retrun the most relevant passages to the query."
+ "We can now query the database with any query we want. In the background, Weaviate uses your Cohere API key to embed the query, then retrun the most relevant passages to the query."
]
},
{
@@ -248,7 +237,7 @@
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
- "version": "3.9.13"
+ "version": "3.8.16"
}
},
"nbformat": 4,
diff --git a/notebooks/images/.DS_Store b/notebooks/images/.DS_Store
new file mode 100644
index 00000000..eab24b34
Binary files /dev/null and b/notebooks/images/.DS_Store differ
diff --git a/notebooks/react_agent_adaptive_rag_cohere.ipynb b/notebooks/react_agent_adaptive_rag_cohere.ipynb
deleted file mode 100644
index 61938521..00000000
--- a/notebooks/react_agent_adaptive_rag_cohere.ipynb
+++ /dev/null
@@ -1,967 +0,0 @@
-{
- "cells": [
- {
- "cell_type": "markdown",
- "metadata": {
- "id": "EUYMLmXAF3W1"
- },
- "source": [
- "# A ReAct agent with Command R+, achieving the same goals as Adaptive RAG\n",
- "\n",
- "Adaptive RAG is a strategy for RAG that unites (1) [query analysis](https://blog.langchain.dev/query-construction/) with (2) [active / self-corrective RAG](https://blog.langchain.dev/agentic-rag-with-langgraph/).\n",
- "\n",
- "In the paper, they report query analysis to route across:\n",
- "- No Retrieval (LLM answers)\n",
- "- Single-shot RAG\n",
- "- Iterative RAG\n",
- "\n",
- "\n",
- "We'll use Command R+, a recent release from Cohere that:\n",
- "- Has strong accuracy on RAG, Tool Use and Agents\n",
- "- Has 128k context\n"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {
- "id": "W8yvOB2MQp07"
- },
- "source": [
- "# Environment"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": 1,
- "metadata": {
- "colab": {
- "base_uri": "https://localhost:8080/"
- },
- "id": "ujQVUvA9QlD4",
- "outputId": "230c6a56-3ee3-4e0d-95af-2a261b8541d6"
- },
- "outputs": [
- {
- "name": "stdout",
- "output_type": "stream",
- "text": [
- "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m812.8/812.8 kB\u001b[0m \u001b[31m6.6 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
- "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m1.8/1.8 MB\u001b[0m \u001b[31m9.5 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
- "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m27.0/27.0 MB\u001b[0m \u001b[31m16.8 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
- "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m1.9/1.9 MB\u001b[0m \u001b[31m28.7 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
- "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m276.8/276.8 kB\u001b[0m \u001b[31m3.4 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
- "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m87.5/87.5 kB\u001b[0m \u001b[31m4.6 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
- "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m145.9/145.9 kB\u001b[0m \u001b[31m8.8 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
- "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m3.1/3.1 MB\u001b[0m \u001b[31m34.4 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
- "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m75.6/75.6 kB\u001b[0m \u001b[31m3.3 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
- "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m49.4/49.4 kB\u001b[0m \u001b[31m2.2 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
- "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m53.0/53.0 kB\u001b[0m \u001b[31m516.7 kB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
- "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m144.8/144.8 kB\u001b[0m \u001b[31m5.4 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
- "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m77.9/77.9 kB\u001b[0m \u001b[31m6.6 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
- "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m58.3/58.3 kB\u001b[0m \u001b[31m5.1 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
- "\u001b[?25h"
- ]
- }
- ],
- "source": [
- "! pip install --quiet langchain langchain_cohere tiktoken faiss-cpu"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": 2,
- "metadata": {
- "id": "K9rrOrKWJaBd"
- },
- "outputs": [],
- "source": [
- "### LLMs\n",
- "import os\n",
- "\n",
- "os.environ[\"COHERE_API_KEY\"] = \"