update multiturn prompt caching to take advantage of implicit cache reads

Author: kevinc13

misc/prompt_caching.ipynb

@@ -45,7 +45,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 4,
+   "execution_count": 1,
    "metadata": {},
    "outputs": [],
    "source": [
@@ -67,14 +67,14 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 5,
+   "execution_count": 2,
    "metadata": {},
    "outputs": [
     {
      "name": "stdout",
      "output_type": "stream",
      "text": [
-      "Fetched 737522 characters from the book.\n",
+      "Fetched 737525 characters from the book.\n",
       "First 500 characters:\n",
       "The Project Gutenberg eBook of Pride and Prejudice\n",
       "This ebook is for the use of anyone anywhere in the United States and\n",
@@ -284,7 +284,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 8,
+   "execution_count": 3,
    "metadata": {},
    "outputs": [
     {
@@ -300,70 +300,77 @@
       "Input tokens (cache read): 0\n",
       "Input tokens (cache write): 187354\n",
       "0.0% of input prompt cached (4 tokens)\n",
-      "Time taken: 22.04 seconds\n",
+      "Time taken: 20.37 seconds\n",
       "\n",
       "Turn 2:\n",
       "User: Who are Mr. and Mrs. Bennet?\n",
-      "Assistant: Mr. and Mrs. Bennet are the parents of five daughters (Jane, Elizabeth, Mary, Kitty, and Lydia) in Pride and Prejudice. Their personalities are quite different from each other:\n",
+      "Assistant: Mr. and Mrs. Bennet are the parents of five daughters (Jane, Elizabeth, Mary, Kitty, and Lydia) in Pride and Prejudice. \n",
       "\n",
-      "Mr. Bennet is an intelligent but rather detached and sarcastic man who often retreats to his library to avoid his wife's dramatic behavior. He is fond of his daughter Elizabeth (Lizzy) in particular due to her wit and good sense. While he can be amusing, he is also somewhat negligent in his duties as a father and head of the household.\n",
+      "Mr. Bennet is an intelligent but detached father who often retreats to his library to avoid his wife's dramatics. He has a satirical wit and tends to be amused by the follies of others, including his own family members. He shows particular fondness for his second daughter Elizabeth, who shares his sharp mind and wit.\n",
       "\n",
-      "Mrs. Bennet is a rather foolish, nervous, and obsessive woman whose main goal in life is to see her daughters married to wealthy men. This preoccupation stems partly from the fact that their estate is entailed to a male heir (Mr. Collins), meaning her daughters will be left with little after Mr. Bennet's death. She often embarrasses her more sensible daughters (particularly Elizabeth and Jane) with her lack of social graces and her obvious attempts at matchmaking.\n",
+      "Mrs. Bennet is a woman primarily focused on getting her five daughters married to wealthy men. She is described as having \"poor nerves\" and is often anxious, dramatic, and somewhat foolish. Her main goal in life is to see her daughters well-married, particularly because their family estate is entailed to a male heir (Mr. Collins), meaning her daughters will be left with little financial security after Mr. Bennet's death. She is often described as lacking sophistication and good judgment, which sometimes embarrasses her more sensible daughters, particularly Elizabeth.\n",
       "\n",
-      "Their marriage is portrayed as an unfortunate example of a match based on superficial attraction (Mr. Bennet was attracted to Mrs. Bennet's youth and beauty) rather than compatibility of character and intellect, serving as a contrast to the more meaningful relationships that develop later in the novel.\n",
+      "Their marriage is portrayed as an ill-matched one, where Mr. Bennet married Mrs. Bennet in his youth because of her beauty, only to discover they were incompatible in terms of intellect and character. This serves as a cautionary tale about marrying without proper consideration of character compatibility.\n",
       "User input tokens: 4\n",
-      "Output tokens: 295\n",
+      "Output tokens: 297\n",
       "Input tokens (cache read): 187354\n",
       "Input tokens (cache write): 36\n",
       "100.0% of input prompt cached (187358 tokens)\n",
-      "Time taken: 8.32 seconds\n",
+      "Time taken: 7.53 seconds\n",
       "\n",
       "Turn 3:\n",
       "User: What is Netherfield Park?\n",
-      "Assistant: Netherfield Park is a large estate near the Bennets' home of Longbourn that is rented by Mr. Bingley at the beginning of the novel. It becomes a central location in the story as Mr. Bingley's arrival there sets many of the novel's key events in motion.\n",
+      "Assistant: Netherfield Park is a large estate near the Bennet family home of Longbourn in the novel. It becomes significant to the plot when it is rented by Mr. Bingley, a wealthy young man who moves into the neighborhood. \n",
       "\n",
-      "The arrival of Mr. Bingley at Netherfield creates much excitement in the neighborhood, particularly for Mrs. Bennet, who sees him as a potential husband for one of her daughters due to his wealth (he has an income of around 4-5 thousand pounds a year). Indeed, it is at Netherfield where the romance between Jane Bennet and Mr. Bingley begins to develop.\n",
+      "The arrival of Mr. Bingley at Netherfield sets much of the novel's plot in motion, as he quickly develops a romantic interest in Jane Bennet, the eldest Bennet daughter. It's also through Netherfield that Elizabeth Bennet first encounters Mr. Darcy, who is staying there as Mr. Bingley's friend.\n",
       "\n",
-      "Netherfield is also significant because it is where Jane falls ill while visiting Mr. Bingley's sisters, leading to Elizabeth coming to nurse her. This extended stay at Netherfield provides important opportunities for Elizabeth to interact with Mr. Darcy, though at this point in the novel their interactions are marked by pride and prejudice on both sides.\n",
+      "Netherfield serves as an important setting for several key scenes in the novel, including:\n",
+      "- The ball where Mr. Darcy first slights Elizabeth\n",
+      "- Jane's illness and subsequent stay at Netherfield (where Elizabeth comes to nurse her)\n",
+      "- Various social interactions between the Bennets and the Bingley-Darcy party\n",
       "\n",
-      "The house and estate represent wealth and social status in the novel, and its occupancy by the Bingleys brings the higher social circles of London into contact with the more provincial society of Meryton and its surroundings.\n",
+      "The estate symbolizes wealth and social status in the novel, and its occupancy by Mr. Bingley represents the possibility of social and financial advancement for the Bennet family through marriage. When Bingley suddenly leaves Netherfield, it creates significant disappointment and disruption in the hopes of the Bennet family, particularly for Jane.\n",
       "User input tokens: 4\n",
-      "Output tokens: 286\n",
+      "Output tokens: 289\n",
       "Input tokens (cache read): 187390\n",
-      "Input tokens (cache write): 306\n",
+      "Input tokens (cache write): 308\n",
       "100.0% of input prompt cached (187394 tokens)\n",
-      "Time taken: 10.73 seconds\n",
+      "Time taken: 6.76 seconds\n",
       "\n",
       "Turn 4:\n",
       "User: What is the main theme of this novel?\n",
       "Assistant: The main theme of \"Pride and Prejudice\" is the interplay between pride and prejudice in relationships, particularly as shown through the central romance between Elizabeth Bennet and Mr. Darcy. However, there are several important related themes:\n",
       "\n",
       "1. Pride and Prejudice:\n",
       "- Darcy's pride in his social position initially makes him appear arrogant and disdainful\n",
-      "- Elizabeth's prejudice against Darcy is based on first impressions and incomplete information\n",
+      "- Elizabeth's prejudice against Darcy based on first impressions and Wickham's false account\n",
       "- Both characters must overcome these flaws to find happiness together\n",
       "\n",
       "2. Marriage and Social Class:\n",
-      "- The novel explores different types of marriages (for love, security, or social advancement)\n",
-      "- Social class differences and their impact on relationships\n",
-      "- The economic necessity of marriage for women in the period\n",
-      "- Contrast between marriages based on true understanding (Elizabeth/Darcy, Jane/Bingley) versus those based on superficial attractions (Lydia/Wickham) or practical considerations (Charlotte/Collins)\n",
+      "- The pressure on young women to marry well for financial security\n",
+      "- The conflict between marrying for love versus social advantage\n",
+      "- Different types of marriages are portrayed (Elizabeth/Darcy, Jane/Bingley, Lydia/Wickham, Charlotte/Collins)\n",
       "\n",
       "3. Reputation and Social Expectations:\n",
-      "- The importance of reputation, particularly for women\n",
-      "- Social manners and proper behavior\n",
-      "- The constraints placed on women in Regency society\n",
+      "- The importance of reputation in Regency society\n",
+      "- How behavior reflects on family honor\n",
+      "- The restrictions placed on women in this period\n",
       "\n",
       "4. Personal Growth and Self-Knowledge:\n",
-      "- Elizabeth and Darcy's journey to better understand themselves and each other\n",
-      "- The importance of overcoming first impressions and preconceptions\n",
-      "- Learning from mistakes and misj\n",
+      "- Elizabeth and Darcy both learn to recognize their own faults\n",
+      "- The importance of overcoming first impressions\n",
+      "- Character development through experience and reflection\n",
+      "\n",
+      "5. Family and Society:\n",
+      "- The role of family connections in determining social status\n",
+      "- The impact of family behavior on individual prospects\n",
+      "- The balance between\n",
       "User input tokens: 4\n",
       "Output tokens: 300\n",
-      "Input tokens (cache read): 187696\n",
-      "Input tokens (cache write): 298\n",
-      "100.0% of input prompt cached (187700 tokens)\n",
-      "Time taken: 7.07 seconds\n"
+      "Input tokens (cache read): 187698\n",
+      "Input tokens (cache write): 301\n",
+      "100.0% of input prompt cached (187702 tokens)\n",
+      "Time taken: 7.13 seconds\n"
      ]
     }
    ],
@@ -403,8 +410,8 @@
     "        user_turns_processed = 0\n",
     "        # Iterate through turns in reverse order\n",
     "        for turn in reversed(self.turns):\n",
-    "            if turn[\"role\"] == \"user\" and user_turns_processed < 2:\n",
-    "                # Add the last two user turns with ephemeral cache control\n",
+    "            if turn[\"role\"] == \"user\" and user_turns_processed < 1:\n",
+    "                # Add the last user turn with ephemeral cache control\n",
     "                result.append({\n",
     "                    \"role\": \"user\",\n",
     "                    \"content\": [\n",
@@ -507,7 +514,7 @@
  ],
  "metadata": {
   "kernelspec": {
-   "display_name": "base",
+   "display_name": "py311",
    "language": "python",
    "name": "python3"
   },
@@ -521,7 +528,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.10.14"
+   "version": "3.11.11"
   }
  },
  "nbformat": 4,