Use genai.protos to access the raw protos. (google#446)

* replace glm reference with genai.protos

* fix links

* Fix more glm references

* genai.protos

* Generative Language API -> Gemini API

Author: MarkDaoust

examples/gemini/python/vectordb_with_chroma/vectordb_with_chroma.ipynb

@@ -138,7 +138,6 @@
         "import pandas as pd\n",
         "\n",
         "import google.generativeai as genai\n",
-        "import google.ai.generativelanguage as glm\n",
         "\n",
         "# Used to securely store your API key\n",
         "from google.colab import userdata\n",

site/en/docs/search_reranking_using_embeddings.ipynb

@@ -184,7 +184,6 @@
         "import textwrap\n",
         "\n",
         "import google.generativeai as genai\n",
-        "import google.ai.generativelanguage as glm\n",
         "\n",
         "import wikipedia\n",
         "from wikipedia.exceptions import DisambiguationError, PageError\n",
@@ -821,7 +820,7 @@
         "In the chat history you can see all 4 steps:\n",
         "\n",
         "1. The user sent the query.\n",
-        "2. The model replied with a `glm.FunctionCall` calling the `wikipedia_search` with a number of relevant searches.\n",
+        "2. The model replied with a `genai.protos.FunctionCall` calling the `wikipedia_search` with a number of relevant searches.\n",
         "3. Because you set `enable_automatic_function_calling=True` when creating the `genai.ChatSession`, it  executed the search function and returned the list of article summaries to the model.\n",
         "4. Folliwing the instructions in the prompt, the model generated a final answer based on those summaries.\n"
       ]
@@ -1044,9 +1043,9 @@
       ],
       "source": [
         "response = chat.send_message(\n",
-        "    glm.Content(\n",
-        "      parts=[glm.Part(\n",
-        "          function_response = glm.FunctionResponse(\n",
+        "    genai.protos.Content(\n",
+        "      parts=[genai.protos.Part(\n",
+        "          function_response = genai.protos.FunctionResponse(\n",
         "            name='wikipedia_search',\n",
         "            response={'result': summaries}\n",
         "          )\n",

site/en/gemini-api/docs/function-calling/python.ipynb

@@ -131,7 +131,6 @@
         "import time\n",
         "\n",
         "import google.generativeai as genai\n",
-        "import google.ai.generativelanguage as glm\n",
         "\n",
         "from IPython import display\n",
         "from IPython.display import Markdown\n",
@@ -206,7 +205,7 @@
         "\n",
         "To use function calling, pass a list of functions to the `tools` parameter when creating a [`GenerativeModel`](https://ai.google.dev/api/python/google/generativeai/GenerativeModel). The model uses the function name, docstring, parameters, and parameter type annotations to decide if it needs the function to best answer a prompt.\n",
         "\n",
-        "> Important: The SDK converts function parameter type annotations to a format the API understands (`glm.FunctionDeclaration`). The API only supports a limited selection of parameter types, and the Python SDK's automatic conversion only supports a subset of that: `AllowedTypes = int | float | bool | str | list['AllowedTypes'] | dict`"
+        "> Important: The SDK converts function parameter type annotations to a format the API understands (`genai.protos.FunctionDeclaration`). The API only supports a limited selection of parameter types, and the Python SDK's automatic conversion only supports a subset of that: `AllowedTypes = int | float | bool | str | list['AllowedTypes'] | dict`"
       ]
     },
     {
@@ -327,13 +326,13 @@
       "source": [
         "Examine the chat history to see the flow of the conversation and how function calls are integrated within it.\n",
         "\n",
-        "The `ChatSession.history` property stores a chronological record of the conversation between the user and the Gemini model. Each turn in the conversation is represented by a [`glm.Content`](https://ai.google.dev/api/python/google/ai/generativelanguage/Content) object, which contains the following information:\n",
+        "The `ChatSession.history` property stores a chronological record of the conversation between the user and the Gemini model. Each turn in the conversation is represented by a [`genai.protos.Content`](https://ai.google.dev/api/python/google/generativeai/protos/Content) object, which contains the following information:\n",
         "\n",
         "*   **Role**: Identifies whether the content originated from the \"user\" or the \"model\".\n",
-        "*   **Parts**: A list of [`glm.Part`](https://ai.google.dev/api/python/google/ai/generativelanguage/Part) objects that represent individual components of the message. With a text-only model, these parts can be:\n",
+        "*   **Parts**: A list of [`genai.protos.Part`](https://ai.google.dev/api/python/google/generativeai/protos/Part) objects that represent individual components of the message. With a text-only model, these parts can be:\n",
         "    *   **Text**: Plain text messages.\n",
-        "    *   **Function Call** ([`glm.FunctionCall`](https://ai.google.dev/api/python/google/ai/generativelanguage/FunctionCall)): A request from the model to execute a specific function with provided arguments.\n",
-        "    *   **Function Response** ([`glm.FunctionResponse`](https://ai.google.dev/api/python/google/ai/generativelanguage/FunctionResponse)): The result returned by the user after executing the requested function.\n",
+        "    *   **Function Call** ([`genai.protos.FunctionCall`](https://ai.google.dev/api/python/google/generativeai/protos/FunctionCall)): A request from the model to execute a specific function with provided arguments.\n",
+        "    *   **Function Response** ([`genai.protos.FunctionResponse`](https://ai.google.dev/api/python/google/generativeai/protos/FunctionResponse)): The result returned by the user after executing the requested function.\n",
         "\n",
         " In the previous example with the mittens calculation, the history shows the following sequence:\n",
         "\n",
@@ -400,7 +399,7 @@
       "source": [
         "While this was all handled automatically, if you need more control, you can:\n",
         "\n",
-        "- Leave the default `enable_automatic_function_calling=False` and process the `glm.FunctionCall` responses yourself.\n",
+        "- Leave the default `enable_automatic_function_calling=False` and process the `genai.protos.FunctionCall` responses yourself.\n",
         "- Or use `GenerativeModel.generate_content`, where you also need to manage the chat history."
       ]
     },
@@ -541,7 +540,7 @@
         "\n",
         "# Build the response parts.\n",
         "response_parts = [\n",
-        "    glm.Part(function_response=glm.FunctionResponse(name=fn, response={\"result\": val}))\n",
+        "    genai.protos.Part(function_response=genai.protos.FunctionResponse(name=fn, response={\"result\": val}))\n",
         "    for fn, val in responses.items()\n",
         "]\n",
         "\n",
@@ -570,18 +569,7 @@
         "AllowedType = (int | float | bool | str | list['AllowedType'] | dict[str, AllowedType]\n",
         "```\n",
         "\n",
-        "The `google.ai.generativelanguage` client library provides access to the low level types giving you full control."
-      ]
-    },
-    {
-      "cell_type": "code",
-      "execution_count": null,
-      "metadata": {
-        "id": "S53E0EE8TBUF"
-      },
-      "outputs": [],
-      "source": [
-        "import google.ai.generativelanguage as glm"
+        "The `google.generativeai.protos` submodule provides access to the low level types giving you full control."
       ]
     },
     {
@@ -648,7 +636,7 @@
         "id": "qFD4U7ym04F5"
       },
       "source": [
-        "This returns the list of `glm.Tool` objects that would be sent to the API. If the printed format is not familiar, it's because these are Google protobuf classes. Each `glm.Tool` (1 in this case) contains a list of `glm.FunctionDeclarations`, which describe a function and its arguments."
+        "This returns the list of `genai.protos.Tool` objects that would be sent to the API. If the printed format is not familiar, it's because these are Google protobuf classes. Each `genai.protos.Tool` (1 in this case) contains a list of `genai.protos.FunctionDeclarations`, which describe a function and its arguments."
       ]
     },
     {
@@ -657,7 +645,7 @@
         "id": "eY6RmFQ76FVu"
       },
       "source": [
-        "Here is a declaration for the same multiply function written using the `glm` classes.\n",
+        "Here is a declaration for the same multiply function written using the `genai.protos` classes.\n",
         "\n",
         "Note that these classes just describe the function for the API, they don't include an implementation of it. So using this doesn't work with automatic function calling, but functions don't always need an implementation."
       ]
@@ -670,16 +658,16 @@
       },
       "outputs": [],
       "source": [
-        "calculator = glm.Tool(\n",
+        "calculator = genai.protos.Tool(\n",
         "    function_declarations=[\n",
-        "      glm.FunctionDeclaration(\n",
+        "      genai.protos.FunctionDeclaration(\n",
         "        name='multiply',\n",
         "        description=\"Returns the product of two numbers.\",\n",
-        "        parameters=glm.Schema(\n",
-        "            type=glm.Type.OBJECT,\n",
+        "        parameters=genai.protos.Schema(\n",
+        "            type=genai.protos.Type.OBJECT,\n",
         "            properties={\n",
-        "                'a':glm.Schema(type=glm.Type.NUMBER),\n",
-        "                'b':glm.Schema(type=glm.Type.NUMBER)\n",
+        "                'a':genai.protos.Schema(type=genai.protos.Type.NUMBER),\n",
+        "                'b':genai.protos.Schema(type=genai.protos.Type.NUMBER)\n",
         "            },\n",
         "            required=['a','b']\n",
         "        )\n",
@@ -753,7 +741,7 @@
         }
       ],
       "source": [
-        "glm.Tool(calculator)"
+        "genai.protos.Tool(calculator)"
       ]
     },
     {
@@ -762,7 +750,7 @@
         "id": "jS6ruiTp6VBf"
       },
       "source": [
-        "Either way, you pass a representation of a `glm.Tool` or list of tools to"
+        "Either way, you pass a representation of a `genai.protos.Tool` or list of tools to"
       ]
     },
     {
@@ -787,7 +775,7 @@
         "id": "517ca06297bb"
       },
       "source": [
-        "Like before the model returns a `glm.FunctionCall` invoking the calculator's `multiply` function:"
+        "Like before the model returns a `genai.protos.FunctionCall` invoking the calculator's `multiply` function:"
       ]
     },
     {
@@ -889,9 +877,9 @@
       "outputs": [],
       "source": [
         "response = chat.send_message(\n",
-        "    glm.Content(\n",
-        "    parts=[glm.Part(\n",
-        "        function_response = glm.FunctionResponse(\n",
+        "    genai.protos.Content(\n",
+        "    parts=[genai.protos.Part(\n",
+        "        function_response = genai.protos.FunctionResponse(\n",
         "          name='multiply',\n",
         "          response={'result': result}))]))"
       ]

site/en/gemini-api/docs/get-started/python.ipynb

@@ -419,7 +419,7 @@
       "source": [
         "Gemini can generate multiple possible responses for a single prompt. These possible responses are called `candidates`, and you can review them to select the most suitable one as the response.\n",
         "\n",
-        "View the response candidates with <a href=\"https://ai.google.dev/api/python/google/ai/generativelanguage/GenerateContentResponse#candidates\"><code>GenerateContentResponse.candidates</code></a>:"
+        "View the response candidates with <a href=\"https://ai.google.dev/api/python/google/generativeai/protos/GenerateContentResponse#candidates\"><code>GenerateContentResponse.candidates</code></a>:"
       ]
     },
     {
@@ -957,7 +957,7 @@
         "id": "AwCqtZ6D4kvk"
       },
       "source": [
-        "`glm.Content` objects contain a list of `glm.Part` objects that each contain either a text (string) or inline_data (`glm.Blob`), where a blob contains binary data and a `mime_type`. The chat history is available as a list of `glm.Content` objects in `ChatSession.history`:"
+        "`genai.protos.Content` objects contain a list of `genai.protos.Part` objects that each contain either a text (string) or inline_data (`genai.protos.Blob`), where a blob contains binary data and a `mime_type`. The chat history is available as a list of `genai.protos.Content` objects in `ChatSession.history`:"
       ]
     },
     {
@@ -1033,7 +1033,7 @@
       "source": [
         "## Count tokens\n",
         "\n",
-        "Large language models have a context window, and the context length is often measured in terms of the **number of tokens**. With the Gemini API, you can determine the number of tokens per any `glm.Content` object. In the simplest case, you can pass a query string to the `GenerativeModel.count_tokens` method as follows:"
+        "Large language models have a context window, and the context length is often measured in terms of the **number of tokens**. With the Gemini API, you can determine the number of tokens per any `genai.protos.Content` object. In the simplest case, you can pass a query string to the `GenerativeModel.count_tokens` method as follows:"
       ]
     },
     {
@@ -1188,9 +1188,9 @@
         "id": "zBg0eNeml3d4"
       },
       "source": [
-        "While the `genai.embed_content` function accepts simple strings or lists of strings, it is actually built around the `glm.Content` type (like <a href=\"https://ai.google.dev/api/python/google/generativeai/GenerativeModel#generate_content\"><code>GenerativeModel.generate_content</code></a>). `glm.Content` objects are the primary units of conversation in the API.\n",
+        "While the `genai.embed_content` function accepts simple strings or lists of strings, it is actually built around the `genai.protos.Content` type (like <a href=\"https://ai.google.dev/api/python/google/generativeai/GenerativeModel#generate_content\"><code>GenerativeModel.generate_content</code></a>). `genai.protos.Content` objects are the primary units of conversation in the API.\n",
         "\n",
-        "While the `glm.Content` object is multimodal, the `embed_content` method only supports text embeddings. This design gives the API the *possibility* to expand to multimodal embeddings."
+        "While the `genai.protos.Content` object is multimodal, the `embed_content` method only supports text embeddings. This design gives the API the *possibility* to expand to multimodal embeddings."
       ]
     },
     {
@@ -1248,7 +1248,7 @@
         "id": "jU8juHCxoUKG"
       },
       "source": [
-        "Similarly, the chat history contains a list of `glm.Content` objects, which you can pass directly to the `embed_content` function:"
+        "Similarly, the chat history contains a list of `genai.protos.Content` objects, which you can pass directly to the `embed_content` function:"
       ]
     },
     {
@@ -1488,18 +1488,7 @@
         "id": "-fthdIItnqki"
       },
       "source": [
-        "Underlying the Python SDK is the <a href=\"https://ai.google.dev/api/python/google/ai/generativelanguage\"><code>google.ai.generativelanguage</code></a> client library:"
-      ]
-    },
-    {
-      "cell_type": "code",
-      "execution_count": null,
-      "metadata": {
-        "id": "l6aafWECnpX6"
-      },
-      "outputs": [],
-      "source": [
-        "import google.ai.generativelanguage as glm"
+        "The [`google.generativeai.protos`](https://ai.google.dev/api/python/google/generativeai/protos) submodule provides access to the low level classes used by the API behind the scenes:"
       ]
     },
     {
@@ -1508,10 +1497,10 @@
         "id": "gm1RWcB3n_n0"
       },
       "source": [
-        "The SDK attempts to convert your message to a `glm.Content` object, which contains a list of `glm.Part` objects that each contain either:\n",
+        "The SDK attempts to convert your message to a `genai.protos.Content` object, which contains a list of `genai.protos.Part` objects that each contain either:\n",
         "\n",
         "1. a <a href=\"https://www.tensorflow.org/text/api_docs/python/text\"><code>text</code></a> (string)\n",
-        "2. `inline_data` (`glm.Blob`), where a blob contains binary `data` and a `mime_type`.\n",
+        "2. `inline_data` (`genai.protos.Blob`), where a blob contains binary `data` and a `mime_type`.\n",
         "\n",
         "You can also pass any of these classes as an equivalent dictionary.\n",
         "\n",
@@ -1530,11 +1519,11 @@
       "source": [
         "model = genai.GenerativeModel('gemini-1.5-flash')\n",
         "response = model.generate_content(\n",
-        "    glm.Content(\n",
+        "    genai.protos.Content(\n",
         "        parts = [\n",
-        "            glm.Part(text=\"Write a short, engaging blog post based on this picture.\"),\n",
-        "            glm.Part(\n",
-        "                inline_data=glm.Blob(\n",
+        "            genai.protos.Part(text=\"Write a short, engaging blog post based on this picture.\"),\n",
+        "            genai.protos.Part(\n",
+        "                inline_data=genai.protos.Blob(\n",
         "                    mime_type='image/jpeg',\n",
         "                    data=pathlib.Path('image.jpg').read_bytes()\n",
         "                )\n",
@@ -1581,9 +1570,9 @@
         "\n",
         "While the `genai.ChatSession` class shown earlier can handle many use cases, it does make some assumptions. If your use case doesn't fit into this chat implementation it's good to remember that `genai.ChatSession` is just a wrapper around <a href=\"https://ai.google.dev/api/python/google/generativeai/GenerativeModel#generate_content\"><code>GenerativeModel.generate_content</code></a>. In addition to single requests, it can handle multi-turn conversations.\n",
         "\n",
-        "The individual messages are `glm.Content` objects or compatible dictionaries, as seen in previous sections. As a dictionary, the message requires `role` and `parts` keys. The `role` in a conversation can either be the `user`, which provides the prompts, or `model`, which provides the responses.\n",
+        "The individual messages are `genai.protos.Content` objects or compatible dictionaries, as seen in previous sections. As a dictionary, the message requires `role` and `parts` keys. The `role` in a conversation can either be the `user`, which provides the prompts, or `model`, which provides the responses.\n",
         "\n",
-        "Pass a list of `glm.Content` objects and it will be treated as multi-turn chat:"
+        "Pass a list of `genai.protos.Content` objects and it will be treated as multi-turn chat:"
       ]
     },
     {