Skip to content

Commit 86ab045

Browse files
committed
docs(vertex): add docs for Vertex AI.
1 parent 60aa8f4 commit 86ab045

File tree

1 file changed

+68
-2
lines changed

1 file changed

+68
-2
lines changed

docs/docs/ai/llm.mdx

Lines changed: 68 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -22,6 +22,7 @@ We support the following types of LLM APIs:
2222
| [OpenAI](#openai) | `LlmApiType.OPENAI` |||
2323
| [Ollama](#ollama) | `LlmApiType.OLLAMA` |||
2424
| [Google Gemini](#google-gemini) | `LlmApiType.GEMINI` |||
25+
| [Vertex AI](#vertex-ai) | `LlmApiType.VERTEX_AI` |||
2526
| [Anthropic](#anthropic) | `LlmApiType.ANTHROPIC` |||
2627
| [Voyage](#voyage) | `LlmApiType.VOYAGE` |||
2728
| [LiteLLM](#litellm) | `LlmApiType.LITE_LLM` |||
@@ -47,6 +48,7 @@ It has the following fields:
4748
See supported LLM APIs in the [LLM API integrations](#llm-api-integrations) section below.
4849
* `model` (type: `str`, required): The name of the LLM model to use.
4950
* `address` (type: `str`, optional): The address of the LLM API.
51+
* `api_config` (optional): Specific configuration for the LLM API. Only needed for specific LLM APIs (see below).
5052

5153

5254
### Text Embedding
@@ -137,7 +139,11 @@ cocoindex.LlmSpec(
137139

138140
### Google Gemini
139141

140-
To use the Gemini LLM API, you need to set the environment variable `GEMINI_API_KEY`.
142+
Google exposes Gemini through Google AI Studio APIs.
143+
Based on [Gemini API recommendation](https://cloud.google.com/ai/gemini?hl=en), this is recommended for experimenting and prototyping purposes.
144+
You may use [Vertex AI](#vertex-ai) for production usages.
145+
146+
To use the Gemini by Google AI Studio API, you need to set the environment variable `GEMINI_API_KEY`.
141147
You can generate the API key from [Google AI Studio](https://aistudio.google.com/apikey).
142148

143149
You can find the full list of models supported by Gemini [here](https://ai.google.dev/gemini-api/docs/models).
@@ -150,7 +156,7 @@ For text generation, a spec looks like this:
150156
```python
151157
cocoindex.LlmSpec(
152158
api_type=cocoindex.LlmApiType.GEMINI,
153-
model="gemini-2.0-flash",
159+
model="gemini-2.5-flash",
154160
)
155161
```
156162

@@ -170,13 +176,73 @@ cocoindex.functions.EmbedText(
170176
)
171177
```
172178

179+
</TabItem>
180+
</Tabs>
181+
173182
All supported embedding models can be found [here](https://ai.google.dev/gemini-api/docs/embeddings#embeddings-models).
174183
Gemini supports task type (optional), which can be found [here](https://ai.google.dev/gemini-api/docs/embeddings#supported-task-types).
175184

176185

186+
### Vertex AI
187+
188+
Google Cloud Vertex AI offers production-level integration with Google Gemini models.
189+
190+
To use the Vertex AI API:
191+
192+
1. Register / login in *Google Cloud*.
193+
2. In [Google Cloud Console](https://console.cloud.google.com/).
194+
- Search for *Vertex AI API*. Enable this API.
195+
- Search for *Billing*. Set the billing account for the current project.
196+
3. Setup [application default credentials (ADC)](https://cloud.google.com/docs/authentication/application-default-credentials).
197+
198+
The easiest way during development is to [install the `gcloud` CLI](https://cloud.google.com/sdk/docs/install-sdk) and run
199+
200+
```sh
201+
gcloud auth application-default login
202+
```
203+
204+
Spec for Vertex AI takes additional `api_config` field, in type `cocoindex.llm.VertexAiConfig` with the following fields:
205+
- `project` (type: `str`, required): The project ID of the Google Cloud project.
206+
- `region` (type: `str`, optional): The region of the Google Cloud project. Use `global` if not specified.
207+
208+
You can find the full list of models supported by Vertex AI [here](https://cloud.google.com/vertex-ai/generative-ai/docs/learn/model-versions).
209+
210+
For text generation, a spec for Vertex AI looks like this:
211+
212+
<Tabs>
213+
<TabItem value="python" label="Python" default>
214+
215+
```python
216+
cocoindex.LlmSpec(
217+
api_type=cocoindex.LlmApiType.VERTEX_AI,
218+
model="gemini-2.0-flash",
219+
api_config=cocoindex.llm.VertexAiConfig(project="your-project-id"),
220+
)
221+
```
222+
177223
</TabItem>
178224
</Tabs>
179225

226+
227+
For text embedding, a spec for Vertex AI looks like this:
228+
229+
<Tabs>
230+
<TabItem value="python" label="Python" default>
231+
232+
```python
233+
cocoindex.functions.EmbedText(
234+
api_type=cocoindex.LlmApiType.VERTEX_AI,
235+
model="text-embedding-005",
236+
task_type="SEMANTICS_SIMILARITY",
237+
api_config=cocoindex.llm.VertexAiConfig(project="your-project-id"),
238+
)
239+
```
240+
241+
</TabItem>
242+
</Tabs>
243+
244+
Vertex AI API supports task type (optional), which can be found [here](https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/text-embeddings-api#parameter-list).
245+
180246
### Anthropic
181247

182248
To use the Anthropic LLM API, you need to set the environment variable `ANTHROPIC_API_KEY`.

0 commit comments

Comments
 (0)