@@ -22,6 +22,7 @@ We support the following types of LLM APIs:
2222| [ OpenAI] ( #openai ) | ` LlmApiType.OPENAI ` | ✅ | ✅ |
2323| [ Ollama] ( #ollama ) | ` LlmApiType.OLLAMA ` | ✅ | ❌ |
2424| [ Google Gemini] ( #google-gemini ) | ` LlmApiType.GEMINI ` | ✅ | ✅ |
25+ | [ Vertex AI] ( #vertex-ai ) | ` LlmApiType.VERTEX_AI ` | ✅ | ✅ |
2526| [ Anthropic] ( #anthropic ) | ` LlmApiType.ANTHROPIC ` | ✅ | ❌ |
2627| [ Voyage] ( #voyage ) | ` LlmApiType.VOYAGE ` | ❌ | ✅ |
2728| [ LiteLLM] ( #litellm ) | ` LlmApiType.LITE_LLM ` | ✅ | ❌ |
@@ -47,6 +48,7 @@ It has the following fields:
4748 See supported LLM APIs in the [ LLM API integrations] ( #llm-api-integrations ) section below.
4849* ` model ` (type: ` str ` , required): The name of the LLM model to use.
4950* ` address ` (type: ` str ` , optional): The address of the LLM API.
51+ * ` api_config ` (optional): Specific configuration for the LLM API. Only needed for specific LLM APIs (see below).
5052
5153
5254### Text Embedding
@@ -137,7 +139,11 @@ cocoindex.LlmSpec(
137139
138140### Google Gemini
139141
140- To use the Gemini LLM API, you need to set the environment variable ` GEMINI_API_KEY ` .
142+ Google exposes Gemini through Google AI Studio APIs.
143+ Based on [ Gemini API recommendation] ( https://cloud.google.com/ai/gemini?hl=en ) , this is recommended for experimenting and prototyping purposes.
144+ You may use [ Vertex AI] ( #vertex-ai ) for production usages.
145+
146+ To use the Gemini by Google AI Studio API, you need to set the environment variable ` GEMINI_API_KEY ` .
141147You can generate the API key from [ Google AI Studio] ( https://aistudio.google.com/apikey ) .
142148
143149You can find the full list of models supported by Gemini [ here] ( https://ai.google.dev/gemini-api/docs/models ) .
@@ -150,7 +156,7 @@ For text generation, a spec looks like this:
150156``` python
151157cocoindex.LlmSpec(
152158 api_type = cocoindex.LlmApiType.GEMINI ,
153- model = " gemini-2.0 -flash" ,
159+ model = " gemini-2.5 -flash" ,
154160)
155161```
156162
@@ -170,13 +176,73 @@ cocoindex.functions.EmbedText(
170176)
171177```
172178
179+ </TabItem >
180+ </Tabs >
181+
173182All supported embedding models can be found [ here] ( https://ai.google.dev/gemini-api/docs/embeddings#embeddings-models ) .
174183Gemini supports task type (optional), which can be found [ here] ( https://ai.google.dev/gemini-api/docs/embeddings#supported-task-types ) .
175184
176185
186+ ### Vertex AI
187+
188+ Google Cloud Vertex AI offers production-level integration with Google Gemini models.
189+
190+ To use the Vertex AI API:
191+
192+ 1 . Register / login in * Google Cloud* .
193+ 2 . In [ Google Cloud Console] ( https://console.cloud.google.com/ ) .
194+ - Search for * Vertex AI API* . Enable this API.
195+ - Search for * Billing* . Set the billing account for the current project.
196+ 3 . Setup [ application default credentials (ADC)] ( https://cloud.google.com/docs/authentication/application-default-credentials ) .
197+
198+ The easiest way during development is to [ install the ` gcloud ` CLI] ( https://cloud.google.com/sdk/docs/install-sdk ) and run
199+
200+ ``` sh
201+ gcloud auth application-default login
202+ ```
203+
204+ Spec for Vertex AI takes additional ` api_config ` field, in type ` cocoindex.llm.VertexAiConfig ` with the following fields:
205+ - ` project ` (type: ` str ` , required): The project ID of the Google Cloud project.
206+ - ` region ` (type: ` str ` , optional): The region of the Google Cloud project. Use ` global ` if not specified.
207+
208+ You can find the full list of models supported by Vertex AI [ here] ( https://cloud.google.com/vertex-ai/generative-ai/docs/learn/model-versions ) .
209+
210+ For text generation, a spec for Vertex AI looks like this:
211+
212+ <Tabs >
213+ <TabItem value = " python" label = " Python" default >
214+
215+ ``` python
216+ cocoindex.LlmSpec(
217+ api_type = cocoindex.LlmApiType.VERTEX_AI ,
218+ model = " gemini-2.0-flash" ,
219+ api_config = cocoindex.llm.VertexAiConfig(project = " your-project-id" ),
220+ )
221+ ```
222+
177223</TabItem >
178224</Tabs >
179225
226+
227+ For text embedding, a spec for Vertex AI looks like this:
228+
229+ <Tabs >
230+ <TabItem value = " python" label = " Python" default >
231+
232+ ``` python
233+ cocoindex.functions.EmbedText(
234+ api_type = cocoindex.LlmApiType.VERTEX_AI ,
235+ model = " text-embedding-005" ,
236+ task_type = " SEMANTICS_SIMILARITY" ,
237+ api_config = cocoindex.llm.VertexAiConfig(project = " your-project-id" ),
238+ )
239+ ```
240+
241+ </TabItem >
242+ </Tabs >
243+
244+ Vertex AI API supports task type (optional), which can be found [ here] ( https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/text-embeddings-api#parameter-list ) .
245+
180246### Anthropic
181247
182248To use the Anthropic LLM API, you need to set the environment variable ` ANTHROPIC_API_KEY ` .
0 commit comments