Vertex Model garden models

[pedroallenrevez]

Is it possible to use other models in the vertex model garden, or is it just gemini models through vertex?

[Kludex]

Do you have a reference to which models are you referring to?

[pedroallenrevez]

LIke mistral and anthropic models through vertex

[pedroallenrevez]

Ok @Kludex, basically you can use the already existing models, but with the vertex counterpart clients of the mistral and anthropic packages.

Feels really silly that there is no docs for this.
There are typing errors, but (mostly) it works.

            if "mistral" in model.identity.id:
                import sys
                from mistralai_gcp.models.usermessage import UserMessage as MistralUserMessage
                sys.modules['pydantic_ai.models.mistral'].MistralUserMessage = MistralUserMessage

                from mistralai_gcp import MistralGoogleCloud
                self._pydantic_model = MistralModel(
                    model.extra['pydantic_ai_alias'],
                    client=MistralGoogleCloud(
                        project_id=provider.project_id, 
                        region=provider.locations[0]
                    ),
                )
            elif "claude" in model.identity.id:
                from anthropic import AsyncAnthropicVertex
                self._pydantic_model = AnthropicModel(
                    model.extra["pydantic_ai_alias"],
                    anthropic_client=AsyncAnthropicVertex(
                        project_id=provider.project_id,
                        region=provider.locations[0]
                    )
                )

[ehaca]

Hey I came looking for this one - is this something one can contribute to? Would be thrilled to help out if wanted/needed/possible :)

[ehaca]

Hey, I had some ideas and couldn't help starting drafting: #1392 🙈

[Kludex]

@ehaca The point here is that we need to use the AnthropicModel instead of the GeminiModel because the API followed is the one in AnthropicModel, right?

[ehaca]

You can use Anthropic's models via Anthropic itself or via GCP/AWS. For the this, the SDK offers dedicated clients that handle auth etc. currently you can plug those clients into anthropic_client and it works, but yields typing errors. I tried just extending the existing Anthropic Provider, but since Anthropic's SDK has dedicated extras for the different clients it was kind of an import mess (?).

[avivex1000]

You can use Anthropic's models via Anthropic itself or via GCP/AWS. For the this, the SDK offers dedicated clients that handle auth etc. currently you can plug those clients into anthropic_client and it works, but yields typing errors. I tried just extending the existing Anthropic Provider, but since Anthropic's SDK has dedicated extras for the different clients it was kind of an import mess (?).

This comment saved me a lot of time, thanks 🙏🏻
Will be happy to see these supported properly in the sdk

[stefannae]

Hi! Below is a snippet that works for me with other models. Tried it using both pydantic-ai 1.0.2 and 0.4.7.

from google.oauth2 import service_account
from google import genai
from pydantic_ai import Agent
from pydantic_ai.models.google import GoogleModel
from pydantic_ai.providers.google import GoogleProvider

service_account_file = ''  # Path to your service account key file
project = ''  # GCP project ID
location = ''  # e.g. 'us-central1'

client = genai.Client(
    vertexai=True,
    location='global',
    project=project,
    credentials=service_account.Credentials.from_service_account_file(
        service_account_file,
        scopes=['https://www.googleapis.com/auth/cloud-platform']
    )
)

provider = GoogleProvider(
    client=client
)

model = GoogleModel(
    # 'gemini-2.5-flash-lite',
    # f'projects/{project}/locations/{location}/publishers/google/models/gemini-2.5-flash-lite',
    f'projects/{project}/locations/{location}/publishers/meta/models/llama-3.3-70b-instruct-maas',
    provider=provider
)

agent = Agent(model)

prompt = "Hello LLM world!"

print(agent.run_sync(prompt))

[DouweM]

@stefannae Great! Would you mind adding that to the docs in a PR?

[stefannae]

Sure @DouweM, I'll add it tomorrow.