Google Service Account authentication still tries to use Application Default Credentials

[Blue9]

Initial Checks

• I confirm that I'm using the latest version of Pydantic AI
• I confirm that I searched for my issue in https://github.com/pydantic/pydantic-ai/issues before opening this issue

Description

The snippet showing how to use a service account JSON file to provide credentials to a Google model still attempts to use Application Default Credentials.

From https://ai.pydantic.dev/models/google/#service-account:

from google.oauth2 import service_account

from pydantic_ai import Agent
from pydantic_ai.models.google import GoogleModel
from pydantic_ai.providers.google import GoogleProvider

credentials = service_account.Credentials.from_service_account_file(
    'path/to/service-account.json',
    scopes=['https://www.googleapis.com/auth/cloud-platform'],
)
provider = GoogleProvider(credentials=credentials)
model = GoogleModel('gemini-1.5-flash', provider=provider)
agent = Agent(model)

This results in the following message:

google.auth.exceptions.DefaultCredentialsError: Your default credentials were not found. To set up Application Default Credentials, see https://cloud.google.com/docs/authentication/external/set-up-adc for more information.

The reason is that we are hitting this line in google-genai which attempts to fall back to default credentials if the project is not provided:

...
if (
    not self.project
    and not self.api_key
    and not validated_http_options.base_url
):
credentials, self.project = load_auth(project=None)  # <- reads default credentials
if not self._credentials:
    self._credentials = credentials
...

This also might explain why the vertexai test is failing.

Fix

The simplest fix is to pass in the project ID when instantiating the GoogleProvider:

GoogleProvider(credentials=credentials, project="<project-id>")

This aligns with Google's examples as well, where the project and location are passed in together (pydantic-ai passes in a default location here so it's not required).

The project ID can be parsed out of the service account JSON file (under the project_id key) so pydantic-ai could handle this under the hood although it's probably easiest to just require the project ID to also be passed in.

Another workaround is to point the GOOGLE_APPLICATION_CREDENTIALS env var to the service account file which works but creates a convoluted flow where the code appears to go through the service account flow but really falls back to the default credential flow which then reads the service account file.

Proposed Changes

I'd be happy to open a PR to make the following changes:

1. Update the docs to include the project argument.
2. Update the vertexai test to parse out project_id from service_account_content and pass it into the provider.
3. Enable the vertexai test.

Python, Pydantic AI & LLM client version

pydantic-ai 0.8.1
google-genai 1.32.0
Python 3.12.11

[DouweM]

@Blue9 Thanks for the report, a PR would be much appreciated! I prefer requiring the project argument over trying to parse it out of the file.

[Blue9]

PR is here @DouweM: #2755

I'm parsing out the project ID in the vertexai test to remain project-agnostic and to avoid hardcoding but if you prefer to pass it in explicitly there let me know.