Unified Exception Handling for Various LLM Models

[timothy-jeong]

Question

Hi there,

First off, I'm currently developing an LLM feature using Pydantic AI, and I think it's a fantastic library. The integration with Logfire makes it even more powerful.

I chose Pydantic AI primarily for its low coupling with individual models and its faster adoption of the latest models compared to services like AWS Bedrock.

The feature I'm preparing to launch uses Gemini 2.0 Flash(and Flash-Lite). I've noticed that when model-specific issues occur, such as rate limits, exceptions specific to that model are raised (e.g., those found in google.genai.errors).

One of the key advantages of using Pydantic AI, in my view, is the ability to quickly swap and test various models, which significantly speeds up development and deployment. However, if I have to handle exceptions for each model individually, it seems to introduce complexity and could hinder development productivity. This feels counterintuitive to the library's goal of abstraction.

Am I missing something here? I looked through the official documentation on exceptions and it appears that there are unified exceptions available. Could you clarify if there is a recommended way to handle errors from different models in a standardized way through Pydantic AI?

Thanks!

Additional Context

Here's the stack trace from when the rate limit was exceeded. The exception class that's clearly visible here is ServerError defined in google.genai.errors, which is what led me to the conclusion I mentioned above.

Traceback (most recent call last):
  ... 
  File "/code/.venv/lib/python3.11/site-packages/pydantic_ai/agent.py", line 494, in run
    async for _ in agent_run:
  File "/code/.venv/lib/python3.11/site-packages/pydantic_ai/agent.py", line 1985, in __anext__
    next_node = await self._graph_run.__anext__()
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/code/.venv/lib/python3.11/site-packages/pydantic_graph/graph.py", line 809, in __anext__
    return await self.next(self._next_node)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/code/.venv/lib/python3.11/site-packages/pydantic_graph/graph.py", line 782, in next
    self._next_node = await node.run(ctx)
                      ^^^^^^^^^^^^^^^^^^^
  File "/code/.venv/lib/python3.11/site-packages/pydantic_ai/_agent_graph.py", line 331, in run
    return await self._make_request(ctx)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/code/.venv/lib/python3.11/site-packages/pydantic_ai/_agent_graph.py", line 391, in _make_request
    model_response = await ctx.deps.model.request(message_history, model_settings, model_request_parameters)
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/code/.venv/lib/python3.11/site-packages/pydantic_ai/models/instrumented.py", line 210, in request
    response = await super().request(messages, model_settings, model_request_parameters)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/code/.venv/lib/python3.11/site-packages/pydantic_ai/models/wrapper.py", line 30, in request
    return await self.wrapped.request(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/code/.venv/lib/python3.11/site-packages/pydantic_ai/models/google.py", line 189, in request
    response = await self._generate_content(messages, False, model_settings, model_request_parameters)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/code/.venv/lib/python3.11/site-packages/pydantic_ai/models/google.py", line 312, in _generate_content
    return await func(model=self._model_name, contents=contents, config=config)  # type: ignore
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/code/.venv/lib/python3.11/site-packages/google/genai/models.py", line 8191, in generate_content
    response = await self._generate_content(
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/code/.venv/lib/python3.11/site-packages/google/genai/models.py", line 7026, in _generate_content
    response = await self._api_client.async_request(
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/code/.venv/lib/python3.11/site-packages/google/genai/_api_client.py", line 1325, in async_request
    result = await self._async_request(
             ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/code/.venv/lib/python3.11/site-packages/google/genai/_api_client.py", line 1270, in _async_request
    return await self._async_retry(  # type: ignore[no-any-return]
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/code/.venv/lib/python3.11/site-packages/tenacity/asyncio/__init__.py", line 111, in __call__
    do = await self.iter(retry_state=retry_state)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/code/.venv/lib/python3.11/site-packages/tenacity/asyncio/__init__.py", line 153, in iter
    result = await action(retry_state)
             ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/code/.venv/lib/python3.11/site-packages/tenacity/_utils.py", line 99, in inner
    return call(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/code/.venv/lib/python3.11/site-packages/tenacity/__init__.py", line 420, in exc_check
    raise retry_exc.reraise()
          ^^^^^^^^^^^^^^^^^^^
  File "/code/.venv/lib/python3.11/site-packages/tenacity/__init__.py", line 187, in reraise
    raise self.last_attempt.result()
          ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/concurrent/futures/_base.py", line 449, in result
    return self.__get_result()
           ^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/concurrent/futures/_base.py", line 401, in __get_result
    raise self._exception
  File "/code/.venv/lib/python3.11/site-packages/tenacity/asyncio/__init__.py", line 114, in __call__
    result = await fn(*args, **kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/code/.venv/lib/python3.11/site-packages/google/genai/_api_client.py", line 1250, in _async_request_once
    await errors.APIError.raise_for_async_response(client_response)
  File "/code/.venv/lib/python3.11/site-packages/google/genai/errors.py", line 161, in raise_for_async_response
    raise ServerError(status_code, response_json, response)
google.genai.errors.ServerError: 503 UNAVAILABLE. {'error': {'code': 503, 'message': 'The service is currently unavailable.', 'status': 'UNAVAILABLE'}}