Skip to content

Commit 4eb553b

Browse files
authored
fix: let pydantic ValidationError bubble up unchanged (#800)
## Description When using `response_format` with a Pydantic model, the clients validate the response internally. If the LLM produces output that doesn't conform to the schema, it raises a `pydantic.ValidationError`. This was incorrectly being converted to `InvalidRequestError` because "validation" matched a regex in the exception translation layer. Now `ValidationError` bubbles up unchanged. ## PR Type - 🐛 Bug Fix ## Relevant issues Fixes #799 ## Note All existing exception handler tests were testing provider specific handling, this is the first "provider"-agnostic exception handling quirk so I created a dedicated testing module just for this, open to other testing strategies for this. ## Checklist - [x] I understand the code I am submitting. - [x] I have added unit tests that prove my fix/feature works - [x] I have run this code locally and verified it fixes the issue. - [x] New and existing tests pass locally - [x] Documentation was updated where necessary - [x] I have read and followed the [contribution guidelines](https://github.com/mozilla-ai/any-llm/blob/main/CONTRIBUTING.md) - **AI Usage:** - [ ] No AI was used. - [x] AI was used for drafting/refactoring. - [ ] This is fully AI-generated.
1 parent 788eea6 commit 4eb553b

File tree

2 files changed

+37
-0
lines changed

2 files changed

+37
-0
lines changed

src/any_llm/utils/exception_handler.py

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -6,6 +6,8 @@
66
import warnings
77
from typing import TYPE_CHECKING, Any, TypeVar
88

9+
from pydantic import ValidationError
10+
911
from any_llm.exceptions import (
1012
AnyLLMError,
1113
AuthenticationError,
@@ -133,8 +135,16 @@ def _handle_exception(exception: Exception, provider_name: str) -> None:
133135
Raises:
134136
AnyLLMError: If unified exceptions are enabled
135137
Exception: The original exception if unified exceptions are disabled
138+
pydantic.ValidationError: Always re-raised unchanged
136139
137140
"""
141+
# When using response_format with a Pydantic model, the SDK validates the response
142+
# internally. If the LLM produces output that doesn't conform to the schema, pydantic
143+
# raises ValidationError. This should bubble up unchanged.
144+
# See: https://github.com/mozilla-ai/any-llm/issues/799
145+
if isinstance(exception, ValidationError):
146+
raise exception
147+
138148
if os.environ.get(ANY_LLM_UNIFIED_EXCEPTIONS_ENV, "").lower() in ("1", "true", "yes", "on"):
139149
converted = convert_exception(exception, provider_name)
140150
raise converted from exception
Lines changed: 27 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,27 @@
1+
import pytest
2+
from pydantic import BaseModel, ValidationError
3+
4+
from any_llm.utils.exception_handler import _handle_exception
5+
6+
7+
def test_validation_error_bubbles_up_unchanged() -> None:
8+
"""Test that pydantic ValidationError bubbles up unchanged.
9+
10+
When using response_format with a Pydantic model, the SDK validates the response
11+
internally. If the LLM produces output that doesn't conform to the schema, pydantic
12+
raises ValidationError. This should bubble up unchanged.
13+
See: https://github.com/mozilla-ai/any-llm/issues/799
14+
"""
15+
16+
class Sample(BaseModel):
17+
value: str
18+
19+
with pytest.raises(ValidationError) as exc_info:
20+
Sample.model_validate({"value": 123})
21+
22+
original = exc_info.value
23+
24+
with pytest.raises(ValidationError) as raised:
25+
_handle_exception(original, "openai")
26+
27+
assert raised.value is original

0 commit comments

Comments
 (0)