Pydantic plugin records errors in validated functions as a validation error

[Kaelten]

Description

When using logfire's pydantic auto instrumentation errors and the @validate_call decorator unexpected logging takes place.

Example:

@validate_call
def my_test_function(value: int) -> None:
    raise RuntimeError

my_test_function(1)

Results in the log message of:

Validation on call raised RuntimeError

Expectations

I would expect the above snippet to not log a validation error at all as the arguments did actually validate successfully.

Workaround

Talking in slack I was given this work around:

@validate_call(config=ConfigDict(plugin_settings={'logfire': {'record': 'off'}}))

I believe this work around has a few failure modes:

1. A bit verbose to have to be setting manually in every instance we'd need to use it to avoid excessive logging
2. Very easy to forget to add in proactively
3. Would also disable logging of -actual- validation errors unless they were captured elsewhere.

Assertion

I believe it's not intuitive for this to be captured as a validation error.

Digging into the implementation I found similar blocks to this:

                span = self._on_enter(input_data)
                try:
                    result = validator(input_data, *args, **kwargs)
                except ValidationError as error:
                    self._on_error_span(span, error)
                    self._count_validation(success=False)
                    raise
                except Exception as exception:
                    self._on_exception_span(span, exception)
                    self._count_validation(success=False)
                    raise
                else:
                    self._on_success(span, result)
                    self._count_validation(success=True)
                    return result

When combined that the validator() call, in this case, implicitly calls the wrapped function it's understandable how we got here.

I'd suggest either in general or when the schema is "on call" the blanket exception logging is skipped.

Python, Logfire & OS Versions, related packages (not required)

logfire="4.14.2"
platform="macOS-26.0.1-arm64-arm-64bit"
python="3.11.8 (main, Feb 20 2024, 15:59:26) [Clang 15.0.0 (clang-1500.1.0.2.5)]"
[related_packages]
requests="2.32.5"
pydantic="2.12.3"
fastapi="0.120.0"
protobuf="5.29.5"
rich="14.2.0"
executing="2.2.1"
opentelemetry-api="1.38.0"
opentelemetry-exporter-otlp-proto-common="1.38.0"
opentelemetry-exporter-otlp-proto-http="1.38.0"
opentelemetry-instrumentation="0.59b0"
opentelemetry-instrumentation-httpx="0.59b0"
opentelemetry-instrumentation-system-metrics="0.59b0"
opentelemetry-proto="1.38.0"
opentelemetry-sdk="1.38.0"
opentelemetry-semantic-conventions="0.59b0"
opentelemetry-util-http="0.59b0"

[Viicos]

I'd suggest either in general or when the schema is "on call" the blanket exception logging is skipped.

The core "issue" is that currently, @validate_call is responsible for calling the decorated function once the arguments have been validated. This all happens in the inner __pydantic_validator__.validate_python() call. Skipping the Exception handling as suggested would drop support for cases where exceptions other than ValidationError are actually raised during validation (what this mechanism is actually supposed to catch):

def raise(v):
    raise RuntimeError

@validate_call
def func(arg: Annotated[int, AfterValidator(raise)]): ...

The clean way to fix this would be for @validate_call to first validate the arguments and get back a two-tuple of (validated_args, validated_kwargs), and after pass those to the wrapped function.

[Kaelten]

That completely makes sense as an even better fix. As long as non-argument-validation errors aren't reported my concern is handled, and doing it in a way that preserves were we'd also get all versions of argument validation issues would be ideal.