Need to configure image `detail` setting

[moritzwilksch]

Description

The detail attribute for images in the OpenAI API should be configurable. Some models (e.g. grok-4-fast through OpenRouter) need to be forced to 'detail': 'high' or will otherwise return unusable results due to downscaling with 'detail': 'auto' on the API provider's side.

detail currently defaults to 'auto' everywhere, either explicitly or implicitly:

pydantic-ai/pydantic_ai_slim/pydantic_ai/models/openai.py

Line 1390 in 79ef2bf

detail='auto',

pydantic-ai/pydantic_ai_slim/pydantic_ai/models/openai.py

Line 1410 in 79ef2bf

responses.ResponseInputImageParam(image_url=item.url, type='input_image', detail='auto')

pydantic-ai/pydantic_ai_slim/pydantic_ai/models/openai.py

Line 756 in 79ef2bf

image_url = ImageURL(url=item.url)

References

No response

[moritzwilksch]

For ImageUrl, we could add another attribute. I’m not sure how to support this in the more agnostic BinaryContent. Would vendor_metadata cover it, or should it live in OpenAI{Chat,Response}ModelSettings? The API lets you set detail per image, which you cannot do if we put it in a settings object. At the same time, this is provider specific, so not all models support it, which argues against putting it on ImageUrl.

EDIT: Opened a PR with extending the model settings, since it is OpenAI(compat)-specific.

[DouweM]

@moritzwilksch This sounds like a great use for vendor_metadata to me, as it's image specific. It's exactly for vendor-specific metadata, like VideoUrl.vendor_metadata mapping to Google's video_metadata, so we should support {ImageUrl,BinaryContent}.vendor_metadata['detail'].

[moritzwilksch]

Thanks, updated the PR