Skip to content

Support for Images in Function Call Outputs #1850

@ihsaanpatel-glean

Description

@ihsaanpatel-glean

Please read this first

  • Have you read the docs?Agents SDK docs
  • Have you searched for related issues? Others may have faced similar issues.

Describe the bug

The openai responses api supports entities like images in function call outputs however the current agents sdk does not, instead it casts all outputs to a string here. The openai python sdk does support the updated output type in the FunctionCallOutput object, however it would need to be upgraded a major version to 2+. Once that upgrade is done this looks like a two line change.

Debug information

  • Agents SDK version: (e.g. v0.0.3) - latest version or any other version
  • Python version (e.g. Python 3.10) - any python version

Repro steps

Have a tool output a list of messages with an image e.g.

[{
  'type': 'input_image',
  'image_url': 'data:image/png;base64,...',
}]

Expected behavior

Do not cast the above into a str, just let it flow through

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions