feat: Add Chat Generator supporting OpenAI Responses API #9808

Amnah199 · 2025-09-22T15:53:56Z

Related Issues

fixes Add support for OpenAI's Responses API #9422

Proposed Changes:

Add a chat generator that uses openai's responses endpoint.
The tools param in init and run methods allows openai and MCP tools besides haystack Tools.
Structured outputs can be enabled by passing text_format in generation_kwargs.
Reasoning can be enabled by setting reasoning params in generation_kwargs.
There are other features in Responses API like background requests etc, which are not supported in this iteration.

How did you test it?

Add new unit tests to test features.
Add new integration tests.

Notes for the reviewer

Using OpenAI tools and MCP tools doesn’t behave like standard function calls.

Tool calls are usually identified by type="function_call". However, for OpenAI/MCP tools, the type value varies depending on the tool being used — for example, type="web_search_call".
Currently, we only return standard ResponseFunctionToolCall objects. As a result, tool calls from OpenAI/MCP tools won’t appear in the ChatMessage, so the user won’t see them.
This is fine, since these tool calls are handled internally by OpenAI to generate the final response. The user already receives the output.

Checklist

I have read the contributors guidelines and the code of conduct
I have updated the related issue with new insights and changes
I added unit tests and updated the docstrings
I've used one of the conventional commit types for my PR title: fix:, feat:, build:, chore:, ci:, docs:, style:, refactor:, perf:, test: and added ! in case the PR includes breaking changes.
I documented my code
I ran pre-commit hooks and fixed any issue

coveralls · 2025-09-22T15:58:14Z

Pull Request Test Coverage Report for Build 18779147146

Warning: This coverage report may be inaccurate.

This pull request's base commit is no longer the HEAD commit of its target branch. This means it includes changes from outside the original pull request, including, potentially, unrelated coverage changes.

For more information on this, see Tracking coverage changes with pull request builds.
To avoid this issue with future PRs, see these Recommended CI Configurations.
For a quick fix, rebase this PR at GitHub. Your next report should be accurate.

Details

0 of 0 changed or added relevant lines in 0 files are covered.
1 unchanged line in 1 file lost coverage.
Overall coverage decreased (-0.9%) to 91.313%

Files with Coverage Reduction	New Missed Lines	%
components/builders/answer_builder.py	1	98.57%

Totals
Change from base Build 18690275685:	-0.9%
Covered Lines:	13581
Relevant Lines:	14873

💛 - Coveralls

…enai-responses

…tack into openai-responses

haystack/dataclasses/chat_message.py

releasenotes/notes/add-openai-responses-chatgenerator-52ca7457a4e61db1.yaml

haystack/components/generators/chat/openai_responses.py

Amnah199 · 2025-10-14T16:21:16Z

haystack/components/generators/chat/openai_responses.py

+        :param tools_strict:
+            Whether to enable strict schema adherence for tool calls. If set to `False`, the model may not exactly
+            follow the schema provided in the `parameters` field of the tool definition. In Response API, tool calls
+            are strict by default.


In Responses, functions are strict by default.
https://platform.openai.com/docs/guides/migrate-to-responses#5-update-function-definitions

haystack/components/generators/chat/openai_responses.py

…enai-responses

vercel · 2025-10-22T11:55:24Z

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment

Project	Deployment	Preview	Comments	Updated (UTC)
haystack-docs	Ignored	Preview		Oct 24, 2025 0:03am

haystack/components/generators/chat/openai_responses.py

vblagoje

@Amnah199 left some initial comments; didn't go into details yet

releasenotes/notes/add-openai-responses-chatgenerator-52ca7457a4e61db1.yaml

vblagoje · 2025-10-24T09:54:25Z

releasenotes/notes/add-openai-responses-chatgenerator-52ca7457a4e61db1.yaml

+    Added the OpenAIResponsesChatGenerator, a new component that integrates OpenAI's Responses API into Haystack.
+    This unlocks several advanced capabilities from the Responses API:
+    - Allowing retrieval of concise summaries of the model's reasoning process.
+    - Allowing the use of native OpenAI or MCP tool formats, along with Haystack Tool objects and Toolset instances.


What else do we support and integrate well from our ecosystem into Responses?

Structured outputs could be one.

vblagoje · 2025-10-24T09:57:52Z

haystack/components/generators/chat/openai_responses.py

+    Completes chats using OpenAI's Responses API.
+
+    It works with the gpt-4 and o-series models and supports streaming responses
+    from OpenAI API. It uses [ChatMessage](https://docs.haystack.deepset.ai/docs/chatmessage)


I'd mention here, including the o3-deep-research,
o4-mini-deep-research, and gpt-5-codex among others. People don't know this in general that you can use these specialized model/agents with this CG. Or do we don't want to mention that at this time, because they need to run usually in background etc

haystack/components/generators/chat/openai_responses.py

vblagoje · 2025-10-24T10:03:53Z

haystack/components/generators/chat/openai_responses.py

+    OpenAI API. Use the `**generation_kwargs` argument when you initialize
+    the component or when you run it. Any parameter that works with
+    `openai.Responses.create` will work here too.
+


Should we mention somewhere here what we don't support i.e background tasks etc

haystack/components/generators/chat/openai_responses.py

Add working ChatGenerator

a0ccca9

github-actions bot added the topic:build/distribution label Sep 22, 2025

rename

8363ae6

github-actions bot added the type:documentation Improvements on the docs label Sep 22, 2025

Amnah199 added 2 commits September 24, 2025 16:20

Merge branch 'main' of https://github.com/deepset-ai/haystack into op…

eba8123

…enai-responses

Improve and add live tests

18ce0e0

github-actions bot added the topic:tests label Sep 24, 2025

Amnah199 added 9 commits October 8, 2025 10:42

Updates

ba14b18

Update the tests

eeec152

Fix errors

64052f7

Add release notes

ef4d0a7

Merge branch 'main' into openai-responses

149a47e

Merge branch 'main' into openai-responses

f3b41f2

Add support for openai tools

50d5feb

Merge branch 'openai-responses' of https://github.com/deepset-ai/hays…

d1bea24

…tack into openai-responses

Remove openai tools test that times out

f2ba387

Amnah199 requested a review from vblagoje October 13, 2025 07:57

fix tool calls

830d086

Amnah199 marked this pull request as ready for review October 13, 2025 10:15

Amnah199 requested a review from a team as a code owner October 13, 2025 10:15

Update release notes

2c866f9

sjrl reviewed Oct 14, 2025

View reviewed changes

haystack/dataclasses/chat_message.py Outdated Show resolved Hide resolved

sjrl reviewed Oct 14, 2025

View reviewed changes

releasenotes/notes/add-openai-responses-chatgenerator-52ca7457a4e61db1.yaml Show resolved Hide resolved

sjrl reviewed Oct 14, 2025

View reviewed changes

haystack/components/generators/chat/openai_responses.py Outdated Show resolved Hide resolved

sjrl reviewed Oct 14, 2025

View reviewed changes

haystack/components/generators/chat/openai_responses.py Outdated Show resolved Hide resolved

sjrl reviewed Oct 14, 2025

View reviewed changes

haystack/components/generators/chat/openai_responses.py Outdated Show resolved Hide resolved

sjrl reviewed Oct 14, 2025

View reviewed changes

haystack/components/generators/chat/openai_responses.py Show resolved Hide resolved

sjrl reviewed Oct 14, 2025

View reviewed changes

haystack/components/generators/chat/openai_responses.py Outdated Show resolved Hide resolved

sjrl reviewed Oct 14, 2025

View reviewed changes

haystack/components/generators/chat/openai_responses.py Outdated Show resolved Hide resolved

Amnah199 marked this pull request as draft October 14, 2025 16:15

Amnah199 added 2 commits October 14, 2025 18:18

PR comments

228a21b

remove edits to chat message

1b0ac65

Amnah199 commented Oct 14, 2025

View reviewed changes

Amnah199 added 3 commits October 14, 2025 18:23

Add a test

96e9343

PR comments

515474a

Send back reasoning to model

9d8aa42

Amnah199 commented Oct 15, 2025

View reviewed changes

haystack/components/generators/chat/openai_responses.py Show resolved Hide resolved

Amnah199 added 3 commits October 21, 2025 12:45

Merge branch 'main' of https://github.com/deepset-ai/haystack into op…

b1d6e80

…enai-responses

Merge branch 'main' of https://github.com/deepset-ai/haystack into op…

9e414a9

…enai-responses

Fix reasoning support

00e6013

Amnah199 added 2 commits October 22, 2025 20:00

Add reasoning support

419ec36

Fix tests

2a7f342

Amnah199 commented Oct 23, 2025

View reviewed changes

haystack/components/generators/chat/openai_responses.py Outdated Show resolved Hide resolved

Amnah199 added 3 commits October 23, 2025 18:45

Refactor

76db039

Simplify methods

0bab968

Fix mypy

8c8e031

Amnah199 marked this pull request as ready for review October 24, 2025 09:10

vblagoje reviewed Oct 24, 2025

View reviewed changes

Amnah199 added 2 commits October 24, 2025 13:45

Stream responses, tool calls etc

9107989

Update docstrings

fe07300

Uh oh!

feat: Add Chat Generator supporting OpenAI Responses API #9808

Are you sure you want to change the base?

feat: Add Chat Generator supporting OpenAI Responses API #9808

Uh oh!

Conversation

Amnah199 commented Sep 22, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Related Issues

Proposed Changes:

How did you test it?

Notes for the reviewer

Checklist

Uh oh!

coveralls commented Sep 22, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Pull Request Test Coverage Report for Build 18779147146

Warning: This coverage report may be inaccurate.

Details

💛 - Coveralls

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Amnah199 Oct 14, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

vercel bot commented Oct 22, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

vblagoje left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

vblagoje Oct 24, 2025

Choose a reason for hiding this comment

Uh oh!

Amnah199 Oct 24, 2025

Choose a reason for hiding this comment

Uh oh!

vblagoje Oct 24, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

vblagoje Oct 24, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Amnah199 commented Sep 22, 2025 •

edited

Loading

coveralls commented Sep 22, 2025 •

edited

Loading

vercel bot commented Oct 22, 2025 •

edited

Loading