Skip to content

Fix: Emit tool_called events immediately in streaming runs #1300

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 6 commits into
base: main
Choose a base branch
from

Conversation

habema
Copy link
Contributor

@habema habema commented Jul 29, 2025

Summary

Resolves #1282

The Solution

The implementation has been updated to emit the semantic tool_called event as soon as the first streaming chunk indicating a tool call is received from the model.

  1. The _run_single_turn_streamed method now inspects raw ResponseOutputItemAddedEvent events as they arrive. If an event's item is a tool call, a corresponding RunItemStreamEvent(name="tool_called") is immediately constructed and put on the queue.

  2. To avoid sending a duplicate event when the full turn results are processed at the end of the stream, a set of emitted_tool_call_ids is maintained for the duration of the turn.

  3. The check for whether an event item is a tool call has been refactored. Instead of using a long, hardcoded tuple of types, it now dynamically creates this tuple from the ToolCallItemTypes alias.

@seratch seratch added bug Something isn't working feature:core labels Jul 29, 2025
@seratch seratch requested review from seratch and rm-openai July 29, 2025 23:31
Copy link
Member

@seratch seratch left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for sending this. It seems this PR modifies the lines of code also changed by #974, which already looks good to me. @rm-openai can you take a look at #974 first and then can you share what you think about this PR?

@habema
Copy link
Contributor Author

habema commented Jul 30, 2025

@seratch My apologies for the duplicate work.

After looking at both, I think this approach might have a couple of advantages:

  • Instead of waiting for the model's full response to finish, this implementation emits the tool_called event the instant the first tool-call chunk arrives in the stream.
  • From a maintainability perspective, it also dynamically checks for any tool-call type, so the code won't need to be updated when more tools are added down the line.

This should make the UI feel more responsive, and the code change itself is a bit smaller and more contained. Happy to chat about it or defer to the other solution if that's preferred.

Copy link
Contributor

This PR is stale because it has been open for 10 days with no activity.

@github-actions github-actions bot added the stale label Aug 10, 2025
@habema
Copy link
Contributor Author

habema commented Aug 11, 2025

Hey @seratch @rm-openai.
Merging #974 resolves this problem, but I still see some advantages/improvements in this implementation. Would it be possible to take a look at it, and either go forward with it or close it.

@seratch
Copy link
Member

seratch commented Aug 11, 2025

While I am confident #974 works well, I am not sure if your code here works without any issues. Would it be possible to enhance the current code with #974 to produce "tool_called" events as you intend here?

@habema
Copy link
Contributor Author

habema commented Aug 11, 2025

It doesnt. There are merge conflicts that were caused by merging #974. I'll fix them briefly.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working feature:core stale
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Bug: run_streamed Buffers tool_call_item Events, Delaying Real-Time Feedback
2 participants