feat: Implement async generator tools #2
Closed
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
* fix: fix loading tools with same tool name * simplify if condition --------- Co-authored-by: Jack Yuan <[email protected]>
…s-agents#773) Our current implementation of AgentTool.stream() has a problem that we don't differentiate between intermediate streaming events and the final ToolResult events. Our only contract is that the last event *must be* be the tool result that is passed to the LLM. Our switch to Typed Events (strands-agents#755) pushes us in the right direction but for backwards compatibility we can't update the signature of `AgentTool.stream()` (nor have we exposed externally TypedEvents yet). That means that if we implemented tool-streaming today, then callers would see non-generator functions yielding both a `ToolStreamEvent` and `ToolResultEvent` even though they're not actually streaming responses. To avoid the odd behavior noted above, we'll special-case SDK-defined functions by allowing them to emit `ToolStreamEvent` and `ToolResultEvent` types directly (bypassing our normal wrapping), since they have the knowledge of when tools are actually generators or not. There's no observable difference in behavior to callers (this is all internal behavior), but this means that when we switch the flip for Tool Streaming, non-generator tools will **not** emit ToolStreamEvents - at least for AgentTool implementations that are in the SDK. Co-authored-by: Mackenzie Zastrow <[email protected]>
…tionEvent & AfterInvocationEvent (strands-agents#782) Co-authored-by: deepyes02 <[email protected]>
Enable decorated tools to be an async generator, enabling streaming of tool events back to to the caller.
zastrowm
force-pushed
the
enable_tool_streaming
branch
from
65f93f3 to
b9be087
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
Enable decorated tools to be an async generator, enabling streaming of tool events back to to the caller. Assuming strands-agents#773 is merged, the changes needed are fairly minor, as currently we're just swallowing tool stream events.
Implements strands-agents#543
Api Bar-raising
The tool event being emitted (as a dictionary) needs to go through bar-raising.
Proposed new Event
(event type is purely our internal name).
ToolStreamEvent:tool_stream_tool_use: ToolUse- the tool use that emitted the eventtool_stream_data: Any- the data that was streamed/yielded from the toolOur existing events are as follows:
InitEventLoopEventinit_event_loop: boolStartEventstart: boolStartEventLoopEventstart_event_loop: boolModelStreamChunkEventevent: StreamEventToolUseStreamEventdelta: ContentBlockDeltacurrent_tool_use: dict[str, Any]TextStreamEventdata: str,delta: ContentBlockDeltaReasoningTextStreamEventreasoningText: str | Nonedelta: ContentBlockDeltareasoning: boolReasoningSignatureStreamEventreasoning_signature: str | None,delta: ContentBlockDelta,reasoning: boolEventLoopThrottleEventevent_loop_throttled_delay: intModelMessageEventmessage: MessageToolResultMessageEventmessage: AnyForceStopEventforce_stop: bool,force_stop_reason: strAgentResultEventresult: AgentResultRelated Issues
strands-agents#543
Documentation PR
Type of Change
Bug fix
New feature
Breaking change
Documentation update
Other (please describe):
Testing
How have you tested the change? Verify that the changes do not break functionality or introduce warnings in consuming repositories: agents-docs, agents-tools, agents-cli
hatch run prepareChecklist
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.