Skip to content

20-70% of concurrently ran chats fail to provide usage data and StreamEvents #435

@MarvellousBee

Description

@MarvellousBee

Minimal example

This is a small script that spawns 10 chats and monitors their output.

https://github.com/MarvellousBee/claude_agent_sdk_bug_report

Expected behavior

I expected chats to return usage data and StreamEvent objects, 100% of the time.
StreamEvent is expected since I passed include_partial_messages=true.
Here are some exaples of this happening, from the script:

Session 2: ✔️ (output_tokens=13) | Message types: AssistantMessage: 1, ResultMessage: 1, StreamEvent: 7, SystemMessage: 1
Session 3: ✔️ (output_tokens=13) | Message types: AssistantMessage: 1, ResultMessage: 1, StreamEvent: 7, SystemMessage: 1
Session 4: ✔️ (output_tokens=13) | Message types: AssistantMessage: 1, ResultMessage: 1, StreamEvent: 7, SystemMessage: 1
Session 5: ✔️ (output_tokens=13) | Message types: AssistantMessage: 1, ResultMessage: 1, StreamEvent: 7, SystemMessage: 1
Session 6: ✔️ (output_tokens=13) | Message types: AssistantMessage: 1, ResultMessage: 1, StreamEvent: 7, SystemMessage: 1

Actual behavior

Sometimes, chats fail with two issues:

  • No StreamEvent objects are returned
  • All usage data is set to 0
Session 7: ❌ ZERO TOKENS
  Response text: '7 + 8 = **15**...' (14 chars)
  Usage: {'input_tokens': 0, 'cache_creation_input_tokens': 0, 'cache_read_input_tokens': 0, 'output_tokens': 0, 'server_tool_use': {'web_search_requests': 0, 'web_fetch_requests': 0}, 'service_tier': 'standard', 'cache_creation': {'ephemeral_1h_input_tokens': 0, 'ephemeral_5m_input_tokens': 0}}
  Message types: AssistantMessage: 1, ResultMessage: 1, SystemMessage: 1
  Messages received:
    [0] SystemMessage
    [1] AssistantMessage
    [2] ResultMessage: is_error=False, output_tokens=0

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions