Skip to content

Message with finish_reason = "max_tokens" can contain toolUse, but the tool use isn't returned #197

@azriel-healthpoint

Description

@azriel-healthpoint

Describe the bug

Sometimes a tool call in a message isn't returned, and instead a message is logged:

Unknown tag in message content toolUse

This was hit when using Haiku 4.5 as an Evalite / autoevals Factuality scorer.

Please complete the following information:

  • Which API you used: [e.g. /chat/completions]
  • Which model you used: au.anthropic.claude-haiku-4-5-20251001-v1:0

I think the API is /chat/completions -- it's this call; the issue stems from here:

if finish_reason == "tool_use":

For the finish_reason, I was receiving "max_tokens", and content was:

[{
  "toolUse": {
    "toolUseId": "tooluse_abcdef123456_abcd1234",
    "name": "select_choice",
    "input": { "choice": "D" }
  }
}]

Expected behavior

I expected the tool use to be returned

Additional context

I'll open a PR (#198) with some code changes that got this to work -- just check for "toolUse" even if the finish_reason isn't "tool_use". I'm not sure if it's the right solution for all cases -- it worked for my case.

If anyone else hits the same issue, it's because maxTokens is hardcoded to 512 in autoevals, and presumably Haiku 4.5's tokenization is different and exceeds that.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions