-
Notifications
You must be signed in to change notification settings - Fork 253
Description
Describe the bug
Sometimes a tool call in a message isn't returned, and instead a message is logged:
Unknown tag in message content toolUse
This was hit when using Haiku 4.5 as an Evalite / autoevals Factuality scorer.
Please complete the following information:
- Which API you used: [e.g. /chat/completions]
- Which model you used:
au.anthropic.claude-haiku-4-5-20251001-v1:0
I think the API is /chat/completions -- it's this call; the issue stems from here:
| if finish_reason == "tool_use": |
For the finish_reason, I was receiving "max_tokens", and content was:
[{
"toolUse": {
"toolUseId": "tooluse_abcdef123456_abcd1234",
"name": "select_choice",
"input": { "choice": "D" }
}
}]Expected behavior
I expected the tool use to be returned
Additional context
I'll open a PR (#198) with some code changes that got this to work -- just check for "toolUse" even if the finish_reason isn't "tool_use". I'm not sure if it's the right solution for all cases -- it worked for my case.
If anyone else hits the same issue, it's because maxTokens is hardcoded to 512 in autoevals, and presumably Haiku 4.5's tokenization is different and exceeds that.