Anthropic JSON Mode requires client to fabricate a nonsensical tool_use block #1034
Description
According to the Anthropic API, structured output is available by submitting a query in "JSON mode":
You can use tools to get Claude produce JSON output that follows a schema, even if you don’t have any intention of running that output through a tool or function.
When using tools in this way:
You usually want to provide a single tool
You should set tool_choice to instruct the model to explicitly use that tool
Remember that the model will pass the input to the tool, so the name of the tool and description should be from the model’s perspective.
That's a little janky, but it's fine. And when I submit a message with the right format, I get back structured output in a tool_use block.
However, I encountered problems while writing some code to engage Anthropic in a chat-style message stream where one of the messages returns structured output. First, when I serialized the structured-output response back into the message stream with some following chat messages, the API complained because I hadn't included the request_id in the tool call for the JSON-output message:
'error': {'type': 'invalid_request_error', 'message': 'messages.5.content.0.tool_use.id: Field required'}
Of course, the call ID doesn't really matter because the tool use is not being used to a call an actual function; it's just forcing Claude to pretend that it's calling a function in order to force it to generate structured output. But if I humor Claude and add ['id': (request ID string)] to the serialized request, I get this:
'error': {'type': 'invalid_request_error', 'message': 'messages.6: tool_useids were found withouttool_resultblocks immediately after: (request ID string). Eachtool_useblock must have a correspondingtool_result block in the next message.'}
So not only do I need to preserve the call ID from the structured output, but I also need to pretend that the schema-definition "tool" is an actual tool and synthesize a fictitious tool_result block with the same call ID, just so that Claude sees that all of its tool calls were fulfilled.
This is nonsense, Anthropic. This janky implementation of JSON mode is causing the API to trip over itself. Other service providers have APIs that support structured output without requiring devs to jump through hoops like this. Please fix.