Implement with_structured_output for ChatBedrock from langchain-aws

[codekiln]

Checked

• I searched existing ideas and did not find a similar one
• I added a very descriptive title
• I've clearly described the feature request and motivation for it

Feature request

1. Please implement with_structured_output for ChatBedrock from langchain-aws
2. Until it is implemented, please update the https://python.langchain.com/v0.2/docs/how_to/structured_output/ with instructions on how to do this yourself, that is, how to make a workaround
3. At a high level, it would be great if there was some way for the docs to explain why langchain-anthropic and langchain-bedrock must have different support for claude, since bedrock is just a different deployment model for anthropic

Motivation

Structured output is possible using Bedrock's Claude deployments:

1. Community | Handling streaming and structured data using Amazon Bedrock
2. Unlocking Structured Outputs with Amazon Bedrock: A Guide to Leveraging Instructor and Anthropic Claude 3 | Medium
3. amazon-bedrock-samples/function-calling/Extracting_structured_json_Bedrock_converse.ipynb at 4cdc7a4c9bb2752b32398005d3bcb4617d7f4cee · aws-samples/amazon-bedrock-samples

It's also possible in langchain-anthropic; see Anthropic function calling for structured LLM outputs - YouTube.

However, in the compatibility table, bedrock says it does not support it: https://python.langchain.com/v0.2/docs/integrations/chat/

Proposal (If applicable)

No response

[leikoilja]

came across this discussion after struggling to make ChatBedrock work with with_structured_output(). docs seems to say it's supported

but the promt() call returns None.

langchain==0.2.5
langchain-aws==0.1.6
langchain-core==0.2.11

@codekiln , have you found any workaround?

[leikoilja]

Seems like bumping to the latest v0.2.7 has fixed it for me: https://github.com/langchain-ai/langchain/releases

[AyushSonuu]

not fixed yet

[yuvaldream]

Any new?

[Mahaswami]

For those who is awaiting a solution for this, a suggestion is to use ChatBedrockConverse which is directly from AWS and supports with_structured_output.

An added note that with_structured_output with Llama is requiring mandatory tool calling requirement so logic needs to be tweaked accordingly

[wondamo]

what do you mean by tweaking the logic

[Mahaswami]

Sorry I perhaps could have been a bit more clearer.

Requesting "withStructuredOutput" should only be indicating that an output format should be something like json. However, when Llama is requested with "withStructuredOutput", is also expecting that the prompt should result in at least one tool call. I believe connecting "withStructuredOutput" to a mandatory tool is a bug.

By Tweaking I meant that for my case I entirely avoided "withStructuredOutput" since I don't have a tool call and converted the output manually into a structured output for my further processing.