withStructuredOutput not working properly with ChatAnthropic #7936

Aman9723 · 2025-04-01T03:42:28Z

Aman9723
Apr 1, 2025

Checked other resources

I added a very descriptive title to this question.
I searched the LangChain documentation with the integrated search.
I used the GitHub search to find a similar question and didn't find it.

Commit to Help

I commit to help with one of those options 👆

Example Code

import { ENV } from "../lib/constants";
import { AIMessage, HumanMessage, SystemMessage } from "@langchain/core/messages";
import { z } from "zod";
import { ChatAnthropic } from "@langchain/anthropic";

const CodeGeneratorResponseSchema = z.object({
    edits: z.string().describe("The text response from the model regarding the edits made to the code or generated code"),
    code: z.string().describe("The newly generated or edited code"),
    suggestions: z
        .array(z.string())
        .optional()
        .describe("Suggestions for the user to improve the game which can be used as prompts for the next iteration")
});

const claudeHaikuModel = new ChatAnthropic({
    apiKey: ENV.ANTHROPIC_API_KEY,
    model: "claude-3-5-haiku-latest"
}).withStructuredOutput(CodeGeneratorResponseSchema, {
    name: "CodeGeneratorResponseSchema",
    includeRaw: true
});

async function run() {
    const res = await claudeHaikuModel.invoke([
        new SystemMessage(
            `You are a specialized game developer. Generate a single HTML file that includes all necessary code (HTML, CSS, and JavaScript) to create a game`
        ),
        new HumanMessage("Generate a chess game"),
        new AIMessage(
            `<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <meta name="viewport" content="width=device-width, initial-scale=1.0">
    <title>Chess Game</title>
    <style>
        body {
            font-family: Arial, sans-serif;
            display: flex;
            justify-content: center;
            align-items: center;
            height: 100vh;
            margin: 0;
            background-color: #f0f0f0;
        }
        #chessboard {
            display: grid;
            grid-template-columns: repeat(8, 50px);
            grid-template-rows: repeat(8, 50px);
            border: 2px solid black;
        }
        .square {
            width: 50px;
            height: 50px;
        }
        .white {
            background-color: #fff;
        }
        .black {
            background-color: #000;
        }
    </style>
</head>
<body>
    <div id="chessboard"></div>
    <script>
        const chessboard = document.getElementById('chessboard');
        for (let row = 0; row < 8; row++) {
            for (let col = 0; col < 8; col++) {
                const square = document.createElement('div');
                square.className = 'square ' + ((row + col) % 2 === 0 ? 'white' : 'black');
                chessboard.appendChild(square);
            }
        }
    </script>
</body>
</html>`
        ),
        new HumanMessage("Improve the game logic")
    ]);
    console.log(res);
}

run();

Description

I'm trying to get structured output with anthropic models.
I expect it to give me an object containing 2 or 3 fields in res.parsed. This is working perfectly fine with o3-mini model of openai.
It sometimes give the object but sometimes gives null.

System Info

"@langchain/anthropic": "^0.3.15",
"@langchain/core": "^0.3.43",
"@langchain/langgraph": "^0.2.62",
"@langchain/openai": "^0.5.2",

OS Name Microsoft Windows 11 Home Single Language

npm: '11.1.0',
node: '22.13.0',

@Aman9723 · 2025-04-01T03:45:40Z

dosubot[bot]
bot Apr 1, 2025

Hello, @Aman9723! I'm here to assist you with any bugs, questions, or contributions you need help with. Let's tackle this together!

I found a similar closed issue that might be relevant to your problem. The issue was titled "withStructuredOutput not returning the desired format" and was resolved by switching to a model that supports structured output, such as ANTHROPIC claude-3-5-sonnet-latest. Additionally, using sectionsSchema.parse(reportSections) for validation can help catch format errors early in the execution ^[1].

In your case, you might want to verify if the claude-3-5-haiku-latest model supports structured output. If it doesn't, consider switching to a model that does, like the claude-3-5-sonnet-latest. Also, ensure that your schema validation is correctly implemented to handle any format inconsistencies.

To continue talking to Dosu, mention @dosu.

^{Help Dosu learn! Give it feedback: Great Response | Irrelevant Answer | Incorrect Sources | Too Verbose | Hallucination | Bug Report | Other}

2 replies

Aman9723 Apr 1, 2025
Author

@dosu
This issue persists with claude-3-5-sonnet-latest and claude-3-7-sonnet-latest models.

dosubot[bot] bot Apr 1, 2025

I couldn't find specific information about limitations or issues with the claude-3-5-sonnet-latest and claude-3-7-sonnet-latest models regarding structured output in the LangChain repository. However, it's important to ensure that the models you are using explicitly support structured output. If the issue persists, you might want to double-check the schema and the configuration used with the withStructuredOutput method to ensure they align with the model's capabilities. Additionally, consider validating the output using the schema to catch any inconsistencies early.

To continue talking to Dosu, mention @dosu.

^{Help Dosu learn! Give it feedback: Great Response | Irrelevant Answer | Incorrect Sources | Too Verbose | Hallucination | Bug Report | Other}

Aman9723 · 2025-04-02T03:07:26Z

Aman9723
Apr 2, 2025
Author

Works fine when using the native sdk of anthropic.

import { ENV } from "../lib/constants";
import Anthropic from "@anthropic-ai/sdk";

const anthropic = new Anthropic({
    apiKey: ENV.ANTHROPIC_API_KEY
});

async function run() {
    const msg = await anthropic.messages.create({
        tool_choice: {
            type: "tool",
            name: "CodeGeneratorResponseSchema"
        },
        model: "claude-3-5-haiku-latest",
        max_tokens: 8_192,
        system: "You are a specialized game developer. Generate a single HTML file that includes all necessary code (HTML, CSS, and JavaScript) to create a game",
        tools: [
            {
                name: "CodeGeneratorResponseSchema",
                description: "Schema for the response containing code edits, generated code, and suggestions",
                input_schema: {
                    type: "object",
                    properties: {
                        edits: {
                            type: "string",
                            description: "The text response from the model regarding the edits made to the code or generated code"
                        },
                        code: {
                            type: "string",
                            description: "The newly generated or edited code"
                        },
                        suggestions: {
                            type: "array",
                            items: { type: "string" },
                            description: "Suggestions for the user to improve the game which can be used as prompts for the next iteration"
                        }
                    },
                    required: ["edits", "code"]
                }
            }
        ],
        messages: [
            { role: "user", content: "Generate a chess game" },
            {
                role: "assistant",
                content: `<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <meta name="viewport" content="width=device-width, initial-scale=1.0">
    <title>Chess Game</title>
    <style>
        body {
            font-family: Arial, sans-serif;
            display: flex;
            justify-content: center;
            align-items: center;
            height: 100vh;
            margin: 0;
            background-color: #f0f0f0;
        }
        #chessboard {
            display: grid;
            grid-template-columns: repeat(8, 50px);
            grid-template-rows: repeat(8, 50px);
            border: 2px solid black;
        }
        .square {
            width: 50px;
            height: 50px;
        }
        .white {
            background-color: #fff;
        }
        .black {
            background-color: #000;
        }
    </style>
</head>
<body>
    <div id="chessboard"></div>
    <script>
        const chessboard = document.getElementById('chessboard');
        for (let row = 0; row < 8; row++) {
            for (let col = 0; col < 8; col++) {
                const square = document.createElement('div');
                square.className = 'square ' + ((row + col) % 2 === 0 ? 'white' : 'black');
                chessboard.appendChild(square);
            }
        }
    </script>
</body>
</html>`
            },
            {
                role: "user",
                content: "Improve the game logic"
            }
        ]
    });

    console.log(msg);
}

run();

0 replies

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

withStructuredOutput not working properly with ChatAnthropic #7936

Uh oh!

{{title}}

Uh oh!

Replies: 2 comments 2 replies

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{title}}

Uh oh!

Select a reply

Uh oh!

withStructuredOutput not working properly with ChatAnthropic #7936

Uh oh!

Aman9723 Apr 1, 2025

Checked other resources

Commit to Help

Example Code

Description

System Info

Replies: 2 comments · 2 replies

Uh oh!

dosubot[bot] bot Apr 1, 2025

Uh oh!

Aman9723 Apr 1, 2025 Author

Uh oh!

dosubot[bot] bot Apr 1, 2025

Uh oh!

Aman9723 Apr 2, 2025 Author

Aman9723
Apr 1, 2025

Replies: 2 comments 2 replies

dosubot[bot]
bot Apr 1, 2025

Aman9723 Apr 1, 2025
Author

Aman9723
Apr 2, 2025
Author