ChatGroq setting wrong toolcall flag in resulting API call.

[schmoasss]

Checked other resources

• This is a bug, not a usage question.
• I added a clear and descriptive title that summarizes this issue.
• I used the GitHub search to find a similar question and didn't find it.
• I am sure that this is a bug in LangChain rather than my code.
• The bug is not resolved by updating to the latest stable version of LangChain (or the specific integration package).
• This is not related to the langchain-community package.
• I posted a self-contained, minimal, reproducible example. A maintainer can copy it and run it AS IS.

Package (Required)

• langchain
• langchain-openai
• langchain-anthropic
• langchain-classic
• langchain-core
• langchain-cli
• langchain-model-profiles
• langchain-tests
• langchain-text-splitters
• langchain-chroma
• langchain-deepseek
• langchain-exa
• langchain-fireworks
• langchain-groq
• langchain-huggingface
• langchain-mistralai
• langchain-nomic
• langchain-ollama
• langchain-perplexity
• langchain-prompty
• langchain-qdrant
• langchain-xai
• Other / not sure / general

Example Code (Python)

from dotenv import load_dotenv
import os
from pydantic import BaseModel, Field
from langchain_groq import ChatGroq
from langchain.messages import HumanMessage, SystemMessage

class Track(BaseModel):
    main_artists: list[str] = Field(description="Names of all unique Main-Artists for this track.")
    title: str = Field(description="Track title as released.")
load_dotenv("../.env")
GROQ_API_KEY = os.environ.get("GROQ_API_KEY")

llm = ChatGroq(
    model="openai/gpt-oss-20b",
    temperature=0,
    api_key=GROQ_API_KEY,
).with_structured_output(Track)

message = [
    SystemMessage("You are a parser for Music-Records respond only with the given json format.\n"
                  "You do not need any Tools!"
                  "DOMAIN INFORMATION:\n"
                  "The input data contains recordings that were broadcasted on radio, tv or similar.\n"
                  "Your task is to parse the data into a usable format.\n"
                  "The data can contain the following fields:\n"
                  "Play Date or Play Time which states the date a song was broadcasts. DO NOT CONFUSE WITH RELEASE DATE"),
    
    HumanMessage("RECORD_TITLE: walk of life; COMPOSER: KNOPFLER; Vocalist: DIRE STRAITS; Interprete Details: nan; Station: FREI-FM; Play Date: 20240901; "
                 "DURATION: 230; Catalogue Details: nan; ISRC: GBF088500675; Label: VERTIGO 5147662; "
                 "Record Number: nan; Date of the record: nan; Country: nan; First Publishing: nan; Album Title: nan; "
                 "MAIN_ARTIST: DIRE STRAITS; Track Number: nan; Genre: nan; Program: nan; Order N°: nan; Brand: nan; "
                 "Label Code: nan; EAN/GTIN: nan;")]

llm.invoke(message)

Error Message and Stack Trace (if applicable)

BadRequestError: Error code: 400 - {'error': {'message': 'Tool choice is required, but model did not call a tool', 'type': 'invalid_request_error', 'code': 'tool_use_failed', 'failed_generation': '\n{\n  "record_title": "walk of life",\n  "composer": "KNOPFLER",\n  "vocalist": "DIRE STRAITS",\n  "interpreter_details": null,\n  "station": "FREI-FM",\n  "play_date": "2024-09-01",\n  "duration_seconds": 230,\n  "catalogue_details": null,\n  "isrc": "GBF088500675",\n  "label": "VERTIGO 5147662",\n  "record_number": null,\n  "date_of_record": null,\n  "country": null,\n  "first_publishing": null,\n  "album_title": null,\n  "main_artists": ["DIRE STRAITS"],\n  "track_number": null,\n  "genre": null,\n  "program": null,\n  "order_n": null,\n  "brand": null,\n  "label_code": null,\n  "ean_gtin": null\n}\n'}}

Description

Im trying to use the langchain groq adapter to parse unstructured data into structured output.

The issue occurs only sometimes and (supposedy) only when the model interprets the prompt as needing no tools.

I took it up with groq support first and they told me the following:

the error is likely happening because your short priming prompt doesn’t trigger a tool call, but the request is still set to require one. The model’s output is valid, but the API flags it as a tool-use failure.
To fix it, try one of these:
If you’re not using tools, remove the tools field or set tool_choice="none".
If you do use tools, set tool_choice="auto" so it’s optional instead of required.
That should stop the error while keeping your prompt working as-is.

I tried adding tool_choice="none" as kwarg in ChatGroq but that only fixed it sometimes.

System Info

System Information

OS: Windows
OS Version: 10.0.26200
Python Version: 3.13.5 (tags/v3.13.5:6cb20a2, Jun 11 2025, 16:15:46) [MSC v.1943 64 bit (AMD64)]

Package Information

langchain_core: 1.0.4
langchain: 1.0.5
langsmith: 0.4.42
langchain_groq: 1.0.1
langchain_ollama: 1.0.0
langgraph_sdk: 0.2.9

Optional packages not installed

langserve

Other Dependencies

groq: 0.34.1
httpx: 0.28.1
jsonpatch: 1.33
langgraph: 1.0.3
ollama: 0.6.0
orjson: 3.11.4
packaging: 25.0
pydantic: 2.12.4
pytest: 8.4.2
pyyaml: 6.0.3
requests: 2.32.5
requests-toolbelt: 1.0.0
tenacity: 9.1.2
typing-extensions: 4.15.0
zstandard: 0.25.0

[schmoasss]

This is a problem of the .with_structured_output() method.

[NeilMathew-git]

Hi! I'd love to take this on — planning to submit a PR this week if available.

[keenborder786]

"You do not need any Tools!" Just remove this line @schmoasss

[schmoasss]

"You do not need any Tools!" Just remove this line @schmoasss

Unfortunately this is not a fix. I also had the same error happen without the "You dont need tools" line. I only added that so its reproducable.

[keenborder786]

@schmoasss I just tested this, and removed this line, it works.

[schmoasss]

@schmoasss I just tested this, and removed this line, it works.

Oh sorry, i forgot to mention. I use the same system prompt (without the no tools statement) on hundreds of queries. Only some of them produce this error. I now switched to the official groq package and havent had the issue since.

[schmoasss]

@schmoasss I just tested this, and removed this line, it works.

Oh sorry, i forgot to mention. I use the same system prompt (without the no tools statement) on hundreds of queries. Only some of them produce this error. I now switched to the official groq package and havent had the issue since.

Also, Im not entirely sure if, when not using tools, getting an error is the desired behavior.

[dumko2001]

I have submitted a fix in #34053.

The issue was that ChatGroq.with_structured_output was not passing additional kwargs (like tool_choice) to the underlying model binder. My PR ensures these arguments are passed through, allowing you to explicitly set tool_choice="auto" to prevent the API 400 error when no tools are called.