AzureChatOpenAI: support for gpt-oss-120b tool calling

[tudoanh]

Checked other resources

• This is a bug, not a usage question. For questions, please use the LangChain Forum (https://forum.langchain.com/).
• I added a clear and descriptive title that summarizes this issue.
• I used the GitHub search to find a similar question and didn't find it.
• I am sure that this is a bug in LangChain rather than my code.
• The bug is not resolved by updating to the latest stable version of LangChain (or the specific integration package).
• I read what a minimal reproducible example is (https://stackoverflow.com/help/minimal-reproducible-example).
• I posted a self-contained, minimal, reproducible example. A maintainer can copy it and run it AS IS.

Example Code

from langchain_openai import AzureChatOpenAI

model_kwargs = {
    "extra_query": {
        "enable-auto-tool-choice": True,
        "tool-call-parser": True
    }
}

model = AzureChatOpenAI(
    azure_endpoint=settings.AZURE_API_BASE,
    api_key=settings.AZURE_OPENAI_API_KEY,
    deployment_name=model_name,
    api_version=settings.AZURE_API_VERSION,
    streaming=streaming,
    tags=tags or [],
    stream_usage=True if streaming else False,
    model_kwargs=model_kwargs
)

Error Message and Stack Trace (if applicable)

BadRequestError: Error code: 400 - {'error': {'code': 'Bad Request', 'message': '{"object":"error","message":"\"auto\" tool choice requires --enable-auto-tool-choice and --tool-call-parser to be set","type":"BadRequestError","param":null,"code":400}', 'status': 400}}

Description

When trying gpt-oss-120b on Azure AI Foundry, it always raised the tool choice require error here. I think it because this model is using new format for messages and template https://github.com/openai/harmony

System Info

System Information

OS: Linux
OS Version: #26~24.04.1-Ubuntu SMP PREEMPT_DYNAMIC Thu Apr 17 19:20:47 UTC 2
Python Version: 3.12.8 (main, Jan 5 2025, 05:33:15) [Clang 19.1.6 ]

Package Information

langchain_core: 0.3.66
langchain: 0.3.26
langsmith: 0.4.4
langchain_mcp_adapters: 0.1.7
langchain_openai: 0.3.27
langchain_text_splitters: 0.3.8

Optional packages not installed

langserve

Other Dependencies

async-timeout<5.0.0,>=4.0.0;: Installed. No version info available.
httpx: 0.28.1
jsonpatch<2.0,>=1.33: Installed. No version info available.
langchain-anthropic;: Installed. No version info available.
langchain-aws;: Installed. No version info available.
langchain-azure-ai;: Installed. No version info available.
langchain-cohere;: Installed. No version info available.
langchain-community;: Installed. No version info available.
langchain-core<0.4,>=0.3.36: Installed. No version info available.
langchain-core<1.0.0,>=0.3.51: Installed. No version info available.
langchain-core<1.0.0,>=0.3.66: Installed. No version info available.
langchain-deepseek;: Installed. No version info available.
langchain-fireworks;: Installed. No version info available.
langchain-google-genai;: Installed. No version info available.
langchain-google-vertexai;: Installed. No version info available.
langchain-groq;: Installed. No version info available.
langchain-huggingface;: Installed. No version info available.
langchain-mistralai;: Installed. No version info available.
langchain-ollama;: Installed. No version info available.
langchain-openai;: Installed. No version info available.
langchain-perplexity;: Installed. No version info available.
langchain-text-splitters<1.0.0,>=0.3.8: Installed. No version info available.
langchain-together;: Installed. No version info available.
langchain-xai;: Installed. No version info available.
langsmith-pyo3: Installed. No version info available.
langsmith>=0.1.17: Installed. No version info available.
langsmith>=0.3.45: Installed. No version info available.
mcp>=1.9.2: Installed. No version info available.
openai-agents: Installed. No version info available.
openai<2.0.0,>=1.86.0: Installed. No version info available.
opentelemetry-api: Installed. No version info available.
opentelemetry-exporter-otlp-proto-http: Installed. No version info available.
opentelemetry-sdk: Installed. No version info available.
orjson: 3.10.18
packaging: 24.2
packaging<25,>=23.2: Installed. No version info available.
pydantic: 2.11.7
pydantic<3.0.0,>=2.7.4: Installed. No version info available.
pydantic>=2.7.4: Installed. No version info available.
pytest: Installed. No version info available.
PyYAML>=5.3: Installed. No version info available.
requests: 2.32.4
requests-toolbelt: 1.0.0
requests<3,>=2: Installed. No version info available.
rich: 14.0.0
SQLAlchemy<3,>=1.4: Installed. No version info available.
tenacity!=8.4.0,<10.0.0,>=8.1.0: Installed. No version info available.
tiktoken<1,>=0.7: Installed. No version info available.
typing-extensions>=4.7: Installed. No version info available.
zstandard: 0.23.0

[lomohov]

Same problem here, Harmony support should be implemented to both Langchain and Langgraph. Very relevant.

[Meliodas-gpu]

I need

[d-r-e]

Yes.

[msb418]

Here’s what you can do to resolve the issue:

⸻

✅ Diagnosis

The error—and your GitHub issue—reveals that certain Azure-hosted models (like gpt-oss-120b, or phi-4-multimodel-instruct) do not support tool calling, even with correct flags set (enable-auto-tool-choice, tool-call-parser). The Azure error response clearly states:

"auto" tool choice requires --enable-auto-tool-choice and --tool-call-parser to be set
— but even when set, the model rejects the call. 

This indicates tool calling is simply not supported on this model.

⸻

🛠 Solution Options

1. Switch to a Supported Model

Models like gpt-4o, or others hosted on Azure that support tool calling, work with enable-auto-tool-choice. To avoid the error:
• Confirm the model deployment supports function/tool calls.
• If not, change to a compatible model (e.g. gpt-4o).
• Re-deploy your Azure model accordingly before calling.

2. Fallback to Manual Tool Calls

If you must use gpt-oss-120b, avoid using tool-calling altogether:
• Use manual chaining logic (e.g. call tools locally based on intent extraction from the LLM response).
• Use tool_choice="none" so LangChain doesn’t attempt auto tool invocation.   

This avoids the 400 error and still enables prompt handling without tool integration.

📘 Sample Code Update
from langchain_openai import AzureChatOpenAI

model = AzureChatOpenAI(
azure_endpoint=...,
api_key=...,
deployment_name="gpt-4o", # ensure using tool-compatible model
api_version=...,
model_kwargs=model_kwargs, # include tool flags if using tool support
)

llm = model.bind_tools(
tools=[...],
tool_choice="auto", # auto tool calling only works if model supports it
)

If using gpt-oss-120b:

llm = model.bind_tools(
tools=[...],
tool_choice="none", # disables tool-calling features
)

🧪 TL;DR
• The error arises because gpt-oss-120b does not support tool calling, regardless of flag settings.  
• Switch to tool-compatible models (like gpt-4o) or disable tool-calling if using unsupported models.
• Update your configuration and test again with the right tool_choice and model.

[lomohov]

That's clearly wrong. @msb418

For example with OpenAI's Agents sdk, this works well.

import asyncio
from openai import AsyncOpenAI
from agents import Agent, Runner, function_tool, OpenAIResponsesModel, set_tracing_disabled

set_tracing_disabled(True)

@function_tool
def multiply(a: int, b: int):
    result = a * b
    print(f"[Tool] multiply({a}, {b}) = {result}")
    return result

@function_tool
def add(a: int, b: int):
    result = a + b
    print(f"[Tool] add({a}, {b}) = {result}")
    return result

async def main():
    agent = Agent(
        name="Assistant",
        instructions="You are a helpful assistant. Always use tools for math expressions.",
        model=OpenAIResponsesModel(
            model="openai/gpt-oss-120b",
            openai_client=AsyncOpenAI(
                base_url="http://localhost:8000/v1",
                api_key="not-needed",
            ),
        ),
        tools=[multiply, add],
    )

    query = "What is 1231238 * 12233 + 913832?"
    result = await Runner.run(agent, query)

    print("\n=== Final Output ===")
    print(result.final_output)

if __name__ == "__main__":
    asyncio.run(main())

python test.py
[Tool] multiply(1231238, 12233) = 15061734454
[Tool] add(15061734454, 913832) = 15062648286

=== Final Output ===
The result is **15,062,648,286**.

Both models clearly support tool calling.

https://cookbook.openai.com/articles/openai-harmony

Harmony follows the structure of Responses API.

It might be a good idea to implement Responses Api support and Harmony format now. As OpenAI is slowly migrating from completions to Responses anyway for quite some time now. It's also superset of completions in terms of functionality. Only audio is not yet supported.

https://platform.openai.com/docs/guides/responses-vs-chat-completions

[tudoanh]

Btw the langchain-openai working fine with the GPT-OSS-120B model from Groq, calling tools without trouble.

[eksno]

@msb418 has to be AI lmao

here's a slimilar issue I raised on pydantic-ai with some insight in the comments: pydantic/pydantic-ai#2439

[eksno]

Main issue from my research is harmony is already supported... badly by the provider! Which means unless you intend to self host and call transformers directly the providers forced abstraction of harmony can't be bypassed.