Add support for GPT-5 Free-Form Function Calling and Context Free Grammar constraints over tools

docs/models/openai.md

@@ -143,6 +143,149 @@ As of 7:48 AM on Wednesday, April 2, 2025, in Tokyo, Japan, the weather is cloud
 
 You can learn more about the differences between the Responses API and Chat Completions API in the [OpenAI API docs](https://platform.openai.com/docs/guides/responses-vs-chat-completions).
 
+### Freeform Function Calling
+
+GPT‑5 can now send raw text payloads - anything from Python scripts to SQL queries - to your custom tool without wrapping the data in JSON using freeform function calling. This differs from classic structured function calls, giving you greater flexibility when interacting with external runtimes such as:
+
+* code execution with sandboxes (Python, C++, Java, …)
+* SQL databases
+* Shell environments
+* Configuration generators
+
+Note that freeform function calling does NOT support parallel tool calling.
+
+You can enable freeform function calling for a tool using the `text_format` parameter when creating your tool. To use this the tool must take a single string argument (other than the runtime context) and the model must be one of the GPT-5 responses models. For example:
+
+```python
+from pydantic_ai import Agent
+from pydantic_ai.models.openai import OpenAIResponsesModel
+
+model = OpenAIResponsesModel('gpt-5')  # (1)!
+agent = Agent(model)
+
+@agent.tool_plain(text_format='text')  # (2)!
+def freeform_tool(sql: str): ...
+```
+
+1. The GPT-5 family (`gpt-5`, `gpt-5-mini`, `gpt-5-nano`) all support freeform function calling.
+2. If the tool or model cannot be used with freeform function calling then it will be invoked in the normal way.
+
+You can read more about this function calling style in the [openai documentation](https://cookbook.openai.com/examples/gpt-5/gpt-5_new_params_and_tools#2-freeform-function-calling).
+
+#### Context Free Grammar
+
+Invoking tools using freeform function calling can result in errors when the tool expectations are not met. For example, a tool that queries an SQL database can only accept valid SQL. The freeform function calling of GPT-5 supports generation of valid SQL for this situation by constraining the generated text using a context free grammar.
+
+A context‑free grammar is a collection of production rules that define which strings belong to a language. Each rule rewrites a non‑terminal symbol into a sequence of terminals (literal tokens) and/or other non‑terminals, independent of surrounding context—hence context‑free. CFGs can capture the syntax of most programming languages and, in OpenAI custom tools, serve as contracts that force the model to emit only strings that the grammar accepts.
+
+The grammar can be written as either a regular expression:
+
+
+```python
+from pydantic_ai import Agent
+from pydantic_ai.models.openai import OpenAIResponsesModel
+from pydantic_ai.tools import FunctionTextFormat
+
+model = OpenAIResponsesModel('gpt-5')  # (1)!
+agent = Agent(model)
+
+timestamp_grammar_definition = r'^\d{4}-(0[1-9]|1[0-2])-(0[1-9]|[12]\d|3[01]) (?:[01]\d|2[0-3]):[0-5]\d$'
+
+@agent.tool_plain(text_format=FunctionTextFormat(syntax='regex', grammar=timestamp_grammar_definition))  # (2)!
+def timestamp_accepting_tool(timestamp: str): ...
+```
+
+1. The GPT-5 family (`gpt-5`, `gpt-5-mini`, `gpt-5-nano`) all support freeform function calling with context free grammar constraints. Unfortunately gpt-5-nano often struggles with these calls.
+2. If the tool or model cannot be used with freeform function calling then it will be invoked in the normal way, which may lead to invalid input.
+
+Or as a [LARK](https://lark-parser.readthedocs.io/en/latest/how_to_use.html) grammar:
+
+```python
+from pydantic_ai import Agent
+from pydantic_ai.models.openai import OpenAIResponsesModel
+from pydantic_ai.tools import FunctionTextFormat
+
+model = OpenAIResponsesModel('gpt-5')  # (1)!
+agent = Agent(model)
+
+timestamp_grammar_definition = r'''
+start: timestamp
+
+timestamp: YEAR "-" MONTH "-" DAY " " HOUR ":" MINUTE
+
+%import common.DIGIT
+
+YEAR: DIGIT DIGIT DIGIT DIGIT
+MONTH: /(0[1-9]|1[0-2])/
+DAY: /(0[1-9]|[12]\d|3[01])/
+HOUR: /([01]\d|2[0-3])/
+MINUTE: /[0-5]\d/
+'''
+
+@agent.tool_plain(text_format=FunctionTextFormat(syntax='lark', grammar=timestamp_grammar_definition))  # (2)!
+def i_like_iso_dates(date: str): ...
+```
+
+1. The GPT-5 family (`gpt-5`, `gpt-5-mini`, `gpt-5-nano`) all support freeform function calling with context free grammar constraints. Unfortunately gpt-5-nano often struggles with these calls.
+2. If the tool or model cannot be used with freeform function calling then it will be invoked in the normal way, which may lead to invalid input.
+
+There is a limit to the grammar complexity that GPT-5 supports, as such it is important to test your grammar.
+
+Freeform function calling, with or without a context free grammar, can be used with the output tool for the agent:
+
+```python
+from pydantic_ai import Agent
+from pydantic_ai.models.openai import OpenAIResponsesModel
+from pydantic_ai.output import ToolOutput
+from pydantic_ai.tools import FunctionTextFormat
+
+sql_grammar_definition = r'''
+start: select_stmt
+select_stmt: "SELECT" select_list "FROM" table ("WHERE" condition ("AND" condition)*)?
+select_list: "*" | column ("," column)*
+table: "users" | "orders"
+column: "id" | "user_id" | "name" | "age"
+condition: column ("=" | ">" | "<") (NUMBER | STRING)
+%import common.NUMBER
+%import common.ESCAPED_STRING -> STRING
+%import common.WS
+%ignore WS
+''' # (1)!
+def database_query(sql: str) -> str:
+    return sql  # (2)!
+
+output_tool = ToolOutput(database_query, text_format=FunctionTextFormat(syntax='lark', grammar=sql_grammar_definition))
+model = OpenAIResponsesModel('gpt-5')
+agent = Agent(model, output_type=output_tool)
+```
+
+1. An inline SQL grammar definition would be quite extensive and so this simplified version has been written, you can find an example SQL grammar [in the openai example](https://cookbook.openai.com/examples/gpt-5/gpt-5_new_params_and_tools#33-example---sql-dialect--ms-sql-vs-postgresql). There are also example grammars in the [lark repo](https://github.com/lark-parser/lark/blob/master/examples/composition/json.lark). Remember that a simpler grammar that matches your DDL will be easier for GPT-5 to work with and will result in fewer semantically invalid results.
+2. Returning the input directly might seem odd, remember that it has been constrained to the provided grammar. This can be useful if you want GPT-5 to generate content according to a grammar that you then use extensively through your program.
+
+##### Best Practices
+
+Lark grammars can be tricky to perfect. While simple grammars perform most reliably, complex grammars often require iteration on the grammar definition itself, the prompt, and the tool description to ensure that the model does not go out of distribution.
+
+* Keep terminals bounded – use /[^.\n]{0,10}*\./ rather than /.*\./. Limit matches both by content (negated character class) and by length ({M,N} quantifier).
+* Prefer explicit char‑classes over . wildcards.
+* Thread whitespace explicitly, e.g. using SP = " ", instead of a global %ignore.
+* Describe your tool: tell the model exactly what the CFG accepts and instruct it to reason heavily about compliance.
+
+Troubleshooting
+
+* API rejects the grammar because it is too complex ➜ Simplify rules and terminals, remove %ignore.*.
+* Unexpected tokens ➜ Confirm terminals aren't overlapping; check greedy lexer.
+* When the model drifts "out‑of‑distribution" (shows up as the model producing excessively long or repetitive outputs, it is syntactically valid but is semantically wrong):
+  - Tighten the grammar.
+  - Iterate on the prompt (add few-shot examples) and tool description (explain the grammar and instruct the model to reason to conform to it).
+  - Experiment with a higher reasoning effort (e.g, bump from medium to high).
+
+Resources:
+
+* [Lark Docs](https://lark-parser.readthedocs.io/en/stable/)
+* [Lark IDE](https://www.lark-parser.org/ide/)
+* [OpenAI Cookbook on CFG](https://cookbook.openai.com/examples/gpt-5/gpt-5_new_params_and_tools#3-contextfree-grammar-cfg)
+
 ## OpenAI-compatible Models
 
 Many providers and models are compatible with the OpenAI API, and can be used with `OpenAIChatModel` in Pydantic AI.

pydantic_ai_slim/pydantic_ai/_output.py

@@ -28,7 +28,7 @@
     ToolOutput,
     _OutputSpecItem,  # type: ignore[reportPrivateUsage]
 )
-from .tools import GenerateToolJsonSchema, ObjectJsonSchema, ToolDefinition
+from .tools import FunctionTextFormat, GenerateToolJsonSchema, ObjectJsonSchema, ToolDefinition
 from .toolsets.abstract import AbstractToolset, ToolsetTool
 
 if TYPE_CHECKING:
@@ -587,6 +587,7 @@ class OutputObjectDefinition:
     name: str | None = None
     description: str | None = None
     strict: bool | None = None
+    text_format: Literal['text'] | FunctionTextFormat | None = None
 
 
 @dataclass(init=False)
@@ -617,6 +618,7 @@ def __init__(
         name: str | None = None,
         description: str | None = None,
         strict: bool | None = None,
+        text_format: Literal['text'] | FunctionTextFormat | None = None,
     ):
         if inspect.isfunction(output) or inspect.ismethod(output):
             self._function_schema = _function_schema.function_schema(output, GenerateToolJsonSchema)
@@ -659,6 +661,7 @@ def __init__(
             description=description,
             json_schema=json_schema,
             strict=strict,
+            text_format=text_format,
         )
 
     async def process(
@@ -916,19 +919,23 @@ def build(
             name = None
             description = None
             strict = None
+            text_format = None
             if isinstance(output, ToolOutput):
                 # do we need to error on conflicts here? (DavidM): If this is internal maybe doesn't matter, if public, use overloads
                 name = output.name
                 description = output.description
                 strict = output.strict
+                text_format = output.text_format
 
                 output = output.output
 
             description = description or default_description
             if strict is None:
                 strict = default_strict
 
-            processor = ObjectOutputProcessor(output=output, description=description, strict=strict)
+            processor = ObjectOutputProcessor(
+                output=output, description=description, strict=strict, text_format=text_format
+            )
             object_def = processor.object_def
 
             if name is None:
@@ -953,6 +960,7 @@ def build(
                 description=description,
                 parameters_json_schema=object_def.json_schema,
                 strict=object_def.strict,
+                text_format=object_def.text_format,
                 outer_typed_dict_key=processor.outer_typed_dict_key,
                 kind='output',
             )

pydantic_ai_slim/pydantic_ai/agent/__init__.py

@@ -8,7 +8,7 @@
 from collections.abc import AsyncIterator, Awaitable, Callable, Iterator, Sequence
 from contextlib import AbstractAsyncContextManager, AsyncExitStack, asynccontextmanager, contextmanager
 from contextvars import ContextVar
-from typing import TYPE_CHECKING, Any, ClassVar, cast, overload
+from typing import TYPE_CHECKING, Any, ClassVar, Literal, cast, overload
 
 from opentelemetry.trace import NoOpTracer, use_span
 from pydantic.json_schema import GenerateJsonSchema
@@ -49,6 +49,7 @@
     DeferredToolResult,
     DeferredToolResults,
     DocstringFormat,
+    FunctionTextFormat,
     GenerateToolJsonSchema,
     RunContext,
     Tool,
@@ -1005,6 +1006,7 @@ def tool(
         require_parameter_descriptions: bool = False,
         schema_generator: type[GenerateJsonSchema] = GenerateToolJsonSchema,
         strict: bool | None = None,
+        text_format: Literal['text'] | FunctionTextFormat | None = None,
         requires_approval: bool = False,
     ) -> Callable[[ToolFuncContext[AgentDepsT, ToolParams]], ToolFuncContext[AgentDepsT, ToolParams]]: ...
 
@@ -1020,6 +1022,7 @@ def tool(
         require_parameter_descriptions: bool = False,
         schema_generator: type[GenerateJsonSchema] = GenerateToolJsonSchema,
         strict: bool | None = None,
+        text_format: Literal['text'] | FunctionTextFormat | None = None,
         requires_approval: bool = False,
     ) -> Any:
         """Decorator to register a tool function which takes [`RunContext`][pydantic_ai.tools.RunContext] as its first argument.
@@ -1065,6 +1068,8 @@ async def spam(ctx: RunContext[str], y: float) -> float:
             schema_generator: The JSON schema generator class to use for this tool. Defaults to `GenerateToolJsonSchema`.
             strict: Whether to enforce JSON schema compliance (only affects OpenAI).
                 See [`ToolDefinition`][pydantic_ai.tools.ToolDefinition] for more info.
+            text_format: Used to invoke the function using free-form function calling (only affects OpenAI).
+                See [`ToolDefinition`][pydantic_ai.tools.ToolDefinition] for more info.
             requires_approval: Whether this tool requires human-in-the-loop approval. Defaults to False.
                 See the [tools documentation](../deferred-tools.md#human-in-the-loop-tool-approval) for more info.
         """
@@ -1075,15 +1080,16 @@ def tool_decorator(
             # noinspection PyTypeChecker
             self._function_toolset.add_function(
                 func_,
-                True,
-                name,
-                retries,
-                prepare,
-                docstring_format,
-                require_parameter_descriptions,
-                schema_generator,
-                strict,
-                requires_approval,
+                takes_ctx=True,
+                name=name,
+                retries=retries,
+                prepare=prepare,
+                docstring_format=docstring_format,
+                require_parameter_descriptions=require_parameter_descriptions,
+                schema_generator=schema_generator,
+                strict=strict,
+                text_format=text_format,
+                requires_approval=requires_approval,
             )
             return func_
 
@@ -1104,6 +1110,7 @@ def tool_plain(
         require_parameter_descriptions: bool = False,
         schema_generator: type[GenerateJsonSchema] = GenerateToolJsonSchema,
         strict: bool | None = None,
+        text_format: Literal['text'] | FunctionTextFormat | None = None,
         requires_approval: bool = False,
     ) -> Callable[[ToolFuncPlain[ToolParams]], ToolFuncPlain[ToolParams]]: ...
 
@@ -1119,6 +1126,7 @@ def tool_plain(
         require_parameter_descriptions: bool = False,
         schema_generator: type[GenerateJsonSchema] = GenerateToolJsonSchema,
         strict: bool | None = None,
+        text_format: Literal['text'] | FunctionTextFormat | None = None,
         requires_approval: bool = False,
     ) -> Any:
         """Decorator to register a tool function which DOES NOT take `RunContext` as an argument.
@@ -1164,6 +1172,8 @@ async def spam(ctx: RunContext[str]) -> float:
             schema_generator: The JSON schema generator class to use for this tool. Defaults to `GenerateToolJsonSchema`.
             strict: Whether to enforce JSON schema compliance (only affects OpenAI).
                 See [`ToolDefinition`][pydantic_ai.tools.ToolDefinition] for more info.
+            text_format: Used to invoke the function using free-form function calling (only affects OpenAI).
+                See [`ToolDefinition`][pydantic_ai.tools.ToolDefinition] for more info.
             requires_approval: Whether this tool requires human-in-the-loop approval. Defaults to False.
                 See the [tools documentation](../deferred-tools.md#human-in-the-loop-tool-approval) for more info.
         """
@@ -1172,15 +1182,16 @@ def tool_decorator(func_: ToolFuncPlain[ToolParams]) -> ToolFuncPlain[ToolParams
             # noinspection PyTypeChecker
             self._function_toolset.add_function(
                 func_,
-                False,
-                name,
-                retries,
-                prepare,
-                docstring_format,
-                require_parameter_descriptions,
-                schema_generator,
-                strict,
-                requires_approval,
+                takes_ctx=False,
+                name=name,
+                retries=retries,
+                prepare=prepare,
+                docstring_format=docstring_format,
+                require_parameter_descriptions=require_parameter_descriptions,
+                schema_generator=schema_generator,
+                strict=strict,
+                text_format=text_format,
+                requires_approval=requires_approval,
             )
             return func_