How should Instructor support the responses API?

[ivanleomk]

Supporting Responses

I think we can support the responses model in a few ways but I would love to know if there are more things that we can integrate into our support for Responses itself that leverage this new API capability.

1. Without In Built Tools : We allow the users to use a new mode and this operates as per original instructor
2. With In-Built Tools : We allow the users to use the tools kwarg and specify what they want to configure (Eg. WebSearch, FileSearch )

The difference is that for 1, we have to allow multiple tool calls and parse it differently ( see below for how this might work ). I think this unlocks a new use-case for instructor which is to combine the web search results or File Search and from there obtain a structured output.

Without In-Built Tools

Without In Built Tools, we can support the responses API by simply modifying the Instructor class and then doing a manual mapping of the kwargs from the ChatCompletions API over to the new responses kwarg.

from openai import OpenAI
from pydantic import BaseModel
import instructor

client = instructor.from_openai(OpenAI(), mode=instructor.Mode.RESPONSE_TOOL_CALL)

class User(BaseModel):
    name: str
    age: int

response = client.chat.completions.create(
    model="gpt-4o-mini",
    messages=[{"role": "user", "content": "Anibal is 23 years old"}],
    response_model=User,
)

print(response)

response = client.responses.create(
    model="gpt-4o-mini",
    input=[{"role": "user", "content": "Anibal is 23 years old"}],
    response_model=User,
)

print(response)

To get the above to work, I needed to override the create method as seen below so that it would accept both a message

Note: Current implementation requires overriding the 'create' method and related methods so we would also need to supporting both response.create and chat.completions.create in future updates.

def create(
        self,
        response_model: type[T] | None,
        messages: list[ChatCompletionMessageParam] | None = None,
        max_retries: int | Retrying | AsyncRetrying = 3,
        validation_context: dict[str, Any] | None = None,
        context: dict[str, Any] | None = None,
        strict: bool = True,
        **kwargs: Any,
    ) -> T | Any | Awaitable[T] | Awaitable[Any]:
        if not messages and not kwargs.get("input"):
            raise ValueError("Either messages or input must be provided")

        if not messages and kwargs.get("input"):
            messages = kwargs.get("input")
            del kwargs["input"]

Under the hood since we’d be using the responses.create method, I think we would be able to support most API methods moving forward.

Using In-Built Tools

• Need to figure out if this works too for computer use but tested with file search and the web search endpoints

When using in-built tools, we can provide multiple tools and allow the model to call a few of them. We can see an example below

from openai import OpenAI
from rich import print
from pydantic import BaseModel
from instructor.function_calls import openai_schema

class Citation(BaseModel):
    file_id: str
    text_content: str

class FinalQuotation(BaseModel):
    price: str
    country: str
    citations: list[Citation]

quotation_model = openai_schema(FinalQuotation)

client = OpenAI()

response = client.responses.create(
    model="gpt-4o-mini",
    input="What is the cost of the Japanese package? Return the final price according to the quotation schema along with the exact file_ids that you used to generate the quotation and the text_content that you used to generate the quotation - add a file-{file_id} to the file id you'll cite",
    tools=[
        {
            "type": "file_search",
            "vector_store_ids": ["vs_67e2dfcc7bfc8191a6366d0fd2efcaf7"],
        },
        {
            **quotation_model.openai_schema,
            "type": "function",
        },
    ],
    include=["file_search_call.results"],
)

file_search = response.output[0]
assert file_search.type == "file_search_call"
file_ids = [file_search.file_id for file_search in file_search.results]
#> ['file-PSwV7xE8W6Wepj4EWJyCM1', 'file-EfEet6Mp2d1us2VGX9XN1v', 'file-Mx7r5LpsStxCEsvYBeHCuc']

quotation = response.output[1]
assert quotation.type == "function_call"
assert quotation.name == "FinalQuotation"
quotation_output = FinalQuotation.model_validate_json(quotation.arguments)
print(quotation_output)
#> FinalQuotation(price='$8,950', country='Japan', citations=[Citation(file_id='file-PSwV7xE8W6Wepj4EWJyCM1', text_content='- **Per Person (Double Occupancy)**: $8,950')])

print(quotation_output.citations)
for citation in quotation_output.citations:
    assert citation.file_id in file_ids

#> [Citation(file_id='file-PSwV7xE8W6Wepj4EWJyCM1', text_content='- **Per Person (Double Occupancy)**: $8,950')]

This is also the same thing for the web search API as we’ll see below when we ask the model to do a quick search for great restaurants in Paris and ask it to return a structured output in the form of a single recomendation.

from openai import OpenAI
from pydantic import BaseModel
from instructor.function_calls import openai_schema
from pydantic import BaseModel
import json
from rich import print

client = OpenAI()

class Citation(BaseModel):
    text_citation: str
    url: str

class Recomendation(BaseModel):
    citations: list[Citation]
    place: str
    reason: str

class TravelGuide(BaseModel):
    reasoning: str
    recomendations: Recomendation

recomendation_model = openai_schema(Recomendation)
travel_guide_model = openai_schema(TravelGuide)

response = client.responses.create(
    model="gpt-4o",
    tools=[
        {"type": "web_search_preview"},
        {
            **travel_guide_model.openai_schema,
            "type": "function",
        },
    ],
    input="Search the web for the latest places that have just opened in the city of Paris France which are hip and trendy. Think briefly through about what they might be good for before summarising your findings by giving a single recomendation using the TravelGuide schema. Provide all citations in the form of URLs that you used to generate the specific recomendation in the TravelGuide schema. ",
)

websearch_result = response.output[1].content[0]
urls = [websearch_result.url for websearch_result in websearch_result.annotations]
print(urls)

[
    'https://parisjetaime.com/eng/media/article/press-inaugurations-a1151?utm_source=openai',
    'https://parisjetaime.com/eng/media/article/press-inaugurations-a1151?utm_source=openai',
    'https://www.ft.com/content/cdfd58d3-0ef0-4c41-a0ad-b59306787c5a?utm_source=openai',
    'https://www.insider-trends.com/the-best-new-retail-in-paris-february-2025/?utm_source=openai'
]

arguments = json.loads(response.output[2].arguments)
print(arguments)
{
    'reasoning': "After analyzing the latest openings in Paris, I chose **Mia Mao** for its unique blend of immersive music and electronic nightlife experience. This club 
represents the cutting-edge of Paris's nightlife, combining innovative sound design with a focus on authenticity and artist engagement. The 'no photo' policy enhances the 
immersive aspect, making it not just a place to visit, but an experience to be fully part of.",
    'recomendations': {
        'citations': [
            {'text_citation': 'parisjetaime.com - Press Inaugurations', 'url': 'https://parisjetaime.com/eng/media/article/press-inaugurations-a1151?utm_source=openai'}
        ],
        'place': 'Mia Mao',
        'reason': 'Mia Mao stands out for its state-of-the-art sound system, artistic light design, and a policy that encourages full engagement with the music and artists, 
offering an authentic and immersive nightlife experience.'
    }
}

for citation in arguments["recomendations"]["citations"]:
    print(citation)
    {'text_citation': 'parisjetaime.com - Press Inaugurations', 'url': 'https://parisjetaime.com/eng/media/article/press-inaugurations-a1151?utm_source=openai'}
    assert citation["url"] in urls

We can see that in both cases, the final result was a structured object which is consistent with the tool schema that we defined earlier. I think that by supporting this specific use case, instructor becomes a way for us to consolidate information which these built-in tools can support

[pablospe]

I was wondering if there have been any updates regarding support for the Responses API.

[pablospe]

I was wondering if there have been any updates regarding support for the Responses API.

I haven't received any Responses, either related to the API or from this thread :P

[ivanleomk]

Haha @pablospe this should be out soon. Was just waiting for a while for people to leave comments

[pablospe]

Great, thank you! I’m happy to help test it. Please let me know if there’s a PR, and I’d be glad to assist with testing.

[pablospe]

Are there any updates regarding support for the Responses API?

[tchklovski]

+1 interested in this

[pablospe]

Is this the way to support Responses API?
https://github.com/567-labs/instructor/blob/main/docs/integrations/openai-responses.md

Maybe this discussion can be closed?