Switch to Responses API for OpenAI

[tpaulshippy]

What this does

As explained here there are numerous reasons to use the newer Responses API instead of the Chat Completions API. Features we get by switching include:

1. Mixing web search with custom tools
2. Conversations with images in and out
3. Certain models like o4-mini-deep-research
4. MCP support through OpenAI

There is one feature not yet available - audio inputs are not supported by the Responses API. So, the library will detect any audio inputs and fall back to the Chat Completions API if they exist.

Type of change

• New feature

Scope check

• I read the Contributing Guide
• This aligns with RubyLLM's focus on LLM communication
• This isn't application-specific logic that belongs in user code
• This benefits most users, not just my specific use case

Quality check

• I ran overcommit --install and all hooks pass
• I tested my changes thoroughly
• I updated documentation if needed
• I didn't modify auto-generated files manually (models.json, aliases.json)

API changes

• No API changes

Related issues

Replaces #290
Should enable resolution of #213

[tpaulshippy]

This change would be backward compatible as it falls back to the chat completion API when you provide audio.

[jaryl]

Just some heads up, support for audio in the Responses API might be coming:

Multimodal from the ground up. Text, images, audio, function calls—all first-class citizens. We didn’t bolt modalities onto a text API; we designed the house with enough bedrooms from day one.
Source: https://developers.openai.com/blog/responses-api/

Don't see it in the docs yet.

[rikkiprince]

I'm interested in Responses API support so we can more easily connect to remote MCP servers.

[radanskoric]

While there seem to be very few technical reasons to switch to the Responses API, it looks like OpenAI is set on pushing it hard. Ignoring the reasons why they're doing this (this is not the place to discuss that) I'm worried that they will start to release new features only on ResponsesAPI in order to force people to switch. I doubt they will deprecate the Completions API but I wouldn't be surprised if they decide to "leave it behind". In that case it would be good to have RubyLLM support ResponsesAPI at least as an optional non-default.

I realise it's a problem due to RubyLLM not being designed to have multiple APIs per provider but is that a direction with less friction? Perhaps "OpenAIResponses" could be introduced as a new separate provider, with OpenAI provider still defaulting to Completions. I know that has some potential for confusion but at least it would make it very explicit that you need to intentionally switch to the Responses API.

[tpaulshippy]

What are the scenarios, aside from audio inputs, where the completions API would provide value? I could change this to offer both options but I'm just not sure why.

[tpaulshippy]

Just some heads up, support for audio in the Responses API might be coming:

Kind of ridiculous that "Coming soon" has been so long.

[radanskoric]

What are the scenarios, aside from audio inputs, where the completions API would provide value? I could change this to offer both options but I'm just not sure why.

I don't think there's any. But there are also very few technical reasons why you'd need to switch to ResponsesAPI, other than OpenAI obviously being intent on pushing it hard as the default choice. Which I think is a good enough reason to add it to RubyLLM.

It's kind of annoying that Audio is not supported because otherwise this could be a change to just switch the RubyLLM backend from Completion to Responses. And I don't think it's an option to just not support audio in RubyLLM so supporting both seem like the only option going forward.

I'm wondering if a way forward would be to keep Responses vs Completions API as an implementation detail: Use Responses by default and transparently fallback to Completions with audio? This would remove the need to refactor RubyLLM to support multiple implementations per provider. It would be just one implementation that happens to use both APIs under the hood.

[tpaulshippy]

Use Responses by default and transparently fallback to Completions with audio?

That was my intent with the current state of this PR. Are you seeing something missing or different?

[radanskoric]

That was my intent with the current state of this PR. Are you seeing something missing or different?

Sorry! I noticed this PR some time ago and missed that development in the meantime. I see now the @using_responses_api = !audio_input?(messages) part. Thank you for pointing it out, that's exactly what I was suggesting. 👏

[IAPark]

I'm a little confused to hear that there aren't any compelling reasons to want to use the response API. Better support for reasoning + being able to use web search while also using function calling seem like big advantages.

[thomaswitt]

@tpaulshippy Thank you for your work.

To be honest, I don't understand why this isn't merged by @crmne. Our reason to switch to ruby_llm was mainly the better support of modern APIs and I honestly can't quite understand why this isn't a priority.

1. OpenAI clearly says (in bold): While Chat Completions remains supported, Responses is recommended for all new projects. So clearly, this is a deprecated API
2. We measured about 10-30% better performance in our internal tests with more complex prompts combined with structured outputs
3. We measured significantly less erratic responses with the new API. This is especially relevant to repetitions in arrays - which can happen according the docs, but with the old API you sometimes get thousands of tokens of the same value. That happens significantly less with the the new API.

So unless there is a reson I am not seeing, I am strongly urging to merge this PR and use the recommended API. It's just a question of time when a new model won't be supported anymore with the Chat Responses API.

[MirkoMignini]

@tpaulshippy thank you for your work on this PR, are you planning to rebase it and clean conflicts? it's a great addition to ruby_llm and hopefully @crmne will merge it when ready! thanks again to both

[tpaulshippy]

@tpaulshippy thank you for your work on this PR, are you planning to rebase it and clean conflicts? it's a great addition to ruby_llm and hopefully @crmne will merge it when ready! thanks again to both

Started taking a look. Need to work a bit on the reasoning part and get cassettes updated. Should be able to push something here soon.

[tpaulshippy]

Did what I could. Had some malloc issues updating cartridges on a couple of the PDF/image specs. Also noticed that we aren't really streaming the reasoning information from OpenAI, but I don't see it happening in main branch either - just getting a bunch of nils.

[thomaswitt]

Here are the first features only available in the responses API:

https://x.com/openaidevs/status/2021286050623373500

Ceterum censeo ruby_LLM API responsorum sustinere debere, @crmne

[cktricky]

Super keen to see this shipped ❤️