Custom Realtime Model Selection

[AnasMations]

This PR introduces the ability to select a custom OpenAI realtime model in the RealtimeClient. Users can now specify the desired model for more flexibility and tailored API usage.

Changes:

• Added support for passing a custom model name to RealtimeClient through the updateSession function

Let me know if there are any further changes or enhancements needed!

[jjmlovesgit]

An excellent and most valuable feature -- Recognition and support to Anas for this work!

[scottfits]

@khorwood-openai or @jhills20 - would you mind taking a look at this?

[mrmuke]

+1

[hawkeye-sama]

if I'm not wrong, it can be done like this? this.realtimeModel = model || 'gpt-4o-realtime-preview-2024-10-01';

[devpras22]

@AnasMations

I’ve been testing your PR #92, which adds support for passing a custom model name via updateSession(), but I’m running into an issue where OpenAI still defaults to gpt-4o-realtime-preview-2024-10-01, even when explicitly setting gpt-4o-mini-realtime-preview-2024-12-17.

🔍 Findings So Far:

1️⃣ Model Name Appears in realtimeModel But Not in Active Session
• When I initialize RealtimeClient({ model: "gpt-4o-mini-realtime-preview-2024-12-17" }), the model correctly appears in client.realtimeModel.
• However, it does not get applied in sessionConfig, meaning OpenAI might not be using it during session creation.

2️⃣ Explicitly Calling updateSession({ model: "gpt-4o-mini-realtime-preview-2024-12-17" }) Still Doesn’t Work
• Even after forcing client.sessionConfig.model = client.realtimeModel and calling updateSession(client.sessionConfig), OpenAI continues defaulting to gpt-4o-realtime-preview-2024-10-01.
• Logs confirm that the session is updating, but the API dashboard still shows the old model being used.

3️⃣ Potential Issue with WebSocket Model Application
• OpenAI might not be honoring the model field in updateSession().
• It’s unclear if the WebSocket API requires a separate "session.update" event for model selection rather than updateSession().

🚀 Next Steps

Can you confirm if:
• OpenAI actually allows model switching via updateSession()?
• A direct WebSocket message (client.realtime.send("session.update", {...})) might be required instead?
• There’s a known issue with session config updates not applying to active requests?

Any guidance would be appreciated! Thanks for your work on this PR—it’s a much-needed feature, and I’d love to help get it fully functional.

[devpras22]

did this actually work for people? is model selection actually functional?

I hardcoded the 4o mini model here - if (!model) {
this.realtimeModel = 'gpt-4o-mini-realtime-preview-2024-12-17';
} else {
this.realtimeModel = model;
} - BUT STILL it reverts to the default 4o model (the old one from 24/10/01) as seen on the openai dashboard.

[djfm]

@AnasMations lol we're about 20 people to have fixed this issue and none of the PRs were accepted 😅. Surely one must be to OpenAI's taste at least...

Eventhough it was a 5 minutes fix tops many of us should probably have looked at the open PR's before making a new, functionally identical one ahah.

@devpras22 the official library no, it's still not gonna work. But pick any of the forks which 1) mention in the title that they fix this issue and 2) are not behind the main branch of OpenAI in terms of commits.

yarn - and probably all other node package managers too now have a way to install a package directly from github. Look at how to do that with the package manager you're using and make it install the library not from OpenAI but from one of the like, 20 forks ?, which fix the same issue...