Improve the types of RealtimeSession configuration #96
Conversation
seratch
commented
Jun 12, 2025
seratch
commented
- inputAudioTranscription
- turnDetection
🦋 Changeset detectedLatest commit: 7d248d3 The changes in this PR will be included in the next version bump. This PR includes changesets to release 3 packages
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
| turnDetection: { | ||
| type: 'semantic_vad', | ||
| eagerness: 'medium', | ||
| create_response: true, |
There was a problem hiding this comment.
changed the example but either works!
|
|
||
| export type RealtimeInputAudioTranscriptionConfig = { | ||
| language?: string; | ||
| model?: 'gpt-4o-transcribe' | 'gpt-4o-mini-transcribe' | 'whisper-1' | string; |
There was a problem hiding this comment.
allowed this property to pass anything else as we may release new models in the future (plus alpha users may use different ones)
There was a problem hiding this comment.
We should do string & {} for better type autocomplete
| return { | ||
| type: c.type, | ||
| create_response: | ||
| 'createResponse' in c |
There was a problem hiding this comment.
if snake_case then camelCase is preferred, happy to change this order
seratch
force-pushed
the
voice-agent-config-types
branch
from
ed4b4de to
f86d0fc
Compare
seratch
changed the title
|
|
||
| // The Realtime API accepts snake_cased keys, so when using this, this SDK coverts the keys to snake_case ones before passing it to the API | ||
| export type RealtimeTurnDetectionConfigCamelCase = { | ||
| type?: 'semantic_vad'; |
There was a problem hiding this comment.
Type can also be server_vad so we should support both.
| prefixPaddingMs?: number; | ||
| silenceDurationMs?: number; | ||
| threshold?: number; | ||
| }; |
There was a problem hiding this comment.
Can we make it acceptable for this to also still take other properties inside of these two settings? Thinking how theoretically you could roll your own Realtime Transport Layer right now with other session config. But also fine to guide people to providerData for that and override this entire property
dkundel-openai
left a comment
dkundel-openai
left a comment
There was a problem hiding this comment.
See open comments
seratch
force-pushed
the
voice-agent-config-types
branch
from
f86d0fc to
7d248d3
Compare
| prefixPaddingMs?: number; | ||
| silenceDurationMs?: number; | ||
| threshold?: number; | ||
| }; |
| threshold, | ||
| ...rest, | ||
| }; | ||
| // Remove undefined values from the config |
There was a problem hiding this comment.
When I verified the behavior, having undefined values could affect the connection establishment, so I added this logic. but if my observation is wrong or is missing something, please feel free to adjust this part.
| const item = event.response.output[event.response.output.length - 1]; | ||
| const textOutput = getLastTextFromAudioOutputMessage(item) ?? ''; | ||
| const itemId = item.id ?? ''; | ||
| const itemId = item?.id ?? ''; |
There was a problem hiding this comment.
This is an unrelated existing bug i found while doing tests
