From ce62e94198d3ee72186d80b1703dddb20ce80c65 Mon Sep 17 00:00:00 2001 From: Bryant Date: Mon, 11 Nov 2024 17:43:18 +1100 Subject: [PATCH 1/2] add realtime api docs --- fern/docs.yml | 2 ++ fern/openai-realtime.mdx | 14 ++++++++++++++ 2 files changed, 16 insertions(+) create mode 100644 fern/openai-realtime.mdx diff --git a/fern/docs.yml b/fern/docs.yml index c5eb56f7c..f44fa7be6 100644 --- a/fern/docs.yml +++ b/fern/docs.yml @@ -242,6 +242,8 @@ navigation: path: tools-calling.mdx - page: Prompting Guide path: prompting-guide.mdx + - page: OpenAI Realtime + path: openai-realtime.mdx - section: Glossary contents: - page: Definitions diff --git a/fern/openai-realtime.mdx b/fern/openai-realtime.mdx new file mode 100644 index 000000000..dedfbdd38 --- /dev/null +++ b/fern/openai-realtime.mdx @@ -0,0 +1,14 @@ +--- +title: OpenAI Realtime +subtitle: You can use OpenAI's newest speech-to-speech model with your Vapi assistants. +slug: openai-realtime +--- + +OpenAI's Realtime API enables developers to create low-latency, voice-driven interactions within applications by leveraging a speech-to-speech model. This API processes audio input and output in real-time, supporting seamless, natural conversations without intermediate text conversion. + +To start using it with your Vapi assistants, select `gpt-4o-realtime-preview-2024-10-01` as your model. +- Please note that only OpenAI voices may be selected while using this model. The voice selection will not act as a TTS (text-to-speech) model, but rather as the voice used within the speech-to-speech model. + + + The Realtime API is currently in beta, and not recommended for production use by OpenAI. We're excited to have you try this new feature and welcome your [feedback](https://discord.com/invite/pUFNcf2WmH) as we continue to refine and improve the experience. + \ No newline at end of file From 89536d0401f9eced6b928c621ed2eec232181346 Mon Sep 17 00:00:00 2001 From: Bryant Date: Mon, 11 Nov 2024 18:21:42 +1100 Subject: [PATCH 2/2] nits --- fern/openai-realtime.mdx | 12 +++++++----- 1 file changed, 7 insertions(+), 5 deletions(-) diff --git a/fern/openai-realtime.mdx b/fern/openai-realtime.mdx index dedfbdd38..e82ac8bee 100644 --- a/fern/openai-realtime.mdx +++ b/fern/openai-realtime.mdx @@ -4,11 +4,13 @@ subtitle: You can use OpenAI's newest speech-to-speech model with your Vapi assi slug: openai-realtime --- -OpenAI's Realtime API enables developers to create low-latency, voice-driven interactions within applications by leveraging a speech-to-speech model. This API processes audio input and output in real-time, supporting seamless, natural conversations without intermediate text conversion. + + The Realtime API is currently in beta, and not recommended for production use by OpenAI. We're excited to have you try this new feature and welcome your [feedback](https://discord.com/invite/pUFNcf2WmH) as we continue to refine and improve the experience. + + +OpenAI’s Realtime API enables developers to use a native speech-to-speech model. Unlike other Vapi configurations which orchestrate a transcriber, model and voice API to simulate speech-to-speech, OpenAI’s Realtime API natively processes audio in and audio out. To start using it with your Vapi assistants, select `gpt-4o-realtime-preview-2024-10-01` as your model. - Please note that only OpenAI voices may be selected while using this model. The voice selection will not act as a TTS (text-to-speech) model, but rather as the voice used within the speech-to-speech model. - - - The Realtime API is currently in beta, and not recommended for production use by OpenAI. We're excited to have you try this new feature and welcome your [feedback](https://discord.com/invite/pUFNcf2WmH) as we continue to refine and improve the experience. - \ No newline at end of file +- Also note that we don’t currently support Knowledge Bases with the Realtime API. +- Lastly, note that our Realtime integration still retains the rest of Vapi's orchestration layer such as the endpointing and interruption models to enable a reliable conversational flow. \ No newline at end of file