diff --git a/fern/assistants/assistant-hooks.mdx b/fern/assistants/assistant-hooks.mdx index 58156bdd8..54760f4e1 100644 --- a/fern/assistants/assistant-hooks.mdx +++ b/fern/assistants/assistant-hooks.mdx @@ -15,7 +15,7 @@ Supported events include: - `customer.speech.interrupted`: When the customer's speech is interrupted - `customer.speech.timeout`: When the customer doesn't speak within a specified time -You can combine actions and add filters to control when hooks trigger. +You can combine actions and add filters to control when hooks trigger. Multiple `customer.speech.timeout` hooks can be attached to an assistant with staggered trigger delay to support different actions at different timing in the conversation. ## How hooks work @@ -145,6 +145,10 @@ Perform multiple actions—say a message, call a function, and transfer the call }] } ``` + + +Use `"oneOf": ["pipeline-error"]` as a catch-all filter for any pipeline-related error reason. + ## Example: Handle speech interruptions @@ -205,9 +209,73 @@ The `customer.speech.timeout` hook supports special options: - `triggerResetMode`: Whether to reset the trigger count when user speaks (default: "never") - -Use `"oneOf": ["pipeline-error"]` as a catch-all filter for any pipeline-related error reason. - +## Example: End call if user hasn't spoken for 30s + +Assistant checks with the user at the 10 and 20s mark from when the user is silent, and ends the call after 30s of silence. + +```json +{ + "hooks": [ + { + "hooks": [ + { + "on": "customer.speech.timeout", + "options": { + "timeoutSeconds": 10, + "triggerMaxCount": 3, + "triggerResetMode": "onUserSpeech" + }, + "do": [ + { + "type": "say", + "exact": "Are you still there? Please let me know how I can help you." + } + ] + }, + { + "on": "customer.speech.timeout", + "options": { + "timeoutSeconds": 20, + "triggerMaxCount": 3, + "triggerResetMode": "onUserSpeech" + }, + "do": [ + { + "type": "say", + "prompt": "The user has not responded in 20s. Based on the above conversation in {{transcript}} ask the user if they need help or if not you will be ending the call" + } + ] + } + ] + }, + { + "hooks": [ + { + "on": "customer.speech.timeout", + "options": { + "timeoutSeconds": 30, + "triggerMaxCount": 3, + "triggerResetMode": "onUserSpeech" + }, + "do": [ + { + "type" : "say", + "exact" : "I'll be ending the call now, please feel free to call back at any time." + }, + { + "type": "tool", + "tool": { + "type": "endCall" + } + } + ] + } + ] + } + ] +} +``` + ## Common use cases diff --git a/fern/assistants/idle-messages.mdx b/fern/assistants/idle-messages.mdx index 3fb113c9b..f3dd7608b 100644 --- a/fern/assistants/idle-messages.mdx +++ b/fern/assistants/idle-messages.mdx @@ -6,7 +6,7 @@ slug: assistants/idle-messages ## Overview -Idle messages automatically prompt users during periods of inactivity to maintain engagement and reduce call abandonment. They work alongside silence timeout messages to handle conversation flow during calls. +Idle messages automatically prompt users during periods of inactivity to maintain engagement and reduce call abandonment. They are set up using [Assistant Hooks](/assistants/assistant-hooks) to trigger on customer silence timeout, and can be configured to say exact messages or use a model-generated message based on the conversation history. **Idle messages help you:** @@ -21,14 +21,14 @@ Idle messages automatically prompt users during periods of inactivity to maintai ## How idle messages work -When a user stops speaking, Vapi starts a timer. After the configured timeout period, it randomly selects and speaks one of your idle messages. This process repeats until either the user responds or the maximum message count is reached. +When a user stops speaking, Vapi starts a timer. Based on the configured timeout periods in `customer.speech.timeout` hooks, the assistant will trigger the action, which can be configured to say messages to the user. Timer starts when user stops speaking - Random message plays after timeout + Fetches a message to say to the user Counter resets when user responds (optional) @@ -37,7 +37,7 @@ When a user stops speaking, Vapi starts a timer. After the configured timeout pe ## Configuration -Configure idle messages in your assistant's `messagePlan`: +Configure idle messages using [Assistant Hooks](/assistants/assistant-hooks). Use the `customer.speech.timeout` hook to send a message when the user is silent for a specified period: ```typescript title="TypeScript (Server SDK)" @@ -45,41 +45,63 @@ import { VapiClient } from "@vapi-ai/server-sdk"; const client = new VapiClient({ token: process.env.VAPI_API_KEY }); -const assistant = await client.assistants.create({ -name: "Support Assistant", -messagePlan: { -idleMessages: [ -"Are you still there?", -"Can I help you with anything else?", -"I'm here whenever you're ready to continue." -], -idleTimeoutSeconds: 15, -idleMessageMaxSpokenCount: 3, -idleMessageResetCountOnUserSpeechEnabled: true -} +await client.assistants.create({ + name: "Support Assistant", + hooks: [ + { + on: "customer.speech.timeout", + options: { + timeoutSeconds: 10, + triggerMaxCount: 3, + triggerResetMode: "onUserSpeech" + }, + do: [ + { + type: "say", + exact: [ + "Are you still there?", + "Can I help you with anything else?", + "I'm here whenever you're ready to continue." + ] + } + ], + name: "idle_message_check" + } + ] }); - -```` +``` ```python title="Python (Server SDK)" from vapi import Vapi +import os client = Vapi(token=os.getenv("VAPI_API_KEY")) assistant = client.assistants.create( name="Support Assistant", - message_plan={ - "idle_messages": [ - "Are you still there?", - "Can I help you with anything else?", - "I'm here whenever you're ready to continue." - ], - "idle_timeout_seconds": 15, - "idle_message_max_spoken_count": 3, - "idle_message_reset_count_on_user_speech_enabled": True - } + hooks=[ + { + "on": "customer.speech.timeout", + "options": { + "timeoutSeconds": 10, + "triggerMaxCount": 3, + "triggerResetMode": "onUserSpeech" + }, + "do": [ + { + "type": "say", + "exact": [ + "Are you still there?", + "Can I help you with anything else?", + "I'm here whenever you're ready to continue." + ] + } + ], + "name": "idle_message_check" + } + ] ) -```` +``` ```bash title="cURL" curl -X POST "https://api.vapi.ai/assistant" \ @@ -87,66 +109,117 @@ curl -X POST "https://api.vapi.ai/assistant" \ -H "Content-Type: application/json" \ -d '{ "name": "Support Assistant", - "messagePlan": { - "idleMessages": [ - "Are you still there?", - "Can I help you with anything else?", - "I'"'"'m here whenever you'"'"'re ready to continue." - ], - "idleTimeoutSeconds": 15, - "idleMessageMaxSpokenCount": 3, - "idleMessageResetCountOnUserSpeechEnabled": true - } + "hooks": [{ + "on": "customer.speech.timeout", + "options": { + "timeoutSeconds": 10, + "triggerMaxCount": 3, + "triggerResetMode": "onUserSpeech" + }, + "do": [{ + "type": "say", + "exact": [ + "Are you still there?", + "Can I help you with anything else?", + "I am here whenever you are ready to continue." + ] + }], + "name": "idle_message_check" + }] }' ``` + + Learn more about hook options and actions in **[Assistant hooks](/assistants/assistant-hooks)**. + + ## Configuration options -### Core settings +### Timeout and triggering + +| Option | Type | Range | Default | Description | +| --------------------- | -------- | --------------- | --------- | ----------- | +| `timeoutSeconds` | number | 1-1000 seconds | 7.5 | How long to wait for customer speech before triggering | +| `triggerMaxCount` | number | 1-10 | 3 | Maximum times the timeout hook triggers per call | +| `triggerResetMode` | string | `never`\|`onUserSpeech` | `never` | Whether to reset the trigger count when the user speaks | -| Parameter | Type | Range | Default | Description | -| ------------------------------------------ | ---------- | ---------------- | ----------- | ----------------------------------------- | -| `idleMessages` | `string[]` | ≤1000 chars each | `undefined` | Array of messages to randomly select from | -| `idleTimeoutSeconds` | `number` | 5-60 seconds | 10 | Timeout before triggering first message | -| `idleMessageMaxSpokenCount` | `number` | 1-10 messages | 3 | Maximum times to repeat messages | -| `idleMessageResetCountOnUserSpeechEnabled` | `boolean` | - | `false` | Reset count when user speaks | +### Message action + +| Field | Type | Description | +| ------------ | ----------------- | ----------- | +| `say.exact` | string or string[]| Speak one of the provided messages verbatim | +| `say.prompt` | string | Use a model-generated response based on the given prompt and context (e.g., `{{transcript}}`) | ### Advanced configuration - + ```json { - "messagePlan": { - "idleMessages": ["Are you still there?"], - "idleTimeoutSeconds": 10 - } + "hooks": [ + { + "on": "customer.speech.timeout", + "options": { "timeoutSeconds": 10 }, + "do": [{ "type": "say", "exact": "Are you still there?" }] + } + ] } ``` ```json { - "messagePlan": { - "idleMessages": ["Hello, are you there?"], - "idleTimeoutSeconds": 15, - "idleMessageMaxSpokenCount": 5, - "idleMessageResetCountOnUserSpeechEnabled": true - } + "hooks": [ + { + "on": "customer.speech.timeout", + "options": { + "timeoutSeconds": 15, + "triggerMaxCount": 5, + "triggerResetMode": "onUserSpeech" + }, + "do": [{ "type": "say", "exact": "Hello, are you there?" }] + } + ] } ``` - + ```json { - "messagePlan": { - "idleMessages": ["Can I help you with anything else?"], - "idleTimeoutSeconds": 20, - "silenceTimeoutMessage": "I'll end our call now. Thank you!" - }, - "silenceTimeoutSeconds": 60 + "hooks": [ + { + "on": "customer.speech.timeout", + "options": { + "timeoutSeconds": 10, + "triggerMaxCount": 3, + "triggerResetMode": "onUserSpeech" + }, + "do": [{ "type": "say", "exact": "Are you still there? Please let me know how I can help you." }] + }, + { + "on": "customer.speech.timeout", + "options": { + "timeoutSeconds": 20, + "triggerMaxCount": 3, + "triggerResetMode": "onUserSpeech" + }, + "do": [{ "type": "say", "prompt": "The user has not responded in 20s. Based on the above conversation in {{transcript}} ask the user if they need help or if not you will be ending the call" }] + }, + { + "on": "customer.speech.timeout", + "options": { + "timeoutSeconds": 30, + "triggerMaxCount": 3, + "triggerResetMode": "onUserSpeech" + }, + "do": [ + { "type": "say", "exact": "I'll be ending the call now, please feel free to call back at any time." }, + { "type": "tool", "tool": { "type": "endCall" } } + ] + } + ] } ``` @@ -154,53 +227,49 @@ curl -X POST "https://api.vapi.ai/assistant" \ ## Multilingual support -Handle multiple languages by creating language-specific assistants or dynamically updating messages: +Handle multiple languages by creating language-specific assistants or dynamically configuring hook messages: ```typescript // English assistant - const enAssistant = await client.assistants.create({ + await client.assistants.create({ name: "EN Support", - messagePlan: { - idleMessages: [ + hooks: [{ + on: "customer.speech.timeout", + options: { timeoutSeconds: 10 }, + do: [{ type: "say", exact: [ "Are you still there?", "Can I help you with anything else?" - ] - } + ] }] + }] }); // Spanish assistant - const esAssistant = await client.assistants.create({ + await client.assistants.create({ name: "ES Support", - messagePlan: { - idleMessages: [ + hooks: [{ + on: "customer.speech.timeout", + options: { timeoutSeconds: 10 }, + do: [{ type: "say", exact: [ "¿Sigues ahí?", "¿Puedo ayudarte con algo más?" - ] - } + ] }] + }] }); ``` ```typescript - async function updateIdleMessagesForLanguage( - assistantId: string, - detectedLanguage: string - ) { - const languageMessages = { - en: ['Are you still there?', 'Can I help you with anything else?'], - es: ['¿Sigues ahí?', '¿Puedo ayudarte con algo más?'], - fr: ['Êtes-vous toujours là?', 'Puis-je vous aider avec autre chose?'] - }; - - await client.assistants.update(assistantId, { - messagePlan: { - idleMessages: languageMessages[detectedLanguage] || languageMessages['en'] - } - }); - } + await client.assistants.create({ + name: "Multi Support", + hooks: [{ + on: "customer.speech.timeout", + options: { timeoutSeconds: 10 }, + do: [{ type: "say", prompt: "Based on the above conversation in {{transcript}} ask the user in the language of the conversation if they need help or if not you will be ending the call. If no conversation is provided, default to English." }] + }] + }); ``` @@ -240,47 +309,29 @@ Choose timeout duration based on your use case: -### Frequency management - -Balance engagement with user experience: - -```json -{ - "idleMessageMaxSpokenCount": 2, - "idleMessageResetCountOnUserSpeechEnabled": true, - "idleTimeoutSeconds": 15 -} -``` - - - Enable `idleMessageResetCountOnUserSpeechEnabled` to give users multiple - chances to engage throughout long conversations. - - ## Troubleshooting ### Messages not triggering - Check that idle messages are properly configured: + Check that idle messages are properly configured in hooks: ```typescript const assistant = await client.assistants.get(assistantId); - console.log('Idle config:', assistant.messagePlan); + console.log('Hook config:', assistant.hooks); ``` - - Account for audio processing delays (2-3 seconds): - ```json - { "idleTimeoutSeconds": 12 } - ``` + + Account for message generation time (1-2 seconds) if using `say.prompt`: - Ensure the maximum count hasn't been reached: + Ensure the maximum count hasn't been reached. If the maximum count is reached, you will see a log in the call logs that mentions hook not triggered because of max count. ```json { - "idleMessageMaxSpokenCount": 5, - "idleMessageResetCountOnUserSpeechEnabled": true + "options": { + "triggerMaxCount": 5, + "triggerResetMode": "onUserSpeech" + } } ``` @@ -307,16 +358,16 @@ Balance engagement with user experience: - **Solution:** This shouldn't happen - idle messages are automatically disabled during tool calls and transfers. If it persists, contact support. + **Solution:** This shouldn't happen - all hooks are automatically disabled during tool calls and transfers. If it persists, contact support. ## Limitations -- **Static content**: Messages cannot be dynamically generated based on conversation context -- **No context awareness**: Messages don't adapt to the current conversation topic -- **Character limits**: Each message is limited to 1000 characters -- **Processing delays**: Account for 2-3 seconds of audio processing time in your timeout settings +- **Generative variability**: Using `say.prompt` produces model-generated text that may vary; use `say.exact` for strict control +- **Trigger limits**: `triggerMaxCount` caps how many times the timeout hook fires per call (1-10) +- **Timeout range**: `timeoutSeconds` supports 1-1000 seconds (default ~7.5s); account for processing delays +- **Processing delays**: Allow 2-3 seconds of audio processing time when choosing timeout values ## Next steps