diff --git a/fern/calls/call-handling-with-vapi-and-twilio.mdx b/fern/calls/call-handling-with-vapi-and-twilio.mdx new file mode 100644 index 000000000..12bf06f58 --- /dev/null +++ b/fern/calls/call-handling-with-vapi-and-twilio.mdx @@ -0,0 +1,274 @@ +This document explains how to handle a scenario where a user is on hold while the system attempts to connect them to a specialist. If the specialist does not pick up within X seconds or if the call hits voicemail, we take an alternate action (like playing an announcement or scheduling an appointment). This solution integrates Vapi.ai for AI-driven conversations and Twilio for call bridging. + +## Problem + +Vapi.ai does not provide a built-in way to keep the user on hold, dial a specialist, and handle cases where the specialist is unavailable. We want: + +1. The user already talking to the AI (Vapi). +2. The AI offers to connect them to a specialist. +3. The user is placed on hold or in a conference room. +4. We dial the specialist to join. +5. If the specialist answers, everyone is merged. +6. If the specialist does not answer (within X seconds or goes to voicemail), we want to either announce "Specialist not available" or schedule an appointment. + +## Solution + +1. An inbound call arrives from Vapi or from the user directly. +2. We store its details (e.g., Twilio CallSid). +3. We send TwiML (or instructions) to put the user in a Twilio conference (on hold). +4. We place a second call to the specialist, also directed to join the same conference. +5. If the specialist picks up, Twilio merges the calls. +6. If not, we handle the no-answer event by playing a message or returning control to the AI for scheduling. + +## Steps to Solve the Problem + +1. **Receive Inbound Call** + + - Twilio posts data to your `/inbound_call`. + - You store the call reference. + - You might also invoke Vapi for initial AI instructions. + +2. **Prompt User via Vapi** + + - The user decides whether they want the specialist. + - If yes, you call an endpoint (e.g., `/connect`). + +3. **Create/Join Conference** + + - In `/connect`, you update the inbound call to go into a conference route. + - The user is effectively on hold. + +4. **Dial Specialist** + + - You create a second call leg to the specialist’s phone. + - A `statusCallback` can detect no-answer or voicemail. + +5. **Detect Unanswered** + + - If Twilio sees a no-answer or failure, your callback logic plays an announcement or signals the AI to schedule an appointment. + +6. **Merge or Exit** + + - If the specialist answers, they join the user. + - If not, the user is taken off hold and the call ends or goes back to AI. + +7. **Use Ephemeral Call (Optional)** + - If you need an in-conference announcement, create a short-lived Twilio call that `` the message to everyone, then ends the conference. + +## Code Example + +Below is a minimal Express.js server aligned for On-Hold Specialist Transfer with Vapi and Twilio. + +1. **Express Setup and Environment** + +```js +const express = require("express"); +const bodyParser = require("body-parser"); +const axios = require("axios"); +const twilio = require("twilio"); + +const app = express(); +app.use(bodyParser.urlencoded({ extended: true })); +app.use(bodyParser.json()); + +// Load important env vars +const { + TWILIO_ACCOUNT_SID, + TWILIO_AUTH_TOKEN, + FROM_NUMBER, + TO_NUMBER, + VAPI_BASE_URL, + PHONE_NUMBER_ID, + ASSISTANT_ID, + PRIVATE_API_KEY, +} = process.env; + +// Create a Twilio client +const client = twilio(TWILIO_ACCOUNT_SID, TWILIO_AUTH_TOKEN); + +// We'll store the inbound call SID here for simplicity +let globalCallSid = ""; +``` + +2. **`/inbound_call` - Handling the Inbound Call** + +```js +app.post("/inbound_call", async (req, res) => { + try { + globalCallSid = req.body.CallSid; + const caller = req.body.Caller; + + // Example: We call Vapi.ai to get initial TwiML + const response = await axios.post( + `${VAPI_BASE_URL || "https://api.vapi.ai"}/call`, + { + phoneNumberId: PHONE_NUMBER_ID, + phoneCallProviderBypassEnabled: true, + customer: { number: caller }, + assistantId: ASSISTANT_ID, + }, + { + headers: { + Authorization: `Bearer ${PRIVATE_API_KEY}`, + "Content-Type": "application/json", + }, + } + ); + + const returnedTwiml = response.data.phoneCallProviderDetails.twiml; + return res.type("text/xml").send(returnedTwiml); + } catch (err) { + return res.status(500).send("Internal Server Error"); + } +}); +``` + +3. **`/connect` - Putting User on Hold and Dialing Specialist** + +```js +app.post("/connect", async (req, res) => { + try { + const protocol = + req.headers["x-forwarded-proto"] === "https" ? "https" : "http"; + const baseUrl = `${protocol}://${req.get("host")}`; + const conferenceUrl = `${baseUrl}/conference`; + + // 1) Update inbound call to fetch TwiML from /conference + await client.calls(globalCallSid).update({ + url: conferenceUrl, + method: "POST", + }); + + // 2) Dial the specialist + const statusCallbackUrl = `${baseUrl}/participant-status`; + + await client.calls.create({ + to: TO_NUMBER, + from: FROM_NUMBER, + url: conferenceUrl, + method: "POST", + statusCallback: statusCallbackUrl, + statusCallbackMethod: "POST", + }); + + return res.json({ status: "Specialist call initiated" }); + } catch (err) { + return res.status(500).json({ error: "Failed to connect specialist" }); + } +}); +``` + +4. **`/conference` - Placing Callers Into a Conference** + +```js +app.post("/conference", (req, res) => { + const VoiceResponse = twilio.twiml.VoiceResponse; + const twiml = new VoiceResponse(); + + // Put the caller(s) into a conference + const dial = twiml.dial(); + dial.conference( + { + startConferenceOnEnter: true, + endConferenceOnExit: true, + }, + "my_conference_room" + ); + + return res.type("text/xml").send(twiml.toString()); +}); +``` + +5. **`/participant-status` - Handling No-Answer or Busy** + +```js +app.post("/participant-status", async (req, res) => { + const callStatus = req.body.CallStatus; + if (["no-answer", "busy", "failed"].includes(callStatus)) { + console.log("Specialist did not pick up:", callStatus); + // Additional logic: schedule an appointment, ephemeral call, etc. + } + return res.sendStatus(200); +}); +``` + +6. **`/announce` (Optional) - Ephemeral Announcement** + +```js +app.post("/announce", (req, res) => { + const VoiceResponse = twilio.twiml.VoiceResponse; + const twiml = new VoiceResponse(); + twiml.say("Specialist is not available. Ending call now."); + + // Join the conference, then end it. + twiml.dial().conference( + { + startConferenceOnEnter: true, + endConferenceOnExit: true, + }, + "my_conference_room" + ); + + return res.type("text/xml").send(twiml.toString()); +}); +``` + +7. **Starting the Server** + +```js +app.listen(3000, () => { + console.log("Server running on port 3000"); +}); +``` + +## How to Test + +1. **Environment Variables** + Set `TWILIO_ACCOUNT_SID`, `TWILIO_AUTH_TOKEN`, `FROM_NUMBER`, `TO_NUMBER`, `VAPI_BASE_URL`, `PHONE_NUMBER_ID`, `ASSISTANT_ID`, and `PRIVATE_API_KEY`. + +2. **Expose Your Server** + + - Use a tool like `ngrok` to create a public URL to port 3000. + - Configure your Twilio phone number to call `/inbound_call` when a call comes in. + +3. **Place a Real Call** + + - Dial your Twilio number from a phone. + - Twilio hits `/inbound_call`, and run Vapi logic. + - Trigger `/connect` to conference the user and dial the specialist. + - If the specialist answers, they join the same conference. + - If they never answer, Twilio eventually calls `/participant-status`. + +4. **Use cURL for Testing** + - **Simulate Inbound**: + ```bash + curl -X POST https:///inbound_call \ + -F "CallSid=CA12345" \ + -F "Caller=+15551112222" + ``` + - **Connect**: + ```bash + curl -X POST https:///connect \ + -H "Content-Type: application/json" \ + -d "{}" + ``` + +## Note on Replacing "Connect" with Vapi Tools + +Vapi offers built-in functions or custom tool calls for placing a second call or transferring, you can replace the manual `/connect` call with that Vapi functionality. The flow remains the same: user is put in a Twilio conference, the specialist is dialed, and any no-answer events are handled. + +## Notes & Limitations + +1. **Voicemail** + If a phone’s voicemail picks up, Twilio sees it as answered. Consider advanced detection or a fallback. + +2. **Concurrent Calls** + Multiple calls at once require storing separate `CallSid`s or similar references. + +3. **Conference Behavior** + `startConferenceOnEnter: true` merges participants immediately; `endConferenceOnExit: true` ends the conference when that participant leaves. + +4. **X Seconds** + Decide how you detect no-answer. Typically, Twilio sets a final `callStatus` if the remote side never picks up. + +With these steps and code, you can integrate Vapi Assistant while using Twilio’s conferencing features to hold, dial out to a specialist, and handle an unanswered or unavailable specialist scenario. diff --git a/fern/docs.yml b/fern/docs.yml index 67b23a358..d09181116 100644 --- a/fern/docs.yml +++ b/fern/docs.yml @@ -254,6 +254,8 @@ navigation: path: calls/call-ended-reason.mdx - page: Live Call Control path: calls/call-features.mdx + - page: On-Hold Specialist Transfer + path: calls/call-handling-with-vapi-and-twilio.mdx - page: Voice Mail Detection path: calls/voice-mail-detection.mdx - section: SIP