From 46b7c4846b85c6f519abdbb509d8bde21b1b2714 Mon Sep 17 00:00:00 2001 From: Arthur Chi Date: Wed, 12 Nov 2025 22:20:50 -0800 Subject: [PATCH 1/4] squad updateS --- fern/assistants/call-recording.mdx | 2 + fern/squads.mdx | 157 +++++++++++++++++++++++++---- fern/tools/handoff.mdx | 1 + 3 files changed, 142 insertions(+), 18 deletions(-) diff --git a/fern/assistants/call-recording.mdx b/fern/assistants/call-recording.mdx index 86a8dd1b8..d05d31599 100644 --- a/fern/assistants/call-recording.mdx +++ b/fern/assistants/call-recording.mdx @@ -265,6 +265,8 @@ Control where each artifact type is stored: ### Dynamic Artifact Control +When handing off between assistants in a Squad, you may choose to change the local context for each assistant via the `contextEngineeringPlan`. By default, only the final context will be used in the artifact and analysis (Structured outputs and success evaluation). To include the full message history across all assistants in the call, set [`artifactPlan.fullMessageHistoryEnabled`](/api-reference/squads/create#request.body.membersOverrides.artifactPlan.fullMessageHistoryEnabled) to true. + In squads with multiple assistants, artifact generation (recording, logging, transcripts) can be controlled per assistant. When assistants are swapped or transferred during a call: - **Recording**: Pauses when `recordingEnabled: false` assistant is active, resumes when `recordingEnabled: true` assistant takes over diff --git a/fern/squads.mdx b/fern/squads.mdx index c8e6eb7f2..486517d69 100644 --- a/fern/squads.mdx +++ b/fern/squads.mdx @@ -4,12 +4,14 @@ subtitle: Use Squads to handle complex workflows and tasks. slug: squads --- -Sometimes, complex workflows are easier to manage with multiple assistants. -You can think of each assistant in a Squad as a leg of a conversation tree. -For example, you might have one assistant for lead qualification, which transfers to another for booking an appointment if they’re qualified. +Squads let you break complex workflows into multiple specialized assistants that hand off to each other during a conversation. Each assistant in a Squad handles a specific part of your workflow—like one assistant for lead qualification that transfers to another for appointment booking. -Prior to Squads you would put all functionality in one assistant, but Squads were added to break up the complexity of larger prompts into smaller specialized assistants with specific tools and fewer goals. -Squads enable calls to transfer assistants mid-conversation, while maintaining full conversation context. +**Why use Squads?** Large, all-in-one assistants with lengthy prompts and extensive context lead to: +- **Higher hallucination rates** - Models lose focus with too many and sometimes conflicting instructions +- **Increased costs** - Longer prompts consume more tokens per request +- **Greater latency** - Processing large contexts takes more time and will increase the latency of your assistant. + +Squads solve this by splitting complex prompts into focused assistants with specific tools and clear goals, while maintaining full conversation context across handoffs. View all configurable properties in the [API Reference](/api-reference/squads/create-squad). @@ -20,8 +22,11 @@ Squads enable calls to transfer assistants mid-conversation, while maintaining f To use Squads, you can create a `squad` when starting a call and specify `members` as a list of assistants and destinations. The first member is the assistant that will start the call, and assistants can be either persistent or transient. -Each assistant should be assigned the relevant assistant transfer destinations. -Transfers are specified by assistant name and are used when the model recognizes a specific trigger. +We recommend using [Handoff Tools](/tools/handoff) to specify which destinations the current assistant can handoff too, and when to handoff to each assistant. Each assistant within the squad can use its saved handoff tools as well as handoff tools from Assistant Overrides (see below). + + + ```json { @@ -29,17 +34,27 @@ Transfers are specified by assistant name and are used when the model recognizes "members": [ { "assistantId": "information-gathering-assistant-id", - "assistantDestinations": [{ - "type": "assistant", - "assistantName": "Appointment Booking", - "message": "Please hold on while I transfer you to our appointment booking assistant.", - "description": "Transfer the user to the appointment booking assistant after they say their name." - }], }, { "assistant": { "name": "Appointment Booking", - ... + "model": { + "provider": "openai", + "model": "gpt-4o", + "toolIds": ["handoff-tool-id"], + "tools": [ + { + "type": "handoff", + "destinations": [ + { + "type": "assistant", + "assistantId": "assistant-123", + "description": "Call this tool when the customer wants to talk about pricing" + } + ] + } + ] + }, }, } ] @@ -47,12 +62,118 @@ Transfers are specified by assistant name and are used when the model recognizes } ``` +## Overrides + +### Assistant Overrides +To override the configuration of a saved assistant without modifying the underlying assistant, use the `assistantsOverrides` to alter individual assistants. For example, if you have disparate assistants with different voices, you can use `assistantOverrides` to make sure all of the assistants are using the same voice without changing the assistant (in case its being used in another squad). + +```json +{ + "squad": { + "members": [ + { + "assistant": { + "name": "Appointment Booking", + "voice": { + "provider": "vapi", + "voiceId": "Elliot", + }, + }, + }, + { + "assistantId": "saved-assistant-id", + "assistantOverrides": { + "voice": { + "provider": "vapi", + "voiceId": "Elliot", + }, + } + }, + ] + } +} +``` + +You may also define inline tools via assistant overrides through the `model` object, so that the assistant will only handoff if it is a part of this squad. +```json +{ + "squad": { + "members": [ + { + "assistant": { + "name": "Appointment Booking", + "voice": { + "provider": "vapi", + "voiceId": "Elliot", + }, + }, + }, + { + "assistantId": "saved-assistant-id", + "assistantOverrides": { + "model": { + "provider": "openai", + "model": "gpt-4o", + "tools:append": [ + { + "type": "handoff", + "destinations": [ + { + "type": "assistant", + "assistantId": "assistant-123", + "description": "Call this tool when the customer wants to talk about pricing" + } + ] + } + ] + }, + } + }, + ] + } +} +``` + + +### Member Overrides +To override the configuration of _all_ assistants without modifying the underlying assistants, use the `memberOverrides`. + +```json +{ + "squad": { + "members": [ + { + "assistant": { + "name": "Appointment Booking", + "voice": { + "provider": "vapi", + "voiceId": "Elliot", + }, + }, + }, + { + "assistantId": "saved-assistant-id", + }, + ], + "memberOverrides": { + "voice": { + "provider": "vapi", + "voiceId": "Elliot", + }, + } + } +} +``` ## Best Practices -The following are some best practices for using Squads to reduce errors: +**Keep assistants focused** - Each assistant should have a single, well-defined responsibility with 1-3 goals maximum. Assign only the tools needed for that specific task. + +**Minimize squad size** - Try to reduce the number of squad members. Only split into separate assistants when there's a clear functional boundary (lead qualification → sales → booking). + +**Define clear handoff conditions** - Write specific handoff descriptions that state exact trigger conditions and what information to collect before transferring. Utilize [variable extraction](/tools/handoff#variable-extraction) to save information and generate summaries during a handoff to pass to other assistants. + +**Engineer context carefully** - Use [context engineering](/tools/handoff#context-engineering) to control what conversation history is passed between assistants. As the context grows throughout the call, you may want to limit message history to reduce tokens, improve performance, and prevent context poisoning. + -- Group assistants by closely related tasks -- Create as few assistants as possible to reduce complexity -- Make sure descriptions for transfers are clear and concise diff --git a/fern/tools/handoff.mdx b/fern/tools/handoff.mdx index 7300b341f..771e4877e 100644 --- a/fern/tools/handoff.mdx +++ b/fern/tools/handoff.mdx @@ -510,6 +510,7 @@ Override the default function definition for more control. You can overwrite the 3. **Model Optimization**: Use multiple tools for OpenAI, single tool for Anthropic 4. **Variable Extraction**: Extract key data before handoff to maintain context 5. **Testing**: Test handoff scenarios thoroughly, including edge cases +6. **Monitoring and Analysis**: Enable [`artifactPlan.fullMessageHistoryEnabled`](api-reference/assistants/create#response.body.artifactPlan.fullMessageHistoryEnabled) to capture the complete message history across all handoffs in your artifacts. See [squad artifact behavior](/assistants/call-recording#squad-and-transfer-behavior) for details. ## Troubleshooting From f2a0de33c1014e2ce944cbd4af40a16cd1b38583 Mon Sep 17 00:00:00 2001 From: Arthur Chi Date: Wed, 12 Nov 2025 22:47:48 -0800 Subject: [PATCH 2/4] add best practices --- fern/squads.mdx | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/fern/squads.mdx b/fern/squads.mdx index 486517d69..d153f091f 100644 --- a/fern/squads.mdx +++ b/fern/squads.mdx @@ -171,9 +171,9 @@ To override the configuration of _all_ assistants without modifying the underlyi **Minimize squad size** - Try to reduce the number of squad members. Only split into separate assistants when there's a clear functional boundary (lead qualification → sales → booking). -**Define clear handoff conditions** - Write specific handoff descriptions that state exact trigger conditions and what information to collect before transferring. Utilize [variable extraction](/tools/handoff#variable-extraction) to save information and generate summaries during a handoff to pass to other assistants. +**Define clear handoff conditions** - Write specific handoff descriptions that state exact trigger conditions and what information to collect before transferring. Make sure to specify this in the assistant's prompt and/or tool description. -**Engineer context carefully** - Use [context engineering](/tools/handoff#context-engineering) to control what conversation history is passed between assistants. As the context grows throughout the call, you may want to limit message history to reduce tokens, improve performance, and prevent context poisoning. +**Engineer context carefully** - Use [context engineering](/tools/handoff#context-engineering) to control what conversation history is passed between assistants. As the context grows throughout the call, you may want to limit message history to reduce tokens, improve performance, and prevent context poisoning. Utilize [variable extraction](/tools/handoff#variable-extraction) to save information and generate summaries during a handoff to pass to other assistants. From ab23b8f537b9a7180544326bc623f772d83295b0 Mon Sep 17 00:00:00 2001 From: Arthur Chi Date: Wed, 12 Nov 2025 23:20:21 -0800 Subject: [PATCH 3/4] second pass --- fern/squads.mdx | 13 ++++++++----- 1 file changed, 8 insertions(+), 5 deletions(-) diff --git a/fern/squads.mdx b/fern/squads.mdx index d153f091f..404804568 100644 --- a/fern/squads.mdx +++ b/fern/squads.mdx @@ -4,10 +4,10 @@ subtitle: Use Squads to handle complex workflows and tasks. slug: squads --- -Squads let you break complex workflows into multiple specialized assistants that hand off to each other during a conversation. Each assistant in a Squad handles a specific part of your workflow—like one assistant for lead qualification that transfers to another for appointment booking. +Squads let you break complex workflows into multiple specialized assistants that hand off to each other during a conversation. Each assistant in a Squad handles a specific part of your workflow; for example, one assistant for lead qualification that transfers to another for appointment booking. **Why use Squads?** Large, all-in-one assistants with lengthy prompts and extensive context lead to: -- **Higher hallucination rates** - Models lose focus with too many and sometimes conflicting instructions +- **Higher hallucination rates** - Models lose focus with too many and potentially conflicting instructions - **Increased costs** - Longer prompts consume more tokens per request - **Greater latency** - Processing large contexts takes more time and will increase the latency of your assistant. @@ -20,14 +20,14 @@ Squads solve this by splitting complex prompts into focused assistants with spec ## Usage To use Squads, you can create a `squad` when starting a call and specify `members` as a list of assistants and destinations. + + The first member is the assistant that will start the call, and assistants can be either persistent or transient. + We recommend using [Handoff Tools](/tools/handoff) to specify which destinations the current assistant can handoff too, and when to handoff to each assistant. Each assistant within the squad can use its saved handoff tools as well as handoff tools from Assistant Overrides (see below). - - ```json { "squad": { @@ -137,6 +137,9 @@ You may also define inline tools via assistant overrides through the `model` obj ### Member Overrides To override the configuration of _all_ assistants without modifying the underlying assistants, use the `memberOverrides`. + +Note: This is `squadOverrides` for [`assistant-request`](api-reference/webhooks/server-message#response.body.messageResponse.AssistantRequest.squadOverrides) + ```json { From 4c4e88386ae6c00022d357768574e275ed359627 Mon Sep 17 00:00:00 2001 From: Arthur Chi Date: Wed, 12 Nov 2025 23:29:31 -0800 Subject: [PATCH 4/4] third pass --- fern/squads.mdx | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/fern/squads.mdx b/fern/squads.mdx index 404804568..de33b9aac 100644 --- a/fern/squads.mdx +++ b/fern/squads.mdx @@ -19,10 +19,10 @@ Squads solve this by splitting complex prompts into focused assistants with spec ## Usage -To use Squads, you can create a `squad` when starting a call and specify `members` as a list of assistants and destinations. +To use Squads, you can create a `squad` when starting a call and specify `members` as a list of assistants and destinations. Assistants can be either persistent or transient. -The first member is the assistant that will start the call, and assistants can be either persistent or transient. +The first member is the assistant that will start the call. We recommend using [Handoff Tools](/tools/handoff) to specify which destinations the current assistant can handoff too, and when to handoff to each assistant. Each assistant within the squad can use its saved handoff tools as well as handoff tools from Assistant Overrides (see below). @@ -65,7 +65,7 @@ We recommend using [Handoff Tools](/tools/handoff) to specify which destinations ## Overrides ### Assistant Overrides -To override the configuration of a saved assistant without modifying the underlying assistant, use the `assistantsOverrides` to alter individual assistants. For example, if you have disparate assistants with different voices, you can use `assistantOverrides` to make sure all of the assistants are using the same voice without changing the assistant (in case its being used in another squad). +To override the configuration of a saved assistant without modifying the underlying assistant, use the `assistantsOverrides` to alter individual assistants. For example, if you have assistants in a squad with different voices, you can use `assistantOverrides` to make sure all of the assistants are using the same voice without changing the assistant (in case it's being used in another squad). ```json { @@ -94,7 +94,7 @@ To override the configuration of a saved assistant without modifying the underly } ``` -You may also define inline tools via assistant overrides through the `model` object, so that the assistant will only handoff if it is a part of this squad. +You may also define inline tools via assistant overrides through the `model` object (using `tools:append`), so that the assistant will only handoff if it is a part of this squad. ```json { "squad": { @@ -136,9 +136,9 @@ You may also define inline tools via assistant overrides through the `model` obj ### Member Overrides -To override the configuration of _all_ assistants without modifying the underlying assistants, use the `memberOverrides`. +To override the configuration of _all_ assistants in a squad without modifying the underlying assistants, use the `memberOverrides`. -Note: This is `squadOverrides` for [`assistant-request`](api-reference/webhooks/server-message#response.body.messageResponse.AssistantRequest.squadOverrides) +Note: This is `squadOverrides` for the [`assistant-request`](api-reference/webhooks/server-message#response.body.messageResponse.AssistantRequest.squadOverrides) webhook response. ```json