Skip to content

Commit c481c39

Browse files
committed
docs: multilingual guides
1 parent 852199f commit c481c39

File tree

11 files changed

+2064
-611
lines changed

11 files changed

+2064
-611
lines changed

fern/assistants/examples/multilingual-agent.mdx

Lines changed: 1012 additions & 561 deletions
Large diffs are not rendered by default.

fern/customization/multilingual.mdx

Lines changed: 44 additions & 46 deletions
Original file line numberDiff line numberDiff line change
@@ -15,9 +15,9 @@ Configure your voice assistant to communicate in multiple languages with automat
1515
- Design language-aware system prompts
1616
- Test and optimize multilingual performance
1717

18-
<Warning>
19-
**Provider Limitations:** Most transcription providers don't support true multilingual mode. Only **Google STT** currently supports automatic language detection and switching within a single conversation.
20-
</Warning>
18+
<Note>
19+
**Multilingual Support:** Multiple providers support automatic language detection. **Deepgram** (Nova 2, Nova 3 with "Multi" setting) and **Google STT** (with "Multilingual" setting) both offer automatic language detection for seamless multilingual conversations.
20+
</Note>
2121

2222
## Configure automatic language detection
2323

@@ -28,12 +28,10 @@ Set up your transcriber to automatically detect and process multiple languages.
2828
1. Navigate to **Assistants** in your [Vapi Dashboard](https://dashboard.vapi.ai/)
2929
2. Create a new assistant or edit an existing one
3030
3. In the **Transcriber** section:
31-
- **Provider**: Select `Google` (only provider with true multilingual support)
32-
- **Model**: Choose Flash or Flash Lite (recommended for speed)
33-
- **Language Detection**: Set to `Automatic multiple languages`
34-
4. **Alternative for limited multilingual**:
35-
- **Deepgram**: Only supports English and Spanish in "Multi" mode
36-
- **Other providers**: Single language only, no auto-detection
31+
- **Provider**: Select `Deepgram` (recommended) or `Google`
32+
- **Model**: For Deepgram, choose `Nova 2` or `Nova 3`; for Google, choose `Latest`
33+
- **Language**: Set to `Multi` (Deepgram) or `Multilingual` (Google)
34+
4. **Other providers**: Single language only, no automatic detection
3735
5. Click **Save** to apply the configuration
3836
</Tab>
3937
<Tab title="TypeScript (Server SDK)">
@@ -42,21 +40,21 @@ Set up your transcriber to automatically detect and process multiple languages.
4240

4341
const vapi = new VapiClient({ token: "YOUR_VAPI_API_KEY" });
4442

45-
// Recommended: Google for true multilingual support
43+
// Recommended: Deepgram for multilingual support
4644
const assistant = await vapi.assistants.create({
4745
name: "Multilingual Assistant",
4846
transcriber: {
49-
provider: "google",
50-
model: "gemini-2.0-flash-lite", // or "gemini-2.0-flash" for better accuracy
51-
languageBehaviour: "automatic multiple languages"
47+
provider: "deepgram",
48+
model: "nova-2", // or "nova-3"
49+
language: "multi"
5250
}
5351
});
5452

55-
// Alternative: Deepgram for English + Spanish only
56-
const limitedMultilingual = {
57-
provider: "deepgram",
58-
model: "nova-2",
59-
languageBehaviour: "Multi" // Only en + es supported
53+
// Alternative: Google for multilingual support
54+
const googleMultilingual = {
55+
provider: "google",
56+
model: "latest",
57+
language: "multilingual"
6058
};
6159
```
6260
</Tab>
@@ -67,56 +65,56 @@ Set up your transcriber to automatically detect and process multiple languages.
6765

6866
client = Vapi(token=os.getenv("VAPI_API_KEY"))
6967

70-
# Recommended: Google for true multilingual support
68+
# Recommended: Deepgram for multilingual support
7169
assistant = client.assistants.create(
7270
name="Multilingual Assistant",
7371
transcriber={
74-
"provider": "google",
75-
"model": "gemini-2.0-flash-lite", # or "gemini-2.0-flash" for better accuracy
76-
"languageBehaviour": "automatic multiple languages"
72+
"provider": "deepgram",
73+
"model": "nova-2", # or "nova-3"
74+
"language": "multi"
7775
}
7876
)
7977

80-
# Alternative: Deepgram for English + Spanish only
81-
limited_multilingual = {
82-
"provider": "deepgram",
83-
"model": "nova-2",
84-
"languageBehaviour": "Multi" # Only en + es supported
78+
# Alternative: Google for multilingual support
79+
google_multilingual = {
80+
"provider": "google",
81+
"model": "latest",
82+
"language": "multilingual"
8583
}
8684
```
8785
</Tab>
8886
<Tab title="cURL">
8987
```bash
90-
# Recommended: Google for true multilingual support
88+
# Recommended: Deepgram for multilingual support
9189
curl -X POST "https://api.vapi.ai/assistant" \
9290
-H "Authorization: Bearer $VAPI_API_KEY" \
9391
-H "Content-Type: application/json" \
9492
-d '{
9593
"name": "Multilingual Assistant",
9694
"transcriber": {
97-
"provider": "google",
98-
"model": "gemini-2.0-flash-lite",
99-
"languageBehaviour": "automatic multiple languages"
95+
"provider": "deepgram",
96+
"model": "nova-2",
97+
"language": "multi"
10098
}
10199
}'
102100

103-
# Alternative: Deepgram for English + Spanish only
101+
# Alternative: Google for multilingual support
104102
curl -X POST "https://api.vapi.ai/assistant" \
105103
-H "Authorization: Bearer $VAPI_API_KEY" \
106104
-H "Content-Type: application/json" \
107105
-d '{
108106
"transcriber": {
109-
"provider": "deepgram",
110-
"model": "nova-2",
111-
"languageBehaviour": "Multi"
107+
"provider": "google",
108+
"model": "latest",
109+
"language": "multilingual"
112110
}
113111
}'
114112
```
115113
</Tab>
116114
</Tabs>
117115

118116
<Note>
119-
**Google STT Performance:** While Google is slower than other providers, use the `gemini-2.0-flash-lite` or `gemini-2.0-flash` models for better speed. These lighter models provide the best balance of multilingual support and performance.
117+
**Provider Performance:** **Deepgram** offers the best balance of speed and multilingual accuracy. **Google** provides broader language support but may be slower. Both providers support automatic language detection within conversations.
120118
</Note>
121119

122120
## Set up multilingual voices
@@ -460,8 +458,8 @@ Validate your configuration with different languages and scenarios.
460458

461459
| Provider | Multilingual Support | Languages | Notes |
462460
|----------|---------------------|-----------|-------|
463-
| **Google STT** | ✅ Full auto-detection | 125+ | **Only provider with true multilingual mode**. Use Flash or Flash Lite models for speed. |
464-
| **Deepgram** | ⚠️ Limited | English + Spanish only | "Multi" mode supports only 2 languages |
461+
| **Deepgram** | ✅ Full auto-detection | 100+ | **Recommended**: Nova 2/Nova 3 with "Multi" language setting |
462+
| **Google STT** | ✅ Full auto-detection | 125+ | Latest models with "Multilingual" language setting |
465463
| **Assembly AI** | ❌ English only | English | No multilingual support |
466464
| **Azure STT** | ❌ Single language | 100+ | Many languages, but no auto-detection |
467465
| **OpenAI Whisper** | ❌ Single language | 90+ | Many languages, but no auto-detection |
@@ -483,10 +481,10 @@ Validate your configuration with different languages and scenarios.
483481
<AccordionGroup>
484482
<Accordion title="Language detection is inaccurate">
485483
**Solutions:**
486-
- Use Google STT (only provider with reliable multilingual support)
487-
- For Deepgram, stick to English + Spanish combinations only
488-
- Use higher-quality audio preprocessing
484+
- Use Deepgram (Nova 2/Nova 3 with "Multi") or Google STT (with "Multilingual")
485+
- Ensure high-quality audio input for better detection accuracy
489486
- Test with native speakers of target languages
487+
- Consider provider-specific language combinations for optimal results
490488
</Accordion>
491489

492490
<Accordion title="Assistant doesn't realize it can speak multiple languages">
@@ -497,12 +495,12 @@ Validate your configuration with different languages and scenarios.
497495
- Avoid generic "multilingual" statements without specifics
498496
</Accordion>
499497

500-
<Accordion title="Google STT is too slow">
498+
<Accordion title="Transcription is too slow">
501499
**Solutions:**
502-
- Use `gemini-2.0-flash-lite` or `gemini-2.0-flash` models instead of standard models
503-
- Consider the speed vs accuracy tradeoff
504-
- For English + Spanish only, use Deepgram's "Multi" mode
505-
- Optimize your audio quality to improve processing speed
500+
- Use Deepgram Nova 2/Nova 3 for optimal speed and multilingual support
501+
- For Google STT, use latest models for better performance
502+
- Consider the speed vs accuracy tradeoff for your use case
503+
- Optimize audio quality and format to improve processing speed
506504
</Accordion>
507505

508506
<Accordion title="Voice quality varies between languages">

fern/docs.yml

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -258,6 +258,9 @@ navigation:
258258
- page: Order management
259259
path: workflows/examples/ecommerce-order-management.mdx
260260
icon: fa-light fa-shopping-cart
261+
- page: Multilingual support
262+
path: workflows/examples/multilingual-support.mdx
263+
icon: fa-light fa-globe
261264

262265
- section: Best practices
263266
contents:

fern/guides.mdx

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -41,6 +41,22 @@ slug: guides
4141
<br />
4242
Create an outbound sales agent that can schedule appointments automatically
4343
</Card>
44+
<Card title="Multilingual Support Workflow" icon="globe" href="/workflows/examples/multilingual-support">
45+
<div className='absolute top-4 right-4'>
46+
<Icon icon="arrow-up-right-from-square" />
47+
</div>
48+
<div class="vapi-badge vapi-badge-workflow">Built with Workflows</div>
49+
<br />
50+
Build a structured multilingual support workflow with language selection and dedicated conversation paths
51+
</Card>
52+
<Card title="Dynamic Multilingual Agent" icon="language" href="/assistants/examples/multilingual-agent">
53+
<div className='absolute top-4 right-4'>
54+
<Icon icon="arrow-up-right-from-square" />
55+
</div>
56+
<div class="vapi-badge vapi-badge-assistant">Built with Assistants</div>
57+
<br />
58+
Build a dynamic agent with automatic language detection and real-time language switching
59+
</Card>
4460
<Card title="Docs Agent" icon="book-open" href="/assistants/examples/docs-agent">
4561
<div className='absolute top-4 right-4'>
4662
<Icon icon="arrow-up-right-from-square" />

fern/providers/model/openai.mdx

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,7 @@ slug: providers/model/openai
77

88
**What is OpenAI?**
99

10-
OpenAI is a leading artificial intelligence research and deployment company dedicated to ensuring that artificial general intelligence (AGI) benefits all of humanity. Founded with the mission to create safe and highly capable AI systems, OpenAI has made significant strides in AI research, producing groundbreaking models like GPT-4, DALL-E, and Codex. These innovations have not only advanced the field of AI but also transformed various industries by providing powerful tools for natural language processing, image generation, and programming assistance.
10+
OpenAI is a leading artificial intelligence research and deployment company dedicated to ensuring that artificial general intelligence (AGI) benefits all of humanity. Founded with the mission to create safe and highly capable AI systems, OpenAI has made significant strides in AI research, producing groundbreaking models like GPT-4o, DALL-E, and Codex. These innovations have not only advanced the field of AI but also transformed various industries by providing powerful tools for natural language processing, image generation, and programming assistance.
1111

1212
**The Evolution of AI Research:**
1313

@@ -19,7 +19,7 @@ OpenAI offers a range of AI-driven products and services designed to meet divers
1919

2020
**GPT Models:**
2121

22-
- OpenAI’s Generative Pre-trained Transformer (GPT) models, including the latest GPT-4, are state-of-the-art in natural language processing. These models can generate human-like text, answer questions, summarize information, and perform various language tasks with high accuracy. GPT-4, in particular, represents a significant leap in AI capabilities, offering improved coherence, context understanding, and creativity.
22+
- OpenAI’s Generative Pre-trained Transformer (GPT) models, including the latest GPT-4o, are state-of-the-art in natural language processing. These models can generate human-like text, answer questions, summarize information, and perform various language tasks with high accuracy. GPT-4o, in particular, represents a significant leap in AI capabilities, offering improved coherence, context understanding, and creativity.
2323

2424
**DALL-E:**
2525

fern/providers/voice/openai.mdx

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,7 @@ slug: providers/voice/openai
77

88
**What is OpenAI?**
99

10-
OpenAI is a leading artificial intelligence research and deployment company dedicated to ensuring that artificial general intelligence (AGI) benefits all of humanity. Founded with the mission to create safe and highly capable AI systems, OpenAI has made significant strides in AI research, producing groundbreaking models like GPT-4, DALL-E, and Codex. These innovations have not only advanced the field of AI but also transformed various industries by providing powerful tools for natural language processing, image generation, and programming assistance.
10+
OpenAI is a leading artificial intelligence research and deployment company dedicated to ensuring that artificial general intelligence (AGI) benefits all of humanity. Founded with the mission to create safe and highly capable AI systems, OpenAI has made significant strides in AI research, producing groundbreaking models like GPT-4o, DALL-E, and Codex. These innovations have not only advanced the field of AI but also transformed various industries by providing powerful tools for natural language processing, image generation, and programming assistance.
1111

1212
**The Evolution of AI Research:**
1313

@@ -19,7 +19,7 @@ OpenAI offers a range of AI-driven products and services designed to meet divers
1919

2020
**GPT Models:**
2121

22-
- OpenAI’s Generative Pre-trained Transformer (GPT) models, including the latest GPT-4, are state-of-the-art in natural language processing. These models can generate human-like text, answer questions, summarize information, and perform various language tasks with high accuracy. GPT-4, in particular, represents a significant leap in AI capabilities, offering improved coherence, context understanding, and creativity.
22+
- OpenAI’s Generative Pre-trained Transformer (GPT) models, including the latest GPT-4o, are state-of-the-art in natural language processing. These models can generate human-like text, answer questions, summarize information, and perform various language tasks with high accuracy. GPT-4o, in particular, represents a significant leap in AI capabilities, offering improved coherence, context understanding, and creativity.
2323

2424
**DALL-E:**
2525

735 KB
Loading
Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,16 @@
1+
customer_id,name,email,phone,preferred_language,region,account_status,support_tier
2+
C001,Sarah Johnson,[email protected],+1-555-0101,english,north_america,active,premium
3+
C002,María González,[email protected],+34-600-123456,spanish,spain,active,standard
4+
C003,Jean Dubois,[email protected],+33-1-23456789,french,france,active,premium
5+
C004,Carlos Restrepo,[email protected],+52-55-12345678,spanish,mexico,active,standard
6+
C005,Emily Chen,[email protected],+1-555-0102,english,north_america,active,enterprise
7+
C006,Sophie Martin,[email protected],+1-514-1234567,french,canada,active,premium
8+
C007,Antonio Silva,[email protected],+34-91-2345678,spanish,spain,inactive,standard
9+
C008,Michael Thompson,[email protected],+44-20-12345678,english,uk,active,standard
10+
C009,Luisa Fernández,[email protected],+57-1-3456789,spanish,colombia,active,premium
11+
C010,Pierre Leblanc,[email protected],+33-4-56789012,french,france,active,standard
12+
C011,Jennifer Davis,[email protected],+1-555-0103,english,north_america,active,standard
13+
C012,Isabel Rodríguez,[email protected],+52-33-45678901,spanish,mexico,suspended,standard
14+
C013,François Moreau,[email protected],+1-418-5678901,french,canada,active,enterprise
15+
C014,David Wilson,[email protected],+1-555-0104,english,north_america,active,premium
16+
C015,Carmen López,[email protected],+34-93-6789012,spanish,spain,active,standard
Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
1+
product_id,name_en,name_es,name_fr,category,price_usd,description_en,description_es,description_fr,availability
2+
P001,CloudSync Pro,CloudSync Pro,CloudSync Pro,cloud_storage,29.99,"Professional cloud storage with 1TB space and advanced sharing","Almacenamiento en la nube profesional con 1TB de espacio y compartición avanzada","Stockage cloud professionnel avec 1 To d'espace et partage avancé",available
3+
P002,SecureVPN Enterprise,VPN Segura Empresarial,VPN Sécurisé Entreprise,security,49.99,"Enterprise-grade VPN with global servers and 24/7 support","VPN de nivel empresarial con servidores globales y soporte 24/7","VPN de niveau entreprise avec serveurs mondiaux et support 24h/24",available
4+
P003,DataAnalytics Suite,Suite de Análisis de Datos,Suite d'Analyse de Données,analytics,99.99,"Complete data analytics platform with AI-powered insights","Plataforma completa de análisis de datos con insights impulsados por IA","Plateforme d'analyse de données complète avec des insights alimentés par l'IA",available
5+
P004,Mobile App Builder,Constructor de Apps Móviles,Constructeur d'Apps Mobiles,development,79.99,"No-code mobile app development platform","Plataforma de desarrollo de aplicaciones móviles sin código","Plateforme de développement d'applications mobiles sans code",available
6+
P005,Smart Backup Pro,Backup Inteligente Pro,Sauvegarde Intelligente Pro,backup,19.99,"Automated backup solution with intelligent scheduling","Solución de respaldo automatizada con programación inteligente","Solution de sauvegarde automatisée avec planification intelligente",available
7+
P006,Team Collaboration Hub,Centro de Colaboración,Hub de Collaboration,collaboration,39.99,"All-in-one team collaboration platform","Plataforma de colaboración en equipo todo-en-uno","Plateforme de collaboration d'équipe tout-en-un",available
8+
P007,AI Assistant Pro,Asistente IA Pro,Assistant IA Pro,ai,149.99,"Advanced AI assistant for business automation","Asistente de IA avanzado para automatización empresarial","Assistant IA avancé pour l'automatisation des entreprises",beta
9+
P008,Network Monitor,Monitor de Red,Moniteur Réseau,monitoring,59.99,"Real-time network monitoring and alerting","Monitoreo de red en tiempo real y alertas","Surveillance réseau en temps réel et alertes",available
10+
P009,Database Optimizer,Optimizador de Base de Datos,Optimiseur de Base de Données,database,89.99,"Database performance optimization tool","Herramienta de optimización de rendimiento de base de datos","Outil d'optimisation des performances de base de données",available
11+
P010,Code Review Assistant,Asistente de Revisión de Código,Assistant de Révision de Code,development,69.99,"Automated code review with quality insights","Revisión automatizada de código con insights de calidad","Révision automatisée du code avec des insights qualité",coming_soon

0 commit comments

Comments
 (0)