💡 Alternative Voice Providers - Reduce ElevenLabs Token Usage

[paulpreibisch]

💡 Alternative Voice Providers - Reduce ElevenLabs Token Usage

Overview

AgentVibes currently uses ElevenLabs exclusively for TTS, which provides excellent quality but consumes API tokens. This discussion explores alternative voice providers that could reduce or eliminate token costs while maintaining good voice quality.

Research Summary

🍎 macOS Built-in Voices (FREE)

Status: ✅ Excellent option for macOS users

Quality: High-quality neural voices with natural sound

• Best Voices: "Zoe (Premium)", "Anna (Premium)"
• 50+ premium voices available for download
• Free - No API costs whatsoever

Implementation:

say -v "Zoe (Premium)" -f textfile.txt --quality 127 --rate 180

Pros:

• ✅ Completely free
• ✅ High quality neural voices
• ✅ No internet required (offline)
• ✅ Already installed on macOS
• ✅ Zero latency

Cons:

• ❌ macOS only
• ❌ Requires downloading premium voices via System Settings

Download: System Settings → Accessibility → Spoken Content → System Voice → Download

---

🪟 Windows 11 Natural Voices (FREE but complicated)

Status: ⚠️ Possible but unstable

Windows 11 includes beautiful neural voices, but they're locked to Narrator and not accessible via standard APIs.

Solution: NaturalVoiceSAPIAdapter (open-source)

How it works:

• Extracts encryption keys from Windows system files
• Makes Narrator voices accessible to any app
• WARNING: Can break with Windows updates

Pros:

• ✅ Free
• ✅ High quality voices
• ✅ Offline/local

Cons:

• ❌ Unstable (Microsoft could break it anytime)
• ❌ Essentially a hack, not official API
• ❌ Windows only
• ❌ Requires adapter installation

---

🐸 Piper TTS (RECOMMENDED - FREE & Open Source)

Status: ⭐ Best cross-platform offline option

Quality: "Google TTS level" even on medium quality setting

Implementation:

pipx install piper-tts
# Download voices from https://huggingface.co/rhasspy/piper-voices

Quality Levels:

• x_low: 16kHz, 5-7M params (fast)
• low: 16kHz, 15-20M params
• medium: 22.05kHz, 15-20M params ⭐ Recommended
• high: 22.05kHz, 28-32M params (best quality)

Voice Samples: https://rhasspy.github.io/piper-samples/

Pros:

• ✅ Cross-platform (Windows, Mac, Linux)
• ✅ Completely free
• ✅ No API costs ever
• ✅ Runs on Raspberry Pi!
• ✅ Excellent quality (comparable to Google TTS)
• ✅ 100% offline
• ✅ Active community, many voices available
• ✅ Fast inference even on CPU

Cons:

• ⚠️ Not quite ElevenLabs quality (but very close)
• ⚠️ Requires initial setup and voice downloads
• ⚠️ No voice cloning features

---

🐸 Coqui TTS (Advanced - FREE)

Status: ✅ Powerful but more complex

Note: Company shut down Dec 2023, but open-source project is maintained by Idiap Research Institute

Special Features:

• Voice cloning from 6 seconds of audio!
• XTTS-v2 supports 17 languages
• <200ms latency for streaming

Installation: pip install coqui-tts

Pros:

• ✅ Free and open-source
• ✅ Voice cloning capability
• ✅ Very natural sounding
• ✅ Multilingual (17 languages)
• ✅ Offline capable
• ✅ Actively maintained fork

Cons:

• ⚠️ More complex setup than Piper
• ⚠️ Requires more resources (GPU recommended)
• ⚠️ Steeper learning curve

---

💰 Commercial API Alternatives

All major providers cost ~$15-16 per million characters:

Provider	Price/1M chars	Free Tier	Notes
Azure Speech	$16	500k/month	Same quality as OpenAI
OpenAI TTS	~$15	None	Natural voices
Amazon Polly	~$16	5M first year	Fine-grained controls
Google TTS	~$16	None	Good integration
ElevenLabs	$21-35/book (subscription)	10k/month	Best quality, ~75ms latency

Key Finding: Azure, OpenAI, Polly, and Google are significantly cheaper than ElevenLabs for equivalent quality!

Azure Speech is particularly attractive:

• ✅ $16 per million vs ElevenLabs subscription
• ✅ 500k free characters/month
• ✅ Same neural voice quality
• ✅ Pay-as-you-go pricing

---

💡 Proposed Plugin Architecture

Voice Provider Selection Feature

Create a plugin system that lets users choose their TTS provider:

/agent-vibes:provider list                    # Show all available providers
/agent-vibes:provider set piper               # Use Piper TTS (offline)
/agent-vibes:provider set macos               # Use macOS voices (Mac only)
/agent-vibes:provider set azure               # Use Azure Speech API
/agent-vibes:provider set elevenlabs          # Use ElevenLabs (current)
/agent-vibes:provider status                  # Show current provider

Implementation Strategy

Phase 1: Core abstraction

• Create voice-provider-manager.sh abstraction layer
• Standardize interface: speak(text, voice, language)
• Keep current ElevenLabs as default

Phase 2: Add providers

• Piper TTS integration (cross-platform)
• macOS say command integration
• Azure Speech API integration

Phase 3: Auto-detection

• Detect platform (macOS/Windows/Linux)
• Detect available providers
• Recommend best option based on:
  • Platform
  • Cost requirements
  • Quality needs
  • Online/offline preference

Benefits

✅ Cost Savings: Users can eliminate ElevenLabs costs
✅ Flexibility: Choose provider based on needs
✅ Offline Support: Piper/macOS work without internet
✅ Quality Options: From free (Piper) to premium (ElevenLabs)
✅ Platform Support: macOS, Windows, Linux all covered

---

🎯 Recommendations

Immediate Actions

1. Add macOS support - Trivial to implement, huge benefit for Mac users
2. Add Piper TTS - Best cross-platform free option
3. Document Azure pricing - Alternative for cloud users

Future Enhancements

1. Provider plugin system - Let users choose and switch providers
2. Auto-fallback - Try free options first, fall back to paid
3. Cost tracking - Show estimated costs per session
4. Voice matching - Map AgentVibes voices to equivalent voices in each provider

---

📊 Cost Comparison Example

Typical coding session (4 hours, ~100 TTS messages):

Provider	Cost	Notes
Piper TTS	$0	Free forever
macOS voices	$0	Free forever (Mac only)
Azure Speech	$0	Free tier covers this
ElevenLabs	~$0.50	From subscription quota

For heavy users (40 hrs/week):

Provider	Monthly Cost	Notes
Piper TTS	$0	Free
macOS voices	$0	Free (Mac only)
Azure Speech	~$5-10	Pay-as-you-go
ElevenLabs Creator	$22	Subscription required

---

🗳️ Community Input

What would you like to see?

• macOS built-in voice support
• Piper TTS integration (cross-platform, free)
• Azure Speech API option
• Voice provider plugin system
• Cost tracking and reporting
• Windows native voice support
• Keep ElevenLabs only (current)

Vote with 👍 on features you want!

---

🔗 Resources

• Piper TTS: https://github.com/rhasspy/piper
• Piper Voice Samples: https://rhasspy.github.io/piper-samples/
• Coqui TTS: https://github.com/idiap/coqui-ai-TTS
• Azure Speech Pricing: https://azure.microsoft.com/pricing/details/cognitive-services/speech-services/
• macOS Say Command: https://romanzipp.com/blog/get-tts-with-natural-voices-on-macos-without-external-tools
• NaturalVoiceSAPIAdapter (Windows): https://github.com/gexgd0419/NaturalVoiceSAPIAdapter

---

What do you think? Should AgentVibes support multiple voice providers?

Share your thoughts, use cases, and provider preferences below! 💬