Any plans for Text-to-Speech (TTS) support?

[salidux]

RubyLLM already supports audio transcription (speech-to-text) via RubyLLM.transcribe, which is great. I'm wondering if there are any plans to add the reverse direction — text-to-speech (TTS).

Both OpenAI and Azure expose TTS APIs that follow patterns similar to the chat/embedding APIs RubyLLM already wraps. Right now I have to drop out of RubyLLM to generate speech audio, using either the ruby-openai gem or raw Faraday calls depending on the provider.

OpenAI TTS (via ruby-openai)

client = OpenAI::Client.new(access_token: ENV["OPENAI_API_KEY"])

mp3_data = client.audio.speech(parameters: {
  model: "gpt-4o-mini-tts",
  input: "Hello, this is a test.",
  voice: "nova",
  response_format: "mp3",
  instructions: "Speak in a calm, professional tone."
})

File.binwrite("output.mp3", mp3_data)

Azure Speech Services (via raw Faraday)

response = Faraday.new(url: "https://eastus.tts.speech.microsoft.com/cognitiveservices/v1") do |f|
  f.response :raise_error
end.post do |req|
  req.headers["Content-Type"] = "application/ssml+xml"
  req.headers["X-Microsoft-OutputFormat"] = "audio-24khz-48kbitrate-mono-mp3"
  req.headers["Ocp-Apim-Subscription-Key"] = ENV["AZURE_API_KEY"]
  req.body = ssml_string
end

mp3_data = response.body

What a RubyLLM API could look like

Something like this would be a natural complement to transcribe:

# Simple
audio = RubyLLM.speak("Hello, this is a test.", voice: "nova")
audio.save("output.mp3")

# With provider selection
audio = RubyLLM.speak("Hello.", voice: "nova", model: "gpt-4o-mini-tts", provider: :openai)

# Azure
audio = RubyLLM.speak("Hello.", voice: "en-US-JennyNeural", provider: :azure)

# SSML input (for fine-grained control over pauses, emphasis, prosody, etc.)
ssml = <<~SSML
  <speak version="1.0" xmlns="http://www.w3.org/2001/10/synthesis"
         xmlns:mstts="https://www.w3.org/2001/mstts" xml:lang="en-US">
    <voice name="en-US-JennyNeural">
      <p><prosody rate="slow">Welcome.</prosody></p>
      <break time="500ms"/>
      <p>Let's get started.</p>
    </voice>
  </speak>
SSML
audio = RubyLLM.speak(ssml, format: :ssml, provider: :azure)

Why SSML matters

Both Azure and OpenAI support SSML to varying degrees, and it's essential for any non-trivial TTS use case — podcast-style multi-voice audio, controlled pacing, emphasis, etc. Azure in particular expects SSML as its native input format. Supporting a format: :ssml option (or auto-detecting <speak> tags) would make RubyLLM viable for these real-world scenarios without falling back to raw HTTP.

Context

I'm using RubyLLM for chat (works great with both OpenAI and Azure Foundry), but for TTS I still need a separate client. It would be nice to consolidate everything under one library, especially since RubyLLM already handles the provider abstraction, configuration, and token tracking so well.

Is this something on the roadmap? Happy to help if there's interest.

[crmne]

Would be a great addition!

And would be great to see a great DSL for creating SSML like RubyLLM::Schema.

Feel free to open an issue about both so we can discuss the design of both before jumping in the code.

The speak interface otherwise looks nice, except it would be nice to autodetect SSML in order to not needing to specify a format!

[salidux]

Thanks @crmne !

Great idea on auto-detecting SSML — that's much cleaner than requiring a format parameter.

I'll open an issue covering both the speak API and the SSML Builder DSL as suggested. The focus will be on speak first (plain text → audio), with SSML and the builder DSL as a natural next phase once the core is solid.

Looking forward to discussing the design!

[crmne]

Better to separate them in two issues for organization!

Thanks!

[salidux]

Issue created: #651