Langfuse+pipecat stt and tss I/O and costing is not coming

[atindraojha]

Describe your question

When i am integrating langfuse with the pipecat voice agent, I am unable to get the input output data in the stt and tts so pricing is also not reflecting there and only llm data and costing is there.
Help me if you have any solutions:

Langfuse Cloud or Self-Hosted?

Langfuse Cloud

If Self-Hosted

No response

If Langfuse Cloud

No response

SDK and integration versions

No response

Pre-Submission Checklist

• I have checked for existing issues/discussions and consulted Langfuse AI.

[inkeep-ai-assistant[bot]]

The issue you're experiencing with STT and TTS input/output and costing not appearing in Langfuse traces when using Pipecat is a known limitation(1).

The Problem

When TTS (text-to-speech) and STT (speech-to-text) models run in Pipecat, they are logged as span observations by default, not as generation observations(1). Cost tracking in Langfuse only works for generation and embedding observation types(1).

The Solution

To enable cost tracking for TTS and STT, you need to log these runs as generation observations instead of generic span observations(1). Here's how:

1. Log TTS/STT as a Generation: Wrap your TTS/STT calls in a Langfuse observation with as_type="generation"(1).

2. Attach Usage Details: After the call, update the generation with usage details that match your custom model definition (e.g., audio_duration, characters_count)(1). The keys in your usage_details dict must exactly match the usage type keys in your Langfuse model definition for cost calculation to work(1).

Example code:

from langfuse import observe, get_client

langfuse = get_client()

@observe(as_type="generation")
def tts_generation(text, **kwargs):
    # Call your TTS model here
    audio, duration, char_count = call_tts(text)
    langfuse.update_current_generation(
        model="your-tts-model-name",
        usage_details={
            "audio_duration": duration,  # e.g., in seconds
            "characters_count": char_count
        }
    )
    return audio

(1)

3. Custom Model Definition: In the Langfuse UI, define a custom model for your TTS/STT with usage types and prices that match the keys you send (e.g., audio_duration, characters_count)(1). Cost will be calculated automatically if the keys and model name match(1).

If your framework only logs TTS/STT as a span, you'll need to manually create a generation observation, as spans do not support cost tracking(1).

📚 Sources:

1. Logging text-to-speech with Langfuse

---

Have another question? Just tag @inkeep.

[jannikmaierhoefer]

Hi @atindraojha, could you share the link to the trace with us? Happy to have a look!

[atindraojha]

This is some trace

ids ::664974f35437cf13afcc5f7b46bd2e45 , 324e5284b0c3609aed7bb82eb3743225, bf96c7b657d48f343cb5948c21467104, 90e0bf1d4b25e95c2da5af1121989609

[atindraojha]

If you need anything , feel free to ask

[maxdeichmann]

Thanks a lot, @hassiebp will look into this once he is back from christmas vacation.

[hassiebp]

Tracking progress on this here: #11573

[dalmia]

hi @maxdeichmann did you get a chance to look into this?