Align live transcription response type with OpenAI Realtime ConversationItem pattern

[rui-ren]

Description

Redesigns LiveAudioTranscriptionResponse to follow the OpenAI Realtime API's ConversationItem shape, enabling forward compatibility with a future WebSocket-based architecture.

Motivation:

• Customers using OpenAI's Realtime API access transcription via result.content[0].transcript
• By adopting this pattern now, customers who write result.Content[0].Text won't need to change their code when we migrate to WebSocket transport
• Aligns with the team's plan to move toward OpenAI Realtime API compatibility

Before:

// Extended AudioCreateTranscriptionResponse from Betalgo
await foreach (var result in session.GetTranscriptionStream())
{
    Console.Write(result.Text);           // inherited from base
    bool final = result.IsFinal;          // custom field
    var segments = result.Segments;       // inherited from base
}

After:

// Own type shaped like OpenAI Realtime ConversationItem
await foreach (var result in session.GetTranscriptionStream())
{
    Console.Write(result.Content[0].Text);       // ConversationItem pattern
    Console.Write(result.Content[0].Transcript); // alias for Text (Realtime compat)
    bool final = result.IsFinal;
    double? start = result.StartTime;
}

Changes:

File	Change
LiveAudioTranscriptionTypes.cs	Removed AudioCreateTranscriptionResponse inheritance. New standalone LiveAudioTranscriptionResponse with Content list + new TranscriptionContentPart type
LiveAudioTranscriptionClient.cs	Updated text checks: .Text → .Content?[0]?.Text
JsonSerializationContext.cs	Registered TranscriptionContentPart, removed AudioCreateTranscriptionResponse.Segment
LiveAudioTranscriptionTests.cs	Updated assertions to match new type shape
Program.cs (sample)	Updated result reading to result.Content?[0]?.Text
README.md	Updated docs and output type table

Key design decisions:

• TranscriptionContentPart has both Text and Transcript (set to the same value) for maximum compatibility with both Whisper and Realtime API patterns
• StartTime/EndTime are top-level on the response (not nested in Segments) — simpler access, maps to Realtime's audio_start_ms/audio_end_ms
• No dependency on Betalgo's ConversationItem — we own the type to avoid carrying unused chat/tool-calling fields
• LiveAudioTranscriptionRaw (Core JSON deserialization) is unchanged — this is purely an SDK presentation change, no Core/neutron-server impact

No breaking changes to: Core API, native interop, audio pipeline, session lifecycle

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
foundry-local	Error		Mar 28, 2026 1:34am

[nenad1002]

We're tracking this?

[rui-ren]

Yes,

/// <summary>
/// Transcription result sent back to SDK via callback during streaming.
/// Must match the SDK's AudioStreamingTranscriptionResult type.
/// </summary>
public record AudioStreamingTranscriptionResult
{
    [JsonPropertyName("is_final")]
    public bool IsFinal { get; init; }

    [JsonPropertyName("text")]
    public string Text { get; init; } = string.Empty;

    [JsonPropertyName("start_time")]
    public double? StartTime { get; init; }

    [JsonPropertyName("end_time")]
    public double? EndTime { get; init; }
}

This is inside our Core code, as you can see, we have the start_time in JSON response.

It is useful for the caller to have timestamp display, subtitle generation etc.