[FEATURE REQ] Be able to retrieve Streaming Reasoning Content

[rogerbarreto]

Describe the feature or improvement you are requesting

Currently we don't have a public and reliable way to extract the streaming thinking/reasoning of reasoning models without resorting to reflection (not AOT friendly) or serialization (extra resource) of a specific chunk.

Additional context

Ideally we should be able to check for the StreamingResponseUpdate type and get the Delta property back if it is a suggested public ResponseReasoningSummaryTextDeltaUpdate type unfortunately we don't have a way to do it today.

[rliberoff]

We are facing the same issue. Actually, our users are requesting to see the chain of thought while the model "thinks or reasons" the proper actions.

[rogerbarreto]

FYI, while the feature isn't public or available yet, it is possible thru breaking glass to get this information, not ideal though.

Workaround (Unblocking)

I have a proposed change to the Microsoft.Extensions.AI.OpenAI with the breaking glass approach below, to be updated as soon this issue is resolved.

// Breaking-Glass capture of streaming reasoning contents
if (streamingUpdate.GetType().Name == "InternalResponseReasoningSummaryTextDeltaEvent")
{
    var responseDeltaElement = JsonSerializer.Deserialize<JsonElement>(JsonSerializer.Serialize(streamingUpdate));

    if (responseDeltaElement.TryGetProperty("delta", out var deltaProperty))
    {
        // Your code with the thinking content in 
        var deltaContent = deltaProperty.GetString()
    }
}

• Add reasoning text content for OpenAI Responses ChatClient Streaming dotnet/extensions#6761

[jsquire]

Hi @rogerbarreto. Thanks for reaching out and we regret that you're experiencing difficulties. Would you please provide additional context around your end-to-end scenario and where you're running into difficulties? Seeing a code snippet of what you're attempting to do would be helpful for us to better understand and potentially offer guidance.

[gaspardpetit]

If I may comment on this, here's a full sample that provides reasoning summaries and encrypted content. It does not compile without altering the client code, but it shows you the path we would need for a clean usage.

#pragma warning disable OPENAI001 // Type is for evaluation purposes only and is subject to change or removal in future updates. Suppress this diagnostic to proceed.
#pragma warning disable OPENAICUA001 // Type is for evaluation purposes only and is subject to change or removal in future updates. Suppress this diagnostic to proceed.


using OpenAI;
using OpenAI.Responses;
using System.ClientModel;
using System.Text;
using System.Text.Json;

class Program
{
	static async Task Main()
	{
		var apiKey = Environment.GetEnvironmentVariable("OPENAI_API_KEY") ?? throw new InvalidOperationException("OPENAI_API_KEY not set.");
		OpenAIResponseClient client = new(
				model: "gpt-5",
				credential: new ApiKeyCredential(apiKey),
				options: new OpenAIClientOptions()
				);

		var options = new ResponseCreationOptions {
			StoredOutputEnabled = false,
			ReasoningOptions = new ResponseReasoningOptions {
				ReasoningEffortLevel = new ResponseReasoningEffortLevel("low"),
				ReasoningSummaryVerbosity = ResponseReasoningSummaryVerbosity.Auto
			},
			TextOptions = new ResponseTextOptions {
				TextFormat = ResponseTextFormat.CreateTextFormat()
			},
			Include = [InternalIncludable.ReasoningEncryptedContent]
		};

		var inputItems = new List<ResponseItem>
		{
			ResponseItem.CreateDeveloperMessageItem("You think a lot, but provide very short responses."),
			ResponseItem.CreateUserMessageItem("pros and cons of using `internal` (c#) in a client sdk")
		};

		StringBuilder contentSb = new();

		OrderedDictionary<string, string> hiddenThoughts = new();
		await foreach (var update in client.CreateResponseStreamingAsync(inputItems, options))
			HandleResponseUpdate(update, hiddenThoughts, contentSb);

		inputItems.AddRange(
			// Note : we can't use `ResponseItem.CreateReasoningItem()` since it only accepts a summary...
			hiddenThoughts.Select(kv => new ReasoningResponseItem(summaryParts: [], encryptedContent: kv.Value, id: kv.Key)
			));

		inputItems.Add(ResponseItem.CreateAssistantMessageItem(contentSb.ToString()));
		inputItems.Add(ResponseItem.CreateUserMessageItem("What else?"));

		await foreach (var update in client.CreateResponseStreamingAsync(inputItems, options))
			HandleResponseUpdate(update, hiddenThoughts, contentSb);
	}

	private static void HandleStreamResponseItem(ResponseItem item, OrderedDictionary<string, string> encryptedThoughts)
	{

		switch (item)
		{
			case InternalCodeInterpreterToolCallItemResource:
			case InternalImageGenToolCallItemResource:
			case InternalLocalShellToolCallItemResource:
			case InternalLocalShellToolCallOutputItemResource:
			case InternalMCPApprovalRequestItemResource:
			case InternalMCPApprovalResponseItemResource:
			case InternalMCPCallItemResource:
			case InternalMCPListToolsItemResource:
			case InternalResponsesAssistantMessage:
			case InternalResponsesDeveloperMessage:
			case InternalResponsesSystemMessage:
			case InternalResponsesUserMessage:
			case InternalUnknownItemResource:
			case InternalUnknownResponsesMessageItemResource:
				// lost of goodies that should be exposed...
				break;

			case ComputerCallOutputResponseItem:
			case ComputerCallResponseItem:
			case FileSearchCallResponseItem:
			case FunctionCallOutputResponseItem:
			case FunctionCallResponseItem:
			case MessageResponseItem:
			case ReferenceResponseItem:
			case WebSearchCallResponseItem:
				// some goodies already available...
				break;

			case ReasoningResponseItem cr:
				// as the stream advances, the same id will be repeated - it's important to only keep the latest value
				encryptedThoughts[cr.Id] = cr.EncryptedContent;

				List<string>? summaries = cr.SummaryParts?.OfType<ReasoningSummaryTextPart>().Select(sp => sp.Text).ToList();
				string encrypted = cr.EncryptedContent;
				Console.Write($"\n<reasoning id='{cr.Id}'>");
				if (summaries != null)
					foreach (string summary in summaries)
						Console.Write($"<summary>{summary}</summary>");
				if (string.IsNullOrEmpty(encrypted) == false)
					Console.Write($"<encrypted>{encrypted}</encrypted>");
				Console.Write("</reasoning>\n");
				break;
			default:
				break;
		}
	}

	private static void HandleResponseUpdate(StreamingResponseUpdate update, OrderedDictionary<string, string> hiddenThoughts, StringBuilder contentSb)
	{
		switch (update)
		{
			case StreamingResponseCreatedUpdate:
			case StreamingResponseInProgressUpdate:
			case InternalResponseReasoningSummaryTextDoneEvent:
			case StreamingResponseOutputTextDoneUpdate:
			case StreamingResponseContentPartDoneUpdate:
			case StreamingResponseContentPartAddedUpdate:
				// ignored
				break;

			case StreamingResponseOutputItemAddedUpdate u:
				// these can actually be ingnored, they will be updated and provided in StreamingResponseCompletedUpdate
				HandleStreamResponseItem(u.Item, hiddenThoughts);
				break;

			case StreamingResponseOutputItemDoneUpdate u:
				// these can actually be ingnored, they will be updated and provided in StreamingResponseCompletedUpdate
				HandleStreamResponseItem(u.Item, hiddenThoughts);
				break;

			case InternalResponseReasoningSummaryPartAddedEvent u:
				Console.WriteLine($"<thinking_summary id='{u.ItemId} index='{u.SummaryIndex}'>");
				break;

			case InternalResponseReasoningSummaryPartDoneEvent u:
				Console.WriteLine($"\n</thinking_summary>\n");
				break;

			case InternalResponseReasoningSummaryTextDeltaEvent u:
				// reasoning summary delta - don't use in in follow up (prefer encrypted content)
				Console.Write(u.Delta);
				break;

			case StreamingResponseOutputTextDeltaUpdate u:
				// response text
				contentSb.Append(u.Delta);
				Console.Write(u.Delta);
				break;

			case StreamingResponseCompletedUpdate u:
				Console.WriteLine();
				Console.WriteLine("\n--- DONE ---");
				Console.WriteLine();
				foreach (ResponseItem? item in u.Response.OutputItems)
					HandleStreamResponseItem(item, hiddenThoughts);

				var usage = u.Response.Usage;
				var payload = new {
					usage = new {
						input_tokens = usage?.InputTokenCount,
						output_tokens = usage?.OutputTokenCount,
						total_tokens = usage?.TotalTokenCount,
						reasoning_tokens = usage?.OutputTokenDetails?.ReasoningTokenCount
					}
				};
				Console.WriteLine(JsonSerializer.Serialize(payload, new JsonSerializerOptions { WriteIndented = true }));
				Console.WriteLine();
				break;
			default:
				Console.WriteLine(update.GetType().Name);
				break;
		}
	}
}

Output looks like this:

<reasoning id='rs_68b8115ed360819592ecc1462c3ec7fb0379010263910ce8'>
<encrypted>gAAAAABouBFe...o1Y0U1mA=</encrypted></reasoning>
<thinking_summary id='rs_68b8115ed360819592ecc1462c3ec7fb0379010263910ce8 index='0'>

**Considering IVT and strong naming**

I'm thinking about how the InternalsVisibleTo (IVT) attribute can leak internal details ...
</thinking_summary>


<reasoning id='rs_68b8115ed360819592ecc1462c3ec7fb0379010263910ce8'>
<summary>
**Considering IVT and strong naming**

I'm thinking about how the InternalsVisibleTo (IVT) attribute can leak internal details for test assemblies ... Let's get this delivered!</summary>
<encrypted>gAAAAABouBFysrExCwkxbWVKp...hiF_Z6GFuw==</encrypted>
</reasoning>

Pros
...

Cons
- Reduced extensibility: Consumers can't subclass, plug in, or reuse internal types; may force awkward workarounds.
...

--- DONE ---


<reasoning id='rs_68b8115ed360819592ecc1462c3ec7fb0379010263910ce8'>
<summary>
**Considering IVT and strong naming**

I'm thinking about how the InternalsVisibleTo (IVT) attribute can leak internal details ... Let's get this delivered!
</summary>
<encrypted>gAAAAABouBF4KPAR_Q...-3d7MhPA==</encrypted>
</reasoning>

{ usage": { "input_tokens"": 37, "output_tokens": 1099, "total_tokens": 1136, "reasoning_tokens": 704 }

<reasoning id='rs_68b8117a016c8195a7448bd1bf46fa070379010263910ce8'>
<encrypted>gAAAAABouBF6UpbljJCp...S5UTo=</encrypted>
</reasoning>

<reasoning id='rs_68b8117a016c8195a7448bd1bf46fa070379010263910ce8'><encrypted>gAAAAABouBGH8InM3D4aZe3...3GxZpoq9tU4kY=</encrypted>
</reasoning>

Additional pros
...
Additional cons
...
--- DONE ---


<reasoning id='rs_68b8117a016c8195a7448bd1bf46fa070379010263910ce8'>
<encrypted>gAAAAABouBGPvHXr...eZ_EinWqh8X0A=</encrypted>
</reasoning>
{ usage": { "input_tokens": 439,  "output_tokens": 1040,  "total_tokens": 1479, "reasoning_tokens": 576  }
}

Summary of what needs to be made public for this to work:

InternalIncludable.ReasoningEncryptedContent
InternalResponseReasoningSummaryPartAddedEvent
InternalResponseReasoningSummaryPartDoneEvent
InternalResponseReasoningSummaryTextDeltaEvent

(but I'd take everything under HandleStreamResponseItem and HandleResponseUpdate that is marked Internal*...)

and new ctor for ReasoningResponseItem that accepts encryptedContent and Id

[rogerbarreto]

Would you please provide additional context around your end-to-end scenario and where you're running into difficulties?

@jsquire, @gaspardpetit Added more context to it.

This is a very desirable feature and basically today we can't stream out thinking contents as a modeled out public API.

switch (streamingUpdate)
{
     // There's no Thinking Delta available like this
     case ResponseReasoningSummaryTextDeltaUpdate u:
content)
        Console.Write(u.Delta);
        break;
}

[jsquire]

Thanks, @gaspardpetit. That was incredibly helpful to understand the full end-to-end scenario. Looking over the SSE types highlighted, they appear in both the OpenAI platform docs and seem to all be covered in the REST API spec. They should move to public.