Preserve function content in SummarizingChatReducer
#6908
Conversation
There was a problem hiding this comment.
Pull Request Overview
Fixes an issue in SummarizingChatReducer where function call content was being removed even when not summarized. The PR preserves function call and result content in the returned message list and improves message boundary detection to avoid orphaning assistant responses from their user messages.
Key changes:
- Preserves
FunctionCallContentandFunctionResultContentin message lists - Implements backward search within threshold window for user message boundaries
- Updates logic to keep complete function call sequences together
Reviewed Changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.
| File | Description |
|---|---|
src/Libraries/Microsoft.Extensions.AI/ChatReduction/SummarizingChatReducer.cs |
Core implementation changes to preserve function content and improve message boundary detection |
test/Libraries/Microsoft.Extensions.AI.Tests/ChatReduction/SummarizingChatReducerTests.cs |
Updated and added tests to verify function content preservation and threshold behavior |
src/Libraries/Microsoft.Extensions.AI/ChatReduction/SummarizingChatReducer.cs
Show resolved
Hide resolved
| // We want to keep complete function call sequences together with their responses | ||
| while (indexOfFirstMessageToKeep > 0) | ||
| { | ||
| if (!unsummarizedMessages[indexOfFirstMessageToKeep - 1].Contents.Any(c => c is FunctionCallContent or FunctionResultContent)) |
There was a problem hiding this comment.
There's also UserInputRequestContent / UserInputResponseContent. Does this need to pay attention to those as well?
| // Start at the ideal cut point (keeping exactly targetCount messages) | ||
| var indexOfFirstMessageToKeep = unsummarizedMessages.Count - targetCount; | ||
|
|
||
| // Move backward to skip over function call/result content at the boundary |
There was a problem hiding this comment.
So is this never getting rid of any FunctionCall/ResultContent? Does that mean in a really long conversation we could end up with lots and lots of these, and eventually still overflow context windows?
2 participants
Summary
Fixes an issue where
SummarizingChatReduceralways removes function call content, even if it didn't get summarized.Description
This PR makes the following changes to
SummarizingChatReducer:FunctionCallContentandFunctionResultContentnow get preserved in the returned message list.targetCount + thresholdmessages.Fixes #6890
Microsoft Reviewers: Open in CodeFlow