Skip to content

Python: fix: coerce dict response_format to typed Azure AI model to fix Azure Monitor compatibility#13912

Open
octo-patch wants to merge 2 commits intomicrosoft:mainfrom
octo-patch:fix/issue-13715-coerce-response-format-for-azure-monitor
Open

Python: fix: coerce dict response_format to typed Azure AI model to fix Azure Monitor compatibility#13912
octo-patch wants to merge 2 commits intomicrosoft:mainfrom
octo-patch:fix/issue-13715-coerce-response-format-for-azure-monitor

Conversation

@octo-patch
Copy link
Copy Markdown

Fixes #13715

Problem

When AzureAIAgent is used inside AgentGroupChat with Azure Monitor (configure_azure_monitor) enabled, all agent executions fail with:

ValueError: Unknown response format <class 'dict'>

The Azure AI telemetry instrumentor (_AIAgentsInstrumentorPreview.agent_api_response_to_str) only handles str, AgentsResponseFormatMode, AgentsResponseFormat, and ResponseFormatJsonSchemaType. When response_format is a plain Python dict (e.g. {"type": "json_object"}), none of the isinstance checks match and the instrumentor raises ValueError.

SK's AgentThreadActions._generate_options passes response_format as-is from the merged options dict, which can be a plain dict when users supply it that way or when agent.definition.response_format returns a raw mapping.

Solution

Add a _coerce_response_format() static method to AgentThreadActions that converts a plain dict to the appropriate typed Azure AI SDK model before it is forwarded to agents.runs.create():

  • {"type": "json_schema", ...}ResponseFormatJsonSchemaType
  • Any other dictAgentsResponseFormat
  • Already-typed values (str, AgentsResponseFormatMode, AgentsResponseFormat, ResponseFormatJsonSchemaType, None) pass through unchanged.

_generate_options now calls _coerce_response_format on the merged response_format value, ensuring the telemetry instrumentor always receives a supported type.

Testing

  • Added unit tests in test_agent_thread_actions.py covering:
    • Passthrough for all already-correct types (None, str, AgentsResponseFormatMode, AgentsResponseFormat, ResponseFormatJsonSchemaType)
    • Plain dict with "type": "json_object"AgentsResponseFormat
    • Plain dict with "type": "json_schema"ResponseFormatJsonSchemaType
    • Unknown non-dict type passes through unchanged
  • All 14 existing unit tests in test_agent_thread_actions.py continue to pass.

octo-patch added 2 commits April 18, 2026 10:04
…n result logging (fixes microsoft#13681)

When a KernelFunction returns a type not registered in AbstractionsJsonContext
(e.g. Microsoft.Extensions.AI.TextContent from MCP tools), and JSON
serialization is attempted with AOT-safe JsonSerializerOptions, a
NotSupportedException is thrown and swallowed, causing the log message
to show "Failed to log function result value" instead of useful content.

This change adds a ToString() fallback in the NotSupportedException catch
block so that result values still produce meaningful log output even when
their runtime type is absent from the source-generated JSON context.
…ix Azure Monitor compatibility

When response_format is passed as a plain Python dict (e.g. {"type":
"json_object"}), the Azure AI telemetry instrumentor raises
ValueError: Unknown response format <class 'dict'> because it only
handles the typed SDK models (AgentsResponseFormat,
ResponseFormatJsonSchemaType, AgentsResponseFormatMode, str).

Add AgentThreadActions._coerce_response_format() to convert plain
dicts to the appropriate typed model before the options dict is
forwarded to agents.runs.create(), preventing the telemetry error
when Azure Monitor is enabled.

Fixes microsoft#13715
@octo-patch octo-patch requested review from a team as code owners April 23, 2026 02:48
@moonbox3 moonbox3 added .NET Issue or Pull requests regarding .NET code python Pull requests for the Python Semantic Kernel kernel Issues or pull requests impacting the core kernel kernel.core labels Apr 23, 2026
@github-actions github-actions Bot changed the title Python: fix: coerce dict response_format to typed Azure AI model to fix Azure Monitor compatibility .Net: Python: fix: coerce dict response_format to typed Azure AI model to fix Azure Monitor compatibility Apr 23, 2026
@github-actions github-actions Bot changed the title .Net: Python: fix: coerce dict response_format to typed Azure AI model to fix Azure Monitor compatibility Python: fix: coerce dict response_format to typed Azure AI model to fix Azure Monitor compatibility Apr 23, 2026
Copy link
Copy Markdown
Contributor

@github-actions github-actions Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated Code Review

Reviewers: 4 | Confidence: 90%

✓ Correctness

This PR adds two independent improvements: (1) a C# fallback to ToString() when JSON serialization fails with NotSupportedException in KernelFunctionLogMessages, and (2) a Python _coerce_response_format method that converts plain dict response_format values to typed Azure SDK models to avoid telemetry errors. Both changes are correct. The C# catch chain properly captures the NotSupportedException and falls back gracefully. The Python coerce method correctly handles all response_format variants — I verified that AgentsResponseFormat(dict) and ResponseFormatJsonSchemaType(dict) constructors work correctly with the installed Azure AI agents SDK (1.2.0b6). The isinstance checks in the passthrough path are also valid. Tests are comprehensive and cover passthrough, conversion, and unknown-type scenarios.

✓ Security Reliability

This PR makes two defensive improvements: (1) a C# logging fallback that uses ToString() when JSON serialization throws NotSupportedException (e.g., in AOT scenarios with unregistered types), and (2) a Python coercion helper that converts plain-dict response_format values into typed Azure AI SDK models to avoid ValueError in telemetry instrumentation. Both changes are well-scoped, maintain existing error-handling patterns, and include good test coverage. No security or reliability issues were identified.

✓ Test Coverage

The PR adds a ToString() fallback in the .NET KernelFunctionLogMessages when JSON serialization throws NotSupportedException, and adds a _coerce_response_format helper in the Python Azure AI agent thread actions to convert plain dicts to typed SDK models. Both changes include new tests. The .NET test covers the happy-path ToString() fallback but not the inner catch block where ToString() itself throws. The Python tests thoroughly cover _coerce_response_format in isolation but don't verify the integration point in _generate_options. Overall, test quality is good with meaningful assertions, but one untested code path in the .NET change is worth noting.

✗ Design Approach

I found one blocking design issue in the .NET change: the new serialization fallback bypasses FunctionResult's own string-conversion abstraction and reaches directly into Value.ToString(), which can produce different output than the rest of Semantic Kernel for culture-sensitive or custom-converted values. The Python change looks directionally reasonable for the telemetry problem, but it currently widens accepted response_format shapes only inside an internal helper while the Azure AI public API still advertises a narrower contract.

Suggestions

  • [dotnet] The inner catch block (lines 171-174 in the diff) that handles the case where ToString() itself throws is not covered by any test. Consider adding a test with a type whose ToString() throws, verifying the log message falls back to "Failed to log function result value" with the original NotSupportedException.
  • [python] Consider moving the response_format normalization to the Azure AI public API boundary as well. AgentThreadActions now accepts/coerces plain dicts internally, but the exposed contract is still AgentsApiResponseFormatOption = str | ResponseFormatJsonSchemaType, so the widened accepted shapes remain undocumented and unvalidated until the internal helper call.

Automated review by octo-patch's agents


[JsonSerializable(typeof(IDictionary<string, object?>))]
[JsonSerializable(typeof(object))]
private sealed partial class RestrictedJsonContext : JsonSerializerContext { }
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Test coverage gap: there is no test for the inner catch block in LogFunctionResultValueInternal (lines 171-174 of KernelFunctionLogMessages.cs). When ToString() itself throws, the code should fall back to logging "Failed to log function result value" with the original NotSupportedException. Consider adding a test with a type whose ToString() throws, using the same RestrictedJsonContext, and asserting that the error message and exception are logged.

Suggested change
private sealed partial class RestrictedJsonContext : JsonSerializerContext { }
[Fact]
public void ItShouldLogFailureMessageWhenToStringAlsoFails()
{
// Arrange
var logger = new Mock<ILogger>();
logger.Setup(l => l.IsEnabled(It.IsAny<LogLevel>()).Returns(true);
var throwingValue = new TypeWhoseToStringThrows();
var functionResult = new FunctionResult(KernelFunctionFactory.CreateFromMethod(() => { }), throwingValue);
var restrictedOptions = RestrictedJsonContext.Default.Options;
// Act
logger.Object.LogFunctionResultValue("p1", "f1", functionResult, restrictedOptions);
// Assert - should fall back to the error message
logger.Verify(l => l.Log(
LogLevel.Trace,
0,
It.Is<It.IsAnyType>((o, _) => o.ToString() == "Function p1-f1 result: Failed to log function result value"),
It.IsAny<NotSupportedException>(),
It.IsAny<Func<It.IsAnyType, Exception?, string>>()));
}
private sealed class TypeWhoseToStringThrows
{
public override string ToString() => throw new InvalidOperationException("ToString failed");
}
[JsonSerializable(typeof(IDictionary<string, object?>))]
[JsonSerializable(typeof(object))]
private sealed partial class RestrictedJsonContext : JsonSerializerContext { }

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

kernel.core kernel Issues or pull requests impacting the core kernel .NET Issue or Pull requests regarding .NET code python Pull requests for the Python Semantic Kernel

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Bug: [Python] AgentGroupChat fails with "Unknown response format <class 'dict'>" when Azure Monitor is enabled

2 participants