.NET: [Bug]: Uploading images to Gemini causes Bad Request

[callendorph]

Description

I used the template defined in this project: Microsoft.Agents.AI.ProjectTemplates

I first tried setting the open-ai client to point at the gemini compatible endpoint https://generativelanguage.googleapis.com/v1beta/openai/. I can build and then run the project, open devUI in the browser and I see this:

I get an arbitrary "Bad Response" with no further information.

I then installed GeminiDotnet to use its IChatClient implementation instead. When I run this I get something similar but it tells me that the issue is the MIME type:

If I look through the code, I find that the issue is here.

On this line, the image's MIME type is converted to "image/*" arbitrarily. If you set a breakpoint there for a Data URI, you will see that the input content is data:image/png;base64,iVBORw0KGgo... with an explicit MIME type.

It seems that this code is coming from something labeled Microsoft.Agents.AI.Hosting.OpenAI which makes me think it is an OpenAI specific interface but that makes the IChatClient as a generic interface pretty ineffectual.

Code Sample

using System.ClientModel;
using System.ComponentModel;
using Microsoft.Agents.AI;
using Microsoft.Agents.AI.DevUI;
using Microsoft.Agents.AI.Hosting;
using Microsoft.Agents.AI.Workflows;
using Microsoft.Extensions.AI;
using OpenAI;
using OpenAI.Chat;
using OpenTelemetry;
using OpenTelemetry.Logs;
using OpenTelemetry.Resources;
using OpenTelemetry.Trace;

using GeminiDotnet;
using GeminiDotnet.Extensions.AI;

var builder = WebApplication.CreateBuilder(args);

// var options = new GeminiClientOptions
// {
//     ApiKey = builder.Configuration["GEMINI_TOKEN"] ?? throw new InvalidOperationException("Missing configuration: GEMINI_TOKEN"),
//     ModelId = "gemini-3-flash-preview",
// };

// var gClient = new GeminiChatClient(options); 

// IChatClient chatClient = new ChatClientBuilder(gClient)
//     .UseFunctionInvocation()
//     .Build();    

var chatClient = new ChatClient(
    //"gemini-3-flash-preview",
    "gemini-2.5-flash",
    new ApiKeyCredential(builder.Configuration["GEMINI_TOKEN"] ?? throw new InvalidOperationException("Missing configuration: GEMINI_TOKEN")),
    new OpenAIClientOptions { Endpoint = new Uri("https://generativelanguage.googleapis.com/v1beta/openai/") })
.AsIChatClient(); 


builder.Services.AddChatClient(chatClient);

builder.AddAIAgent("writer", "You write short stories (300 words or less) about the specified topic.");

builder.AddAIAgent("editor", (sp, key) => new ChatClientAgent(
    chatClient,
    name: key,
    instructions: "You edit short stories to improve grammar and style, ensuring the stories are less than 300 words. Once finished editing, you select a title and format the story for publishing.",
    tools: [AIFunctionFactory.Create(FormatStory)]
));

builder.AddWorkflow("publisher", (sp, key) => AgentWorkflowBuilder.BuildSequential(
    workflowName: key,
    agents:
    [
        sp.GetRequiredKeyedService<AIAgent>("writer"),
        sp.GetRequiredKeyedService<AIAgent>("editor")
    ]
)).AddAsAIAgent("publisher-agent");

// Register services for OpenAI responses and conversations (also required for DevUI)
builder.Services.AddOpenAIResponses();
builder.Services.AddOpenAIConversations();

var app = builder.Build();
//app.UseHttpsRedirection();

// Map endpoints for OpenAI responses and conversations (also required for DevUI)
app.MapOpenAIResponses();
app.MapOpenAIConversations();

if (builder.Environment.IsDevelopment())
{
    // Map DevUI endpoint to /devui
    app.MapDevUI();
}

app.Run();

[Description("Formats the story for publication, revealing its title.")]
string FormatStory(string title, string story) => $"""
    **Title**: {title}

    {story}
    """;

Error Messages / Stack Traces

No stacktrace in the output.

Package Versions

Microsoft.Agents.AI: 1.0.0-preview.251219.1

.NET Version

.Net 10.0

Additional Context

No response

[callendorph]

Minor addendum - I can make text only requests and they complete successfully.