Skip to content

Commit 0e8291e

Browse files
tsuzaki430gr2m
andauthored
chore,docs(provider/azure):azure responses image-generation streaming is enabled (#10391)
## Background The image_generation tool's streaming mode is now available on Microsoft Azure platform , https://learn.microsoft.com/en-us/azure/ai-foundry/openai/how-to/dall-e?view=foundry-classic&tabs=gpt-image-1#streaming ## Summary After image_generation became enabled with streaming on the Azure side, streamText worked as expected without requiring any changes to doStream in the AI SDK. Therefore, I concluded that we can close this issue by adding examples and tests for Next.js, and updating the documentation accordingly. - fix documents. - rework azure CICD test for image_generation. - fix example in examples/ai-core. - add example in examples/next-openai. ## Manual Verification - add cicd test for package @ai-sdk/azure build. - streamText with image generation in `examples/ai-core` `pnpm tsx src/stream-text/azure-image-generation-tool.ts` - next.js useChat client test. `examples/next-openai/app/test-azure-image-generation/page.tsx` `http://localhost:3000/test-azure-image-generation` <img width="666" height="972" alt="image" src="https://github.com/user-attachments/assets/4bd1b649-d48e-4b8d-8ccf-d28000ca2280" /> ## Related Issues Fixes #9064 --------- Co-authored-by: tsuzaki430 <tsuzaki430@users.noreply.github.com> Co-authored-by: Gregor Martynus <39992+gr2m@users.noreply.github.com>
1 parent 92cd7ea commit 0e8291e

10 files changed

Lines changed: 463 additions & 141 deletions

File tree

content/providers/01-ai-sdk-providers/04-azure.mdx

Lines changed: 11 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -421,18 +421,9 @@ const result = await generateText({
421421

422422
#### Image Generation Tool
423423

424-
<Note type="warning">
425-
Azure OpenAI Responses API
426-
[image_generation](https://learn.microsoft.com/en-us/azure/ai-foundry/openai/how-to/responses?tabs=python-secure#image-generation-preview)
427-
is now Preview release(Not GA release).Image tool does not currently support
428-
streaming mode. You can not use the image tool with `streamText` currently.
429-
</Note>
430-
431424
Azure OpenAI's Responses API supports multi-modal image generation as a provider-defined tool.
432425
Availability is restricted to specific models (for example, `gpt-5` variants).
433426

434-
You can use the image tool with `generateText`.
435-
436427
```ts
437428
import { createAzure } from '@ai-sdk/azure';
438429
import { generateText } from 'ai';
@@ -459,6 +450,17 @@ for (const toolResult of result.staticToolResults) {
459450
}
460451
```
461452

453+
<Note>
454+
The tool must be named `image_generation` when using Azure OpenAI's image
455+
generation functionality. This name is required by Azure OpenAI's API
456+
specification and cannot be customized.
457+
</Note>
458+
459+
<Note>
460+
The 'image_generation' tool is only supported with the default responses API,
461+
and is not supported when using 'azure.chat' or 'azure.completion'
462+
</Note>
463+
462464
<Note>
463465
To use image_generation, you must first create an image generation model. You
464466
must add a deployment specification to the header

examples/ai-core/src/stream-text/azure-image-generation-tool.ts

Lines changed: 0 additions & 21 deletions
Original file line numberDiff line numberDiff line change
@@ -4,27 +4,6 @@ import { presentImages } from '../lib/present-image';
44
import { run } from '../lib/run';
55
import { convertBase64ToUint8Array } from '../lib/convert-base64';
66

7-
/**
8-
*
9-
* *** NOTICE ***
10-
* The image_generation function is currently preview(Not GA).
11-
* Unfortunately ,This example code does not work, now.
12-
* Because image_generation tool is not supported stream mode on Azure OpenAI, yet.
13-
* So it doesn't work on streamText function.
14-
*
15-
* This example finish error with this message.
16-
* "ImageGen as a tool is not supported in streaming mode."
17-
*
18-
*
19-
* ` The Responses API image generation tool does not currently support streaming mode. `
20-
* link:
21-
* https://learn.microsoft.com/en-us/azure/ai-foundry/openai/how-to/responses?tabs=python-secure#image-generation-preview
22-
*
23-
*
24-
* When updated on Azure , it will work on streamText function in the future.
25-
* And then this example code will be fixed.
26-
*/
27-
287
run(async () => {
298
const azure = createAzure({
309
headers: {
Lines changed: 24 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,24 @@
1+
import { createAzure, azure } from '@ai-sdk/azure';
2+
import { InferAgentUIMessage, ToolLoopAgent } from 'ai';
3+
4+
export const azureImageGenerationAgent = new ToolLoopAgent({
5+
model: createAzure({
6+
headers: {
7+
'x-ms-oai-image-generation-deployment': 'gpt-image-1', // use your own image model deployment
8+
},
9+
})('gpt-4.1-mini'),
10+
tools: {
11+
image_generation: azure.tools.imageGeneration({
12+
partialImages: 3,
13+
quality: 'low',
14+
size: '1024x1024',
15+
}),
16+
},
17+
onStepFinish: ({ request }) => {
18+
console.log(JSON.stringify(request.body, null, 2));
19+
},
20+
});
21+
22+
export type AzureImageGenerationMessage = InferAgentUIMessage<
23+
typeof azureImageGenerationAgent
24+
>;
Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
1+
import { createAgentUIStreamResponse } from 'ai';
2+
import { azureImageGenerationAgent } from '@/agent/azure-image-generation-agent';
3+
4+
export async function POST(req: Request) {
5+
const body = await req.json();
6+
7+
return createAgentUIStreamResponse({
8+
agent: azureImageGenerationAgent,
9+
uiMessages: body.messages,
10+
});
11+
}
Lines changed: 40 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,40 @@
1+
'use client';
2+
3+
import { AzureImageGenerationMessage } from '@/agent/azure-image-generation-agent';
4+
import ChatInput from '@/components/chat-input';
5+
import ImageGenerationView from '@/components/tool/openai-image-generation-view';
6+
import { useChat } from '@ai-sdk/react';
7+
import { DefaultChatTransport } from 'ai';
8+
9+
export default function TestOpenAIImageGeneration() {
10+
const { status, sendMessage, messages } =
11+
useChat<AzureImageGenerationMessage>({
12+
transport: new DefaultChatTransport({
13+
api: '/api/chat-azure-image-generation',
14+
}),
15+
});
16+
17+
return (
18+
<div className="flex flex-col py-24 mx-auto w-full max-w-md stretch">
19+
<h1 className="mb-4 text-xl font-bold">
20+
Azure OpenAI Image Generation Test
21+
</h1>
22+
23+
{messages.map(message => (
24+
<div key={message.id} className="whitespace-pre-wrap">
25+
{message.role === 'user' ? 'User: ' : 'AI: '}
26+
{message.parts.map((part, index) => {
27+
switch (part.type) {
28+
case 'text':
29+
return <div key={index}>{part.text}</div>;
30+
case 'tool-image_generation':
31+
return <ImageGenerationView key={index} invocation={part} />;
32+
}
33+
})}
34+
</div>
35+
))}
36+
37+
<ChatInput status={status} onSubmit={text => sendMessage({ text })} />
38+
</div>
39+
);
40+
}

packages/azure/src/__fixtures__/azure-image-generation-tool.1.chunks.txt

Lines changed: 41 additions & 0 deletions
Large diffs are not rendered by default.

packages/azure/src/__fixtures__/azure-image-generation-tool.1.json

Lines changed: 88 additions & 0 deletions
Large diffs are not rendered by default.

packages/azure/src/__fixtures__/openai-image-generation-tool.1.json

Lines changed: 0 additions & 96 deletions
This file was deleted.

packages/azure/src/__snapshots__/azure-openai-provider.test.ts.snap

Lines changed: 227 additions & 14 deletions
Large diffs are not rendered by default.

packages/azure/src/azure-openai-provider.test.ts

Lines changed: 21 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1252,7 +1252,7 @@ describe('responses', () => {
12521252
let result: LanguageModelV3GenerateResult;
12531253

12541254
beforeEach(async () => {
1255-
prepareJsonFixtureResponse('openai-image-generation-tool.1');
1255+
prepareJsonFixtureResponse('azure-image-generation-tool.1');
12561256

12571257
result = await createModel('test-deployment').doGenerate({
12581258
prompt: TEST_PROMPT,
@@ -1764,4 +1764,24 @@ describe('responses', () => {
17641764
).toMatchSnapshot();
17651765
});
17661766
});
1767+
describe('image generation tool', () => {
1768+
it('should stream image generation tool results include', async () => {
1769+
prepareChunksFixtureResponse('azure-image-generation-tool.1');
1770+
const result = await createModel('test-deployment').doStream({
1771+
prompt: TEST_PROMPT,
1772+
tools: [
1773+
{
1774+
type: 'provider',
1775+
id: 'openai.image_generation',
1776+
name: 'image_generation',
1777+
args: {},
1778+
},
1779+
],
1780+
});
1781+
1782+
expect(
1783+
await convertReadableStreamToArray(result.stream),
1784+
).toMatchSnapshot();
1785+
});
1786+
});
17671787
});

0 commit comments

Comments
 (0)