Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
127 changes: 127 additions & 0 deletions src/content/docs/workers-ai/configuration/ai-sdk.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,127 @@
---
pcx_content_type: configuration
title: AI SDK
sidebar:
order: 3

---

Workers AI can be used with the [AI SDK](https://sdk.vercel.ai/) for JavaScript and TypeScript codebases.

## Setup

Install the [`workers-ai-provider` provider](https://sdk.vercel.ai/providers/community-providers/cloudflare-workers-ai):

```bash
npm install workers-ai-provider
```

Then, add an AI binding in your Workers project `wrangler.toml` file:

```toml
[ai]
binding = "AI"
```

## Models

The AI SDK can be configured to work with [any AI model](/workers-ai/models/).

```js
import { createWorkersAI } from 'workers-ai-provider';

const workersai = createWorkersAI({ binding: env.AI });

// Choose any model: https://developers.cloudflare.com/workers-ai/models/
const model = workersai('@cf/meta/llama-3.1-8b-instruct', {});
```

## Generate Text

Once you have selected your model, you can generate text from a given prompt.

```js
import { createWorkersAI } from 'workers-ai-provider';
import { generateText } from 'ai';

type Env = {
AI: Ai;
};

export default {
async fetch(_: Request, env: Env) {
const workersai = createWorkersAI({ binding: env.AI });
const result = await generateText({
model: workersai('@cf/meta/llama-2-7b-chat-int8'),
prompt: 'Write a 50-word essay about hello world.',
});

return new Response(result.text);
},
};
```

## Stream Text

For longer responses, consider streaming responses to provide as the generation completes.

```js
import { createWorkersAI } from 'workers-ai-provider';
import { streamText } from 'ai';

type Env = {
AI: Ai;
};

export default {
async fetch(_: Request, env: Env) {
const workersai = createWorkersAI({ binding: env.AI });
const result = streamText({
model: workersai('@cf/meta/llama-2-7b-chat-int8'),
prompt: 'Write a 50-word essay about hello world.',
});

return result.toTextStreamResponse({
headers: {
// add these headers to ensure that the
// response is chunked and streamed
'Content-Type': 'text/x-unknown',
'content-encoding': 'identity',
'transfer-encoding': 'chunked',
},
});
},
};
```

## Generate Structured Objects

You can provide a Zod schema to generate a structured JSON response.

```js
import { createWorkersAI } from 'workers-ai-provider';
import { generateObject } from 'ai';
import { z } from 'zod';

type Env = {
AI: Ai;
};

export default {
async fetch(_: Request, env: Env) {
const workersai = createWorkersAI({ binding: env.AI });
const result = await generateObject({
model: workersai('@cf/meta/llama-3.1-8b-instruct'),
prompt: 'Generate a Lasagna recipe',
schema: z.object({
recipe: z.object({
ingredients: z.array(z.string()),
description: z.string(),
}),
}),
});

return Response.json(result.object);
},
};
```
1 change: 1 addition & 0 deletions src/content/docs/workers-ai/get-started/rest-api.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -72,3 +72,4 @@ By completing this guide, you have created a Cloudflare account (if you did not
## Related resources

- [Models](/workers-ai/models/) - Browse the Workers AI models catalog.
- [AI SDK](/workers-ai/configuration/ai-sdk) - Learn how to integrate with an AI model.
Original file line number Diff line number Diff line change
Expand Up @@ -141,3 +141,4 @@ By finishing this tutorial, you have created a Worker, connected it to Workers A

- [Cloudflare Developers community on Discord](https://discord.cloudflare.com) - Submit feature requests, report bugs, and share your feedback directly with the Cloudflare team by joining the Cloudflare Discord server.
- [Models](/workers-ai/models/) - Browse the Workers AI models catalog.
- [AI SDK](/workers-ai/configuration/ai-sdk) - Learn how to integrate with an AI model.
Loading