Skip to content
Closed
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
83 changes: 62 additions & 21 deletions src/content/docs/workers-ai/configuration/bindings.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,6 @@ pcx_content_type: configuration
title: Workers Bindings
sidebar:
order: 1

---

import { Type, MetaInfo } from "~/components";
Expand Down Expand Up @@ -40,33 +39,75 @@ To configure a Workers AI binding in your Pages Function, you must use the Cloud
`async env.AI.run()` runs a model. Takes a model as the first parameter, and an object as the second parameter.

```javascript
const answer = await env.AI.run('@cf/meta/llama-3.1-8b-instruct', {
prompt: "What is the origin of the phrase 'Hello, World'"
const answer = await env.AI.run("@cf/meta/llama-3.1-8b-instruct", {
prompt: "What is the origin of the phrase 'Hello, World'",
});
```

**Parameters**
```javascript
const answer = await env.AI.run("@cf/meta/llama-3.1-8b-instruct", {
prompt: "What is the origin of the phrase 'Hello, World'",
stream: true,
});

return new Response(answer, {
headers: { "content-type": "text/event-stream" },
});
```

**Parameters**

* `model` <Type text="string" /> <MetaInfo text="required" />
- `model` <Type text="string" /> <MetaInfo text="required" />

* The model to run.
- The model to run.

**Supported options**

* `stream` <Type text="boolean" /> <MetaInfo text="optional" />
* Returns a stream of results as they are available.



```javascript
const answer = await env.AI.run('@cf/meta/llama-3.1-8b-instruct', {
prompt: "What is the origin of the phrase 'Hello, World'",
stream: true
});

return new Response(answer, {
headers: { "content-type": "text/event-stream" }
});
```
- `prompt` <Type text="string" /> <MetaInfo text="optional" />
- Text prompt for the text-generation (maxLength: 131072, minLength: 1).
- `raw` <Type text="boolean" /> <MetaInfo text="optional" />
- If true, a chat template is not applied and you must adhere to the specific model's expected formatting.
- `stream` <Type text="boolean" /> <MetaInfo text="optional" />
- If true, the response will be streamed back incrementally using SSE, Server Sent Events.
- `max_tokens` <Type text="number" /> <MetaInfo text="optional" />
- The maximum number of tokens to generate in the response.
- `temperature` <Type text="number" /> <MetaInfo text="optional" />
- Controls the randomness of the output; higher values produce more random results (maximum: 5, minimum: 0).
- `top_p` <Type text="number" /> <MetaInfo text="optional" />
- Adjusts the creativity of the AI's responses by controlling how many possible words it considers. Lower values make outputs more predictable; higher values allow for more varied and creative responses (maximum: 2, minimum: 0).
- `top_k` <Type text="number" /> <MetaInfo text="optional" />
- Limits the AI to choose from the top 'k' most probable words. Lower values make responses more focused; higher values introduce more variety and potential surprises (maximum: 50, minimum: 1).
- `seed` <Type text="number" /> <MetaInfo text="optional" />
- Random seed for reproducibility of the generation (maximum: 9999999999, minimum: 1).
- `repetition_penalty` <Type text="number" /> <MetaInfo text="optional" />
- Penalty for repeated tokens; higher values discourage repetition (maximum: 2, minimum: 0).
- `frequency_penalty` <Type text="number" /> <MetaInfo text="optional" />
- Decreases the likelihood of the model repeating the same lines verbatim (maximum: 2, minimum: 0).
- `presence_penalty` <Type text="number" /> <MetaInfo text="optional" />
- Increases the likelihood of the model introducing new topics (maximum: 2, minimum: 0).
- `messages` <Type text="{
role: &quot;user&quot; | &quot;assistant&quot; | &quot;system&quot; | &quot;tool&quot; | (string & NonNullable<unknown>);
content: string;
name?: string;
}[]" /> <MetaInfo text="optional" /> \* An array of message objects representing the conversation history.
- `tools` <Type text="{
type: &quot;function&quot; | (string & NonNullable<unknown>);
function: {
name: string;
description: string;
parameters?: {
type: &quot;object&quot; | (string & NonNullable<unknown>);
properties: {
[key: string]: {
type: string;
description?: string;
};
};
required: string[];
};
};
}[]" /> <MetaInfo text="optional" /> \* A list of tools available for the assistant to use.
- `functions` <Type text="{
name: string;
code: string;
}[]" /> <MetaInfo text="optional" /> \* A list of functions available for the assistant to use.
Loading