Skip to content

Commit 345fd5b

Browse files
celsocelso
authored
Longer context windows (#20220)
1 parent 22a3a9b commit 345fd5b

File tree

60 files changed

+1308
-1193
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

60 files changed

+1308
-1193
lines changed

‎public/calls/static/calls-api-2024-05-21.yaml‎

Lines changed: 1 addition & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -502,7 +502,7 @@ components:
502502
errorCode:
503503
type: string
504504
errorDescription:
505-
type: string
505+
type: string
506506
requiresImmediateRenegotiation:
507507
type: boolean
508508
GetSessionStateResponse:
@@ -538,6 +538,3 @@ components:
538538
type: string
539539
sessionDescription:
540540
$ref: "#/components/schemas/SessionDescription"
541-
542-
543-

‎src/components/models/ModelFeatures.tsx‎

Lines changed: 118 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,118 @@
1+
import type { WorkersAIModelsSchema } from "~/schemas";
2+
3+
const ModelFeatures = ({ model }: { model: WorkersAIModelsSchema }) => {
4+
const nf = new Intl.NumberFormat("en-US");
5+
const properties: any = {};
6+
model.properties.forEach((property: any) => {
7+
properties[property.property_id] = property.value;
8+
});
9+
10+
return (
11+
<>
12+
{Object.keys(properties).length ? (
13+
<>
14+
<table>
15+
<thead>
16+
<tr>
17+
<>
18+
<th>Features</th>
19+
<th />
20+
</>
21+
</tr>
22+
</thead>
23+
<tbody>
24+
{properties.planned_deprecation_date && (
25+
<tr>
26+
<td>
27+
{Date.now() >
28+
Math.floor(
29+
new Date(properties.planned_deprecation_date).getTime() /
30+
1000,
31+
)
32+
? "Deprecated"
33+
: "Planned Deprecation"}
34+
</td>
35+
<td>
36+
{new Date(
37+
properties.planned_deprecation_date,
38+
).toLocaleDateString("en-US")}
39+
</td>
40+
</tr>
41+
)}
42+
{properties.context_window && (
43+
<tr>
44+
<td>
45+
Context Window
46+
<a href="/workers-ai/glossary/">
47+
<span className="external-link"> ↗</span>
48+
</a>
49+
</td>
50+
<td>{nf.format(properties.context_window)} tokens</td>
51+
</tr>
52+
)}
53+
{properties.terms && (
54+
<tr>
55+
<td>Terms and License</td>
56+
<td>
57+
<a href={properties.terms} target="_blank">
58+
link<span className="external-link"> ↗</span>
59+
</a>
60+
</td>
61+
</tr>
62+
)}
63+
{properties.info && (
64+
<tr>
65+
<td>More information</td>
66+
<td>
67+
<a href={properties.info} target="_blank">
68+
link<span className="external-link"> ↗</span>
69+
</a>
70+
</td>
71+
</tr>
72+
)}
73+
{properties.max_input_tokens && (
74+
<tr>
75+
<td>Maximum Input Tokens</td>
76+
<td>{nf.format(properties.max_input_tokens)}</td>
77+
</tr>
78+
)}
79+
{properties.output_dimensions && (
80+
<tr>
81+
<td>Output Dimensions</td>
82+
<td>{nf.format(properties.output_dimensions)}</td>
83+
</tr>
84+
)}
85+
{properties.function_calling && (
86+
<tr>
87+
<td>
88+
Function calling{" "}
89+
<a href="/workers-ai/function-calling">
90+
<span className="external-link"> ↗</span>
91+
</a>
92+
</td>
93+
<td>Yes</td>
94+
</tr>
95+
)}
96+
{properties.lora && (
97+
<tr>
98+
<td>LoRA</td>
99+
<td>Yes</td>
100+
</tr>
101+
)}
102+
{properties.beta && (
103+
<tr>
104+
<td>Beta</td>
105+
<td>Yes</td>
106+
</tr>
107+
)}
108+
</tbody>
109+
</table>
110+
</>
111+
) : (
112+
false
113+
)}
114+
</>
115+
);
116+
};
117+
118+
export default ModelFeatures;

‎src/content/changelog/workers-ai/2025-02-24-context-windows.mdx‎

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
1+
---
2+
title: Workers AI larger context windows
3+
description: Updated Workers AI models with larger context windows
4+
date: 2025-02-24T11:00:00Z
5+
---
6+
7+
We've updated the Workers AI text generation models to include context windows and limits definitions and changed our APIs to estimate and validate the number of tokens in the input prompt, not the number of characters.
8+
9+
This update allows developers to use larger context windows when interacting with Workers AI models, which can lead to better and more accurate results.
10+
11+
Our [catalog page](/workers-ai/models/) provides more information about each model's supported context window.

‎src/content/glossary/workers-ai.yaml‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -4,6 +4,10 @@ entries:
44
- term: Workers AI
55
general_definition: |-
66
[Workers AI](/workers-ai/) is a Cloudflare service that enables running machine learning models on Cloudflare's global network, utilizing serverless GPUs. It allows developers to integrate AI capabilities into their applications using Workers, Pages, or via the REST API.
7+
- term: Context Window
8+
general_definition: In generative AI, the context window is the sum of the number of input, reasoning, and completion or response tokens a model supports. You can find the context window limit on each [model page](/workers-ai/models/).
9+
- term: Maximum Tokens
10+
general_definition: In generative AI, the user-defined property `max_tokens` defines the maximum number of tokens at which the model should stop responding. This limit cannot exceed the context window.
711
- term: Serverless GPUs
812
general_definition: |-
913
[Serverless GPUs](/workers-ai/) are graphics processing units provided by Cloudflare in a serverless environment, enabling scalable and efficient execution of machine learning models without the need for managing underlying hardware.

‎src/content/glossary/workers.yaml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -131,4 +131,4 @@ entries:
131131
132132
- term: wrangler.toml / wrangler.json / wrangler.jsonc
133133
general_definition: |-
134-
The [configuration](/workers/wrangler/configuration/) used to customize the development and deployment setup for a Worker or a Pages Function.
134+
The [configuration](/workers/wrangler/configuration/) used to customize the development and deployment setup for a Worker or a Pages Function.

‎src/content/products/automatic-platform-optimization.yaml‎

Lines changed: 1 addition & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -8,8 +8,7 @@ product:
88

99
meta:
1010
title: Cloudflare Automatic Platform Optimization docs
11-
description:
12-
Serve your WP site from Cloudflare's edge, ensuring improved performance.
11+
description: Serve your WP site from Cloudflare's edge, ensuring improved performance.
1312
author: "@cloudflare"
1413

1514
resources:

‎src/content/release-notes/api-deprecations.yaml‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -9,11 +9,11 @@ entries:
99
title: Cloudflare DWeb Resolver
1010
description: |-
1111
Deprecation date: July 1, 2025
12-
12+
1313
The Cloudflare DWeb Resolver experiment is ending.
14-
14+
1515
Deprecated APIs:
16-
16+
1717
- DoH resolver on resolver.cloudflare-eth.com
1818
1919
- publish_date: "2025-03-23"

‎src/content/release-notes/api-shield.yaml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -14,7 +14,7 @@ entries:
1414
title: API Authentication Posture
1515
description: |-
1616
Customers will see per-endpoint authentication details inside API Shield's [Endpoint Management](/api-shield/management-and-monitoring/) for zones with configured session identifiers.
17-
17+
1818
- publish_date: "2024-12-19"
1919
title: Automatically applied endpoint risk labels
2020
description: |-

‎src/content/release-notes/dns.yaml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -29,4 +29,4 @@ entries:
2929
- Automatically quoted TXT content upon save if no quotes exist in the record content field.
3030
- publish_date: "2024-10-07"
3131
title: API support for per-record CNAME flattening
32-
description: Paid zones now have the option to flatten specific CNAME records. When using the API, specify the setting `cname_flatten` as `true` or `false`. Refer to the [documentation](/dns/cname-flattening/set-up-cname-flattening/#per-record) for details.
32+
description: Paid zones now have the option to flatten specific CNAME records. When using the API, specify the setting `cname_flatten` as `true` or `false`. Refer to the [documentation](/dns/cname-flattening/set-up-cname-flattening/#per-record) for details.

‎src/content/release-notes/durable-objects.yaml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -60,4 +60,4 @@ entries:
6060
- publish_date: "2024-02-15"
6161
title: Optional `alarmInfo` parameter for Durable Object Alarms
6262
description: |-
63-
Durable Objects [Alarms](/durable-objects/api/alarms/) now have a new `alarmInfo` argument that provides more details about an alarm invocation, including the `retryCount` and `isRetry` to signal if the alarm was retried.
63+
Durable Objects [Alarms](/durable-objects/api/alarms/) now have a new `alarmInfo` argument that provides more details about an alarm invocation, including the `retryCount` and `isRetry` to signal if the alarm was retried.

0 commit comments

Comments
 (0)