Skip to content

Commit dc700fb

Browse files
daisyfaithaumahyperlint-ai[bot]Oxyjun
authored
hid overviews on ai gateway (#16476)
* hid overviews * spelling error fixed * hid overview on pages * Update src/content/docs/ai-gateway/providers/vertex.mdx Co-authored-by: hyperlint-ai[bot] <154288675+hyperlint-ai[bot]@users.noreply.github.com> * Update src/content/docs/ai-gateway/tutorials/index.mdx Fixed hide overview Co-authored-by: Jun Lee <[email protected]> * Corrected issues on overview --------- Co-authored-by: hyperlint-ai[bot] <154288675+hyperlint-ai[bot]@users.noreply.github.com> Co-authored-by: Jun Lee <[email protected]>
1 parent 1d3c79c commit dc700fb

File tree

21 files changed

+227
-234
lines changed

21 files changed

+227
-234
lines changed

.vscode/settings.json

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,5 +2,6 @@
22
"astro.content-intellisense": true,
33
"editor.formatOnSave": true,
44
"editor.defaultFormatter": "esbenp.prettier-vscode",
5-
"typescript.tsdk": "node_modules/typescript/lib"
5+
"typescript.tsdk": "node_modules/typescript/lib",
6+
"cSpell.enableFiletypes": ["mdx"]
67
}

src/content/docs/ai-gateway/configuration/index.mdx

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -2,11 +2,12 @@
22
pcx_content_type: navigation
33
title: Configuration
44
sidebar:
5+
group:
6+
hideIndex: true
57
order: 5
6-
78
---
89

9-
import { DirectoryListing } from "~/components"
10+
import { DirectoryListing } from "~/components";
1011

1112
Configure your AI Gateway with multiple options and customizations.
1213

src/content/docs/ai-gateway/demos.mdx

Lines changed: 12 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -2,16 +2,24 @@
22
pcx_content_type: navigation
33
title: Architectures
44
sidebar:
5-
order: 5
6-
5+
group:
6+
hideIndex: true
7+
order: 5
78
---
89

9-
import { GlossaryTooltip, ResourcesBySelector } from "~/components"
10+
import { GlossaryTooltip, ResourcesBySelector } from "~/components";
1011

1112
Learn how you can use AI Gateway within your existing architecture.
1213

1314
## Reference architectures
1415

1516
Explore the following <GlossaryTooltip term="reference architecture">reference architectures</GlossaryTooltip> that use AI Gateway:
1617

17-
<ResourcesBySelector types={["reference-architecture","design-guide","reference-architecture-diagram"]} products={["AI Gateway"]} />
18+
<ResourcesBySelector
19+
types={[
20+
"reference-architecture",
21+
"design-guide",
22+
"reference-architecture-diagram",
23+
]}
24+
products={["AI Gateway"]}
25+
/>

src/content/docs/ai-gateway/get-started.mdx

Lines changed: 10 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -4,20 +4,19 @@ title: Get started
44
updated: 2024-06-19
55
sidebar:
66
order: 2
7-
87
---
98

10-
11-
12-
import { Details, DirectoryListing, LinkButton, Render } from "~/components"
9+
import { Details, DirectoryListing, LinkButton, Render } from "~/components";
1310

1411
In this guide, you will learn how to create your first AI Gateway. You can create multiple gateways to control different applications.
1512

1613
## Prerequisites
1714

1815
Before you get started, you need a Cloudflare account.
1916

20-
<LinkButton variant="primary" href="https://dash.cloudflare.com/sign-up">Sign up</LinkButton>
17+
<LinkButton variant="primary" href="https://dash.cloudflare.com/sign-up">
18+
Sign up
19+
</LinkButton>
2120

2221
## Create gateway
2322

@@ -31,37 +30,33 @@ Next, connect your AI provider to your gateway.
3130

3231
AI Gateway offers multiple endpoints for each Gateway you create - one endpoint per provider, and one Universal Endpoint.
3332

34-
3533
<Details header="Supported providers">
3634

3735
<DirectoryListing folder="ai-gateway/providers" />
3836

39-
4037
</Details>
4138

4239
If you do not have a provider preference, start with one of our dedicated tutorials.
4340

44-
4541
<Details header="Onboarding tutorials">
4642

47-
* [OpenAI](/ai-gateway/tutorials/deploy-aig-worker/)
48-
* [Worker AI](/ai-gateway/tutorials/create-first-aig-workers/)
49-
43+
- [OpenAI](/ai-gateway/tutorials/deploy-aig-worker/)
44+
- [Worker AI](/ai-gateway/tutorials/create-first-aig-workers/)
5045

5146
</Details>
5247

5348
## View analytics
5449

5550
Now that your provider is connected to the AI Gateway, you can view analytics for requests going through your gateway.
5651

57-
<Render file="analytics-overview" /> <br/>
52+
<Render file="analytics-overview" /> <br />
5853

5954
<Render file="analytics-dashboard" />
6055

6156
<Render file="analytics-warning" />
6257

6358
## Next steps
6459

65-
* Learn more about [caching](/ai-gateway/configuration/caching/) for faster requests and cost savings and [rate limiting](/ai-gateway/configuration/rate-limiting/) to control how your application scales.
66-
* Explore how to specify model or provider [fallbacks](/ai-gateway/configuration/fallbacks/) for resiliency.
67-
* Learn how to use low-cost, open source models on [Workers AI](/ai-gateway/providers/workersai/) - our AI inference service.
60+
- Learn more about [caching](/ai-gateway/configuration/caching/) for faster requests and cost savings and [rate limiting](/ai-gateway/configuration/rate-limiting/) to control how your application scales.
61+
- Explore how to specify model or provider [fallbacks](/ai-gateway/configuration/fallbacks/) for resiliency.
62+
- Learn how to use low-cost, open source models on [Workers AI](/ai-gateway/providers/workersai/) - our AI inference service.

src/content/docs/ai-gateway/providers/anthropic.mdx

Lines changed: 10 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -1,10 +1,9 @@
11
---
22
title: Anthropic
33
pcx_content_type: get-started
4-
54
---
65

7-
[Anthropic](https://www.anthropic.com/) helps build reliable, interpretable, and steerable AI systems.
6+
[Anthropic](https://www.anthropic.com/) helps build reliable, interpretable, and steerable AI systems.
87

98
## Endpoint
109

@@ -29,25 +28,25 @@ curl https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id}/anthropic/v1
2928
If you are using the `@anthropic-ai/sdk`, you can set your endpoint like this:
3029

3130
```js title="JavaScript"
32-
import Anthropic from '@anthropic-ai/sdk';
31+
import Anthropic from "@anthropic-ai/sdk";
3332

3433
const apiKey = env.ANTHROPIC_API_KEY;
35-
const accountId = '{account_id}';
36-
const gatewayId = '{gateway_id}';
34+
const accountId = "{account_id}";
35+
const gatewayId = "{gateway_id}";
3736
const baseURL = `https://gateway.ai.cloudflare.com/v1/${accountId}/${gatewayId}/anthropic`;
3837

3938
const anthropic = new Anthropic({
40-
apiKey,
41-
baseURL,
39+
apiKey,
40+
baseURL,
4241
});
4342

44-
const model = 'claude-3-opus-20240229';
43+
const model = "claude-3-opus-20240229";
4544
const messages = [{ role: "user", content: "What is Cloudflare?" }];
4645
const maxTokens = 1024;
4746

4847
const message = await anthropic.messages.create({
49-
model,
50-
messages,
51-
max_tokens: maxTokens,
48+
model,
49+
messages,
50+
max_tokens: maxTokens,
5251
});
5352
```

src/content/docs/ai-gateway/providers/azureopenai.mdx

Lines changed: 14 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,6 @@
11
---
22
title: Azure OpenAI
33
pcx_content_type: get-started
4-
54
---
65

76
[Azure OpenAI](https://azure.microsoft.com/en-gb/products/ai-services/openai-service/) allows you apply natural language algorithms on your data.
@@ -14,11 +13,11 @@ pcx_content_type: get-started
1413

1514
When making requests to Azure OpenAI, you will need:
1615

17-
* AI Gateway account ID
18-
* AI Gateway gateway name
19-
* Azure OpenAI API key
20-
* Azure OpenAI resource name
21-
* Azure OpenAI deployment name (aka model name)
16+
- AI Gateway account ID
17+
- AI Gateway gateway name
18+
- Azure OpenAI API key
19+
- Azure OpenAI resource name
20+
- Azure OpenAI deployment name (aka model name)
2221

2322
## URL structure
2423

@@ -45,19 +44,18 @@ If you are using the `openai-node` library, you can set your endpoint like this:
4544
```js title="JavaScript"
4645
import OpenAI from "openai";
4746

48-
const resource = 'xxx';
49-
const model = 'xxx';
50-
const apiVersion = 'xxx';
47+
const resource = "xxx";
48+
const model = "xxx";
49+
const apiVersion = "xxx";
5150
const apiKey = env.AZURE_OPENAI_API_KEY;
52-
const accountId = '{account_id}';
53-
const gatewayId = '{gateway_id}';
51+
const accountId = "{account_id}";
52+
const gatewayId = "{gateway_id}";
5453
const baseURL = `https://gateway.ai.cloudflare.com/v1/${accountId}/${gatewayId}/azure-openai/${resource}/${model}`;
5554

5655
const azure_openai = new OpenAI({
57-
apiKey,
58-
baseURL,
59-
defaultQuery: { 'api-version': apiVersion },
60-
defaultHeaders: { 'api-key': apiKey },
56+
apiKey,
57+
baseURL,
58+
defaultQuery: { "api-version": apiVersion },
59+
defaultHeaders: { "api-key": apiKey },
6160
});
62-
6361
```

src/content/docs/ai-gateway/providers/bedrock.mdx

Lines changed: 64 additions & 57 deletions
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,6 @@ pcx_content_type: get-started
44
sidebar:
55
badge:
66
text: Beta
7-
87
---
98

109
[Amazon Bedrock](https://aws.amazon.com/bedrock/) allows you to build and scale generative AI applications with foundation models.
@@ -22,66 +21,74 @@ With Bedrock, you will need to sign the URL before you make requests to AI Gatew
2221
## Example:
2322

2423
```typescript
25-
import { AwsClient } from 'aws4fetch'
24+
import { AwsClient } from "aws4fetch";
2625

2726
interface Env {
28-
accessKey: string;
29-
secretAccessKey: string;
27+
accessKey: string;
28+
secretAccessKey: string;
3029
}
3130

3231
export default {
33-
async fetch(request: Request, env: Env, ctx: ExecutionContext): Promise<Response> {
34-
35-
// replace with your configuration
36-
const cfAccountId = "{account_id}";
37-
const gatewayName = "{gateway_id}";
38-
const region = 'us-east-1';
39-
40-
// added as secrets (https://developers.cloudflare.com/workers/configuration/secrets/)
41-
const accessKey = env.accessKey;
42-
const secretKey = env.secretAccessKey;
43-
44-
const requestData = {
45-
inputText: "What does ethereal mean?"
46-
};
47-
48-
const headers = {
49-
'Content-Type': 'application/json'
50-
};
51-
52-
// sign the original request
53-
const stockUrl = new URL("https://bedrock-runtime.us-east-1.amazonaws.com/model/amazon.titan-embed-text-v1/invoke")
54-
55-
const awsClient = new AwsClient({
56-
accessKeyId: accessKey,
57-
secretAccessKey: secretKey,
58-
region: region,
59-
service: "bedrock"
60-
});
61-
62-
const presignedRequest = await awsClient.sign(stockUrl.toString(), {
63-
method: "POST",
64-
headers: headers
65-
});
66-
67-
// change the signed request's host to AI Gateway
68-
const stockUrlSigned = new URL(presignedRequest.url);
69-
stockUrlSigned.host = "gateway.ai.cloudflare.com"
70-
stockUrlSigned.pathname = `/v1/${cfAccountId}/${gatewayName}/aws-bedrock/bedrock-runtime/${region}/model/amazon.titan-embed-text-v1/invoke`
71-
72-
// make request
73-
const response = await fetch(stockUrlSigned, {
74-
method: 'POST',
75-
headers: presignedRequest.headers,
76-
body: JSON.stringify(requestData)
77-
})
78-
79-
if (response.ok && response.headers.get('content-type')?.includes('application/json')) {
80-
const data = await response.json();
81-
return new Response(JSON.stringify(response));
82-
} else {
83-
return new Response("Invalid response", { status: 500 });
84-
}
85-
},
32+
async fetch(
33+
request: Request,
34+
env: Env,
35+
ctx: ExecutionContext,
36+
): Promise<Response> {
37+
// replace with your configuration
38+
const cfAccountId = "{account_id}";
39+
const gatewayName = "{gateway_id}";
40+
const region = "us-east-1";
41+
42+
// added as secrets (https://developers.cloudflare.com/workers/configuration/secrets/)
43+
const accessKey = env.accessKey;
44+
const secretKey = env.secretAccessKey;
45+
46+
const requestData = {
47+
inputText: "What does ethereal mean?",
48+
};
49+
50+
const headers = {
51+
"Content-Type": "application/json",
52+
};
53+
54+
// sign the original request
55+
const stockUrl = new URL(
56+
"https://bedrock-runtime.us-east-1.amazonaws.com/model/amazon.titan-embed-text-v1/invoke",
57+
);
58+
59+
const awsClient = new AwsClient({
60+
accessKeyId: accessKey,
61+
secretAccessKey: secretKey,
62+
region: region,
63+
service: "bedrock",
64+
});
65+
66+
const presignedRequest = await awsClient.sign(stockUrl.toString(), {
67+
method: "POST",
68+
headers: headers,
69+
});
70+
71+
// change the signed request's host to AI Gateway
72+
const stockUrlSigned = new URL(presignedRequest.url);
73+
stockUrlSigned.host = "gateway.ai.cloudflare.com";
74+
stockUrlSigned.pathname = `/v1/${cfAccountId}/${gatewayName}/aws-bedrock/bedrock-runtime/${region}/model/amazon.titan-embed-text-v1/invoke`;
75+
76+
// make request
77+
const response = await fetch(stockUrlSigned, {
78+
method: "POST",
79+
headers: presignedRequest.headers,
80+
body: JSON.stringify(requestData),
81+
});
82+
83+
if (
84+
response.ok &&
85+
response.headers.get("content-type")?.includes("application/json")
86+
) {
87+
const data = await response.json();
88+
return new Response(JSON.stringify(response));
89+
} else {
90+
return new Response("Invalid response", { status: 500 });
91+
}
92+
},
8693
};
8794
```

src/content/docs/ai-gateway/providers/cohere.mdx

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,6 @@
11
---
22
title: Cohere
33
pcx_content_type: get-started
4-
54
---
65

76
[Cohere](https://cohere.com/) build AI models designed to solve real-world business challenges.

0 commit comments

Comments
 (0)