You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
This tutorial provides a step-by-step guide on how to leverage Cloudflare Workers AI Gatewayto create a functional and secure AI Assistant Wrapper. This way, Cloudflare Zero Trust customers are able to enforce Secure Web Gateway controls on how their users interact with AI agents, including Remote Browser Isolation, enforcement of DLP Profiles to prevent sensitive data leakage to the agents, and content scanning controls to avoid getting answers from AI agents that violate internal corporate guidelines.
10
+
This tutorial explains how to use [Cloudflare AI Gateway](/ai-gateway/) and Zero Trust to create a functional and secure AI agent wrapper. Cloudflare Zero Trust admins can enforce [Gateway](/cloudflare-one/policies/gateway/) controls on how their users interact with AI agents, including executing AI agents in an isolated browser with [Browser Isolation](/cloudflare-one/policies/browser-isolation/), enforcing [Data Loss Prevention](/cloudflare-one/policies/data-loss-prevention/) profiles to prevent sensitive data exfiltration, and scanning content to avoid answers from AI agents that violate internal corporate guidelines. Creating an AI agent wrapper is also an effective way to enforce tenant control if you have an enterprise plan of a specific AI provider, such as ChatGPT Enterprise.
11
11
12
-
This is also an effective way to enforce tenant control if you happen to be a customer of premium tier of a specific AI provider such as ChatGPT Enterprise.
12
+
This tutorial uses ChatGPT as an example AI agent.
13
13
14
14
## Prerequisites
15
15
16
-
- API key to authenticate the consumption of the desired AI provider, such as OpenAI API key for ChatGPT.
16
+
- API key for your desired AI provider, such as an [OpenAI API key](https://platform.openai.com/api-keys) for ChatGPT.
17
17
18
-
## 1. Create an AI Gateway
18
+
## 1. Create an AI gateway
19
+
20
+
First, create an AI gateway to control an AI app.
19
21
20
22
1. Log into the [Cloudflare dashboard](https://dash.cloudflare.com/) and select your account.
21
23
2. Go to **AI** > **AI Gateway**.
22
-
3. Select **Create gateway**.
23
-
4.Give your Gateway a name, and toggle the options you want.
24
+
3. Select **Create Gateway**.
25
+
4.Name your gateway.
24
26
5. Select **Create**.
25
-
6. Follow [our documentation](https://developers.cloudflare.com/ai-gateway/get-started/) to understand how to proxy queries to your AI agent of choice via your **AI Gateway**.
26
-
7. (Optional) The [**Authenticated Gateway**](https://developers.cloudflare.com/ai-gateway/configuration/authentication/) feature ensures your AI Gateway can only be called securely by enforcing a token in the form of a request header `cf-aig-authorization`.
27
-
1. To do this, go to **AI** > **AI Gateway** > **Settings** and toggle **Authenticated Gateway**.
28
-
2. When creating your Worker, you will need to pass this token through when calling your AI Gateway.
27
+
6. Configure your desired options for the gateway.
28
+
7.[Connect your AI provider](/ai-gateway/get-started/#connect-application) to proxy queries to your AI agent of choice using your AI gateway.
29
+
8. Turn on [Authenticated Gateway](/ai-gateway/configuration/authentication/). The Authenticated Gateway feature ensures your AI gateway can only be called securely by enforcing a token in the form of a request header `cf-aig-authorization`.
30
+
1. Go to **AI** > **AI Gateway**.
31
+
2. Select your AI gateway, then go to **Settings**.
32
+
3. Turn on **Authenticated Gateway**, then choose **Confirm**.
33
+
4. Select **Create authentication token**, then select **Create an AI Gateway authentication token**.
34
+
5. Configure your token and copy the token value. When creating your Worker, you will need to pass this token when calling your AI gateway.
35
+
36
+
For more information, refer to [Getting started with AI Gateway](/ai-gateway/get-started/).
29
37
30
-
### (Optional) Use Guardrails to block unsafe or inappropriate content
38
+
### 2. (Optional) Use Guardrails to block unsafe or inappropriate content
31
39
32
-
[**Guardrails**](https://developers.cloudflare.com/ai-gateway/guardrails/) is an AI Gateway built-in security feature that allows Cloudflare to identify unsafe or inappropriate content in prompts and responses based on selected categories.
40
+
[Guardrails](/ai-gateway/guardrails/) is an built-in AI Gateway security feature that allows Cloudflare to identify unsafe or inappropriate content in prompts and responses based on selected categories.
33
41
34
42
1. Go to **AI** > **AI Gateway**.
35
-
2. Select your Gateway.
36
-
3. Go to the **Guardrails** tab.
37
-
4.Select the toggle to enable Guardrails.
38
-
5. Select the categories you would like to filter, for both prompts and responses.
43
+
2. Select your AI gateway.
44
+
3. Go to **Guardrails**.
45
+
4.Turn on Guardrails.
46
+
5. Select **Change** to configure the categories you would like to filter for both prompts and responses.
39
47
40
-
## 2. Create a Worker to serve a wrapper
48
+
## 3. Create a Worker to serve a wrapper
41
49
42
-
In order to build the Worker, you will need to choose if you want to build it locally using [**Wrangler**](https://developers.cloudflare.com/workers/wrangler/install-and-update/) or remotely using the [**Dash**](https://dash.cloudflare.com/).
50
+
In order to build the Worker, you will need to choose if you want to build it locally using [Wrangler](/workers/wrangler/install-and-update/) or remotely using the [dashboard](https://dash.cloudflare.com/).
43
51
44
-
### Option 1: Build a Worker locally using Wrangler
52
+
<Tabs> <TabItemlabel="Wrangler">
45
53
46
54
1. Log in to your Cloudlare account:
47
55
48
-
```bash
49
-
wrangler login
50
-
```
56
+
```bash
57
+
wrangler login
58
+
```
51
59
52
60
2. Initiate the project locally:
53
61
54
-
```bash
55
-
mkdir ai-agent-wrapper
56
-
cd ai-agent-wrapper
57
-
wrangler init
58
-
```
62
+
```bash
63
+
mkdir ai-agent-wrapper
64
+
cd ai-agent-wrapper
65
+
wrangler init
66
+
```
59
67
60
68
3. Create a `wrangler.toml` configuration file:
61
69
62
-
```toml
63
-
name = "ai-agent-wrapper"
64
-
main = "src/index.js"
65
-
compatibility_date = "2023-10-30"
70
+
```toml
71
+
name = "ai-agent-wrapper"
72
+
main = "src/index.js"
73
+
compatibility_date = "2023-10-30"
66
74
67
-
[vars]
68
-
# Add any environment variables here
69
-
```
75
+
[vars]
76
+
# Add any environment variables here
77
+
```
70
78
71
79
4. Add your AI provider API Key as a secret:
72
80
73
-
```bash
74
-
wrangler secret put OPENAI_API_KEY
75
-
```
81
+
```bash
82
+
wrangler secret put OPENAI_API_KEY
83
+
```
76
84
77
85
5. The Worker can now be built using the `index.js` file that was created by **Wrangler**.
78
86
79
-
### Option 2: Build a Worker remotely using the Dash
87
+
</TabItem> <TabItemlabel="Dashboard">
80
88
81
89
1. Log into the [Cloudflare dashboard](https://dash.cloudflare.com/) and select your account.
82
90
2. Go to **Workers & Pages** > **Workers & Pages**.
@@ -88,9 +96,13 @@ wrangler secret put OPENAI_API_KEY
88
96
8. Choose **Secret** as the type, give your variable a name such as `OPENAI_API_KEY`, and past the value of your AI provider API key in the **Value** field.
89
97
9. The Worker can now be built by using the online code editor by clicking the **Edit code** icon at the top-right.
90
98
99
+
</TabItem> </Tabs>
100
+
91
101
### Build the Worker
92
102
93
-
The following is an example starter Worker that serves a simple front-end to allow a user to interact with an AI provider behind **AI Gateway**. For this example, Open AI was used as the provider.
103
+
The following is an example starter Worker that serves a simple front-end to allow a user to interact with an AI provider behind **AI Gateway**. For this example, OpenAI was used as the provider:
Notice that the **Account ID** and the **Gateway ID** need to be replaced in the AI Gateway endpoint. You can add these as environment variables or Secrets. If you chose to use the **Authenticated Gateway** option when creating your AI Gateway, make sure to also add your token as a Secret and pass its value through to the AI Gateway in the form of the `cf-aig-authorization` header.
330
344
331
345
This Worker can serve as a starting point and extra functionality can be built on top.
@@ -334,7 +348,7 @@ This Worker can serve as a starting point and extra functionality can be built o
334
348
335
349
Once the Worker is code is completed, it is almost ready to be published. We now need to make the Worker addressable via a hostname that can be controlled by Cloudlare Access.
336
350
337
-
#### Option 1: Worker built locally using Wrangler
351
+
<Tabs> <TabItemlabel="Wrangler">
338
352
339
353
Edit the `wrangler.toml` configuration file and add the following information to ensure that the worker is only accessible via the custom hostname.
340
354
@@ -355,7 +369,7 @@ routes = [
355
369
356
370
To publish the worker do: `wrangler publish`.
357
371
358
-
#### Option 2: Worker built remotely using the Dash
372
+
</TabItem> <TabItemlabel="Dashboard">
359
373
360
374
If the worker was built remotely using the online code editor available in the **Dash**, it can be published by clicking **Deploy**.
361
375
@@ -364,13 +378,15 @@ In order to ensure that the worker is only accessible via the custom hostname, d
364
378
2. Go to **Workers & Pages** > **Workers & Pages**.
365
379
3. Select your Worker.
366
380
4. Select the **Settings** tab.
367
-
5. Within the **Domains & Routes** section, select **+ Add**.
381
+
5. Within the **Domains & Routes** section, select **Add**.
368
382
6. Choose **Custom domain (Recommended)** (**Route** is also a viable option).
369
383
7. Add your desired custom domain name.
370
384
8. Select **Add domain**.
371
385
372
386
The Worker now sits behind an addressable public hostname. Make sure to disable both your **workers.dev** and **Preview URLs** so that the Worker can only be accessed via its **Custom domain**.
373
387
388
+
</TabItem> </Tabs>
389
+
374
390
## Secure the AI Agent Wrapper using Access
375
391
376
392
We need to secure our AI Agent Wrapper to ensure that only trusted users can access it. We will use [**Access**](https://developers.cloudflare.com/cloudflare-one/policies/access/) to achieve this.
@@ -393,7 +409,7 @@ If you use another gateway for web filtering, try to replicate a similar policy.
393
409
394
410
1. In [Zero Trust](https://one.dash.cloudflare.com/), go to **Gateway** > **Firewall policies**.
395
411
2. Select **HTTP**.
396
-
3. Select **+ Add a policy**.
412
+
3. Select **Add a policy**.
397
413
4. Under the **Traffic** section, select **Add condition**.
398
414
5. Select **Content Categories**.
399
415
6. Select **Artificial Intelligence**.
@@ -416,10 +432,10 @@ Since you have full control over access to your AI Agent wrapper, you can enforc
416
432
2. Ensure that the DLP profiles you wish to enforce are properly configured.
417
433
3. Then, go to **Gateway** > **Firewall policies**.
418
434
4. Select **HTTP**.
419
-
5. Select **+ Add a policy**.
435
+
5. Select **Add a policy**.
420
436
6. Under the **Traffic** section, select **Add condition**.
421
437
7. Select **Host** and enter the custom domain of your AI Agent Wrapper.
422
-
8. Select **+ Add** to add another condition.
438
+
8. Select **Add** to add another condition.
423
439
9. Select **DLP Profile** and choose the DLP profiles you would like to enforce.
424
440
10. Add any other conditions that apply to your environment.
425
441
11. Under **Action**, select **Block**.
@@ -454,7 +470,7 @@ Organizations that adopt Cloudflare to secure access to AI agents as exemplified
454
470
455
471
All [access events](https://developers.cloudflare.com/cloudflare-one/insights/logs/audit-logs/) to the AI wrapper and [DLP violations](https://developers.cloudflare.com/cloudflare-one/insights/logs/gateway-logs/#http-logs) will be logged in Cloudflare's dashboard. In addition, AI gateway provides [visibility](https://developers.cloudflare.com/ai-gateway/observability/logging/) into user prompts, model response, token usage and costs.
456
472
457
-
Finally, all the logs can be easily exported to external systems using [logpush](https://developers.cloudflare.com/logs/).
473
+
Finally, all the logs can be easily exported to external systems using [Logpush](https://developers.cloudflare.com/logs/).
0 commit comments