Skip to content

Commit fd402d7

Browse files
daisyfaithaumahyperlint-ai[bot]RebeccaTamachiro
authored and
harshil1712
committed
[AIG] How to add human feedback using API (#17745)
* How to add human feedback using API * Grammar * Update src/content/docs/ai-gateway/evaluations/add-human-feedback-api.mdx grammar Co-authored-by: hyperlint-ai[bot] <154288675+hyperlint-ai[bot]@users.noreply.github.com> * api link * Update src/content/docs/ai-gateway/evaluations/add-human-feedback-api.mdx Co-authored-by: Rebecca Tamachiro <[email protected]> * Update src/content/docs/ai-gateway/evaluations/add-human-feedback-api.mdx Co-authored-by: Rebecca Tamachiro <[email protected]> * Update src/content/docs/ai-gateway/evaluations/add-human-feedback-api.mdx Co-authored-by: Rebecca Tamachiro <[email protected]> * Update src/content/docs/ai-gateway/evaluations/add-human-feedback-api.mdx Co-authored-by: Rebecca Tamachiro <[email protected]> * Update src/content/docs/ai-gateway/evaluations/add-human-feedback-api.mdx Co-authored-by: Rebecca Tamachiro <[email protected]> * minor fixes * edits * Update src/content/docs/ai-gateway/evaluations/add-human-feedback-api.mdx Co-authored-by: Rebecca Tamachiro <[email protected]> --------- Co-authored-by: hyperlint-ai[bot] <154288675+hyperlint-ai[bot]@users.noreply.github.com> Co-authored-by: Rebecca Tamachiro <[email protected]>
1 parent dd157f2 commit fd402d7

File tree

1 file changed

+152
-0
lines changed

1 file changed

+152
-0
lines changed

‎src/content/docs/ai-gateway/evaluations/add-human-feedback-api.mdx‎

Lines changed: 152 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,152 @@
1+
---
2+
pcx_content_type: how-to
3+
title: Add Human Feedback using API
4+
sidebar:
5+
order: 4
6+
---
7+
8+
This guide will walk you through the steps of adding human feedback to an AI Gateway request using the Cloudflare API. You will learn how to retrieve the relevant request logs, and submit feedback using the API.
9+
10+
## 1. Create an API Token
11+
12+
1. [Create an API token](/fundamentals/api/get-started/create-token/) with the following permissions:
13+
14+
- `AI Gateway - Read`
15+
- `AI Gateway - Edit`
16+
17+
2. Get your [Account ID](/fundamentals/setup/find-account-and-zone-ids/).
18+
3. Using that API token and Account ID, send a [`POST` request](/api/operations/aig-config-create-gateway) to the Cloudflare API.
19+
20+
## 2. Using the API Token
21+
22+
Once you have the token, you can use it in API requests by adding it to the authorization header as a bearer token. Here is an example of how to use it in a request:
23+
24+
```bash
25+
curl "https://api.cloudflare.com/client/v4/accounts/{account_id}/ai-gateway/gateways/{gateway_id}/logs" \
26+
--header "Authorization: Bearer {your_api_token}"
27+
```
28+
29+
In the request above:
30+
31+
- Replace `{account_id}` and `{gateway_id}` with your specific Cloudflare account and gateway details.
32+
- Replace `{your_api_token}` with the API token you just created.
33+
34+
## 3. Retrieve the `cf-aig-log-id`
35+
36+
The `cf-aig-log-id` is a unique identifier for the specific log entry to which you want to add feedback. Below are two methods to obtain this identifier.
37+
38+
### Method 1: Locate the `cf-aig-log-id` in the request response
39+
40+
This method allows you to directly find the `cf-aig-log-id` within the body of the response returned by the AI Gateway. This is the most straightforward approach if you have access to the original API response.
41+
42+
The steps below outline how to do this.
43+
44+
1. **Make a Request to the AI Gateway**: This could be a request your application sends to the AI Gateway. Once the request is made, the response will contain various pieces of metadata.
45+
2. **Check the Response Body**: The response body will include a field named `cf-aig-log-id`. This is the identifier you will need to submit feedback.
46+
47+
In the example below, the `cf-aig-log-id` is `01JADMCQQQBWH3NXZ5GCRN98DP`.
48+
49+
```json
50+
{
51+
"status": "success",
52+
"data": {
53+
"response": "Sample response data",
54+
"cf-aig-log-id": "01JADMCQQQBWH3NXZ5GCRN98DP"
55+
}
56+
}
57+
```
58+
59+
### Method 2: Retrieve the `cf-aig-log-id` via API (GET request)
60+
61+
If you don't have the `cf-aig-log-id` in the response body or you need to access it after the fact, you can retrieve it by querying the logs using the [Cloudflare API](/api/operations/aig-config-list-gateway-logs).
62+
63+
The steps below outline how to do this.
64+
65+
1. **Send a GET Request to Retrieve Logs**: You can query the AI Gateway logs for a specific time frame or for a specific request. The request will return a list of logs, each containing its own `id`.
66+
Here is an example request:
67+
68+
```bash
69+
GET https://api.cloudflare.com/client/v4/accounts/{account_id}/ai-gateway/gateways/{gateway_id}/logs
70+
```
71+
72+
Replace `{account_id}` and `{gateway_id}` with your specific account and gateway details.
73+
74+
2. **Search for the Relevant Log**: In the response from the GET request, locate the specific log entry for which you would like to submit feedback. Each log entry will include the `id`.
75+
76+
In the example below, the `id` is `01JADMCQQQBWH3NXZ5GCRN98DP`.
77+
78+
```json
79+
{
80+
"result": [
81+
{
82+
"id": "01JADMCQQQBWH3NXZ5GCRN98DP",
83+
"cached": true,
84+
"created_at": "2019-08-24T14:15:22Z",
85+
"custom_cost": true,
86+
"duration": 0,
87+
"id": "string",
88+
"metadata": "string",
89+
"model": "string",
90+
"model_type": "string",
91+
"path": "string",
92+
"provider": "string",
93+
"request_content_type": "string",
94+
"request_type": "string",
95+
"response_content_type": "string",
96+
"status_code": 0,
97+
"step": 0,
98+
"success": true,
99+
"tokens_in": 0,
100+
"tokens_out": 0
101+
}
102+
],
103+
"result_info": {
104+
"count": 0,
105+
"max_cost": 0,
106+
"max_duration": 0,
107+
"max_tokens_in": 0,
108+
"max_tokens_out": 0,
109+
"max_total_tokens": 0,
110+
"min_cost": 0,
111+
"min_duration": 0,
112+
"min_tokens_in": 0,
113+
"min_tokens_out": 0,
114+
"min_total_tokens": 0,
115+
"page": 0,
116+
"per_page": 0,
117+
"total_count": 0
118+
},
119+
"success": true
120+
}
121+
```
122+
123+
## 4. Submit feedback via PATCH request
124+
125+
Once you have both the API token and the `cf-aig-log-id`, you can send a PATCH request to submit feedback. Use the following URL format, replacing the `{account_id}`, `{gateway_id}`, and `{log_id}` with your specific details:
126+
127+
```bash
128+
PATCH https://api.cloudflare.com/client/v4/accounts/{account_id}/ai-gateway/gateways/{gateway_id}/logs/{log_id}
129+
```
130+
131+
Add the following in the request body to submit positive feedback:
132+
133+
```json
134+
{
135+
"feedback": 1
136+
}
137+
```
138+
139+
Add the following in the request body to submit negative feedback:
140+
141+
```json
142+
{
143+
"feedback": -1
144+
}
145+
```
146+
147+
## 5. Verify the feedback submission
148+
149+
You can verify the feedback submission in two ways:
150+
151+
- **Through the [Cloudflare dashboard ](https://dash.cloudflare.com)**: check the updated feedback on the AI Gateway interface.
152+
- **Through the API**: Send another GET request to retrieve the updated log entry and confirm the feedback has been recorded.

0 commit comments

Comments
 (0)