From b7597781b6f4624dc4196374e13346994776ab95 Mon Sep 17 00:00:00 2001 From: daisyfaithauma Date: Thu, 24 Oct 2024 00:00:27 +0100 Subject: [PATCH 01/12] How to add human feedback using API --- .../evaluations/add-human-feedback-api.mdx | 150 ++++++++++++++++++ 1 file changed, 150 insertions(+) create mode 100644 src/content/docs/ai-gateway/evaluations/add-human-feedback-api.mdx diff --git a/src/content/docs/ai-gateway/evaluations/add-human-feedback-api.mdx b/src/content/docs/ai-gateway/evaluations/add-human-feedback-api.mdx new file mode 100644 index 00000000000000..8a35ee5a44e251 --- /dev/null +++ b/src/content/docs/ai-gateway/evaluations/add-human-feedback-api.mdx @@ -0,0 +1,150 @@ +--- +pcx_content_type: how-to +title: Add Human Feedback using API +sidebar: + order: 4 +--- + +This guide will walk you through the steps of adding human feedback to an AI Gateway request using the Cloudflare API. You'll learn how to retrieve the relevant request logs, and submit feedback using the API. + +## Step 1. Create an API Token + +1. [Create an API token](/fundamentals/api/get-started/create-token/) with the following permissions: + +- `AI Gateway - Read` +- `AI Gateway - Edit` + +2. Get your [Account ID](/fundamentals/setup/find-account-and-zone-ids/). +3. Using that API token and Account ID, send a [`POST` request](/api/operations/aig-config-create-gateway) to to the Cloudflare API. + +## Step 2. Using the API Token + +Once you have the token, you can use it in API requests by adding it to the authorization header as a bearer token. Here is an example of how to use it in a request: + +```bash +curl -X GET "https://api.cloudflare.com/client/v4/accounts/{account_id}/ai-gateway/gateways/{gateway_id}/logs" \ +-H "Authorization: Bearer {your_api_token}" +``` + +In the request above: + +- Replace `{account_id}` and `{gateway_id}` with your specific Cloudflare account and gateway details. +- Replace `{your_api_token}` with the API token you just created. + +## Step 3. Retrieving the `cf-aig-log-id` + +The `cf-aig-log-id` is a unique identifier for the specific log entry to which you want to add feedback. Below are two methods to obtain this identifier. + +### Method 1: Locate the `cf-aig-log-id` in the request response + +This method allows you to directly find the `cf-aig-log-id` within the body of the response returned by the AI Gateway. This is the most straightforward approach if you have access to the original API response. + +The steps below outline how to do this. + +1. **Make a Request to the AI Gateway**: This could be a request your application sends to the AI Gateway. Once the request is made, the response will contain various pieces of metadata. +2. **Check the Response Body**: The response body will include a field named `cf-aig-log-id`. This is the identifier you will need to submit feedback. + +In the example below, the cf-aig-log-id is `01JADMCQQQBWH3NXZ5GCRN98DP`. + +```json +{ + "status": "success", + "data": { + "response": "Sample response data", + "cf-aig-log-id": "01JADMCQQQBWH3NXZ5GCRN98DP" + } +} +``` + +### Method 2: Retrieve the `cf-aig-log-id` via API (GET request) + +If you don't have the `cf-aig-log-id` in the response body or you need to access it after the fact, you can retrieve it by querying the logs using the Cloudflare API. + +The steps below outline how to do this. + +1. **Send a GET Request to Retrieve Logs**: You can query the AI Gateway logs for a specific time frame or for a specific request. The request will return a list of logs, each containing its own `id`. + Here is an example request: + +```bash +GET https://api.cloudflare.com/client/v4/accounts/{account_id}/ai-gateway/gateways/{gateway_id}/logs +``` + +Replace `{account_id}` and `{gateway_id}` with your specific account and gateway details. + +2. **Search for the Relevant Log**: In the response from the GET request, locate the specific log entry for which you would like to submit feedback. Each log entry will include the `id`. + +In the example below, the `id` is `01JADMCQQQBWH3NXZ5GCRN98DP`. + +```json +{ + "result": [ + { + "id": "01JADMCQQQBWH3NXZ5GCRN98DP", + "cached": true, + "created_at": "2019-08-24T14:15:22Z", + "custom_cost": true, + "duration": 0, + "id": "string", + "metadata": "string", + "model": "string", + "model_type": "string", + "path": "string", + "provider": "string", + "request_content_type": "string", + "request_type": "string", + "response_content_type": "string", + "status_code": 0, + "step": 0, + "success": true, + "tokens_in": 0, + "tokens_out": 0 + } + ], + "result_info": { + "count": 0, + "max_cost": 0, + "max_duration": 0, + "max_tokens_in": 0, + "max_tokens_out": 0, + "max_total_tokens": 0, + "min_cost": 0, + "min_duration": 0, + "min_tokens_in": 0, + "min_tokens_out": 0, + "min_total_tokens": 0, + "page": 0, + "per_page": 0, + "total_count": 0 + }, + "success": true +} +``` + +## Step 4. Submit Feedback via PATCH Request + +Once you have both the API token and the `cf-aig-log-id`, you can send a PATCH request to submit feedback. Use the following URL format, replacing the `{account_id}`, `{gateway_id}`, and `{log_id}` with your specific details: + +```bash +PATCH https://api.cloudflare.com/client/v4/accounts/{account_id}/ai-gateway/gateways/{gateway_id}/logs/{log_id} +``` + +Add the following in the request body to submit positive feedback: + +```json +{ + "feedback": 1 +} +``` + +Add the following in the request body to submit negative feedback: + +```json +{ + "feedback": -1 +} +``` + +## Step 5. You can verify the feedback submission in two ways: + +- **Through the [Cloudflare dashboard ](https://dash.cloudflare.com)**: check the updated feedback on the AI Gateway interface. +- **Through the API**: Send another GET request to retrieve the updated log entry and confirm the feedback has been recorded. From 0e3e2d90f6058c19fb66b5d6f5ff1c9014d0b8ea Mon Sep 17 00:00:00 2001 From: daisyfaithauma Date: Thu, 24 Oct 2024 00:02:57 +0100 Subject: [PATCH 02/12] Grammar --- .../docs/ai-gateway/evaluations/add-human-feedback-api.mdx | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/content/docs/ai-gateway/evaluations/add-human-feedback-api.mdx b/src/content/docs/ai-gateway/evaluations/add-human-feedback-api.mdx index 8a35ee5a44e251..935d52a958f0dd 100644 --- a/src/content/docs/ai-gateway/evaluations/add-human-feedback-api.mdx +++ b/src/content/docs/ai-gateway/evaluations/add-human-feedback-api.mdx @@ -5,7 +5,7 @@ sidebar: order: 4 --- -This guide will walk you through the steps of adding human feedback to an AI Gateway request using the Cloudflare API. You'll learn how to retrieve the relevant request logs, and submit feedback using the API. +This guide will walk you through the steps of adding human feedback to an AI Gateway request using the Cloudflare API. You will learn how to retrieve the relevant request logs, and submit feedback using the API. ## Step 1. Create an API Token From a5cb416a0c6868d635e962e867ecabe39a139665 Mon Sep 17 00:00:00 2001 From: daisyfaithauma Date: Thu, 24 Oct 2024 00:04:47 +0100 Subject: [PATCH 03/12] Update src/content/docs/ai-gateway/evaluations/add-human-feedback-api.mdx grammar Co-authored-by: hyperlint-ai[bot] <154288675+hyperlint-ai[bot]@users.noreply.github.com> --- .../docs/ai-gateway/evaluations/add-human-feedback-api.mdx | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/content/docs/ai-gateway/evaluations/add-human-feedback-api.mdx b/src/content/docs/ai-gateway/evaluations/add-human-feedback-api.mdx index 935d52a958f0dd..ba14d4098655d6 100644 --- a/src/content/docs/ai-gateway/evaluations/add-human-feedback-api.mdx +++ b/src/content/docs/ai-gateway/evaluations/add-human-feedback-api.mdx @@ -15,7 +15,7 @@ This guide will walk you through the steps of adding human feedback to an AI Gat - `AI Gateway - Edit` 2. Get your [Account ID](/fundamentals/setup/find-account-and-zone-ids/). -3. Using that API token and Account ID, send a [`POST` request](/api/operations/aig-config-create-gateway) to to the Cloudflare API. +3. Using that API token and Account ID, send a [`POST` request](/api/operations/aig-config-create-gateway) to the Cloudflare API. ## Step 2. Using the API Token From ca6a82a95d99ee42114e1676f521e0fb385afe4f Mon Sep 17 00:00:00 2001 From: daisyfaithauma Date: Thu, 24 Oct 2024 13:02:23 +0100 Subject: [PATCH 04/12] api link --- .../docs/ai-gateway/evaluations/add-human-feedback-api.mdx | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/content/docs/ai-gateway/evaluations/add-human-feedback-api.mdx b/src/content/docs/ai-gateway/evaluations/add-human-feedback-api.mdx index 935d52a958f0dd..bc5b2ae5ddea49 100644 --- a/src/content/docs/ai-gateway/evaluations/add-human-feedback-api.mdx +++ b/src/content/docs/ai-gateway/evaluations/add-human-feedback-api.mdx @@ -58,7 +58,7 @@ In the example below, the cf-aig-log-id is `01JADMCQQQBWH3NXZ5GCRN98DP`. ### Method 2: Retrieve the `cf-aig-log-id` via API (GET request) -If you don't have the `cf-aig-log-id` in the response body or you need to access it after the fact, you can retrieve it by querying the logs using the Cloudflare API. +If you don't have the `cf-aig-log-id` in the response body or you need to access it after the fact, you can retrieve it by querying the logs using the [Cloudflare API](/api/operations/aig-config-list-gateway-logs). The steps below outline how to do this. From 27e751cd7a1d6914a3087b63d222bae270e20d5d Mon Sep 17 00:00:00 2001 From: daisyfaithauma Date: Thu, 24 Oct 2024 16:04:13 +0100 Subject: [PATCH 05/12] Update src/content/docs/ai-gateway/evaluations/add-human-feedback-api.mdx Co-authored-by: Rebecca Tamachiro <62246989+RebeccaTamachiro@users.noreply.github.com> --- .../docs/ai-gateway/evaluations/add-human-feedback-api.mdx | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/content/docs/ai-gateway/evaluations/add-human-feedback-api.mdx b/src/content/docs/ai-gateway/evaluations/add-human-feedback-api.mdx index e31e35a81c1bde..0d4ba4b72f9416 100644 --- a/src/content/docs/ai-gateway/evaluations/add-human-feedback-api.mdx +++ b/src/content/docs/ai-gateway/evaluations/add-human-feedback-api.mdx @@ -44,7 +44,7 @@ The steps below outline how to do this. 1. **Make a Request to the AI Gateway**: This could be a request your application sends to the AI Gateway. Once the request is made, the response will contain various pieces of metadata. 2. **Check the Response Body**: The response body will include a field named `cf-aig-log-id`. This is the identifier you will need to submit feedback. -In the example below, the cf-aig-log-id is `01JADMCQQQBWH3NXZ5GCRN98DP`. +In the example below, the `cf-aig-log-id` is `01JADMCQQQBWH3NXZ5GCRN98DP`. ```json { From bee4327c17030759ab9bf18e6682516d07be9f2b Mon Sep 17 00:00:00 2001 From: daisyfaithauma Date: Thu, 24 Oct 2024 16:04:38 +0100 Subject: [PATCH 06/12] Update src/content/docs/ai-gateway/evaluations/add-human-feedback-api.mdx Co-authored-by: Rebecca Tamachiro <62246989+RebeccaTamachiro@users.noreply.github.com> --- .../docs/ai-gateway/evaluations/add-human-feedback-api.mdx | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/src/content/docs/ai-gateway/evaluations/add-human-feedback-api.mdx b/src/content/docs/ai-gateway/evaluations/add-human-feedback-api.mdx index 0d4ba4b72f9416..a60112fda9cb25 100644 --- a/src/content/docs/ai-gateway/evaluations/add-human-feedback-api.mdx +++ b/src/content/docs/ai-gateway/evaluations/add-human-feedback-api.mdx @@ -144,7 +144,9 @@ Add the following in the request body to submit negative feedback: } ``` -## Step 5. You can verify the feedback submission in two ways: +## Step 5. Verify the feedback submission + +You can verify the feedback submission in two ways: - **Through the [Cloudflare dashboard ](https://dash.cloudflare.com)**: check the updated feedback on the AI Gateway interface. - **Through the API**: Send another GET request to retrieve the updated log entry and confirm the feedback has been recorded. From dc23f55cfd3b102c1339ee3d3cdc663d7040b1a2 Mon Sep 17 00:00:00 2001 From: daisyfaithauma Date: Thu, 24 Oct 2024 16:04:47 +0100 Subject: [PATCH 07/12] Update src/content/docs/ai-gateway/evaluations/add-human-feedback-api.mdx Co-authored-by: Rebecca Tamachiro <62246989+RebeccaTamachiro@users.noreply.github.com> --- .../docs/ai-gateway/evaluations/add-human-feedback-api.mdx | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/content/docs/ai-gateway/evaluations/add-human-feedback-api.mdx b/src/content/docs/ai-gateway/evaluations/add-human-feedback-api.mdx index a60112fda9cb25..59a5f1e39e1053 100644 --- a/src/content/docs/ai-gateway/evaluations/add-human-feedback-api.mdx +++ b/src/content/docs/ai-gateway/evaluations/add-human-feedback-api.mdx @@ -120,7 +120,7 @@ In the example below, the `id` is `01JADMCQQQBWH3NXZ5GCRN98DP`. } ``` -## Step 4. Submit Feedback via PATCH Request +## Step 4. Submit feedback via PATCH request Once you have both the API token and the `cf-aig-log-id`, you can send a PATCH request to submit feedback. Use the following URL format, replacing the `{account_id}`, `{gateway_id}`, and `{log_id}` with your specific details: From 38ea2c58145647ce572063621de81052c9a7d407 Mon Sep 17 00:00:00 2001 From: daisyfaithauma Date: Thu, 24 Oct 2024 16:05:01 +0100 Subject: [PATCH 08/12] Update src/content/docs/ai-gateway/evaluations/add-human-feedback-api.mdx Co-authored-by: Rebecca Tamachiro <62246989+RebeccaTamachiro@users.noreply.github.com> --- .../docs/ai-gateway/evaluations/add-human-feedback-api.mdx | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/content/docs/ai-gateway/evaluations/add-human-feedback-api.mdx b/src/content/docs/ai-gateway/evaluations/add-human-feedback-api.mdx index 59a5f1e39e1053..3ea9aa4a905c53 100644 --- a/src/content/docs/ai-gateway/evaluations/add-human-feedback-api.mdx +++ b/src/content/docs/ai-gateway/evaluations/add-human-feedback-api.mdx @@ -31,7 +31,7 @@ In the request above: - Replace `{account_id}` and `{gateway_id}` with your specific Cloudflare account and gateway details. - Replace `{your_api_token}` with the API token you just created. -## Step 3. Retrieving the `cf-aig-log-id` +## Step 3. Retrieve the `cf-aig-log-id` The `cf-aig-log-id` is a unique identifier for the specific log entry to which you want to add feedback. Below are two methods to obtain this identifier. From 42d641f2248714270df2f436ee27bc66cae5a423 Mon Sep 17 00:00:00 2001 From: daisyfaithauma Date: Thu, 24 Oct 2024 16:07:01 +0100 Subject: [PATCH 09/12] Update src/content/docs/ai-gateway/evaluations/add-human-feedback-api.mdx Co-authored-by: Rebecca Tamachiro <62246989+RebeccaTamachiro@users.noreply.github.com> --- .../docs/ai-gateway/evaluations/add-human-feedback-api.mdx | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/src/content/docs/ai-gateway/evaluations/add-human-feedback-api.mdx b/src/content/docs/ai-gateway/evaluations/add-human-feedback-api.mdx index 3ea9aa4a905c53..78d404c02b01e7 100644 --- a/src/content/docs/ai-gateway/evaluations/add-human-feedback-api.mdx +++ b/src/content/docs/ai-gateway/evaluations/add-human-feedback-api.mdx @@ -22,9 +22,8 @@ This guide will walk you through the steps of adding human feedback to an AI Gat Once you have the token, you can use it in API requests by adding it to the authorization header as a bearer token. Here is an example of how to use it in a request: ```bash -curl -X GET "https://api.cloudflare.com/client/v4/accounts/{account_id}/ai-gateway/gateways/{gateway_id}/logs" \ --H "Authorization: Bearer {your_api_token}" -``` +curl "https://api.cloudflare.com/client/v4/accounts/{account_id}/ai-gateway/gateways/{gateway_id}/logs" \ +--header "Authorization: Bearer {your_api_token}" In the request above: From 3d1cce0d70a4663418d7262aff0c350274295fc2 Mon Sep 17 00:00:00 2001 From: daisyfaithauma Date: Thu, 24 Oct 2024 16:11:12 +0100 Subject: [PATCH 10/12] minor fixes --- .../evaluations/add-human-feedback-api.mdx | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/src/content/docs/ai-gateway/evaluations/add-human-feedback-api.mdx b/src/content/docs/ai-gateway/evaluations/add-human-feedback-api.mdx index 78d404c02b01e7..ec75fe01a450fb 100644 --- a/src/content/docs/ai-gateway/evaluations/add-human-feedback-api.mdx +++ b/src/content/docs/ai-gateway/evaluations/add-human-feedback-api.mdx @@ -7,7 +7,7 @@ sidebar: This guide will walk you through the steps of adding human feedback to an AI Gateway request using the Cloudflare API. You will learn how to retrieve the relevant request logs, and submit feedback using the API. -## Step 1. Create an API Token +## 1. Create an API Token 1. [Create an API token](/fundamentals/api/get-started/create-token/) with the following permissions: @@ -17,11 +17,11 @@ This guide will walk you through the steps of adding human feedback to an AI Gat 2. Get your [Account ID](/fundamentals/setup/find-account-and-zone-ids/). 3. Using that API token and Account ID, send a [`POST` request](/api/operations/aig-config-create-gateway) to the Cloudflare API. -## Step 2. Using the API Token +## 2. Using the API Token Once you have the token, you can use it in API requests by adding it to the authorization header as a bearer token. Here is an example of how to use it in a request: -```bash +````bash curl "https://api.cloudflare.com/client/v4/accounts/{account_id}/ai-gateway/gateways/{gateway_id}/logs" \ --header "Authorization: Bearer {your_api_token}" @@ -53,7 +53,7 @@ In the example below, the `cf-aig-log-id` is `01JADMCQQQBWH3NXZ5GCRN98DP`. "cf-aig-log-id": "01JADMCQQQBWH3NXZ5GCRN98DP" } } -``` +```` ### Method 2: Retrieve the `cf-aig-log-id` via API (GET request) @@ -119,7 +119,7 @@ In the example below, the `id` is `01JADMCQQQBWH3NXZ5GCRN98DP`. } ``` -## Step 4. Submit feedback via PATCH request +## 4. Submit feedback via PATCH request Once you have both the API token and the `cf-aig-log-id`, you can send a PATCH request to submit feedback. Use the following URL format, replacing the `{account_id}`, `{gateway_id}`, and `{log_id}` with your specific details: @@ -143,7 +143,7 @@ Add the following in the request body to submit negative feedback: } ``` -## Step 5. Verify the feedback submission +## 5. Verify the feedback submission You can verify the feedback submission in two ways: From 5c6eddb59674cae76b7db3d88d570fc203fb63ee Mon Sep 17 00:00:00 2001 From: daisyfaithauma Date: Thu, 24 Oct 2024 16:59:31 +0100 Subject: [PATCH 11/12] edits --- .../docs/ai-gateway/evaluations/add-human-feedback-api.mdx | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/src/content/docs/ai-gateway/evaluations/add-human-feedback-api.mdx b/src/content/docs/ai-gateway/evaluations/add-human-feedback-api.mdx index ec75fe01a450fb..17f41ab6ee697b 100644 --- a/src/content/docs/ai-gateway/evaluations/add-human-feedback-api.mdx +++ b/src/content/docs/ai-gateway/evaluations/add-human-feedback-api.mdx @@ -21,9 +21,10 @@ This guide will walk you through the steps of adding human feedback to an AI Gat Once you have the token, you can use it in API requests by adding it to the authorization header as a bearer token. Here is an example of how to use it in a request: -````bash +```bash curl "https://api.cloudflare.com/client/v4/accounts/{account_id}/ai-gateway/gateways/{gateway_id}/logs" \ --header "Authorization: Bearer {your_api_token}" +``` In the request above: @@ -53,7 +54,7 @@ In the example below, the `cf-aig-log-id` is `01JADMCQQQBWH3NXZ5GCRN98DP`. "cf-aig-log-id": "01JADMCQQQBWH3NXZ5GCRN98DP" } } -```` +``` ### Method 2: Retrieve the `cf-aig-log-id` via API (GET request) From 295d67d830eeccd93d86672a7e30410fc5aee4b2 Mon Sep 17 00:00:00 2001 From: daisyfaithauma Date: Thu, 24 Oct 2024 17:29:22 +0100 Subject: [PATCH 12/12] Update src/content/docs/ai-gateway/evaluations/add-human-feedback-api.mdx Co-authored-by: Rebecca Tamachiro <62246989+RebeccaTamachiro@users.noreply.github.com> --- .../docs/ai-gateway/evaluations/add-human-feedback-api.mdx | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/content/docs/ai-gateway/evaluations/add-human-feedback-api.mdx b/src/content/docs/ai-gateway/evaluations/add-human-feedback-api.mdx index 17f41ab6ee697b..a74c9c2710e384 100644 --- a/src/content/docs/ai-gateway/evaluations/add-human-feedback-api.mdx +++ b/src/content/docs/ai-gateway/evaluations/add-human-feedback-api.mdx @@ -31,7 +31,7 @@ In the request above: - Replace `{account_id}` and `{gateway_id}` with your specific Cloudflare account and gateway details. - Replace `{your_api_token}` with the API token you just created. -## Step 3. Retrieve the `cf-aig-log-id` +## 3. Retrieve the `cf-aig-log-id` The `cf-aig-log-id` is a unique identifier for the specific log entry to which you want to add feedback. Below are two methods to obtain this identifier.