|
| 1 | +--- |
| 2 | +pcx_content_type: how-to |
| 3 | +title: Add Human Feedback |
| 4 | +sidebar: |
| 5 | + order: 3 |
| 6 | + badge: |
| 7 | + text: Beta |
| 8 | +--- |
| 9 | + |
| 10 | +Human feedback is a valuable metric to assess the performance of your AI models. By incorporating human feedback, you can gain deeper insights into how the model's responses are perceived and how well it performs from a user-centric perspective. This feedback can then be used in evaluations to calculate performance metrics, driving optimization and ultimately enhancing the reliability, accuracy, and efficiency of your AI application. |
| 11 | + |
| 12 | +Human feedback measures the performance of your dataset based on direct human input. The metric is calculated as the percentage of positive feedback (thumbs up) given on logs, which are annotated in the Logs tab of the Cloudflare dashboard. This feedback helps refine model performance by considering real-world evaluations of its output. |
| 13 | + |
| 14 | +This tutorial will guide you through the process of adding human feedback to your evaluations in AI Gateway. |
| 15 | + |
| 16 | +## 1. Log into the dashboard |
| 17 | + |
| 18 | +1. Log into the [Cloudflare dashboard](https://dash.cloudflare.com/) and select your account. |
| 19 | +2. Navigate to the AI Gateway section. |
| 20 | + |
| 21 | +## 2. Access the Logs tab |
| 22 | + |
| 23 | +1. In the dashboard, go to the Logs tab under the AI Gateway section. |
| 24 | +2. The Logs tab displays all logs associated with your datasets. These logs show key information, including: |
| 25 | + - Timestamp: When the interaction occurred. |
| 26 | + - Status: Whether the request was successful, cached, or failed. |
| 27 | + - Model: The model used in the request. |
| 28 | + - Tokens: The number of tokens consumed by the response. |
| 29 | + - Cost: The cost based on token usage. |
| 30 | + - Duration: The time taken to complete the response. |
| 31 | + - Feedback: Where you can provide human feedback on each log. |
| 32 | + |
| 33 | +## 3. Provide human feedback |
| 34 | + |
| 35 | +1. In the Logs tab, click on the log entry you want to review. This expands the log, allowing you to see more detailed information. |
| 36 | +2. In the expanded log, you can view additional details such as: |
| 37 | + - The user prompt. |
| 38 | + - The model's response. |
| 39 | + - HTTP response details. |
| 40 | + - Endpoint information. |
| 41 | +3. You will see two icons: |
| 42 | + - Thumbs up: Indicates positive feedback. |
| 43 | + - Thumbs down: Indicates negative feedback. |
| 44 | +4. Click either the thumbs up or thumbs down icon based on how you rate the model's response for that particular log entry. |
| 45 | + |
| 46 | +## 4. Evaluate human feedback |
| 47 | + |
| 48 | +1. After providing feedback on your logs, it becomes part of the evaluation process. |
| 49 | + - When you run an evaluation (as outlined in the [Set Up Evaluations](/ai-gateway/evaluations/set-up-evaluations/) guide), the human feedback metric will be calculated based on the percentage of logs that received thumbs-up feedback. |
| 50 | + - Note: You need to select human feedback as an evaluator to receive its metrics. |
| 51 | +2. The human feedback score is displayed as a percentage, showing how much of the dataset's responses were rated positively by human reviewers. |
| 52 | + |
| 53 | +## 5. Review results |
| 54 | + |
| 55 | +After running the evaluation,review results on the Evaluations tab. |
| 56 | +You will be able to see the model's performance based on cost, speed, and now human feedback, represented as the percentage of positive feedback (thumbs up). |
| 57 | + |
| 58 | +For more information on running evaluations, refer to the official Cloudflare documentation: |
| 59 | + |
| 60 | +- [Set Up Evaluations](/ai-gateway/evaluations/set-up-evaluations/) |
0 commit comments