Skip to content

Commit 25170bb

Browse files
Merge pull request #1489 from sdgilley/patch-11
Update copilot-sdk-evaluate.md
2 parents f243a5e + 3ba59cd commit 25170bb

File tree

1 file changed

+26
-24
lines changed

1 file changed

+26
-24
lines changed

articles/ai-studio/tutorials/copilot-sdk-evaluate.md

Lines changed: 26 additions & 24 deletions
Original file line numberDiff line numberDiff line change
@@ -107,37 +107,39 @@ In Part 1 of this tutorial series, you created an **.env** file that specifies t
107107

108108
### Interpret the evaluation output
109109

110-
In the console output, you see for each question an answer and the summarized metrics. (You might see different columns in your output.)
110+
In the console output, you see an answer for each question, followed by a table with summarized metrics. (You might see different columns in your output.)
111111

112-
If you weren't able to increase the tokens per minute limit for your model, you might see some time-out errors, which are expected. The evaluation script is designed to handle these errors and continue running.
112+
If you weren't able to increase the tokens per minute limit for your model, you might see some time-out errors, which are expected. The evaluation script is designed to handle these errors and continue running.
113113
114-
```txt
114+
> [!NOTE]
115+
> You may also see many `WARNING:opentelemetry.attributes:` - these can be safely ignored and do not affect the evaluation results.
116+
117+
```Text
115118
====================================================
116119
'-----Summarized Metrics-----'
117-
{'groundedness.gpt_groundedness': 2.230769230769231,
118-
'groundedness.groundedness': 2.230769230769231}
120+
{'groundedness.gpt_groundedness': 1.6666666666666667,
121+
'groundedness.groundedness': 1.6666666666666667}
119122
'-----Tabular Result-----'
120-
outputs.response ... outputs.groundedness.groundedness_reason
121-
0 Could you please specify which tent you are as... ... The RESPONSE fails to engage with the specific...
122-
1 Could you please specify which camping table y... ... The RESPONSE does not utilize any of the infor...
123-
2 Sorry, I only can answer queries related to ou... ... The RESPONSE does not relate to the CONTEXT at...
124-
3 To properly care for your TrailWalker Hiking S... ... The RESPONSE provides care instructions for th...
125-
4 The TrailMaster X4 Tent is from the OutdoorLiv... ... The RESPONSE accurately identifies the brand o...
126-
5 The TrailMaster X4 Tent comes with an included... ... The RESPONSE accurately reflects information f...
127-
6 Sorry, I only can answer queries related to ou... ... The RESPONSE does not relate to the CONTEXT at...
128-
7 The TrailBlaze Hiking Pants are crafted from h... ... The RESPONSE accurately reflects part of the i...
129-
8 The color of the TrailBlaze Hiking Pants is de... ... The RESPONSE accurately mentions the color of ...
130-
9 Sorry, I only can answer queries related to ou... ... The RESPONSE is entirely unrelated to the CONT...
131-
10 Sorry, I only can answer queries related to ou... ... The RESPONSE does not reference or relate to a...
132-
11 The material for the PowerBurner Camping Stove... ... The RESPONSE does not contradict the CONTEXT b...
133-
12 Sorry, I only can answer queries related to ou... ... The RESPONSE does not reference or relate to a...
134-
135-
[13 rows x 7 columns]
136-
'View evaluation results in AI Studio: xxxxxx'
123+
outputs.response ... line_number
124+
0 Could you specify which tent you are referring... ... 0
125+
1 Could you please specify which camping table y... ... 1
126+
2 Sorry, I only can answer queries related to ou... ... 2
127+
3 Could you please clarify which aspects of care... ... 3
128+
4 Sorry, I only can answer queries related to ou... ... 4
129+
5 The TrailMaster X4 Tent comes with an included... ... 5
130+
6 (Failed) ... 6
131+
7 The TrailBlaze Hiking Pants are crafted from h... ... 7
132+
8 Sorry, I only can answer queries related to ou... ... 8
133+
9 Sorry, I only can answer queries related to ou... ... 9
134+
10 Sorry, I only can answer queries related to ou... ... 10
135+
11 The PowerBurner Camping Stove is designed with... ... 11
136+
12 Sorry, I only can answer queries related to ou... ... 12
137+
138+
[13 rows x 8 columns]
139+
('View evaluation results in AI Studio: '
140+
'https://xxxxxxxxxxxxxxxxxxxxxxx')
137141
```
138142
139-
> [!NOTE]
140-
> You may see `WARNING:opentelemetry.attributes:` - these can be safely ignored and do not affect the evaluation results.
141143
142144
### View evaluation results in AI Studio
143145

0 commit comments

Comments
 (0)