Skip to content

Commit 68d368e

Browse files
authored
Evaluate function evaluator_config reverses evaluator name and column_mapping
The code expects: ``` result = evaluate( data=dataset, evaluators=evaluators, # column mapping evaluator_config={ "default": { "column_mapping": { "query": "${data.question}", "response": "${data.final_answer}", "ground_truth": "${data.gold_final_answer}", "context": "${data.context}", } } }, ) ``` The doc says: ``` from askwiki import askwiki result = evaluate( data="data.jsonl", target=askwiki, evaluators={ "relevance": relevance_eval }, evaluator_config={ "column_mapping": { "default": { "query": "${data.queries}" "context": "${outputs.context}" "response": "${outputs.response}" } } } ) ```
1 parent d84fb33 commit 68d368e

File tree

1 file changed

+4
-4
lines changed

1 file changed

+4
-4
lines changed

articles/ai-studio/how-to/develop/evaluate-sdk.md

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -432,8 +432,8 @@ result = evaluate(
432432
},
433433
# column mapping
434434
evaluator_config={
435-
"column_mapping": {
436-
"relevance": {
435+
"relevance": {
436+
"column_mapping": {
437437
"query": "${data.queries}"
438438
"ground_truth": "${data.ground_truth}"
439439
"response": "${outputs.response}"
@@ -560,8 +560,8 @@ result = evaluate(
560560
"relevance": relevance_eval
561561
},
562562
evaluator_config={
563-
"column_mapping": {
564-
"default": {
563+
"default": {
564+
"column_mapping": {
565565
"query": "${data.queries}"
566566
"context": "${outputs.context}"
567567
"response": "${outputs.response}"

0 commit comments

Comments
 (0)