Skip to content

Commit c3bd145

Browse files
salma-elshafeySalma Elshafey
andauthored
Modify max_completion_tokens parameter for tool call accuracy evaluator (#42544)
* support 5 levels, evaluate all tools at once * Update sample notebook and change log * Add missing import * Modify test cases to match the new output format * Modify other test file to match the new output format * Fixed parsing of results * Change key name in output * Spell check fixes * Minor prompt update * Update result key to tool_call_accuracy * Delete test_new_evaluator.ipynb * Added field names and messages as constants * Additional note in prompt * Re-add the temperature to the prompty file * Removed 'applicable' field and print statement * Move excess/missing tool calls fields under additional details * Typo fix and removal of redundant field in the prompt * Modify per_tool_call_details field's name to details * Revert "Modify per_tool_call_details field's name to details" This reverts commit 2c3ce50. * Revert 'Merge branch 'main' into selshafey/improve_tool_call_accuracy' * Black reformat * Reformat with black * To re-trigger build pipelines * Add notebook for bugbash * modify bugbash notebook * Add support for built-in tools for Tool Call Accuracy Evaluator * Remove bugbash notebook * Resolve issues with merge * Fix id value * Use existing built-in tool definitions * Run black * Prompt modifications * Add test cases for built-in tools * Handle converter format * Add test cases for converter format * Support only converter format * Revert tool definitions to be required, run black * Change max_completion_tokens to max_tokens * merged with main * re-merge with main * run black --------- Co-authored-by: Salma Elshafey <[email protected]>
1 parent d0d6eb9 commit c3bd145

File tree

1 file changed

+2
-1
lines changed

1 file changed

+2
-1
lines changed

sdk/evaluation/azure-ai-evaluation/azure/ai/evaluation/_evaluators/_tool_call_accuracy/tool_call_accuracy.prompty

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,8 @@ description: Evaluates Tool Call Accuracy for tool used by agent
44
model:
55
api: chat
66
parameters:
7-
max_completion_tokens: 3000
7+
temperature: 0.0
8+
max_tokens: 3000
89
top_p: 1.0
910
presence_penalty: 0
1011
frequency_penalty: 0

0 commit comments

Comments
 (0)