2024-06-05: Update to new AI Chat Protocol, increase flexibility
This release both updates the evaluator tool to assume that the chat backend conforms to the new Microsoft AI Chat Protocol but also adds two new properties to the config JSON to allow for use with backends using the older protocol or other protocols.
Just add these fields and customize the JMESPath expressions as needed:
"target_response_answer_jmespath": "choices[0].message.content",
"target_response_context_jmespath": "choices[0].context.data_points.text"
What's Changed
- Bump promptflow-evals from 0.2.0.dev0 to 0.3.0 by Dependabot (@dependabot) in #88
- Avoid rate-limiting, improve --changed by Pamela Fox (@pamelafox) in #91
- Update to new Protocol response format, use JMESPath expressions by Pamela Fox (@pamelafox) in #92
Full Changelog: 2024-05-13...2024-06-05