Skip to content

2024-06-05: Update to new AI Chat Protocol, increase flexibility

Choose a tag to compare

@pamelafox pamelafox released this 05 Jun 18:28
· 93 commits to main since this release
3aa7f95

This release both updates the evaluator tool to assume that the chat backend conforms to the new Microsoft AI Chat Protocol but also adds two new properties to the config JSON to allow for use with backends using the older protocol or other protocols.

Just add these fields and customize the JMESPath expressions as needed:

    "target_response_answer_jmespath": "choices[0].message.content",
    "target_response_context_jmespath": "choices[0].context.data_points.text"

What's Changed

Full Changelog: 2024-05-13...2024-06-05