-
Notifications
You must be signed in to change notification settings - Fork 1
Open
Labels
contributions-welcomeGood for people looking to contributeGood for people looking to contributedocumentationImprovements or additions to documentationImprovements or additions to documentation
Description
Is your feature request related to a problem? Please describe.
The dokimos-examples module could benefit from additional example evaluators, which showcase more common evaluation patterns. This would allow developers to explore more real-world evaluation scenarios beyond exact match and regex in the repo.
Describe the solution you'd like
The examples module you be extended with some custom implementations for:
- semantic similarity / deduplication of context chunks
- A length-based evaluator that makes sure outputs are in certain length ranges
Metadata
Metadata
Assignees
Labels
contributions-welcomeGood for people looking to contributeGood for people looking to contributedocumentationImprovements or additions to documentationImprovements or additions to documentation