Skip to content

Add more evaluator examples to dokimos-examples module #4

@fkapsahili

Description

@fkapsahili

Is your feature request related to a problem? Please describe.

The dokimos-examples module could benefit from additional example evaluators, which showcase more common evaluation patterns. This would allow developers to explore more real-world evaluation scenarios beyond exact match and regex in the repo.

Describe the solution you'd like

The examples module you be extended with some custom implementations for:

  • semantic similarity / deduplication of context chunks
  • A length-based evaluator that makes sure outputs are in certain length ranges

Metadata

Metadata

Assignees

No one assigned

    Labels

    contributions-welcomeGood for people looking to contributedocumentationImprovements or additions to documentation

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions