Add TruthfulQA benchmarkspec asset

[Copilot]

Adds benchmarkspec for TruthfulQA dataset (817 questions measuring model truthfulness on common misconceptions across 38 categories). Restructures evaluator schema to support C# JObject deserialization.

Changes

• New benchmarkspec: assets/benchmarkspecs/builtin/truthful_qa/

  • spec.yaml: Full benchmark definition with string_check evaluator
  • asset.yaml: Asset metadata

• Simplified evaluator schema: Reduced to id + testingCriteria for JObject compatibility

  evaluator:
    id: "azureml://registries/azureml/evaluators/builtin.string_check/versions/2"
    testingCriteria:
      type: "string_check"
      input: "{{sample.output_text}}"
      operation: "eq"
      reference: "{{item.Best_Answer}}"
      name: "TruthfulQA"

Dataset Configuration

• Source: domenicrosati/TruthfulQA (HuggingFace)
• Schema: 7 string columns (Type, Category, Question, Best_Answer, Correct_Answers, Incorrect_Answers, Source)
• Evaluator: Exact string match against Best_Answer
• License: Apache 2.0

---

💬 We'd love your input! Share your thoughts on Copilot coding agent in our 2 minute survey.

[github-actions]

Test Results for assets-test

0 tests   0 ✅  0s ⏱️
0 suites  0 💤
0 files    0 ❌

Results for commit 303953c.

♻️ This comment has been updated with latest results.