Search results

Reproducing HELM Arabic

user question

Status: Open.

#4046 In stanford-crfm/helm;

· zaidalyafeai opened on Jan 30, 2026

Much higher WER for gpt4o-mini, compared with published leaderboard

user question

Status: Open.

#3971 In stanford-crfm/helm;

· samrae7 opened on Jan 4, 2026

MMLU evaluation fails with Qwen3

user question

Status: Open.

#3942 In stanford-crfm/helm;

· shaafsalman opened on Nov 27, 2025

Question: Specifying a custom set of metrics to helm-run.

Status: Open.

#3915 In stanford-crfm/helm;

· Erotemic opened on Nov 6, 2025

Refactoring the Contamination Scenario

Status: Open.

#3914 In stanford-crfm/helm;

· IriedsonSouto opened on Nov 4, 2025

Duplication of main metrics

Status: Open.

#3847 In stanford-crfm/helm;

· yifanmai opened on Aug 26, 2025

Missing HEIM permutations

Status: Open.

#3824 In stanford-crfm/helm;

· Erotemic opened on Aug 21, 2025

Duplicate entries in med_dialog dataset

MedHELM

Status: Open.

#3746 In stanford-crfm/helm;

· bkj opened on Jul 15, 2025

Bug: ValueError: tokenizer_name is None when adding a new Hugging Face model due to prod_env behavior

user question

Status: Open.

#3709 In stanford-crfm/helm;

· ztmyjs opened on Jul 3, 2025

Proposal: Integrate Psychometric-Based Question Validity Tools into HELM

Status: Open.

#3645 In stanford-crfm/helm;

· yuhengtu opened on Jun 6, 2025

Links on scenario leaderboard table go to all runs instead of scenario runs

Status: Open.

#3569 In stanford-crfm/helm;

· yifanmai opened on May 8, 2025

Issue with link to instances beyond first page

Status: Open.

#3425 In stanford-crfm/helm;

· liamjxu opened on Mar 12, 2025