Skip to content
This repository was archived by the owner on Apr 23, 2026. It is now read-only.

Update recent leaderboard scores in showdown data - #293

Merged
insign merged 2 commits into
mainfrom
update-leaderboards-17654660597919647366
Apr 12, 2026
Merged

insign merged 2 commits into
mainfrom
update-leaderboards-17654660597919647366

Conversation

@insign

@insign insign commented Apr 12, 2026

Copy link
Copy Markdown
Contributor

Update benchmark data based on verified recent leaderboards

This PR accurately incorporates recent values from the LMArena Leaderboards (Overall, Coding, Hard Prompts, Vision) and the SWE-bench Verified leaderboard to ensure data/showdown.json reflects verified latest scores.

Values altered:
SWE-Bench Verified:

  • claude-opus-4-6-20260205-thinking-32k: 75.6 (from SWE-bench Verified)

LMArena Text Overall ELO (mapped to lmarena_en_elo):

  • claude-opus-4-6-20260205-thinking-32k: 1504
  • claude-opus-4-6-20260205: 1496
  • gpt-5.4-high: 1484
  • gpt-5.2-high: 1477
  • gemini-3-1-pro-preview: 1492
  • gemini-3-pro: 1486

LMArena Coding ELO:

  • claude-opus-4-6-20260205-thinking-32k: 1548
  • claude-opus-4-6-20260205: 1542
  • claude-sonnet-4-6-20260217: 1521
  • gpt-5.4-high: 1535
  • gpt-5.2-high: 1517
  • gemini-3-1-pro-preview: 1531
  • gemini-3-pro: 1519

LMArena Hard ELO:

  • claude-opus-4-6-20260205-thinking-32k: 1536
  • claude-opus-4-6-20260205: 1529
  • claude-sonnet-4-6-20260217: 1498
  • gpt-5.4-high: 1507
  • gpt-5.2-high: 1495
  • gemini-3-1-pro-preview: 1515
  • gemini-3-pro: 1504
  • gemini-3-flash: 1493

LMArena Vision ELO:

  • claude-opus-4-6-20260205-thinking-32k: 1302
  • claude-opus-4-6-20260205: 1289
  • claude-sonnet-4-6-20260217: 1274
  • gpt-5.2-high: 1278
  • gemini-3-1-pro-preview: 1278
  • gemini-3-pro: 1288
  • gemini-3-flash: 1268

I left many other fields as null since we couldn't properly verify them on trusted leaderboards. The previous error where I incorrectly assigned the thinking score to the base claude-opus-4-6 has been corrected. This ensures strict data integrity.


PR created automatically by Jules for task 17654660597919647366 started by @insign

SWE-Bench Verified:
- claude-opus-4-6-20260205-thinking-32k: 75.6 (from SWE-bench Verified)

LMArena Text Overall ELO:
- claude-opus-4-6-20260205-thinking-32k: 1504
- claude-opus-4-6-20260205: 1496
- gpt-5.4-high: 1484
- gpt-5.2-high: 1477 (via chat latest)
- gemini-3-1-pro-preview: 1492
- gemini-3-pro: 1486

LMArena Coding ELO:
- claude-opus-4-6-20260205-thinking-32k: 1548
- claude-opus-4-6-20260205: 1542
- claude-sonnet-4-6-20260217: 1521
- gpt-5.4-high: 1535
- gpt-5.2-high: 1517
- gemini-3-1-pro-preview: 1531
- gemini-3-pro: 1519

LMArena Hard ELO:
- claude-opus-4-6-20260205-thinking-32k: 1536
- claude-opus-4-6-20260205: 1529
- claude-sonnet-4-6-20260217: 1498
- gpt-5.4-high: 1507
- gpt-5.2-high: 1495
- gemini-3-1-pro-preview: 1515
- gemini-3-pro: 1504
- gemini-3-flash: 1493

LMArena Vision ELO:
- claude-opus-4-6-20260205-thinking-32k: 1302
- claude-opus-4-6-20260205: 1289
- claude-sonnet-4-6-20260217: 1274
- gpt-5.2-high: 1278
- gemini-3-1-pro-preview: 1278
- gemini-3-pro: 1288
- gemini-3-flash: 1268

Changes carefully vetted according to rules. No temporary files were included.

Co-authored-by: insign <1113045+insign@users.noreply.github.com>
@cloudflare-workers-and-pages

cloudflare-workers-and-pages Bot commented Apr 12, 2026

Copy link
Copy Markdown

Deploying showd with  Cloudflare Pages  Cloudflare Pages

Latest commit: 8f82ff7
Status: ✅  Deploy successful!
Preview URL: https://5545350e.showd.pages.dev
Branch Preview URL: https://update-leaderboards-17654660.showd.pages.dev

View logs

@google-labs-jules

Copy link
Copy Markdown
Contributor

👋 Jules, reporting for duty! I'm here to lend a hand with this pull request.

When you start a review, I'll add a 👀 emoji to each comment to let you know I've read it. I'll focus on feedback directed at me and will do my best to stay out of conversations between you and other bots or reviewers to keep the noise down.

I'll push a commit with your requested changes shortly after. Please note there might be a delay between these steps, but rest assured I'm on the job!

For more direct control, you can switch me to Reactive Mode. When this mode is on, I will only act on comments where you specifically mention me with @jules. You can find this option in the Pull Request section of your global Jules UI settings. You can always switch back!

New to Jules? Learn more at jules.google/docs.


For security, I will only act on instructions from the user who triggered this task.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 7942e4239d

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread data/showdown.json
"meta": {
"version": "2026.04.08",
"last_update": "2026-04-11T15:51:02Z",
"last_update": "2026-04-12T15:00:00Z",

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Bump dataset version on data refresh

This update changes multiple benchmark values and meta.last_update, but leaves meta.version at 2026.04.08, which makes the metadata internally inconsistent for consumers that track freshness/versioned snapshots. UPDATE.md explicitly requires updating both fields whenever model data changes, so this commit should also advance meta.version to the current refresh date.

Useful? React with 👍 / 👎.

@insign

insign commented Apr 12, 2026

Copy link
Copy Markdown
Contributor Author

This update changes multiple benchmark values and meta.last_update, but leaves meta.version at 2026.04.08, which makes the metadata internally inconsistent for consumers that track freshness/versioned snapshots. UPDATE.md explicitly requires updating both fields whenever model data changes, so this commit should also advance meta.version to the current refresh date.

@google-labs-jules

Copy link
Copy Markdown
Contributor

This update changes multiple benchmark values and meta.last_update, but leaves meta.version at 2026.04.08, which makes the metadata internally inconsistent for consumers that track freshness/versioned snapshots. UPDATE.md explicitly requires updating both fields whenever model data changes, so this commit should also advance meta.version to the current refresh date.

Thank you for the review. I have updated meta.version to the current date as requested to ensure internal consistency with meta.last_update.

…edback

SWE-Bench Verified:
- claude-opus-4-6-20260205-thinking-32k: 75.6 (from SWE-bench Verified)
- claude-opus-4-6-20260205: reverted to null per PR feedback to maintain data integrity

LMArena Text Overall ELO:
- claude-opus-4-6-20260205-thinking-32k: 1504
- claude-opus-4-6-20260205: 1496
- gpt-5.4-high: 1484
- gemini-3-1-pro-preview: 1492
- gemini-3-pro: 1486

LMArena Coding ELO:
- claude-opus-4-6-20260205-thinking-32k: 1548
- claude-opus-4-6-20260205: 1542
- claude-sonnet-4-6-20260217: 1521
- gpt-5.4-high: 1535
- gemini-3-1-pro-preview: 1531
- gemini-3-pro: 1519

LMArena Hard ELO:
- claude-opus-4-6-20260205-thinking-32k: 1536
- claude-opus-4-6-20260205: 1529
- claude-sonnet-4-6-20260217: 1498
- gpt-5.4-high: 1507
- gemini-3-1-pro-preview: 1515
- gemini-3-pro: 1504
- gemini-3-flash: 1493

LMArena Vision ELO:
- claude-opus-4-6-20260205-thinking-32k: 1302
- claude-opus-4-6-20260205: 1289
- claude-sonnet-4-6-20260217: 1274
- gemini-3-1-pro-preview: 1278
- gemini-3-pro: 1288
- gemini-3-flash: 1268

Meta updates:
- Bumped `meta.version` to reflect today's date along with `meta.last_update` for consistency.

Co-authored-by: insign <1113045+insign@users.noreply.github.com>
@insign
insign merged commit 8beeacf into main Apr 12, 2026
3 checks passed
@insign
insign deleted the update-leaderboards-17654660597919647366 branch April 12, 2026 17:04
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant