Skip to content

Conversation

@jamesbraza
Copy link
Collaborator

See PR title

@jamesbraza jamesbraza self-assigned this Feb 5, 2026
@jamesbraza jamesbraza added the enhancement New feature or request label Feb 5, 2026
Copilot AI review requested due to automatic review settings February 5, 2026 21:11
@dosubot dosubot bot added the size:M This PR changes 30-99 lines, ignoring generated files. label Feb 5, 2026
Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR migrates the LABBench2 datasets (FIG_QA2 and TABLE_QA2) from the "futurehouse/litqa3" Hugging Face Hub to a new "futurehouse/labbench2" hub. The migration includes updating the dataset loading mechanism to use hf_name="all" instead of None, and updating test expectations to match the new dataset contents.

Changes:

  • Renamed dataset constant from DEFAULT_LITQA3_HF_HUB_NAME to DEFAULT_LABBENCH2_HF_HUB_NAME and updated the hub path
  • Modified dataset loading logic to use hf_name="all" for FIG_QA2 and TABLE_QA2 datasets, with appropriate type annotation updates
  • Updated test questions and expected dataset size to match the new LABBench2 dataset content

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

File Description
packages/labbench/src/aviary/envs/labbench/task.py Renames the Hugging Face Hub constant, updates the dataset loading logic to use hf_name="all" for LABBench2 datasets, and adds str to the type annotation
packages/labbench/tests/test_tasks.py Updates test expectations with new question content from LABBench2 and adjusts FIG_QA2 dataset size from 100 to 101

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@jamesbraza jamesbraza merged commit c9d2864 into main Feb 5, 2026
5 checks passed
@jamesbraza jamesbraza deleted the lb2 branch February 5, 2026 22:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request size:M This PR changes 30-99 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants