Skip to content

Conversation

@daniellepintz
Copy link
Contributor

This will make the dataloader deliver the data in the same order each time.

Test Plan

python -m apps.grpo.main --config apps/grpo/qwen3_1_7b.yaml

Added a breakpoint, and printed out the prompt in pdb:
Screenshot 2025-12-01 at 4 39 26 PM

Before, prompt was different across runs; after this PR prompt is the same across runs.

@meta-cla meta-cla bot added the CLA Signed This label is managed by the Meta Open Source bot. label Dec 1, 2025
@casteryh
Copy link
Contributor

casteryh commented Dec 1, 2025

LGTM. can you verify that this makes the dataloader deterministic beyond the first epoch too? you can test it by truncating the dataset for example.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CLA Signed This label is managed by the Meta Open Source bot.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants