[Post Mortem] Check equal issue for open division + check accuracy run covers all the dataset #2170
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Contributor
|
MLCommons CLA bot All contributors have signed the MLCommons CLA ✍️ ✅ |
pgmpablo157321
force-pushed
the
open_checker_fix
branch
3 times, most recently
from
89faf3d to
fff3e42
Compare
|
Kenneth Leach: How does the change for checking accuracy affect the accuracy check run time? Pablo: It shouldn't. Currently, accuracy check runs over the whole data set. Last time, some accuracy tests weren't done for the whole data set, and the submission checker didn't catch it. This check avoids that oversight. Shouldn't change the time to run if you've been doing that correctly. We've run it against prior submissions, and they all pass. |
pgmpablo157321
force-pushed
the
open_checker_fix
branch
from
fff3e42 to
1c873d0
Compare
mrmhodak
approved these changes
May 20, 2025
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Fix #2173 Fix #2174