-
Notifications
You must be signed in to change notification settings - Fork 21
Description
In legacy, I can't find what the intended behavior is when you have multiple custom graders.
Problemtools has an at least 11-year old comment made by @niemela stating "# TODO: check that all graders give same result" (maybe not even him, since it the PR message is "Moving project from Mercurial"). AFAICT, there's no way to run a specific grader: it's either custom or default... but then, what's the point of having multiple graders if they all must have the exact same output?
Unless someone provides a good reason, could we not just retroactively enforce a limit of at most one grader per problem package? For existing problem packages, the fix is trivial (are there even such problem packages??): just remove all but one grader, as they should all behave the same..... and if they don't all behave the same, just throw them out and rewrite the grader from scratch, as it's definitely broken (if you even need it).