diff --git a/spec/2025-09.md b/spec/2025-09.md index 7f72a67a..6c0b476b 100644 --- a/spec/2025-09.md +++ b/spec/2025-09.md @@ -670,7 +670,7 @@ Their goal is to ensure the integrity and quality of the test data and validatio The files under `invalid_input` are invalid inputs. Unlike in `sample` and `secret`, there are no `.ans` files. -Each `tc.in` under `invalid_input` must be rejected by at least one input validator. +Each `tc.in` under `invalid_input` must be rejected by at least one input validator. (Other input validators may reject the test case, accept it, or fail to operate properly.) The relevant settings from [Test Case Configuration](#test-case-configuration) (`input_validator_args`, and `description`) can be set in `tc.yaml` for one test case, or `test_group.yaml` for all test cases in the same directory. @@ -985,12 +985,25 @@ first pass will be used. The input validator may output debug information on stdout and stderr. This information may be displayed to the user upon invocation of the validator. -### Exit codes +### Reporting Validation Results -The input validator must exit with code 42 on successful validation. -Any other exit code means that the input file could not be confirmed as valid. +The input validator can accept a test case as valid, reject a test case, or fail to operate properly. Although validation fails in both of the latter cases, they should be clearly distinguished in any diagnostics reported to the user. -#### Dependencies +The Checktestdata and VIVA APIs define their mechanisms for reporting validation results. Input validator programs report a result via exit code. + +#### Exit Codes for Input Validator Programs + +An input validator program must return with one of the following codes: +- the input validator must exit with code 42 on successful validation; +- the validator must exit with code 43 to reject the test case. + +The purpose of these somewhat exotic exit codes is to avoid conflict with other exit codes that results when the validator crashes. +For instance, if the validator is written in Java, any unhandled exception results in the program crashing with an exit code of 1, +making it unsuitable to assign a meaning to this exit code. + +An input validator program that does not compile or run, crashes, or returns with any exit code other than 42 or 43 fails to operate properly. + +### Dependencies The validator **must not** read any files outside those defined in the Invocation section. Its result **must** depend only on these files and the arguments. @@ -1166,9 +1179,6 @@ A validator program must report its judgement by exiting with specific exit code Any other exit code, **including 0**, indicates that the validator did not operate properly, and the judging system invoking the validator must take measures to report this to contest personnel. -The purpose of these somewhat exotic exit codes is to avoid conflict with other exit codes that results when the validator crashes. -For instance, if the validator is written in Java, any unhandled exception results in the program crashing with an exit code of 1, -making it unsuitable to assign a judgement meaning to this exit code. ### Reporting Additional Feedback