Skip to content
Open
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
26 changes: 18 additions & 8 deletions spec/2025-09.md
Original file line number Diff line number Diff line change
Expand Up @@ -670,7 +670,7 @@ Their goal is to ensure the integrity and quality of the test data and validatio

The files under `invalid_input` are invalid inputs.
Unlike in `sample` and `secret`, there are no `.ans` files.
Each `tc.in` under `invalid_input` must be rejected by at least one input validator.
Each `tc.in` under `invalid_input` must be rejected by at least one input validator. (Other input validators may reject the test case, accept it, or fail to operate properly.)

The relevant settings from [Test Case Configuration](#test-case-configuration) (`input_validator_args`, and `description`) can be set in `tc.yaml` for one test case, or `test_group.yaml` for all test cases in the same directory.

Expand Down Expand Up @@ -985,12 +985,25 @@ first pass will be used.
The input validator may output debug information on stdout and stderr.
This information may be displayed to the user upon invocation of the validator.

### Exit codes
### Reporting Validation Results

The input validator must exit with code 42 on successful validation.
Any other exit code means that the input file could not be confirmed as valid.
The input validator can accept a test case as valid, reject a test case, or fail to operate properly. Although validation fails in both of the latter cases, they should be clearly distinguished in any diagnostics reported to the user.

#### Dependencies
The Checktestdata and VIVA APIs define their mechanisms for reporting validation results. Input validator programs report a result via exit code.

#### Exit Codes for Input Validator Programs

An input validator program must return with one of the following codes:
- the input validator must exit with code 42 on successful validation;
- the validator must exit with code 43 to reject the test case.

The purpose of these somewhat exotic exit codes is to avoid conflict with other exit codes that results when the validator crashes.
For instance, if the validator is written in Java, any unhandled exception results in the program crashing with an exit code of 1,
making it unsuitable to assign a meaning to this exit code.

An input validator program that does not compile or run, crashes, or returns with any exit code other than 42 or 43 fails to operate properly.

### Dependencies

The validator **must not** read any files outside those defined in the Invocation section.
Its result **must** depend only on these files and the arguments.
Expand Down Expand Up @@ -1166,9 +1179,6 @@ A validator program must report its judgement by exiting with specific exit code

Any other exit code, **including 0**, indicates that the validator did not operate properly,
and the judging system invoking the validator must take measures to report this to contest personnel.
The purpose of these somewhat exotic exit codes is to avoid conflict with other exit codes that results when the validator crashes.
For instance, if the validator is written in Java, any unhandled exception results in the program crashing with an exit code of 1,
making it unsuitable to assign a judgement meaning to this exit code.

### Reporting Additional Feedback

Expand Down