From cf5abed025592d76c22d98e8b7c9a051ca00d946 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Fredrik=20Niemel=C3=A4?= Date: Mon, 1 Sep 2025 11:46:12 +0400 Subject: [PATCH] Clarify test data config for validator test data --- spec/2023-07-draft.md | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/spec/2023-07-draft.md b/spec/2023-07-draft.md index 091d86a4..d2d7e0cd 100644 --- a/spec/2023-07-draft.md +++ b/spec/2023-07-draft.md @@ -666,10 +666,12 @@ Their goal is to ensure the integrity and quality of the test data and validatio #### Invalid Input -The files under `invalid_input` are invalid inputs. +The files under `invalid_input` are invalid inputs. Unlike in `sample` and `secret`, there are no `.ans` files. Each `tc.in` under `invalid_input` must be rejected by at least one input validator. +The relevant settings from [Test Case Configuration](#test-case-configuration) (`input_validator_args`, and `description`) can be set in `tc.yaml` for one test case, or `test_group.yaml` for all test cases in the same directory. + #### Invalid Output The test cases in `invalid_output` describe invalid outputs for non-interactive problems. @@ -683,7 +685,7 @@ In particular, for any test case in `invalid_output/`, for example `invalid_outp tc.in tc.ans dir [arguments] < tc.out # MUST FAIL ``` -The directory `invalid_output` must be organized into a tree-like structure similar to `secret` and may contain arguments in `test_group.yaml` files that are passed to the validators. +The relevant settings from [Test Case Configuration](#test-case-configuration) (`input_validator_args`, `output_validator_args`, and `description`) can be set in `tc.yaml` for one test case, or `test_group.yaml` for all test cases in the same directory. ### Valid Output @@ -700,7 +702,7 @@ In particular, for any test case in `valid_output/`, for example `valid_output/t tc.in tc.ans dir [arguments] < tc.out # MUST PASS ``` -The directory `valid_output` must be organized into a tree-like structure similar to `secret` and may contain arguments in `test_group.yaml` files that are passed to the validators. +The relevant settings from [Test Case Configuration](#test-case-configuration) (`input_validator_args`, `output_validator_args`, and `description`) can be set in `tc.yaml` for one test case, or `test_group.yaml` for all test cases in the same directory. ### Samples