Clarify dependent groups #487
Open
+14
−16
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
hairez
suggested changes
Sep 2, 2025
| Each such test data group must have `pass-fail` aggregation. | ||
| The group specifying `require_pass` will only be run if the group being referred to receives an accepted verdict for all test cases in the group. | ||
| If the group is not run, the group score is 0, and it did not receive an accepted verdict for all test cases in the group. | ||
| The `require_pass` is set to the name of a single test data group, or `sample`, or a sequence of such names, or `sample`. |
There was a problem hiding this comment.
A bit confusing thatsample is mentioned twice here.
Suggestion:
Suggested change
| The `require_pass` is set to the name of a single test data group, or `sample`, or a sequence of such names, or `sample`. | |
| The `require_pass` is set to the name of a single test data group, or `sample`, or a sequence of such names. |
There was a problem hiding this comment.
Yours read a lot better, but technically sample is not defined to be a "name". That said... I'm inclined to merge your change.
Anybody thinks differently?
| A path consists of zero or more directory names followed by a directory or file name, with a `/` character separating consecutive names. Each name must conform to the [general requirements](#general-requirements) on directory and file names and the specified test data group must exist. | ||
|
|
||
| The path of a group, relative to the `data/` folder, must come later lexicographically than the paths of all test cases and groups it depends on. | ||
| A test data group, or `secret` may specify that it depends on some other test data groups, or `sample` by setting `require_pass`. |
There was a problem hiding this comment.
As I understand it, secret can only depend on sample, right? Maybe that should be specified.
And sample cannot depend on anything else, since it is the first group. But it is allowed to have a require_pass key.
Suggested change
| A test data group, or `secret` may specify that it depends on some other test data groups, or `sample` by setting `require_pass`. | |
| A test data group, or `secret` may specify that it depends on some other test data groups. `secret` may only depend on `sample`. `sample` cannot depend on any other group, but it may still have a `require_pass` key. |
There was a problem hiding this comment.
That is correct, and those are good points.
|
|
||
| The path of a group, relative to the `data/` folder, must come later lexicographically than the paths of all test cases and groups it depends on. | ||
| A test data group, or `secret` may specify that it depends on some other test data groups, or `sample` by setting `require_pass`. | ||
| Each such test data group must have `pass-fail` aggregation. |
There was a problem hiding this comment.
I got a bit confused if it was the groups that depend on others, or the groups dependent on that needs a pass-fail aggregation.
We should specify that this only applies to the test data groups being dependent on.
Suggested change
| Each such test data group must have `pass-fail` aggregation. | |
| Each group mentioned under `require_pass` must have `pass-fail` aggregation. |
| A test data group, or `secret` may specify that it depends on some other test data groups, or `sample` by setting `require_pass`. | ||
| Each such test data group must have `pass-fail` aggregation. | ||
| The group specifying `require_pass` will only be run if the group being referred to receives an accepted verdict for all test cases in the group. | ||
| If the group is not run, the group score is 0, and it did not receive an accepted verdict for all test cases in the group. |
There was a problem hiding this comment.
and it did not receive an accepted verdict for all test cases in the group.
A bit confusing. I assume this is to clarify that even if a group
| @@ -576,7 +577,7 @@ Key | Type | Default | |||
| `max_score` | Integer or `unbounded` | 100 in `secret`, otherwise `unbounded` | The maximum possible score of the test data group. Must be a non-negative integer or `unbounded`. This key is only permitted for the `secret` group and its subgroups. | |||
| `score_aggregation` | `pass-fail`, `sum`, or `min` | `sum` in `secret`, otherwise `pass-fail` | How the score is determined based on the scores of the contained groups or test cases. [See Result Aggregation](#result-aggregation). This key is only permitted for the `secret` group and its subgroups. | |||
| `static_validation_score` | Integer or `pass-fail` | | The maximum score of the static validation test case, or `pass-fail`. See [Static Validator](#static-validator). | |||
| `require_pass` | String or sequence of strings | empty sequence | Other test data groups whose test cases a submission must pass in order to receive a score for this test group. [See Result Aggregation](#result-aggregation). This key is only permitted for the `secret` group and its subgroups. | |||
| `require_pass` | String or sequence of strings | empty sequence | Names of other test data groups, or `sample`, whose test cases a submission must pass in order to receive a score for this test group. [See Result Aggregation](#result-aggregation). This key is only permitted for the `secret` group and its subgroups. | |||
There was a problem hiding this comment.
This key is only permitted for the `secret` group and its subgroups.
This is not correct. It is technically also allowed for sample, as mentioned further down under "Group Dependencies".
Also it should not refer to Result Aggregation, and refer directly to Group Dependencies.
Suggested change
| `require_pass` | String or sequence of strings | empty sequence | Names of other test data groups, or `sample`, whose test cases a submission must pass in order to receive a score for this test group. [See Result Aggregation](#result-aggregation). This key is only permitted for the `secret` group and its subgroups. | |
| `require_pass` | String or sequence of strings | empty sequence | Names of other test data groups, or `sample`, whose test cases a submission must pass in order to receive a score for this test group. See [Group Dependencies](#group-dependencies). |
Comment on lines
+594
to
+595
| The name of a test data group is the path under secret. | ||
| For example, the test data group defined by the directory `data/secret/group1` has the name `secret/group1`. |
There was a problem hiding this comment.
Maybe move the lines
The name of a test data group is the path under secret.
For example, the test data group defined by the directory `data/secret/group1` has the name `secret/group1`.
under
That is, if there are any directories under `data/secret/` there must not be any `.in` files directly in `data/secret/` and vice versa.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Fixes #463