Skip to content

Commit c9d5ed3

Browse files
Etienne Vouganiemela
authored andcommitted
Subgroups of secret must have pass-fail aggregation if secret does
Subgroups of secret do not have maximum scores if secret has pass-fail aggregation
1 parent 39b671b commit c9d5ed3

File tree

1 file changed

+5
-1
lines changed

1 file changed

+5
-1
lines changed

spec/2023-07-draft.md

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1297,10 +1297,14 @@ The goal of each submission is to maximize this score.
12971297
Given a submission, scores are determined for test cases, [test data groups](#test-data-groups), and `secret` (which is the score of the submission itself).
12981298
The scoring behavior is configured for `secret` and each test data group by `max_score`, `score_aggregation`, and `require_pass` in its [`test_group.yaml`](#test-data-configuration).
12991299

1300+
The value of `score_aggregation` determines how the score of `secret` or a test data group is computed; see [Scoring Test Groups](#scoring-test-groups).
13001301
The default value of `score_aggregation` is `sum` for `secret` and `pass-fail` for test data groups.
1302+
If `secret` has `pass-fail` aggregation, then so must all of its test data groups.
1303+
1304+
The `require_pass` key may be used to indicate that `secret` or a test data group should only be scored if all test cases in `sample` or an earlier test data group received an accepted verdict; see [Required Dependent Groups](#required-dependent-groups).
13011305
The default value of `require_pass` is an empty sequence.
13021306

1303-
For `secret`, all test data groups, and every test case in a group with `sum` or `min` aggregation, there is a maximum possible score.
1307+
For `secret`, all test data groups with `sum` or `min` aggregation, and every test case in a group with `sum` or `min` aggregation, there is a maximum possible score.
13041308
The default value of `max_score` for `secret` is 100.
13051309
The default value of `max_score` for test data groups is `unbounded`.
13061310
Test data groups may only have `unbounded` maximum score if `secret` is unbounded.

0 commit comments

Comments
 (0)