Implement 'simplified judgements'

[nickygerritsen]

The current draft Contest Information Access Policy (CIAP) has a new thing called 'Simplified judgement type' for judgements. The main and only use for this is to give teams access to a simplified judgement type of judgements of submissions of other teams (a lot of 'of' here...) for submissions made before the freeze (or up to 3 accepted ones in the freeze for 'the Bill rule'). So instead of seeing that a submissions got WA, you can only see it was RE. I.e. teams would only see AC and RE (and maaayybeee CE if it doesn't give a penalty?) for these submissions.

It was discussed during the CIAP meetings that this probably is best handled by something in the CCS spec, so hence this issue.

One solution could be to add a simplified_judgement_id to the judgement type object. But this makes encoding this in /access a bit hard.

Another way could be to add simplified_judgement_id to the judgement endpoint and exposing that (but not judgement_id) to teams.

Maybe there are even more options...

[deboer-tim]

I would prefer not to use /judgements for a few reasons:

• it's fairly verbose/a lot of extra data for what it is.
• it implies (even by accident) that WA could be simplified to RE in one case and something else (or not simplified) in another, and this should never happen.
• since judgement types come before judgements, there's no way to filter them out if there's ever an access policy that should only see the simplified judgement types.

I don't understand the issue with access and using /judgement_types?

Considering a hypothetical account that should only see the simplified types and never the specific ones, the CDS wouldn't pass through any judgement types that had a simplified type, and would switch the type of all judgements to the corresponding simplified type. For team accounts, the CDS would only do the type switching, and only for judgements from other teams.

[nickygerritsen]

I would prefer not to use /judgements for a few reasons:

• it's fairly verbose/a lot of extra data for what it is.
• it implies (even by accident) that WA could be simplified to RE in one case and something else (or not simplified) in another, and this should never happen.
• since judgement types come before judgements, there's no way to filter them out if there's ever an access policy that should only see the simplified judgement types.

I don't understand the issue with access and using /judgement_types?

Considering a hypothetical account that should only see the simplified types and never the specific ones, the CDS wouldn't pass through any judgement types that had a simplified type, and would switch the type of all judgements to the corresponding simplified type. For team accounts, the CDS would only do the type switching, and only for judgements from other teams.

Hmmm I think you might be right. It feels a bit weird to say 'account A will get different data for this field than account B' but I don't think there is anything against it.

[eldering]

One solution could be to add a simplified_judgement_id to the judgement type object. But this makes encoding this in /access a bit hard.

I'm not sure I understand this option. I don't think this is only about adding a simplified_judgement_id property to judgement_types, because that endpoint is basically public information: everyone may know (and knows) that judgement types like WA and TLE exist.

So does this option imply implicit rewriting of each judgement object's judgement_type_id property for users of the API who don't/shouldn't have access the original value of that object's judgement_type_id?

To be honest, I don't like that as it implies that we expose different values for the very same object's properties depending on who asks. That requires more complicated logic for example for the event feed if you cache/store event data as it is generated, as this data must now be remapped depending on the user.

I think that it is cleaner to add an extra property simplified_judgement_id to judgements that everyone can access (as long as you can access judgements) and then handle permission to get the original judgement_type_id depending on who asks.

I think with the rewriting option (assuming my interpretation is correct), things like comparing API responses to different users would be less clear.
Also, if a restricted user would at a later point (say after thawing the contest) be able to see the real judgement_type_id values, then that would be really messy from an event feed perspective in terms of changes. Although admittedly, also the second option would be a bit messy, but at least only require adding the original judgement_type_id, not rewriting it.

[deboer-tim]

One solution could be to add a simplified_judgement_id to the judgement type object. But this makes encoding this in /access a bit hard.

I'm not sure I understand this option. I don't think this is only about adding a simplified_judgement_id property to judgement_types, because that endpoint is basically public information: everyone may know (and knows) that judgement types like WA and TLE exist.

Correct, it's not only about this (although there are lots of other things that are public information that you still can't see, like judgements during the freeze or teams and reaction videos...).

So does this option imply implicit rewriting of each judgement object's judgement_type_id property for users of the API who don't/shouldn't have access the original value of that object's judgement_type_id?

Yes.

To be honest, I don't like that as it implies that we expose different values for the very same object's properties depending on who asks. That requires more complicated logic for example for the event feed if you cache/store event data as it is generated, as this data must now be remapped depending on the user.

I think that it is cleaner to add an extra property simplified_judgement_id to judgements that everyone can access (as long as you can access judgements) and then handle permission to get the original judgement_type_id depending on who asks.

Fair point on not liking rewriting, but this proposal has the same drawbacks as my first response above, i.e. being verbose and a judgement type potentially being simplified to different values for different submissions. But another drawback is that it makes things for difficult for clients - e.g. any tool built to handle judgements today would fail when judgement_type_id doesn't exist on some runs. Every client would need to use the simplified type as a fallback value.

I think with the rewriting option (assuming my interpretation is correct), things like comparing API responses to different users would be less clear.

Isn't it the opposite? In the other proposal you just compare the one property: client A got judgement type x and client B got type y, done. In this proposal you also need to check if they didn't get a judgement type but did get a simplified judgement type.

Also, if a restricted user would at a later point (say after thawing the contest) be able to see the real judgement_type_id values, then that would be really messy from an event feed perspective in terms of changes. Although admittedly, also the second option would be a bit messy, but at least only require adding the original judgement_type_id, not rewriting it.

If we're talking about sending new events to 'unfilter' the simplified types, then all of the 2000+ judgements need to be resent so it's a wash to me either way.