Always make message extensions an optional value?

[jbeyerstedt]

While working with V2X messages, I came across this definition inside the "ETSI-ITS-DSRC" module:

RequestorDescription ::= SEQUENCE {
  id                VehicleID,
  type              RequestorType OPTIONAL,
  position          RequestorPositionVector OPTIONAL,
  name              DescriptiveName OPTIONAL,
  routeName         DescriptiveName OPTIONAL,
  transitStatus     TransitVehicleStatus OPTIONAL,
  transitOccupancy  TransitVehicleOccupancy OPTIONAL,
  transitSchedule   DeltaTime OPTIONAL,
  regional          SEQUENCE (SIZE(1..4)) OF RegionalExtension {{Reg-RequestorDescription}} OPTIONAL,
  ...,
  ocit OcitRequestorDescriptionContainer -- Extension for OCIT in V2.2.1
}

The RequestorDescription contains a message extension in form of the ocit field which isn't marked as OPTIONAL. So the resulting Rust code looks like this:

#[derive(AsnType, Debug, Clone, Decode, Encode, PartialEq, Eq, Hash)]
#[rasn(automatic_tags)]
#[non_exhaustive]
pub struct RequestorDescription {
    pub id: VehicleID,
    #[rasn(identifier = "type")]
    pub r_type: Option<RequestorType>,
    pub position: Option<RequestorPositionVector>,
    pub name: Option<DescriptiveName>,
    #[rasn(identifier = "routeName")]
    pub route_name: Option<DescriptiveName>,
    #[rasn(identifier = "transitStatus")]
    pub transit_status: Option<TransitVehicleStatus>,
    #[rasn(identifier = "transitOccupancy")]
    pub transit_occupancy: Option<TransitVehicleOccupancy>,
    #[rasn(identifier = "transitSchedule")]
    pub transit_schedule: Option<DeltaTime>,
    pub regional: Option<RequestorDescriptionRegional>,
    #[rasn(extension_addition)]
    pub ocit: OcitRequestorDescriptionContainer,
}

I my point of view the ocit field should be an Option<> since we could also get an "old" message without the extension present. Or how is the parser supposed to fill these fields when they are not present in the message.
JER parsing for example fails if the "ocit" key is missing which is not what I expect from a message extension.

But I'm not sure what the ASN.1 standard has intended when using a message extension with a non-optional value. It's quite an odd choice and only kind-of fine in this case because OcitRequestorDescriptionContainer only contains optional values so we would have a sane default value for "ocit". But that can't be an assumption the Parser relies on in my point of view.

ADDENDUM:
According to ITU-T X.680 (02/2021), chapter 6 the decoder shall not fail when an extension addition is not present:

When decoding an extensible type, a decoder may detect:
a) the absence of expected extension additions in a sequence or set type; or
b) the presence of arbitrary unexpected extension additions above those defined (if any) in a sequence or set type, or of an unknown alternative in a choice type, or an unknown enumeration in an enumerated type, or of an unexpected length or value of a type whose constraint is extensible.

In formal terms, an abstract syntax defined by the extensible type X contains not only the values of type X, but also the values of all types that are extension-related to X. Thus, the decoding process never signals an error when either of the above situations (a or b) is detected. The action that is taken in each situation is determined by the ASN.1 specifier.

NOTE – Frequently the action will be to ignore the presence of unexpected additional extensions, and to use a default value or a "missing" indicator for expected extension additions that are absent.

Unexpected extension additions detected by a decoder in an extensible type can later be included in a subsequent encoding of that type (for transmission back to the sender, or to some third party), provided that the same transfer syntax is used on the subsequent transmission.

[jbeyerstedt]

I looked into ITU-T X.680 (02/2021) and it seems like an extension addition with just a single non-optional field is encoded the same as if it were inside an extension group. So

RequestorDescription ::= SEQUENCE {
  id                VehicleID,
  type              RequestorType OPTIONAL,
  ...,
  ocit OcitRequestorDescriptionContainer -- Extension for OCIT in V2.2.1
}

and

RequestorDescription ::= SEQUENCE {
  id                VehicleID,
  type              RequestorType OPTIONAL,
  ...,
[[
  ocit OcitRequestorDescriptionContainer -- Extension for OCIT in V2.2.1
]]
}

will give the same UPER output since the sequence created by the extension group contains no preamble bits as it doesn't have an extension, not optional fields. In the end, this is the same as making the field optional since a None value will be serialized as missing extension addition (by the current implementation) in both cases.

There are two issues though:

• We wouldn't comply to the optional requirement in ITU-T X.680 (02/2021), chapter 25.15 to serialize the extension addition when it's known to the encoder. But I don't think that this is a real-world issue since a receiver couldn't differentiate this from a sender which only knows the "old" specification.

"ComponentType"s that are extension additions but not contained within an "ExtensionAdditionGroup" should always be encoded if they are not marked OPTIONAL or DEFAULT, except when the abstract value is being relayed from a sender that is using an earlier version of the abstract syntax in which the "ComponentType" is not defined.

• BER (as well as CER/ DER) encodings "ignore" extension markers and handle added fields as if they were in the message. So by making it "OPTIONAL"/ Option<> we may omit the field even though it should technically be there.

This only leads to non-compliant messages when the singular extension addition is followed up by more extensions which kinda make this extension mandatory when the subsequent extensions are filled in a serialize message. But I would be fine to pass that responsibility to the user and detailed protocol definition.