[wip] esdk rust spec changes

README.md

@@ -42,6 +42,11 @@ run `./ci/prettify.sh write`.
 If you want to check them without writing,
 run `./ci/prettify.sh check`.
 
+## Conventions used in these specifications
+
+The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL"
+in these specifications are to be interpreted as described in [RFC2119](https://tools.ietf.org/html/rfc2119) unless otherwise noted in a particular specification.
+
 ## Generate Duvet Reports
 
 [Duvet](https://github.com/awslabs/aws-encryption-sdk-specification/issues/240) is a tool that can be used to ensure specification is documented alongside code.

client-apis/client.md

@@ -3,20 +3,6 @@
 
 # Client
 
-## Version
-
-0.1.0
-
-## Implementations
-
-| Language   | Confirmed Compatible with Spec Version | Minimum Version Confirmed | Implementation                                                                                                                        |
-| ---------- | -------------------------------------- | ------------------------- | ------------------------------------------------------------------------------------------------------------------------------------- |
-| C          | 0.1.0                                  | 0.1.0                     | [session.c](https://github.com/aws/aws-encryption-sdk-c/blob/master/source/session.c)                                                 |
-| NodeJS     | 0.1.0                                  | 0.1.0                     | [index.ts](https://github.com/aws/aws-encryption-sdk-javascript/blob/master/modules/client-node/src/index.ts)                         |
-| Browser JS | 0.1.0                                  | 0.1.0                     | [index.ts](https://github.com/aws/aws-encryption-sdk-javascript/blob/master/modules/client-browser/src/index.ts)                      |
-| Python     | 0.1.0                                  | 0.1.0                     | [\_\_init\_\_.py](https://github.com/aws/aws-encryption-sdk-python/blob/master/src/aws_encryption_sdk/__init__.py)                    |
-| Java       | 0.1.0                                  | 0.1.0                     | [AwsCrypto.java](https://github.com/aws/aws-encryption-sdk-java/blob/master/src/main/java/com/amazonaws/encryptionsdk/AwsCrypto.java) |
-
 ## Overview
 
 This document describes the client experience for the AWS Encryption SDK.
@@ -30,11 +16,10 @@ because anything a client encrypts can be decrypted by the same client.
 
 ## Initialization
 
-On client initialization,
-the caller MUST have the option to provide a:
-
-- [commitment policy](#commitment-policy)
-- [maximum number of encrypted data keys](#maximum-number-of-encrypted-data-keys)
+- On client initialization,
+  the caller MUST have the option to provide a [commitment policy](#commitment-policy).
+- On client initialization,
+  the caller MUST have the option to provide a [maximum number of encrypted data keys](#maximum-number-of-encrypted-data-keys).
 
 If no [commitment policy](#commitment-policy) is provided the default MUST be [REQUIRE_ENCRYPT_REQUIRE_DECRYPT](../framework/algorithm-suites.md#require_encrypt_require_decrypt).
 If no [maximum number of encrypted data keys](#maximum-number-of-encrypted-data-keys) is provided
@@ -50,7 +35,6 @@ The AWS Encryption SDK MUST use the ESDK [commitment policies](../framework/comm
 
 A AWS Encryption SDK message can contain multiple encrypted data keys.
 This is the maximum number of encrypted data keys that the client will attempt to unwrap.
-Callers MUST have a way to disable this limit.
 
 ## Operation
 

client-apis/encrypt.md

@@ -266,13 +266,31 @@ specified by the [algorithm suite](../framework/algorithm-suites.md), with the f
 - The cipherkey MUST be the derived data key
 - The plaintext MUST be an empty byte array
 
+The serialized bytes MUST NOT be released until the entire message header has been serialized.
+If this operation is streaming the encrypted message and
+the entire message header has been serialized,
+the serialized message header MUST be released.
+
+The encrypted message output by the Encrypt operation MUST have a message header equal
+to the message header calculated in this step.
+If the message headers are not equal, the Encrypt operation MUST fail.
+
+If the algorithm suite contains a signature algorithm and
+this operation is [streaming](streaming.md) the encrypted message output to the caller,
+this operation MUST input the serialized header to the signature algorithm as soon as it is serialized,
+such that the serialized header isn't required to remain in memory to [construct the signature](#construct-the-signature).
+
+#### V2 Authentication Tag
+
 With the authentication tag calculated,
 if the message format version associated with the [algorithm suite](../framework/algorithm-suites.md#supported-algorithm-suites) is 2.0,
 this operation MUST serialize the [message header authentication](../data-format/message-header.md#header-authentication-version-2-0) with the following specifics:
 
 - [Authentication Tag](../data-format/message-header.md#authentication-tag): MUST have the value
   of the authentication tag calculated above.
 
+#### V1 Authentication Tag
+
 With the authentication tag calculated,
 if the message format version associated with the [algorithm suite](../framework/algorithm-suites.md#supported-algorithm-suites) is 1.0
 this operation MUST serialize the [message header authentication](../data-format/message-header.md#header-authentication-version-1-0) with the following specifics:
@@ -282,20 +300,6 @@ this operation MUST serialize the [message header authentication](../data-format
 - [Authentication Tag](../data-format/message-header.md#authentication-tag): MUST have the value
   of the authentication tag calculated above.
 
-The serialized bytes MUST NOT be released until the entire message header has been serialized.
-If this operation is streaming the encrypted message and
-the entire message header has been serialized,
-the serialized message header MUST be released.
-
-The encrypted message output by the Encrypt operation MUST have a message header equal
-to the message header calculated in this step.
-If the message headers are not equal, the Encrypt operation MUST fail.
-
-If the algorithm suite contains a signature algorithm and
-this operation is [streaming](streaming.md) the encrypted message output to the caller,
-this operation MUST input the serialized header to the signature algorithm as soon as it is serialized,
-such that the serialized header isn't required to remain in memory to [construct the signature](#construct-the-signature).
-
 ## Construct the body
 
 The encrypted message output by the Encrypt operation MUST have a message body equal
@@ -345,8 +349,8 @@ with the following inputs:
     [Message Body AAD](../data-format/message-body-aad.md).
   - The [sequence number](../data-format/message-body-aad.md#sequence-number) MUST be the sequence
     number of the frame being encrypted.
-    If this is the first frame sequentially, this value MUST be 1.
-    Otherwise, this value MUST be 1 greater than the value of the sequence number
+    If this is the first frame sequentially, the sequence number value MUST be 1.
+    Otherwise, the sequence number value MUST be 1 greater than the value of the sequence number
     of the previous frame.
   - The [content length](../data-format/message-body-aad.md#content-length) MUST have a value
     equal to the length of the plaintext being encrypted.
@@ -365,9 +369,15 @@ with the following inputs:
 
 This operation MUST serialize a regular frame or final frame with the following specifics:
 
+- Final frame only: [Sequence Number End](../data-format/message-body.md#sequence-number-end) field: This MUST
+  be the sequence number end value.
+  The Sequence Number End MUST only be serialized for the final frame.
 - [Sequence Number](../data-format/message-body.md#sequence-number): MUST be the sequence number of this frame,
   as determined above.
 - [IV](../data-format/message-body.md#iv): MUST be the IV used when calculating the encrypted content above
+- Final frame only: [Encrypted Content Length](../data-format/message-body.md#encrypted-content-length) field: This MUST
+  be the encrypted content length value
+  The Encrypted Content Length MUST only be serialized for the final frame.
 - [Encrypted Content](../data-format/message-body.md#encrypted-content): MUST be the encrypted content calculated above.
 - [Authentication Tag](../data-format/message-body.md#authentication-tag): MUST be the authentication tag
   output when calculating the encrypted content above.

data-format/message-body.md

@@ -3,17 +3,6 @@
 
 # Message Body
 
-## Version
-
-See [Message Version](message.md#version).
-
-## Implementations
-
-- [C](https://github.com/awslabs/aws-encryption-sdk-c/blob/master/include/aws/cryptosdk/private/framefmt.h)
-- [JavaScript](https://github.com/awslabs/aws-encryption-sdk-javascript/blob/master/modules/serialize/src/types.ts)
-- [Python](https://github.com/aws/aws-encryption-sdk-python/blob/master/src/aws_encryption_sdk/structures.py)
-- [Java](https://github.com/aws/aws-encryption-sdk-java/blob/master/src/main/java/com/amazonaws/encryptionsdk/model/ContentType.java)
-
 ## Overview
 
 The message body is a component of the [message](message.md).  
@@ -27,11 +16,6 @@ The structure of the body depends on the content type:
 
 ## Definitions
 
-### Conventions used in this document
-
-The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL"
-in this document are to be interpreted as described in [RFC 2119](https://tools.ietf.org/html/rfc2119).
-
 ## Structure
 
 The following sections describe the format of the message body for each content type.
@@ -41,7 +25,7 @@ The following sections describe the format of the message body for each content
 Non-framed data is a sequence of encrypted bytes along with the [initialization vector (IV)](#iv)
 and body [authentication tag](#authentication-tag).
 
-The following describes the fields that form non-framed data.  
+The following table is a non-normative representation of the normative requirements in this secion.
 The bytes are appended in the order shown.
 
 | Field                                                 | Length (bytes)                           | Interpreted as |
@@ -51,25 +35,49 @@ The bytes are appended in the order shown.
 | [Encrypted Content](#encrypted-content)               | Variable                                 | Bytes          |
 | [Authentication Tag](#authentication-tag)             | Variable                                 | Bytes          |
 
-#### IV
+Non-framed data MUST be serialized (written) as, in order,
+IV,
+Encrypted Content Length,
+Encrypted Content,
+and Authentication Tag.
+
+Non-framed data MUST be deserialized (read) as, in order,
+IV,
+Encrypted Content Length,
+Encrypted Content,
+and Authentication Tag.
+
+#### Non-Framed Data IV
 
 The initialization vector to use with the encryption algorithm.
-The IV MUST be a unique IV within the message.
 
-#### Encrypted Content Length
+When writing a message, the IV MUST be a unique IV within the message.
+When writing a message, the operation MUST serialize the IV to be [IV Length](message-header.md#iv-length) bytes.
+
+When reading a message, the operation MUST deserialize [IV Length](message-header.md#iv-length) bytes and interpret it as the IV.
+When reading a message, the deserialized IV MUST be interpreted as bytes.
+
+#### Non-Framed Data Encrypted Content Length
 
 The length of the encrypted content.  
 The length MUST NOT be greater than `2^36 - 32`, or 64 gibibytes (64 GiB),
 due to restrictions imposed by the [implemented algorithms](../framework/algorithm-suites.md).
+When serializing the encrypted content length to a message, the length of the serialized encrypted content length MUST be 8 bytes.
+The encrypted content length MUST be serialized as a Uint64.
+When reading the encrypted content length from a message, the encrypted content length MUST be interpreted as a Uint64.
 
-#### Encrypted Content
+#### Non-Framed Data Encrypted Content
 
 The encrypted data as returned by the [encryption algorithm](../framework/algorithm-suites.md#encryption-algorithm).
+The length of the serialized encrypted content MUST be equal to the value of the [Encrypted Content Length](#encrypted-content-length) field.
+The encrypted content MUST be interpreted as bytes.
 
-#### Authentication Tag
+#### Non-Framed Data Authentication Tag
 
-The authentication value for the body.  
+The authentication value for the body.
 It is used to authenticate the message body.
+The length of the serialized authentication tag MUST be equal to the [authentication tag length](../framework/algorithm-suites.md#authentication-tag-length) of the [algorithm suite](../framework/algorithm-suites.md) specified by the [Algorithm Suite ID](message-header.md#algorithm-suite-id) field.
+The authentication tag MUST be interpreted as bytes.
 
 ### Framed Data
 
@@ -89,8 +97,7 @@ Note:
 
 All frames except the [Final Frame](#final-frame) are "Regular Frames".
 
-The following describes the fields that form the Regular Frame Body Structure.  
-The bytes are appended in the order shown.
+The following table is a non-normative representation of the normative requirements in this section.
 
 | Field                                     | Length (bytes)                                                                                               | Interpreted as |
 | ----------------------------------------- | ------------------------------------------------------------------------------------------------------------ | -------------- |
@@ -99,30 +106,42 @@ The bytes are appended in the order shown.
 | [Encrypted Content](#encrypted-content)   | Variable                                                                                                     | Bytes          |
 | [Authentication Tag](#authentication-tag) | Algorithm suite ID's [Authentication Tag Length](../framework/algorithm-suites.md#authentication-tag-length) | Bytes          |
 
-##### Sequence Number
+A regular frame MUST be serialized as, in order,
+Sequence Number,
+IV,
+Encrypted Content,
+and Authentication Tag.
+
+##### Regular Frame Sequence Number
 
 The Frame sequence number.  
 It is an incremental counter number for the frames.  
 Framed Data MUST start at Sequence Number 1.  
 Subsequent frames MUST be in order and MUST contain an increment of 1 from the previous frame.
+When serializing the sequence number to a message, the length of the serialized sequence number MUST be 4 bytes.
+The sequence number MUST be serialized as a UInt32.
+When reading the sequence number from a message, the sequence number MUST be interpreted as a UInt32.
 
-##### IV
+##### Regular Frame IV
 
 The initialization vector (IV) for the frame.  
 Each frame in the [Framed Data](#framed-data) MUST include an IV that is unique within the message.
 The IV length MUST be equal to the IV length of the algorithm suite specified by the [Algorithm Suite ID](message-header.md#algorithm-suite-id) field.
+The IV MUST be interpreted as bytes.
 Note: This IV is different from the [Header IV](message-header.md#iv).
 
-##### Encrypted Content
+##### Regular Frame Encrypted Content
 
 The encrypted data for each frame, as returned by the [encryption algorithm](../framework/algorithm-suites.md#encryption-algorithm).  
 The length of the encrypted content of a Regular Frame MUST be equal to the Frame Length.
+The encrypted content MUST be interpreted as bytes.
 
-##### Authentication Tag
+##### Regular Frame Authentication Tag
 
 The authentication value for the frame.  
 The authentication tag length MUST be equal to the authentication tag length of the algorithm suite
 specified by the [Algorithm Suite ID](message-header.md#algorithm-suite-id) field.
+The authentication tag MUST be interpreted as bytes.
 
 #### Final Frame
 
@@ -148,8 +167,7 @@ For example, in the case that the length of the Plaintext is equal to the Frame
   The first frame is a regular frame,
   and the second frame is the final frame with a content length of 0.
 
-The following describes the fields that form the Final Frame Body Structure.  
-The bytes are appended in the order shown.
+The following table is a non-normative representation of the normative requirements in this section.
 
 | Field                                                   | Length (bytes)                                                                                               | Interpreted as |
 | ------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------ | -------------- |
@@ -160,38 +178,62 @@ The bytes are appended in the order shown.
 | [Encrypted Content](#encrypted-content)                 | Variable                                                                                                     | Bytes          |
 | [Authentication Tag](#authentication-tag)               | Algorithm suite ID's [Authentication Tag Length](../framework/algorithm-suites.md#authentication-tag-length) | Bytes          |
 
+A final frame MUST be serialized as, in order,
+Sequence Number End,
+Sequence Number,
+IV,
+Encrypted Content Length,
+Encrypted Content,
+and Authentication Tag.
+
+This means a final frame MUST be a regular frame with the addition of the serialized
+Sequence Number End
+and Encrypted Content Length.
+
 ##### Sequence Number End
 
 An indicator for the Final Frame.  
 The value MUST be encoded as the 4 bytes `FF FF FF FF` in hexadecimal notation.
+The length of the serialized sequence number end MUST be 4 bytes.
+The sequence number end MUST be interpreted as bytes.
 
-##### Sequence Number
+##### Final Frame Sequence Number
 
-The Frame Sequence Number.  
-It is an incremental counter number for the frames.
+The Frame sequence number for the final frame.
 The Final Frame Sequence number MUST be equal to the total number of frames in the Framed Data.
+The Final Frame Sequence Number MUST be serialized to a message the same way as the
+[Regular Frame Sequence Number](#regular-frame-sequence-number).
+The Final Frame Sequence Number MUST be interpreted from a message the same way as the
+[Regular Frame Sequence Number](#regular-frame-sequence-number).
 
-##### IV
+##### Final Frame IV
 
 The initialization vector for the final frame.  
 The IV MUST be a unique IV within the message.  
 The IV length MUST be equal to the IV length of the [algorithm suite](../framework/algorithm-suites.md) that generated the message.
+The IV MUST be interpreted as bytes.
 Note: This IV is different from the [Header IV](message-header.md#iv).
 
-##### Encrypted Content Length
+##### Final Frame Encrypted Content Length
 
 The length of the encrypted content.
+When serializing the encrypted content length to a message, the length of the serialized encrypted content length field MUST be 4 bytes.
+The encrypted content length MUST be serialized as a UInt32.
+When reading the encrypted content length from a message, the encrypted content length MUST be interpreted as a UInt32.
 
-##### Encrypted Content
+##### Final Frame Encrypted Content
 
 The encrypted data for the final frame, as returned by the [encryption algorithm](../framework/algorithm-suites.md#encryption-algorithm).
+The length of the serialized encrypted content MUST be equal to the value of the [Encrypted Content Length](#encrypted-content-length-1) field.
+The encrypted content MUST be interpreted as bytes.
 
-##### Authentication Tag
+##### Final Frame Authentication Tag
 
 The authentication value for the final frame.  
 It is used to authenticate the final frame.  
 The authentication tag length MUST be equal to the authentication tag length of the algorithm suite
 specified by the [Algorithm Suite ID](message-header.md#algorithm-suite-id) field.
+The authentication tag MUST be interpreted as bytes.
 
 ## Example Usage
 

data-format/message-footer.md

@@ -40,6 +40,8 @@ The bytes are appended in the order shown.
 ### Signature Length
 
 The length of the signature.
+This length of the signature length field MUST be 2 bytes.
+The signature length field MUST be interpreted as a UInt16.
 
 ### Signature