feat(core/eventStreams): add event stream mixin for HttpProtocols, schema-serde

[kuhe]

Issue #, if available:
#1600

Description of changes:

PR moves event stream handling of schema-serde to a separate submodule called @smithy/core/eventStreams. It is asynchronously loaded by HttpProtocol extender classes when the functionality is needed. This should help with chunking in bundlers.

Additionally, the initial-request and initial-response message types for RPC protocols are handled by this implementation, whereas they are not handled by the existing model-ignorant codegen approach.

The implementation reuses the @smithy/eventstream-serde-universal stream and codec functionality, replacing only the part that was code-generated in protocol files, i.e. calls to the client.config.eventStreamMarshaller.

Old code example:

const de_CallAnalyticsTranscriptResultStream = (
  output: any,
  context: __SerdeContext & __EventStreamSerdeContext
): AsyncIterable<CallAnalyticsTranscriptResultStream> => {
  return context.eventStreamMarshaller.deserialize(output, async (event) => {
    if (event["UtteranceEvent"] != null) {
      return {
        UtteranceEvent: await de_UtteranceEvent_event(event["UtteranceEvent"], context),
      };
    }
    // etc.

New code example:

(In the PR diff)
context: within a schema-serde Protocol implementation

    const ns: NormalizedSchema = ...
    const dataObject: any = {};

    const eventStreamMember = ns.getEventStreamMember();
    if (eventStreamMember) {
      dataObject[eventStreamMember] = await this.deserializeEventStream({
        response,
        responseSchema: ns,
        initialResponseContainer: dataObject,
      });
    }
    dataObject.$metadata = this.deserializeMetadata(response);
    return dataObject;

[kuhe]

What is an event stream?

This information is available in the Smithy spec: https://smithy.io/2.0/spec/streaming.html.

An event stream is an input or output struct having a member which is a union that has the @streaming trait.
This contrasts with a "data stream", which is a streaming blob (like S3.GetObject.Body).

As seen in the PR diff, an event stream is an (http2 or other) message that is made up of headers and a body.

There are always at least a few default headers like :event-type, :content-type, :message-type. Additional headers can be set by the @eventHeader trait. There is also an @eventPayload trait. If this trait is present on a shape member, all other members of that shape must have the @eventHeader trait.

Here is a small illustration of the relationship among the 3 critical components: the Smithy model, the user input, and the message serialization.

Model

Smithy pseudo-code:

struct MyOperationInput {
  eventStream: EventStream
}

union EventStream {
  MemberA: { A: string }
  MemberB: { B: blob }
  MemberWithPayload: { 
    @eventPayload
    C: blob
  }
  MemberWithHeaders: { 
    @eventHeader
    H: string
  }
}

User Input

client.myOperation({
  eventStream: {
    async *[Symbol.asyncIterator]() {
      yield { A: "hi" };
      yield { B: new Uint8Array([0,1,2,3]) };
      yield { C: new Uint8Array([0,1,2,3]) };
      yield { H: "headerValue" }
    }
  }
});

Serialization

(actual serialization is in bytes, and depends on the wire format)

This example uses JSON as the format.
Here the blobs appear as base64 due to the standard serialization behavior of JSON protocols in AWS.

Note the distinction between the event type B, an implicit payload which serializes the entire object and uses the document content type of the protocol, and the event type C, an explicit payload which serializes only the payload member of the object and uses a special content type.

----
:content-type: "application/json"
:event-type: "A"
:message-type: "event"

{ "A": "hi" }
----
:content-type: "application/json"
:event-type: "B"
:message-type: "event"

{ "B": "AAECAw==" }
----
:content-type: "application/octet-stream"
:event-type: "C"
:message-type: "event"

AAECAw==
----
:content-type: "application/json"
:event-type: "H"
:message-type: "event"
H: "headerValue"