Merge remote-tracking branch 'origin/main' into release

Author: ianbotsf

builder.json

@@ -21,6 +21,11 @@
     "test_steps": [
         "{gradlew} test allTests"
     ],
+    "upstream": [
+        {
+            "name": "aws-crt-kotlin"
+        }
+    ],
     "downstream": [
         { "name": "aws-sdk-kotlin" }    
     ]

docs/design/event-streams.md

@@ -89,15 +89,19 @@ structure ThrottlingError {}
 
 ### Event Stream Type Representation
 
-The members of an operation input or output that target a stream will be represented with an asynchronous [Flow](https://kotlinlang.org/docs/reference/coroutines/flow.html) 
-from the `kotlinx-coroutines-core` library. `Flow` is a natural fit for representing asynchronous streams. 
+The members of an operation input or output that target a stream will be represented with an asynchronous 
+[Flow](https://kotlinlang.org/docs/reference/coroutines/flow.html) from the `kotlinx-coroutines-core` library. 
+`Flow` is a natural fit for representing asynchronous streams. 
 
-`Flow` was chosen for pagination and already in use as part of our public API contract. Any alternative to this would require a custom but similar type that doesn't play well with
-the rest of the coroutine ecosystem. There is also prior art for representing streaming requests and responses, see [gRPC Kotlin](https://github.com/grpc/grpc-kotlin).
+`Flow` was chosen for pagination and already in use as part of our public API contract. Any alternative to this 
+would require a custom but similar type that doesn't play well with the rest of the coroutine ecosystem. There is
+also prior art for representing streaming requests and responses, see [gRPC Kotlin](https://github.com/grpc/grpc-kotlin).
 
 The following types and service would be generated. 
 
-NOTE: only the input and output types are shown, the other structures or unions in the model would be generated as described in [Kotlin Smithy Design](kotlin-smithy-sdk.md).
+NOTE: only the input and output types are shown, the other structures or unions in the model would be generated as
+described in [Kotlin Smithy Design](kotlin-smithy-sdk.md) with the exception of event stream members targeting errors
+which is described below in more detail.
 
 #### Input Event Streams
 
@@ -120,7 +124,8 @@ class PublishMessagesRequest private constructor(builder: Builder){
 
 #### Output Event Streams
 
-Output event streams would be modeled the same way as input streams. The response object would have a `Flow<T>` field that represents the response stream.
+Output event streams would be modeled the same way as input streams. The response object would have a `Flow<T>` 
+field that represents the response stream.
 
 ```kt
 class SubscribeToMovementsResponse private constructor(builder: Builder){
@@ -137,20 +142,81 @@ class SubscribeToMovementsResponse private constructor(builder: Builder){
 ```
 
 
-Modeling the event stream as a field of the request or response allows for [initial messages](https://awslabs.github.io/smithy/1.0/spec/core/stream-traits.html#initial-messages) 
-to be implemented. If we directly returned or took a `Flow<T>` as the input or output type we would not be able to represent the initial request or response fields when present.
+Modeling the event stream as a field of the request or response allows for 
+[initial messages](https://awslabs.github.io/smithy/1.0/spec/core/stream-traits.html#initial-messages) to be 
+implemented. If we directly returned or took a `Flow<T>` as the input or output type we would not be able to 
+represent the initial request or response fields when present.
 
 
+#### Event Stream Error Representation
+
+Event stream unions may model exceptions that can appear on the stream. These exceptions are terminal messages that 
+are intended to be surfaced to the client using idiomatic error handling mechanisms of the target language. Thus,
+the modeled errors a consumer may see on the stream are part of the overall union that makes up the possible events.
+
+NOTE: the set of errors on the operation MAY not be the same set of errors modeled on the event stream.
+
+
+Using the example from above: 
+
+```
+@streaming
+union MovementEvents {
+    up: Movement,
+    down: Movement,
+    left: Movement,
+    right: Movement,
+    throttlingError: ThrottlingError
+}
+
+```
+
+The default representation of a union (as documented in [Kotlin Smithy Design](kotlin-smithy-sdk.md)) is generated as:
+
+```kotlin
+sealed class MovementEvents {
+    data class Up(val value: Movement): MovementEvents()
+    data class Down(val value: Movement): MovementEvents()
+    data class Left(val value: Movement): MovementEvents()
+    data class Right(val value: Movement): MovementEvents()
+    data class ThrottlingError(val value: ThrottlingError): MovementEvents()
+    object SdkUnknown : MovementEvents()
+}
+```
+
+This is undesirable though since event stream errors are terminal and end the stream. Keeping them in the set of 
+possible events also means it may be easier for consumers to ignore errors depending on what events they are looking 
+for (e.g. by having a catch all `else` branch they may inadvertently ignore an error and think the stream completed 
+successfully).
+
+
+Event stream unions will be special-cased to filter out variants targeting error shapes. When these errors are 
+emitted by the service on the stream they will be converted to the appropriate modeled exception and thrown rather 
+than being emitted on the stream the consumer sees.
+
+As an example, the generated event stream union will look like this (note the absence of `ThrottlingError`):
+
+```kotlin
+sealed class MovementEvents {
+    data class Up(val value: Movement): MovementEvents()
+    data class Down(val value: Movement): MovementEvents()
+    data class Left(val value: Movement): MovementEvents()
+    data class Right(val value: Movement): MovementEvents()
+    object SdkUnknown : MovementEvents()
+}
+```
+
 ### **Service and Usage**
 
 NOTE: There are types and internal details here not important to the design of how customers will interact with 
-streaming requests/responses (e.g. serialization/deserialization). 
-Those details are subject to change and not part of this design document. The focus here is on the way 
-streaming is exposed to a customer.
+streaming requests/responses (e.g. serialization/deserialization). Those details are subject to change and not part of
+this design document. The focus here is on the way streaming is exposed to a customer.
 
 
-The signatures generated match that of binary streaming requests and responses. Notably that output streams take a lambda instead of returning the response directly (see [binary-streaming design](binary-streaming.md) which discusses this pattern).
-The response (and event stream) are only valid in that scope, after which the resources consumed by the stream are closed and no longer valid.
+The signatures generated match that of binary streaming requests and responses. Notably that output streams take a 
+lambda instead of returning the response directly (see [binary-streaming design](binary-streaming.md) which 
+discusses this pattern). The response (and event stream) are only valid in that scope, after which the resources 
+consumed by the stream are closed and no longer valid.
 
 
 ```kt
@@ -206,7 +272,6 @@ fun main() = runBlocking{
                 is MovementEvents.Down,
                 is MovementEvents.Left,
                 is MovementEvents.Right -> handleMovement(event)
-                is MovementEvents.ThrottlingError -> throw event.throttlingError
                 else -> error("unknown event type: $event")
             }
         }
@@ -218,8 +283,9 @@ fun main() = runBlocking{
 private fun handleMovement(event: MovementEvents) { ... }
 ```
 
-Accepting a lambda matches what is generated for binary streams (see [binary-streaming design](binary-streaming.md)) and will provide a consistent API experience as well
-as the same benefits to the SDK (properly scoped lifetime for resources). 
+Accepting a lambda matches what is generated for binary streams (see [binary-streaming design](binary-streaming.md)) 
+and will provide a consistent API experience as well as the same benefits to the SDK (properly scoped lifetime 
+for resources). 
 
 
 # Appendix
@@ -228,9 +294,10 @@ as the same benefits to the SDK (properly scoped lifetime for resources).
 ## Java Interop
 
 `Flow<T>` is not easily consumable directly from Java due to the `suspend` nature of it. JetBrains provides 
-[reactive adapters](https://github.com/Kotlin/kotlinx.coroutines/tree/master/reactive) that can be used to convert rxJava and JDK-9 
-reactive streams to or from an equivalent `Flow`. Users would be responsible for creating a shim layer using these primitives provided
-by JetBrains which would allow them to expose the Kotlin functions however they see fit to their applications. 
+[reactive adapters](https://github.com/Kotlin/kotlinx.coroutines/tree/master/reactive) that can be used to convert
+rxJava and JDK-9 reactive streams to or from an equivalent `Flow`. Users would be responsible for creating a shim 
+layer using these primitives provided by JetBrains which would allow them to expose the Kotlin functions however 
+they see fit to their applications. 
 
 
 ## Additional References
@@ -243,4 +310,5 @@ by JetBrains which would allow them to expose the Kotlin functions however they
 
 # Revision history
 
+* 02/17/2022 - Remove errors from generated event stream
 * 01/19/2022 - Created

docs/design/paginators.md

@@ -1,7 +1,7 @@
 # Pagination Design
 
 * **Type**: Design
-* **Author(s)**: Aaron Todd
+* **Author(s)**: Aaron Todd, Ken Gilmer
 
 # Abstract
 
@@ -88,13 +88,16 @@ public API for the response of a paginated operation. Runtime library code is
 not required for `Flow`-based paginators, as all functionality will be provided
 by the Kotlin standard lib and the `kotlinx-coroutines-core` library.
 
-Pagination by default will always generate an extension function that returns
-a `Flow` over the normal operation output type.
+Each operation that supports pagination would receive an extension function off the 
+service client interface that returns a `Flow` over the normal operation output type.
+The generated function name uses the operation name plus the suffix `Paginated`. 
+This way the paginated and non-paginated operations should show up next to eachother 
+in the IDE to promote discoverability.
 
 An example of this is demonstrated below (codegen):
 
 ```kotlin
-fun LambdaClient.paginateListFunctions(initialRequest: ListFunctionsRequest): Flow<ListFunctionsResponse> =
+fun LambdaClient.listFunctionsPaginated(initialRequest: ListFunctionsRequest): Flow<ListFunctionsResponse> =
     flow {
         var cursor: String? = null
         var isFirstPage: Boolean = true
@@ -124,6 +127,13 @@ works from the base response flow. The targeted type will provide iteration to
 the `item` element for the user automatically. The name of the generated
 function comes from the name of the member.
 
+Additionally, the `item` specified in the model may be deeply nested and involve
+operations and types with many words. Generating a single function that combines
+the operation, some word to indicate pagination, plus the nested item's name
+often produces unreadable function signatures. Instead the member name targeted
+by the `items` will be used. Thas has the advantage of matching the output structure
+type property name.
+
 Here is an example that follows from the previous example (codegen):
 
 ```kotlin
@@ -135,18 +145,6 @@ fun Flow<ListFunctionsResponse>.functions(): Flow<FunctionConfiguration> =
     }
 ```
 
-### Creating a Paginator
-
-Each operation that supports pagination would receive an extension function (
-based from the service client interface) to create a paginator. The function
-generated always begins with the suffix `Paginated`, helping with
-the `discoverablity`
-design consideration, by allowing users to quickly view all paginatable
-operations by specifying this literal suffix in the IDE. Once customers are
-familiar with these two prefixes, discovering what pagination options are
-available for a given client or paginated response is simple and consistent.
-
-Using the model in the introduction would produce the following:
 
 ### Examples
 
@@ -157,20 +155,13 @@ An example of driving a paginator and processing response instances:
 ```kotlin
 suspend fun rawPaginationExample(client: LambdaClient) {
     lambdaClient
-        .paginateListFunctions(ListFunctionsRequest {})
+        .listFunctionsPaginated(ListFunctionsRequest {})
         .collect { response ->
             response.functions?.forEach { functionConfiguration ->
                 println(functionConfiguration.functionName)
             }
         }
 }
-
-fun Flow<ListFunctionsResponse>.items(): Flow<FunctionConfiguration> =
-   transform() { response ->
-      response.functions?.forEach {
-         emit(it)
-      }
-   }
 ```
 
 #### Usage - Iterating over Modeled Item
@@ -180,12 +171,6 @@ transform we are able to iterate over the nested element. As transforms can be
 applied to any member of a type, users may extend the abstraction as needed
 simply by providing their own flow transforms.
 
-Additionally, the `item` specified in the model may be deeply nested and involve
-operations and types with many words. Generating a single function that combines
-the operation, some word to indicate pagination, plus the nested item's name
-often produces unreadable function signatures. In this approach, we generate
-more legible API by breaking the extraction of the nested items into a separate
-function, which always uses the name of the member referenced as the item.
 
 ```kotlin
 lambdaClient
@@ -381,6 +366,7 @@ future.
 
 # Revision history
 
+* 03/10/2022 - Fix function names
 * 12/15/2021 - Adapted to favor Flow
 * 8/23/2021 - Initial upload
 * 2/05/2021 - Created

gradle.properties

@@ -39,3 +39,6 @@ kotlinLoggingVersion=2.0.3
 
 # logging - JVM
 slf4jVersion=1.7.30
+
+# crt
+crtKotlinVersion=0.5.4-SNAPSHOT

runtime/build.gradle.kts

@@ -43,7 +43,6 @@ subprojects {
     // don't configure anything
     if (!needsConfigure) return@subprojects
 
-    // TODO - the group to publish under needs negotiated still
     group = "aws.smithy.kotlin"
     version = sdkVersion
 

runtime/crt-util/build.gradle.kts

@@ -0,0 +1,47 @@
+/*
+ * Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
+ * SPDX-License-Identifier: Apache-2.0.
+ */
+
+buildscript {
+    val atomicFuVersion: String by project
+    repositories {
+        mavenCentral()
+    }
+
+    dependencies {
+        classpath("org.jetbrains.kotlinx:atomicfu-gradle-plugin:$atomicFuVersion")
+    }
+}
+
+apply(plugin = "kotlinx-atomicfu")
+
+description = "Utilities for working with AWS CRT Kotlin"
+extra["displayName"] = "Smithy :: Kotlin :: CRT :: Util"
+extra["moduleName"] = "aws.smithy.kotlin.runtime.crt"
+
+val atomicFuVersion: String by project
+val coroutinesVersion: String by project
+val crtKotlinVersion: String by project
+
+kotlin {
+    sourceSets {
+        commonMain {
+            dependencies {
+                api(project(":runtime:runtime-core"))
+                api("aws.sdk.kotlin.crt:aws-crt-kotlin:$crtKotlinVersion")
+                api(project(":runtime:protocol:http"))
+                implementation("org.jetbrains.kotlinx:atomicfu:$atomicFuVersion")
+            }
+        }
+        commonTest {
+            dependencies {
+                implementation("org.jetbrains.kotlinx:kotlinx-coroutines-test:$coroutinesVersion")
+            }
+        }
+
+        all {
+            languageSettings.optIn("aws.smithy.kotlin.runtime.util.InternalApi")
+        }
+    }
+}