Merge feature/http 1.x to main (#4484)

## Motivation and Context
<!--- Why is this change required? What problem does it solve? -->
<!--- If it fixes an open issue, please link to the issue here -->
All Smithy-rs crates, for both servers and clients, now use the `1.x`
version of the `http` crate for all internal processing. Utility methods
are still provided for users to convert between SDK types and types from
both the `0.x` and `1.x` versions of the `http` crate.

## Description
<!--- Describe your changes in detail -->
There are too many changes to describe in detail, but this work has been
split over many other PRs to the `feature/http-1.x` branch. You can see
all of those PRs
[here](https://github.com/smithy-lang/smithy-rs/pulls?q=is%3Apr%20is%3Aclosed%20base%3Afeature%2Fhttp-1.x).


## Testing
<!--- Please describe in detail how you tested your changes -->
<!--- Include details of your testing environment, and the tests you ran
to -->
<!--- see how your change affects other areas of the code, etc. -->
All CI tests pass with the exception of the semver failures (see below).
These failures are all related to the `aws-smithy-compression` crate
which is pre-1.0 and had a semver incompatible version bump, so that
shouldn't be an issue.

```
checking aws-smithy-compression...failed!
--- failure feature_missing: package feature removed or renamed ---

Description:
A feature has been removed from this package's Cargo.toml. This will break downstream crates which enable that feature.
        ref: https://doc.rust-lang.org/cargo/reference/semver.html#cargo-feature-remove
       impl: https://github.com/obi1kenobi/cargo-semver-checks/tree/v0.41.0/src/lints/feature_missing.ron

Failed in:
  feature http-body-1-x in the package's Cargo.toml
  feature http-body-0-4-x in the package's Cargo.toml

--- failure inherent_method_missing: pub method removed or renamed ---

Description:
A publicly-visible method or associated fn is no longer available under its prior name. It may have been renamed or removed entirely.
        ref: https://doc.rust-lang.org/cargo/reference/semver.html#item-remove
       impl: https://github.com/obi1kenobi/cargo-semver-checks/tree/v0.41.0/src/lints/inherent_method_missing.ron

Failed in:
  CompressionAlgorithm::into_impl_http_body_0_4_x, previously in file /home/build/workspace/smithy-rs/target/semver-checks/git-base/8c895dfa8dcf82eaffbf2ce44708ff8b2985b58f/aws-sdk/sdk/aws-smithy-compression/src/lib.rs:168

--- failure module_missing: pub module removed or renamed ---

Description:
A publicly-visible module cannot be imported by its prior path. A `pub use` may have been removed, or the module may have been renamed, removed, or made non-public.
        ref: https://doc.rust-lang.org/cargo/reference/semver.html#item-remove
       impl: https://github.com/obi1kenobi/cargo-semver-checks/tree/v0.41.0/src/lints/module_missing.ron

Failed in:
  mod aws_smithy_compression::body::compress::http_body_0_4_x, previously in file /home/build/workspace/smithy-rs/target/semver-checks/git-base/8c895dfa8dcf82eaffbf2ce44708ff8b2985b58f/aws-sdk/sdk/aws-smithy-compression/src/body.rs:41
  mod aws_smithy_compression::http::http_body_1_x, previously in file /home/build/workspace/smithy-rs/target/semver-checks/git-base/8c895dfa8dcf82eaffbf2ce44708ff8b2985b58f/aws-sdk/sdk/aws-smithy-compression/src/http.rs:49
  mod aws_smithy_compression::http::http_body_0_4_x, previously in file /home/build/workspace/smithy-rs/target/semver-checks/git-base/8c895dfa8dcf82eaffbf2ce44708ff8b2985b58f/aws-sdk/sdk/aws-smithy-compression/src/http.rs:10

--- failure trait_missing: pub trait removed or renamed ---

Description:
A publicly-visible trait cannot be imported by its prior path. A `pub use` may have been removed, or the trait itself may have been renamed or removed entirely.
        ref: https://doc.rust-lang.org/cargo/reference/semver.html#item-remove
       impl: https://github.com/obi1kenobi/cargo-semver-checks/tree/v0.41.0/src/lints/trait_missing.ron

Failed in:
  trait aws_smithy_compression::http::http_body_1_x::CompressRequest, previously in file /home/build/workspace/smithy-rs/target/semver-checks/git-base/8c895dfa8dcf82eaffbf2ce44708ff8b2985b58f/aws-sdk/sdk/aws-smithy-compression/src/http.rs:54
  trait aws_smithy_compression::http::http_body_1_x::CloneCompressRequest, previously in file /home/build/workspace/smithy-rs/target/semver-checks/git-base/8c895dfa8dcf82eaffbf2ce44708ff8b2985b58f/aws-sdk/sdk/aws-smithy-compression/src/http.rs:65
  trait aws_smithy_compression::http::http_body_0_4_x::CloneCompressRequest, previously in file /home/build/workspace/smithy-rs/target/semver-checks/git-base/8c895dfa8dcf82eaffbf2ce44708ff8b2985b58f/aws-sdk/sdk/aws-smithy-compression/src/http.rs:26
  trait aws_smithy_compression::http::http_body_0_4_x::CompressRequest, previously in file /home/build/workspace/smithy-rs/target/semver-checks/git-base/8c895dfa8dcf82eaffbf2ce44708ff8b2985b58f/aws-sdk/sdk/aws-smithy-compression/src/http.rs:15
```


## Checklist
<!--- If a checkbox below is not applicable, then please DELETE it
rather than leaving it unchecked -->
- [x] For changes to the smithy-rs codegen or runtime crates, I have
created a changelog entry Markdown file in the `.changelog` directory,
specifying "client," "server," or both in the `applies_to` key.
- [x] For changes to the AWS SDK, generated SDK code, or SDK runtime
crates, I have created a changelog entry Markdown file in the
`.changelog` directory, specifying "aws-sdk-rust" in the `applies_to`
key.

----

_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._

---------

Co-authored-by: ysaito1001 <awsaito@amazon.com>
Co-authored-by: Russell Cohen <rcoh@amazon.com>
Co-authored-by: AWS SDK Rust Bot <aws-sdk-rust-primary@amazon.com>
Co-authored-by: AWS SDK Rust Bot <97246200+aws-sdk-rust-ci@users.noreply.github.com>
Co-authored-by: vcjana <vcjana@amazon.com>
Co-authored-by: Jason Gin <67525213+jasgin@users.noreply.github.com>
Co-authored-by: Aaron Todd <aajtodd@users.noreply.github.com>
Co-authored-by: greenwoodcm <greenwd@amazon.com>
Co-authored-by: Jason Gin <jasongin88@gmail.com>
Co-authored-by: Aaron J Todd <todaaron@amazon.com>
Co-authored-by: Anna H <annahay@amazon.com>
Co-authored-by: Ariel Ben-Yehuda <ariel.byd@gmail.com>
Co-authored-by: Ariel Ben-Yehuda <arielby@amazon.com>
Co-authored-by: Fahad Zubair <fahadzubair@gmail.com>
Co-authored-by: Fahad Zubair <fahadzub@amazon.com>

Author: 13 people

.changelog/http-1x.md

@@ -0,0 +1,18 @@
+---
+applies_to:
+  - aws-sdk-rust
+  - client
+  - server
+authors:
+  - landonxjames
+  - drganjoo
+references:
+  - smithy-rs#4484
+breaking: false
+new_feature: true
+bug_fix: false
+---
+
+All Smithy-rs crates, for both servers and clients, now use the 1.x version of
+the `http` crate for all internal processing. Utility methods are still provided
+for users to convert between SDK types and both of the `http` 0.x and 1.x types.

.github/workflows/pull-request-bot.yml

@@ -118,15 +118,15 @@ jobs:
         ./gradlew -Paws.services=+sts,+sso,+ssooidc,+signin,+transcribestreaming,+dynamodb :aws:sdk:assemble
 
         # Copy the Server runtime crate(s) in
-        cp -r rust-runtime/aws-smithy-http-server rust-runtime/aws-smithy-http-server-python rust-runtime/aws-smithy-http-server-typescript aws/sdk/build/aws-sdk/sdk
+        cp -r rust-runtime/aws-smithy-http-server rust-runtime/aws-smithy-http-server-python rust-runtime/aws-smithy-http-server-typescript rust-runtime/aws-smithy-legacy-http-server aws/sdk/build/aws-sdk/sdk
 
         pushd aws/sdk/build/aws-sdk
 
         # Remove example crates from workspace
         sed -i '/examples/d' Cargo.toml
 
         # Add server runtime crates to the workspace
-        sed -i 's/"sdk\/sts",/"sdk\/sts","sdk\/aws-smithy-http-server","sdk\/aws-smithy-http-server-python","sdk\/aws-smithy-http-server-typescript",/' Cargo.toml
+        sed -i 's/"sdk\/sts",/"sdk\/sts","sdk\/aws-smithy-http-server","sdk\/aws-smithy-http-server-python","sdk\/aws-smithy-http-server-typescript","sdk\/aws-smithy-legacy-http-server",/' Cargo.toml
 
         RUSTDOCFLAGS="--cfg docsrs" cargo +${{ env.rust_nightly_version }} doc --all-features --no-deps
 

aws/SDK_CHANGELOG.next.json

@@ -109,4 +109,4 @@
     }
   ],
   "aws-sdk-model": []
-}
+}

aws/codegen-aws-sdk/src/main/kotlin/software/amazon/smithy/rustsdk/AwsChunkedContentEncodingDecorator.kt

@@ -28,7 +28,8 @@ class AwsChunkedContentEncodingDecorator : ClientCodegenDecorator {
     // This decorator must decorate after any of the following:
     // - HttpRequestChecksumDecorator
     // - HttpRequestCompressionDecorator
-    override val order: Byte = (minOf(HttpRequestChecksumDecorator.ORDER, HttpRequestCompressionDecorator.ORDER) - 1).toByte()
+    override val order: Byte =
+        (minOf(HttpRequestChecksumDecorator.ORDER, HttpRequestCompressionDecorator.ORDER) - 1).toByte()
 
     override fun operationCustomizations(
         codegenContext: ClientCodegenContext,
@@ -81,8 +82,8 @@ private fun RuntimeConfig.awsChunked() =
         InlineAwsDependency.forRustFile(
             "aws_chunked", visibility = Visibility.PUBCRATE,
             CargoDependency.Bytes,
-            CargoDependency.Http,
-            CargoDependency.HttpBody,
+            CargoDependency.Http1x,
+            CargoDependency.HttpBody1x,
             CargoDependency.Tracing,
             AwsCargoDependency.awsRuntime(this).withFeature("http-02x"),
             CargoDependency.smithyRuntimeApiClient(this),

aws/codegen-aws-sdk/src/main/kotlin/software/amazon/smithy/rustsdk/AwsPresigningDecorator.kt

@@ -170,8 +170,8 @@ class AwsPresigningDecorator internal constructor(
             rustCrate.mergeFeature(
                 Feature(
                     "http-1x",
-                    default = false,
-                    listOf("dep:http-body-1x", "aws-smithy-runtime-api/http-1x"),
+                    default = true,
+                    listOf("aws-smithy-runtime-api/http-1x"),
                 ),
             )
         }
@@ -204,6 +204,14 @@ class AwsPresignedFluentBuilderMethod(
             "SdkError" to RuntimeType.sdkError(runtimeConfig),
         )
 
+    // Presigning requires both http version features since it pub exposes into_http_02x_request
+    // and into_http_1x_request functions
+    private val smithyRuntimeApi =
+        CargoDependency.smithyRuntimeApiClient(codegenContext.runtimeConfig)
+            .withFeature("http-02x")
+            .withFeature("http-1x")
+            .toType()
+
     override fun section(section: FluentClientSection): Writable =
         writable {
             if (section is FluentClientSection.FluentBuilderImpl && section.operationShape.hasTrait(PresignableTrait::class.java)) {
@@ -219,7 +227,7 @@ class AwsPresignedFluentBuilderMethod(
                     *codegenScope,
                     "OpError" to section.operationErrorType,
                     "RawResponseType" to
-                        RuntimeType.smithyRuntimeApiClient(runtimeConfig)
+                        smithyRuntimeApi
                             .resolve("client::orchestrator::HttpResponse"),
                 ) {
                     renderPresignedMethodBody(section)
@@ -305,13 +313,13 @@ class AwsPresignedFluentBuilderMethod(
             "OperationError" to section.operationErrorType,
             "RuntimePlugins" to RuntimeType.runtimePlugins(runtimeConfig),
             "SharedInterceptor" to
-                RuntimeType.smithyRuntimeApiClient(runtimeConfig).resolve("client::interceptors")
+                smithyRuntimeApi.resolve("client::interceptors")
                     .resolve("SharedInterceptor"),
             "SigV4PresigningRuntimePlugin" to
                 AwsRuntimeType.presigningInterceptor(runtimeConfig)
                     .resolve("SigV4PresigningRuntimePlugin"),
             "StopPoint" to RuntimeType.smithyRuntime(runtimeConfig).resolve("client::orchestrator::StopPoint"),
-            "USER_AGENT" to CargoDependency.Http.toType().resolve("header::USER_AGENT"),
+            "USER_AGENT" to CargoDependency.Http1x.toType().resolve("header::USER_AGENT"),
             "alternate_presigning_serializer" to
                 writable {
                     if (presignableOp.hasModelTransforms()) {
@@ -334,7 +342,7 @@ class AwsPresignedFluentBuilderMethod(
                             "Layer" to smithyTypes.resolve("config_bag::Layer"),
                             "RuntimePlugin" to RuntimeType.runtimePlugin(codegenContext.runtimeConfig),
                             "SharedRequestSerializer" to
-                                RuntimeType.smithyRuntimeApiClient(codegenContext.runtimeConfig)
+                                smithyRuntimeApi
                                     .resolve("client::ser_de::SharedRequestSerializer"),
                         )
                     }

aws/codegen-aws-sdk/src/main/kotlin/software/amazon/smithy/rustsdk/AwsRuntimeType.kt

@@ -36,6 +36,8 @@ object AwsRuntimeType {
         RuntimeType.forInlineDependency(
             InlineAwsDependency.forRustFile(
                 "presigning", visibility = Visibility.PUBLIC,
+                // Requires http_02x because of existing pub *_http_02x_request methods
+                CargoDependency.Http0x,
                 CargoDependency.Http1x,
                 CargoDependency.HttpBody1x,
             ),

aws/codegen-aws-sdk/src/main/kotlin/software/amazon/smithy/rustsdk/HttpRequestChecksumDecorator.kt

@@ -41,10 +41,11 @@ internal fun RuntimeConfig.awsInlineableHttpRequestChecksum() =
         InlineAwsDependency.forRustFile(
             "http_request_checksum", visibility = Visibility.PUBCRATE,
             CargoDependency.Bytes,
-            CargoDependency.Http,
-            CargoDependency.HttpBody,
+            CargoDependency.Http1x,
+            CargoDependency.HttpBody1x,
+            CargoDependency.HttpBodyUtil01x.toDevDependency(),
             CargoDependency.Tracing,
-            AwsCargoDependency.awsRuntime(this).withFeature("http-02x"),
+            AwsCargoDependency.awsRuntime(this).withFeature("http-1x"),
             CargoDependency.smithyChecksums(this),
             CargoDependency.smithyHttp(this),
             CargoDependency.smithyRuntimeApiClient(this),

aws/codegen-aws-sdk/src/main/kotlin/software/amazon/smithy/rustsdk/HttpResponseChecksumDecorator.kt

@@ -40,8 +40,6 @@ private fun RuntimeConfig.awsInlineableHttpResponseChecksum() =
         InlineAwsDependency.forRustFile(
             "http_response_checksum", visibility = Visibility.PUBCRATE,
             CargoDependency.Bytes,
-            CargoDependency.Http,
-            CargoDependency.HttpBody,
             CargoDependency.Tracing,
             CargoDependency.smithyChecksums(this),
             CargoDependency.smithyHttp(this),

aws/codegen-aws-sdk/src/main/kotlin/software/amazon/smithy/rustsdk/IntegrationTestDependencies.kt

@@ -21,7 +21,7 @@ import software.amazon.smithy.rust.codegen.core.rustlang.CargoDependency.Compani
 import software.amazon.smithy.rust.codegen.core.rustlang.CargoDependency.Companion.Hound
 import software.amazon.smithy.rust.codegen.core.rustlang.CargoDependency.Companion.Http1x
 import software.amazon.smithy.rust.codegen.core.rustlang.CargoDependency.Companion.HttpBody1x
-import software.amazon.smithy.rust.codegen.core.rustlang.CargoDependency.Companion.HttpBodyUtil
+import software.amazon.smithy.rust.codegen.core.rustlang.CargoDependency.Companion.HttpBodyUtil01x
 import software.amazon.smithy.rust.codegen.core.rustlang.CargoDependency.Companion.SerdeJson
 import software.amazon.smithy.rust.codegen.core.rustlang.CargoDependency.Companion.Smol
 import software.amazon.smithy.rust.codegen.core.rustlang.CargoDependency.Companion.TempFile
@@ -180,7 +180,7 @@ class S3TestDependencies(private val runtimeConfig: RuntimeConfig) : LibRsCustom
             addDependency(FuturesUtil.toDevDependency())
             addDependency(HdrHistogram)
             addDependency(HttpBody1x.toDevDependency().copy(optional = false))
-            addDependency(HttpBodyUtil.toDevDependency().copy(optional = false))
+            addDependency(HttpBodyUtil01x.toDevDependency().copy(optional = false))
             addDependency(Smol)
             addDependency(TempFile)
             addDependency(TracingAppender)

aws/codegen-aws-sdk/src/test/kotlin/software/amazon/smithy/rustsdk/HttpChecksumTest.kt

@@ -172,7 +172,7 @@ internal class HttpChecksumTest {
                 Feature(
                     "http-1x",
                     default = false,
-                    listOf("dep:http-body-1x", "aws-smithy-runtime-api/http-1x"),
+                    listOf("aws-smithy-runtime-api/http-1x"),
                 ),
             )
 
@@ -188,7 +188,8 @@ internal class HttpChecksumTest {
                         use #{pretty_assertions}::assert_eq;
                         use #{SdkBody};
                         use std::io::Write;
-                        use http_body::Body;
+                        use http_body_1x::Body;
+                        use http_body_util::BodyExt;
                         use #{HttpRequest};
                         use #{UaAssert};
                         use #{UaExtract};
@@ -364,12 +365,16 @@ internal class HttpChecksumTest {
                     );
                     assert_eq!(headers.get("content-encoding").unwrap(), "aws-chunked");
 
-                    let mut body = request.body().try_clone().expect("body is retryable");
+                    let body = request
+                    .body()
+                    .try_clone()
+                    .expect("body is retryable")
+                    .collect()
+                    .await
+                    .expect("body is collectable");
 
                     let mut body_data = bytes::BytesMut::new();
-                    while let Some(data) = body.data().await {
-                        body_data.extend_from_slice(&data.unwrap())
-                    }
+                    body_data.extend_from_slice(&body.to_bytes());
 
                     let body_string = std::str::from_utf8(&body_data).unwrap();
                     assert!(body_string.contains("x-amz-checksum-$algoLower:${testDef.trailerChecksum}"));