Skip to content

Add support for aws-lc-rs #576

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
faysou wants to merge 1 commit into
base: main
Choose a base branch
from
Draft

Add support for aws-lc-rs #576

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 13 additions & 6 deletions Cargo.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -46,6 +46,7 @@ url = "2.2"
walkdir = { version = "2", optional = true }

# Cloud storage support
aws-lc-rs = { version = "1.15", default-features = false, features = ["non-fips", "ring-io"], optional = true }
base64 = { version = "0.22", default-features = false, features = ["std"], optional = true }
form_urlencoded = { version = "1.2", optional = true }
http-body-util = { version = "0.1.2", optional = true }
Expand All @@ -54,7 +55,7 @@ hyper = { version = "1.2", default-features = false, optional = true }
md-5 = { version = "0.10.6", default-features = false, optional = true }
quick-xml = { version = "0.38.0", features = ["serialize", "overlapped-lists"], optional = true }
rand = { version = "0.9", default-features = false, features = ["std", "std_rng", "thread_rng"], optional = true }
reqwest = { version = "0.12", default-features = false, features = ["rustls-tls-native-roots", "http2"], optional = true }
reqwest = { version = "0.12", default-features = false, features = ["http2", "rustls-tls-webpki-roots-no-provider"], optional = true }
Copy link
Contributor

@tustvold tustvold Dec 13, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a breaking change that will likely break most clients

Copy link
Author

@faysou faysou Dec 13, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It doesn't, it allows users to choose their own implementation of rustls instead of forcing them to use native tls which is harder to compile as it requires openssl.

Copy link
Contributor

@tustvold tustvold Dec 13, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

native-roots doesn't depend on openssl, it concerns where it sources the trust store.

ring = { version = "0.17", default-features = false, features = ["std"], optional = true }
rustls-pki-types = { version = "1.9", default-features = false, features = ["std"], optional = true }
serde = { version = "1.0", default-features = false, features = ["derive"], optional = true }
Expand All @@ -71,14 +72,20 @@ wasm-bindgen-futures = "0.4.18"

[features]
default = ["fs"]
cloud = ["serde", "serde_json", "quick-xml", "hyper", "reqwest", "reqwest/stream", "chrono/serde", "base64", "rand", "ring", "http-body-util", "form_urlencoded", "serde_urlencoded"]
azure = ["cloud", "httparse"]
fs = ["walkdir"]
gcp = ["cloud", "rustls-pki-types"]
aws = ["cloud", "md-5"]
http = ["cloud"]
cloud = ["serde", "serde_json", "quick-xml", "hyper", "reqwest", "reqwest/stream", "chrono/serde", "base64", "rand", "http-body-util", "form_urlencoded", "serde_urlencoded"]
azure = ["cloud", "httparse", "ring"]
gcp = ["cloud", "rustls-pki-types", "ring"]
aws = ["cloud", "md-5", "ring"]
http = ["cloud", "ring"]
azure-aws-lc = ["cloud", "httparse", "aws-lc"]
gcp-aws-lc = ["cloud", "rustls-pki-types", "aws-lc"]
aws-aws-lc = ["cloud", "md-5", "aws-lc"]
http-aws-lc = ["cloud", "aws-lc"]
Comment on lines +76 to +84
Copy link
Contributor

@tustvold tustvold Dec 13, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not a fan of this explosion of feature flags, it will make testing really complicated

Copy link
Author

@faysou faysou Dec 13, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The aws-lc flags are not necessary, it just avoids to redefine dependencies for users who want to use aws-lc. Another possibility is to use ring in the default feature.

Copy link
Author

@faysou faysou Dec 13, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

From my past experience using native tls required openssl, then I'm not an expert on this. But using rustls no provider allows more control on which rustls crypto provider is used.

Copy link
Contributor

@tustvold tustvold Dec 13, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

https://crates.io/crates/rustls-native-certs/0.8.2/dependencies

tls-webpki-roots = ["reqwest?/rustls-tls-webpki-roots"]
integration = ["rand"]
ring = ["dep:ring"]
aws-lc = ["dep:aws-lc-rs"]

[dev-dependencies] # In alphabetical order
hyper = { version = "1.2", features = ["server"] }
Expand Down
5 changes: 3 additions & 2 deletions src/aws/client.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,7 @@ use crate::client::s3::{
InitiateMultipartUploadResult, ListResponse, PartMetadata,
};
use crate::client::{GetOptionsExt, HttpClient, HttpError, HttpResponse};
use crate::crypto;
use crate::list::{PaginatedListOptions, PaginatedListResult};
use crate::multipart::PartId;
use crate::{
Expand All @@ -43,6 +44,8 @@ use async_trait::async_trait;
use base64::Engine;
use base64::prelude::BASE64_STANDARD;
use bytes::{Buf, Bytes};
use crypto::digest;
use crypto::digest::Context;
use http::header::{
CACHE_CONTROL, CONTENT_DISPOSITION, CONTENT_ENCODING, CONTENT_LANGUAGE, CONTENT_LENGTH,
CONTENT_TYPE,
Expand All @@ -52,8 +55,6 @@ use itertools::Itertools;
use md5::{Digest, Md5};
use percent_encoding::{PercentEncode, utf8_percent_encode};
use quick_xml::events::{self as xml_events};
use ring::digest;
use ring::digest::Context;
use serde::{Deserialize, Serialize};
use std::sync::Arc;

Expand Down
41 changes: 33 additions & 8 deletions src/client/builder.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -63,7 +63,12 @@ impl HttpRequestBuilder {
}
}

#[cfg(any(feature = "aws", feature = "azure"))]
#[cfg(any(
feature = "aws",
feature = "azure",
feature = "aws-aws-lc",
feature = "azure-aws-lc"
))]
pub(crate) fn from_parts(client: HttpClient, request: HttpRequest) -> Self {
Self {
client,
Expand Down Expand Up @@ -116,7 +121,7 @@ impl HttpRequestBuilder {
self
}

#[cfg(feature = "aws")]
#[cfg(any(feature = "aws", feature = "aws-aws-lc"))]
pub(crate) fn headers(mut self, headers: http::HeaderMap) -> Self {
use http::header::{Entry, OccupiedEntry};

Expand Down Expand Up @@ -151,7 +156,7 @@ impl HttpRequestBuilder {
self
}

#[cfg(feature = "gcp")]
#[cfg(any(feature = "gcp", feature = "gcp-aws-lc"))]
pub(crate) fn bearer_auth(mut self, token: &str) -> Self {
let value = HeaderValue::try_from(format!("Bearer {token}"));
match (value, &mut self.request) {
Expand All @@ -165,7 +170,7 @@ impl HttpRequestBuilder {
self
}

#[cfg(feature = "gcp")]
#[cfg(any(feature = "gcp", feature = "gcp-aws-lc"))]
pub(crate) fn json<S: serde::Serialize>(mut self, s: S) -> Self {
match (serde_json::to_vec(&s), &mut self.request) {
(Ok(json), Ok(request)) => {
Expand All @@ -177,7 +182,15 @@ impl HttpRequestBuilder {
self
}

#[cfg(any(test, feature = "aws", feature = "gcp", feature = "azure"))]
#[cfg(any(
test,
feature = "aws",
feature = "gcp",
feature = "azure",
feature = "aws-aws-lc",
feature = "gcp-aws-lc",
feature = "azure-aws-lc"
))]
pub(crate) fn query<T: serde::Serialize + ?Sized>(mut self, query: &T) -> Self {
let mut error = None;
if let Ok(ref mut req) = self.request {
Expand Down Expand Up @@ -205,7 +218,12 @@ impl HttpRequestBuilder {
self
}

#[cfg(any(feature = "gcp", feature = "azure"))]
#[cfg(any(
feature = "gcp",
feature = "azure",
feature = "gcp-aws-lc",
feature = "azure-aws-lc"
))]
pub(crate) fn form<T: serde::Serialize>(mut self, form: T) -> Self {
let mut error = None;
if let Ok(ref mut req) = self.request {
Expand All @@ -226,7 +244,14 @@ impl HttpRequestBuilder {
self
}

#[cfg(any(feature = "aws", feature = "gcp", feature = "azure"))]
#[cfg(any(
feature = "aws",
feature = "gcp",
feature = "azure",
feature = "aws-aws-lc",
feature = "gcp-aws-lc",
feature = "azure-aws-lc"
))]
pub(crate) fn body(mut self, b: impl Into<HttpRequestBody>) -> Self {
if let Ok(r) = &mut self.request {
*r.body_mut() = b.into();
Expand All @@ -239,7 +264,7 @@ impl HttpRequestBuilder {
}
}

#[cfg(any(test, feature = "azure"))]
#[cfg(any(test, feature = "azure", feature = "azure-aws-lc"))]
pub(crate) fn add_query_pairs<I, K, V>(uri: &mut Uri, query_pairs: I)
where
I: IntoIterator,
Expand Down
18 changes: 16 additions & 2 deletions src/client/header.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -71,7 +71,14 @@ pub(crate) enum Error {
}

/// Extracts a PutResult from the provided [`HeaderMap`]
#[cfg(any(feature = "aws", feature = "gcp", feature = "azure"))]
#[cfg(any(
feature = "aws",
feature = "gcp",
feature = "azure",
feature = "aws-aws-lc",
feature = "gcp-aws-lc",
feature = "azure-aws-lc"
))]
pub(crate) fn get_put_result(
headers: &HeaderMap,
version: &str,
Expand All @@ -82,7 +89,14 @@ pub(crate) fn get_put_result(
}

/// Extracts a optional version from the provided [`HeaderMap`]
#[cfg(any(feature = "aws", feature = "gcp", feature = "azure"))]
#[cfg(any(
feature = "aws",
feature = "gcp",
feature = "azure",
feature = "aws-aws-lc",
feature = "gcp-aws-lc",
feature = "azure-aws-lc"
))]
pub(crate) fn get_version(headers: &HeaderMap, version: &str) -> Result<Option<String>, Error> {
Ok(match headers.get(version) {
Some(x) => Some(
Expand Down
9 changes: 8 additions & 1 deletion src/client/http/body.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -196,7 +196,14 @@ impl HttpResponseBody {
String::from_utf8(b.into()).map_err(|e| HttpError::new(HttpErrorKind::Decode, e))
}

#[cfg(any(feature = "aws", feature = "gcp", feature = "azure"))]
#[cfg(any(
feature = "aws",
feature = "gcp",
feature = "azure",
feature = "aws-aws-lc",
feature = "gcp-aws-lc",
feature = "azure-aws-lc"
))]
pub(crate) async fn json<B: serde::de::DeserializeOwned>(self) -> Result<B, HttpError> {
let b = self.bytes().await?;
serde_json::from_slice(&b).map_err(|e| HttpError::new(HttpErrorKind::Decode, e))
Expand Down
77 changes: 68 additions & 9 deletions src/client/mod.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -30,26 +30,59 @@ pub(crate) mod mock_server;

pub(crate) mod retry;

#[cfg(any(feature = "aws", feature = "gcp", feature = "azure"))]
#[cfg(any(
feature = "aws",
feature = "gcp",
feature = "azure",
feature = "aws-aws-lc",
feature = "gcp-aws-lc",
feature = "azure-aws-lc"
))]
pub(crate) mod pagination;

pub(crate) mod get;

#[cfg(any(feature = "aws", feature = "gcp", feature = "azure"))]
#[cfg(any(
feature = "aws",
feature = "gcp",
feature = "azure",
feature = "aws-aws-lc",
feature = "gcp-aws-lc",
feature = "azure-aws-lc"
))]
pub(crate) mod list;

#[cfg(any(feature = "aws", feature = "gcp", feature = "azure"))]
#[cfg(any(
feature = "aws",
feature = "gcp",
feature = "azure",
feature = "aws-aws-lc",
feature = "gcp-aws-lc",
feature = "azure-aws-lc"
))]
pub(crate) mod token;

pub(crate) mod header;

#[cfg(any(feature = "aws", feature = "gcp"))]
#[cfg(any(
feature = "aws",
feature = "gcp",
feature = "aws-aws-lc",
feature = "gcp-aws-lc"
))]
pub(crate) mod s3;

pub(crate) mod builder;
mod http;

#[cfg(any(feature = "aws", feature = "gcp", feature = "azure"))]
#[cfg(any(
feature = "aws",
feature = "gcp",
feature = "azure",
feature = "aws-aws-lc",
feature = "gcp-aws-lc",
feature = "azure-aws-lc"
))]
pub(crate) mod parts;
pub use http::*;

Expand Down Expand Up @@ -723,7 +756,14 @@ impl ClientOptions {
/// In particular:
/// * Allows HTTP as metadata endpoints do not use TLS
/// * Configures a low connection timeout to provide quick feedback if not present
#[cfg(any(feature = "aws", feature = "gcp", feature = "azure"))]
#[cfg(any(
feature = "aws",
feature = "gcp",
feature = "azure",
feature = "aws-aws-lc",
feature = "gcp-aws-lc",
feature = "azure-aws-lc"
))]
pub(crate) fn metadata_options(&self) -> Self {
self.clone()
.with_allow_http(true)
Expand Down Expand Up @@ -926,7 +966,14 @@ where
}
}

#[cfg(any(feature = "aws", feature = "azure", feature = "gcp"))]
#[cfg(any(
feature = "aws",
feature = "azure",
feature = "gcp",
feature = "aws-aws-lc",
feature = "azure-aws-lc",
feature = "gcp-aws-lc"
))]
mod cloud {
use super::*;
use crate::RetryConfig;
Expand All @@ -952,7 +999,12 @@ mod cloud {
}

/// Override the minimum remaining TTL for a cached token to be used
#[cfg(any(feature = "aws", feature = "gcp"))]
#[cfg(any(
feature = "aws",
feature = "gcp",
feature = "aws-aws-lc",
feature = "gcp-aws-lc"
))]
pub(crate) fn with_min_ttl(mut self, min_ttl: Duration) -> Self {
self.cache = self.cache.with_min_ttl(min_ttl);
self
Expand Down Expand Up @@ -983,7 +1035,14 @@ mod cloud {
}

use crate::client::builder::HttpRequestBuilder;
#[cfg(any(feature = "aws", feature = "azure", feature = "gcp"))]
#[cfg(any(
feature = "aws",
feature = "azure",
feature = "gcp",
feature = "aws-aws-lc",
feature = "azure-aws-lc",
feature = "gcp-aws-lc"
))]
pub(crate) use cloud::*;

#[cfg(test)]
Expand Down
2 changes: 1 addition & 1 deletion src/client/retry.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -277,7 +277,7 @@ impl RetryableRequestBuilder {
}

/// Set whether this request should be retried on a 409 Conflict response.
#[cfg(feature = "aws")]
#[cfg(any(feature = "aws", feature = "aws-aws-lc"))]
pub(crate) fn retry_on_conflict(mut self, retry_on_conflict: bool) -> Self {
self.request.retry_on_conflict = retry_on_conflict;
self
Expand Down
2 changes: 1 addition & 1 deletion src/client/s3.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -92,7 +92,7 @@ pub(crate) struct InitiateMultipartUploadResult {
pub upload_id: String,
}

#[cfg(feature = "aws")]
#[cfg(any(feature = "aws", feature = "aws-aws-lc"))]
#[derive(Debug, Deserialize)]
#[serde(rename_all = "PascalCase")]
pub(crate) struct CopyPartResult {
Expand Down
7 changes: 6 additions & 1 deletion src/client/token.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -52,7 +52,12 @@ impl<T> Default for TokenCache<T> {

impl<T: Clone + Send> TokenCache<T> {
/// Override the minimum remaining TTL for a cached token to be used
#[cfg(any(feature = "aws", feature = "gcp"))]
#[cfg(any(
feature = "aws",
feature = "gcp",
feature = "aws-aws-lc",
feature = "gcp-aws-lc"
))]
pub(crate) fn with_min_ttl(self, min_ttl: Duration) -> Self {
Self { min_ttl, ..self }
}
Expand Down
Loading