Skip to content

RUST-1529 Use AWS SDK for sigv4 signing #1438

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 49 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 48 commits
Commits
Show all changes
49 commits
Select commit Hold shift + click to select a range
bb11533
Wip - add path for authentication using MongoDB URI behind aws-sdk-au…
JamieTsai1024 Jul 23, 2025
3d78f80
Get auth credentials through env variables using AWS SDK
JamieTsai1024 Jul 23, 2025
1f4c590
test URI/environment variable credentials
isabelatkinson Jul 23, 2025
6fe85bc
Add conditional imports for aws_config modules behind aws-sdk-auth fe…
JamieTsai1024 Jul 23, 2025
b6e1d9a
Try to pin aws_sdk dependency versions to fix tests
JamieTsai1024 Jul 23, 2025
18c3bc8
Merge branch 'main' into RUST-1529-no-ff
JamieTsai1024 Jul 24, 2025
915b087
Remove aws-sdk-auth ff
JamieTsai1024 Jul 25, 2025
f5a65af
fix dependencies for MSRV
isabelatkinson Jul 25, 2025
e2a5080
Merge branch 'main' into RUST-1529-no-ff
JamieTsai1024 Jul 25, 2025
cf3f3a5
Fix lint on AwsCredential unused fields
JamieTsai1024 Jul 25, 2025
8df943f
Fix lint
JamieTsai1024 Jul 25, 2025
6b9d129
Uncomment tests
JamieTsai1024 Jul 25, 2025
e99fff2
Revert comments in config.yml
JamieTsai1024 Jul 25, 2025
73d3a82
Clean up comments
JamieTsai1024 Jul 25, 2025
dd94b48
Remove unused imports and replace expect
JamieTsai1024 Jul 26, 2025
e10a3ba
Make dependencies optional and comment back patchable: false
JamieTsai1024 Jul 26, 2025
f365624
Move get credentials logic to a public method
JamieTsai1024 Jul 26, 2025
408ca2f
Change return type of get_aws_credentials from AwsCredential to aws_c…
JamieTsai1024 Jul 26, 2025
f528c00
Fix lint and imports
JamieTsai1024 Jul 26, 2025
678cdcb
Move import behind aws-auth feature flag
JamieTsai1024 Jul 26, 2025
90d5156
Add logs
JamieTsai1024 Jul 27, 2025
eac9bcf
Add logs
JamieTsai1024 Jul 28, 2025
cb26899
Add default-https-client and rt-tokio to aws-config dependencies
JamieTsai1024 Jul 28, 2025
725d373
Comment out tests for credential caching
JamieTsai1024 Jul 28, 2025
b4c06e8
Merge branch 'main' into RUST-1529-no-ff
JamieTsai1024 Jul 29, 2025
46e171d
Update Cargo.lock
JamieTsai1024 Jul 29, 2025
0cd8dfa
Fix cargo.lock
JamieTsai1024 Jul 29, 2025
a482936
revert unneeded changes in cargo.lock
isabelatkinson Jul 29, 2025
2416644
Uncomment and add ignore tags, remove dbg logs
JamieTsai1024 Jul 30, 2025
ee4f602
Merge branch 'RUST-1529-no-ff' of https://github.com/JamieTsai1024/mo…
JamieTsai1024 Jul 30, 2025
6890532
Remove comment
JamieTsai1024 Jul 30, 2025
f2a4a62
Exclude aws-sdk-sts from Cargo.toml, add it manually during MSRV check
JamieTsai1024 Jul 30, 2025
d41a5b7
Wip - compute authorization header
JamieTsai1024 Jul 30, 2025
cd470fb
Merge branch 'RUST-1529-no-ff' into RUST-1529-aws-sdk-signature
JamieTsai1024 Jul 30, 2025
f841841
Uncomment original compute_authorization_header after adding #[allow(…
JamieTsai1024 Jul 30, 2025
c64a610
Move aws sdk credential getting logic after original commented impl
JamieTsai1024 Jul 30, 2025
06183ea
Merge branch 'RUST-1529-no-ff' into RUST-1529-aws-sdk-signature
JamieTsai1024 Jul 30, 2025
e5549b2
Change service to sts and add content-length header for request
JamieTsai1024 Jul 30, 2025
12b9c33
Remove from_sdk_creds and dereferences
JamieTsai1024 Jul 30, 2025
9a7d14c
Clean up sigv4 function
JamieTsai1024 Jul 30, 2025
c1b3f82
Clean up unwrap, refactor
JamieTsai1024 Jul 30, 2025
55fb305
Merge branch 'main' into RUST-1529-aws-sdk-signature
JamieTsai1024 Aug 1, 2025
2d71fc4
Fix merge - update Document::encode_to_vec to to_writer
JamieTsai1024 Aug 1, 2025
2bffc01
Fix error propagation, remove string conversions
JamieTsai1024 Aug 7, 2025
9c96dde
Update headers error propagation
JamieTsai1024 Aug 8, 2025
070b5b5
Merge branch 'main' into RUST-1529-aws-sdk-signature
JamieTsai1024 Aug 8, 2025
dde6a5c
Upload cargo.lock
JamieTsai1024 Aug 8, 2025
92d0cd3
Update formatting
JamieTsai1024 Aug 8, 2025
7018a69
Remove async
JamieTsai1024 Aug 8, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .evergreen/config.yml
Original file line number Diff line number Diff line change
Expand Up @@ -223,7 +223,7 @@ buildvariants:

- name: aws-auth
display_name: "AWS Authentication"
patchable: false
# patchable: false
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

will uncomment once PR is approved

run_on:
- ubuntu2004-small
expansions:
Expand Down
50 changes: 26 additions & 24 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

13 changes: 12 additions & 1 deletion Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,7 @@ dns-resolver = ["dep:hickory-resolver", "dep:hickory-proto"]
cert-key-password = ["dep:pem", "dep:pkcs8"]

# Enable support for MONGODB-AWS authentication.
aws-auth = ["dep:reqwest", "dep:aws-config", "dep:aws-types", "dep:aws-credential-types"]
aws-auth = ["dep:reqwest", "dep:aws-config", "dep:aws-types", "dep:aws-credential-types", "dep:aws-sigv4", "dep:http"]

# Enable support for on-demand Azure KMS credentials.
azure-kms = ["dep:reqwest"]
Expand Down Expand Up @@ -138,6 +138,17 @@ version = "1.2.4"
optional = true
default-features = false

[dependencies.aws-sigv4]
version = "1.3.3"
optional = true
default-features = false
features = ["sign-http"]

[dependencies.http]
version = "1.3"
optional = true
default-features = false

[dependencies.bson2]
git = "https://github.com/mongodb/bson-rust"
branch = "2.15.x"
Expand Down
154 changes: 142 additions & 12 deletions src/client/auth/aws.rs
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,15 @@ use aws_config::BehaviorVersion;
#[cfg(feature = "aws-auth")]
use aws_credential_types::{provider::ProvideCredentials, Credentials};

#[cfg(feature = "aws-auth")]
use aws_sigv4::{
http_request::{sign, SignableBody, SignableRequest, SigningSettings},
sign::v4::SigningParams,
};

#[cfg(feature = "aws-auth")]
use http::Request;

const AWS_ECS_IP: &str = "169.254.170.2";
const AWS_EC2_IP: &str = "169.254.169.254";
const AWS_LONG_DATE_FMT: &str = "%Y%m%dT%H%M%SZ";
Expand Down Expand Up @@ -117,24 +126,32 @@ async fn authenticate_stream_inner(
let creds = get_aws_credentials(credential).await.map_err(|e| {
Error::authentication_error(MECH_NAME, &format!("failed to get creds: {e}"))
})?;
let aws_credential = AwsCredential::from_sdk_creds(creds);
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In a previous PR, we introduced from_sdk_creds(...) to convert from aws_credential_types::Credentials to locally defined AwsCredential type. Now that we've converted the rest of the authentication to use the AWS SDK, we no longer need this conversion


let date = Utc::now();

let authorization_header = aws_credential.compute_authorization_header(
// Generate authorization header using original implementation without AWS SDK
// let authorization_header = aws_credential.compute_authorization_header(
// date,
// &server_first.sts_host,
// &server_first.server_nonce,
// )?;

// let mut client_second_payload = doc! {
// "a": authorization_header,
// "d": date.format(AWS_LONG_DATE_FMT).to_string(),
// };

// if let Some(security_token) = aws_credential.session_token {
// client_second_payload.insert("t", security_token);
// }

let client_second_payload = compute_aws_sigv4_payload(
creds,
date,
&server_first.sts_host,
&server_first.server_nonce,
)?;

let mut client_second_payload = doc! {
"a": authorization_header,
"d": date.format(AWS_LONG_DATE_FMT).to_string(),
};

if let Some(security_token) = aws_credential.session_token {
client_second_payload.insert("t", security_token);
}
)
.await?;

let mut client_second_payload_bytes = vec![];
client_second_payload.to_writer(&mut client_second_payload_bytes)?;
Expand Down Expand Up @@ -197,6 +214,119 @@ pub(crate) async fn get_aws_credentials(credential: &Credential) -> Result<Crede
}
}

pub async fn compute_aws_sigv4_payload(
creds: Credentials,
date: DateTime<Utc>,
host: &str,
server_nonce: &[u8],
) -> Result<Document> {
let region = if host == "sts.amazonaws.com" {
"us-east-1"
} else {
let parts: Vec<_> = host.split('.').collect();
parts.get(1).copied().unwrap_or("us-east-1")
};

let url = format!("https://{host}");
let date_str = date.format("%Y%m%dT%H%M%SZ").to_string();
let body_str = "Action=GetCallerIdentity&Version=2011-06-15";
let body_bytes = body_str.as_bytes();
let nonce_b64 = base64::encode(server_nonce);

// Create the HTTP request
let mut builder = Request::builder()
.method("POST")
.uri(&url)
.header("host", host)
.header("content-type", "application/x-www-form-urlencoded")
.header("content-length", body_bytes.len())
.header("x-amz-date", &date_str)
.header("x-mongodb-gs2-cb-flag", "n")
.header("x-mongodb-server-nonce", &nonce_b64);

if let Some(token) = creds.session_token() {
builder = builder.header("x-amz-security-token", token);
}

let mut request = builder.body(body_str.to_string()).map_err(|e| {
Error::authentication_error(MECH_NAME, &format!("Failed to build request: {e}"))
})?;

let service = "sts";
let identity = creds.into();

// Set up signing parameters
let signing_settings = SigningSettings::default();
let signing_params = SigningParams::builder()
.identity(&identity)
.region(region)
.name(service)
.time(date.into())
.settings(signing_settings)
.build()
.map_err(|e| {
Error::authentication_error(MECH_NAME, &format!("Failed to build signing params: {e}"))
})?
.into();
let headers: Result<Vec<_>> = request
.headers()
.iter()
.map(|(k, v)| {
let v = v.to_str().map_err(|_| {
Error::authentication_error(
MECH_NAME,
"Failed to convert header value to valid UTF-8",
)
})?;
Ok((k.as_str(), v))
})
.collect();

let signable_request = SignableRequest::new(
request.method().as_str(),
request.uri().to_string(),
headers?.into_iter(),
SignableBody::Bytes(request.body().as_bytes()),
)
.map_err(|e| {
Error::authentication_error(MECH_NAME, &format!("Failed to create SignableRequest: {e}"))
})?;

let (signing_instructions, _signature) = sign(signable_request, &signing_params)
.map_err(|e| Error::authentication_error(MECH_NAME, &format!("Signing failed: {e}")))?
.into_parts();
Comment on lines +294 to +296
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: since we're not using the returned signature, I think this can call output rather than into_parts

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks Isabel! I tried switching to .output(), but ran into a borrow checker error when calling apply_to_request_http1x on line 291. I double-checked the method signature and saw that it needs an owned value which we get with .into_parts(), whereas .output() returns a reference.

The .into_parts() also matches what I saw in the two references we looked at (1, 2). Let me know if you had a different approach in mind!

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ah I missed the reference vs. owned value distinction, feel free to leave as-is

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sounds good, thanks!

signing_instructions.apply_to_request_http1x(&mut request);

let headers = request.headers();
let authorization_header = headers
.get("authorization")
.ok_or_else(|| Error::authentication_error(MECH_NAME, "Missing authorization header"))?
.to_str()
.map_err(|e| {
Error::authentication_error(MECH_NAME, &format!("Invalid header value: {e}"))
})?;

let token_header = headers
.get("x-amz-security-token")
.map(|v| {
v.to_str().map_err(|e| {
Error::authentication_error(MECH_NAME, &format!("Invalid token header: {e}"))
})
})
.transpose()?;

let mut payload = doc! {
"a": authorization_header,
"d": date_str,
};

if let Some(token) = token_header {
payload.insert("t", token);
}

Ok(payload)
}

/// Contains the credentials for MONGODB-AWS authentication.
// RUST-1529 note: dead_code tag added to avoid unused warnings on expiration field
#[allow(dead_code)]
Expand Down