Override service region through environment variables

[skeggse]

Some workloads operate in a very restricted network context, and need to communicate with AWS APIs. Sometimes, that includes cross-region access. Consider the following scenario:

• workload running in us-east-1, and uses AssumeRoleWithWebIdentity
• workload needs to write an S3 object to a bucket in eu-central-1, and has network access to a PrivateLink endpoint in eu-central-1 over a VPC peering connection
• workload does not have access to the global STS endpoint nor the eu-central-1 STS endpoint

We can configure one aspect of this behavior to work as desired, namely with AWS_STS_REGIONAL_ENDPOINTS=regional environment variable. This makes us use the regional STS endpoint. However, the default credential provider chain for SDKs still inherits the region from the configured client, meaning that we try to communicate with STS in eu-central-1 if we configure the S3 client to talk to that region.

The Service-specific endpoints feature looked like it would solve this problem: I could force supported SDKs to use only the local endpoint by configuring it through environment variables! This even works for the AssumeRoleWithWebIdentity call itself: since that request isn't signed, it doesn't matter that the STS client is configured to use a different region than the one we're sending the actual assume role request to. However, for STS requests that are signed, and where we're not explicitly passing in an endpoint URL, the signatures end up not matching:

An error occurred (SignatureDoesNotMatch) when calling the GetCallerIdentity operation: Credential should be scoped to a valid region.

So, a request (not sure if feature or bug): provide a way to explicitly override the region used for a specific service, or at least for STS since that's the backbone of practically all other AWS requests. Alternately, if there's an easy way to do this today, I'd love to hear about it (and maybe get it explicitly documented)!

References:

• Override service endpoint URL through configuration files/environment variables aws-sdk#229
• STSClient does not use environment variable for region controls #5105

[aBurmeseDev]

Hi @skeggse - apologies for the wait here.

In your scenario, where you need to use the STS service in a different region than the S3 service, you can achieve this by creating separate clients for each service and configuring the appropriate endpoint or region for each client.

Here's an example:

const { S3Client, PutObjectCommand } = require('@aws-sdk/client-s3');
const { STSClient, AssumeRoleWithWebIdentityCommand } = require('@aws-sdk/client-sts');

// Configure the S3 client with the desired region
const s3Client = new S3Client({ region: 'eu-central-1' });

// Configure the STS client with a custom endpoint URL or region
const stsClient = new STSClient({
  region: 'us-east-1', // Or any other region where you have access to the STS endpoint
  endpoint: 'https://sts.us-east-1.amazonaws.com' // Explicitly set the STS endpoint URL
});

// Use the stsClient to assume the role
const assumeRoleCommand = new AssumeRoleWithWebIdentityCommand({
  // Provide the required parameters for AssumeRoleWithWebIdentity
});

stsClient.send(assumeRoleCommand)
  .then((assumeRoleResponse) => {
    // Use the assumed role credentials with the S3 client
    const s3Credentials = assumeRoleResponse.Credentials;
    const s3Client = new S3Client({
      region: 'eu-central-1',
      credentials: {
        accessKeyId: s3Credentials.AccessKeyId,
        secretAccessKey: s3Credentials.SecretAccessKey,
        sessionToken: s3Credentials.SessionToken,
      },
    });

    // Now you can use the s3Client to perform operations in the eu-central-1 region
    const putObjectCommand = new PutObjectCommand({
      Bucket: 'your-bucket-name',
      Key: 'object-key',
      Body: 'Hello, world!',
    });

    return s3Client.send(putObjectCommand);
  })
  .then((putObjectResponse) => {
    console.log('Object uploaded successfully:', putObjectResponse);
  })
  .catch((error) => {
    console.error('Error:', error);
  });

Hope it helps and let us know if there's any follow-up questions!
Best,
John

[skeggse]

Aha, thanks for pointing out the unclear component in my question!

The strategy you've outlined works fine if:

• you only need to make a single request once
• you have the engineering capacity to modify all the applications that need to communicate in this way

Neither of these are true for us.

• We need to use a provider chain rather than one-off assume-role commands, because our applications run for longer than an hour.
• We need to configure this via config or environment variables, because otherwise it's cost-prohibitive to implement/resolve.

This is a blocker to certain VPC security controls for multi-region applications.

[skeggse]

@aBurmeseDev how come we moved this to aws-sdk-js-v3? This request isn't specific to the JS SDK, and the JS SDK is pretty low on my list of SDKs where I'd want this functionality.

[skeggse]

@aBurmeseDev can we move this back to aws/aws-sdk?

edit: well, I guess it's archived now for some reason? Maybe we can move this to boto3 then.

[aBurmeseDev]

@skeggse apologies for the long silence here.

As you may have noticed, aws/aws-sdk repo has been archived which is why I moved the issue back here.

Based on your earlier comments, I understand that your request is not specific to the JavaScript SDK but rather relates to boto3 or maybe cross-SDK? However, I'm still unclear about the specifics of your request. Could you please provide more details about your workflow or the SDK(s) you're working with to help us better understand your ask?

[github-actions]

This issue has not received a response in 1 week. If you still think there is a problem, please leave a comment to avoid the issue from automatically closing.