Behavior of `@aws-sdk/credential-providers` `fromTemporaryCredentials` has changed and results in `InvalidClientTokenId`

[jandppw]

Checkboxes for prior research

• I've gone through Developer Guide and API reference
• I've checked AWS Forums and StackOverflow.
• I've searched for previous similar issues and didn't find any solution.

Describe the bug

When upgrading @aws-sdk/credential-providers from 3.721.0 to 3.742.0, our tests fail when fromTemporaryCredentials is called.

According to credential-providers/CHANGELOG.md, there are changes to fromTemporaryCredentials in 3.731.0 and 3.734.0.

Regression Issue

• Select this option if this issue appears to be a regression.

SDK version number

@aws-sdk/credential-providers@3.742.0

Which JavaScript Runtime is this issue in?

Node.js

Details of the browser/Node.js/ReactNative version

v22.13.1

Reproduction Steps

The relevant code is:

…
const fromTemporaryCredentialsOptions = {
  params: {
    RoleArn: `arn:aws:iam::${id}:role/${rolePathAndName}`,
    // Member must have length less than or equal to 64
    RoleSessionName: roleSessionName,
    DurationSeconds: 1800
  },
  clientConfig: { region, profile }
}
const credentialsProvider = fromTemporaryCredentials(fromTemporaryCredentialsOptions)
const credentials = await credentialsProvider()
return { region, credentials } // awsV3ClientConfig

This is general code, that has the intention to get temporary credentials where the "user" gets necessary permissions by assuming a role (RoleArn). The user‘s credentials should be found in ~/.aws/credentials under the given profile. The returned credentials are then used in different places to create a client to perform commands, e.g.,

const apiGatewayClient = new APIGatewayClient(awsV3ClientConfig)

instance = new S3Client(awsV3ClientConfig)

(but we don’t get there).

Observed Behavior

exception (STSServiceException):
InvalidClientTokenId: The security token included in the request is invalid.
    at throwDefaultError (…/node_modules/@smithy/smithy-client/dist-cjs/index.js:867:20)
    at …/node_modules/@smithy/smithy-client/dist-cjs/index.js:876:5
    at de_CommandError (…/node_modules/@aws-sdk/nested-clients/dist-cjs/submodules/sts/index.js:299:14)
    at process.processTicksAndRejections (node:internal/process/task_queues:105:5)
    at async …/node_modules/@smithy/middleware-serde/dist-cjs/index.js:35:20
    at async …/node_modules/@smithy/core/dist-cjs/index.js:167:18
    at async …/node_modules/@smithy/middleware-retry/dist-cjs/index.js:321:38
    at async …/node_modules/@aws-sdk/middleware-logger/dist-cjs/index.js:33:22
    at async …/node_modules/@aws-sdk/credential-providers/dist-cjs/fromTemporaryCredentials.base.js:108:33
    at async clientConfig (…/common/awsV3ClientConfig.js:45:27)

Expected Behavior

Return the correct credentials for the role as before.

Possible Solution

No response

Additional Information/Context

No response

[jandppw]

• The problem does not occur when all other updates are applied, and @aws-sdk/credential-providers@3.721.0 is used.
• @aws-sdk/credential-providers@3.731.0 has the issue.
• @aws-sdk/credential-providers@3.726.0 does not have the issue.
• @aws-sdk/credential-providers@3.729.0 does not have the issue.

@aws-sdk/credential-providers@3.730.0 does have an issue, but here the error says

Error: `credentials` is missing
    at credentialsProvider (node_modules/@aws-sdk/credential-providers/node_modules/@aws-sdk/core/dist-cjs/submodules/httpAuthSchemes/index.js:211:15)
    at boundCredentialsProvider (node_modules/@aws-sdk/credential-providers/node_modules/@aws-sdk/core/dist-cjs/submodules/httpAuthSchemes/index.js:215:71)
    at …/node_modules/@smithy/core/dist-cjs/index.js:85:23
    at process.processTicksAndRejections (node:internal/process/task_queues:105:5)
    at async …/node_modules/@aws-sdk/credential-providers/node_modules/@aws-sdk/middleware-logger/dist-cjs/index.js:33:22
    at async …/node_modules/@aws-sdk/credential-providers/dist-cjs/fromTemporaryCredentials.js:50:33
    at async clientConfig (common/awsV3ClientConfig.js:45:27)

In @aws-sdk/credential-providers@3.731.0, the error changes to the InvalidClientTokenId described above.

[kuhe]

Mocking of @aws-sdk/client-sts needs to be transferred to @aws-sdk/nested-clients/sts, if your test does this.

what is the output from the credential provider log?

import { fromTemporaryCredentials } from "@aws-sdk/credential-providers";


const provider = fromTemporaryCredentials({
  logger: console,
  params: {
    RoleArn: "arn:aws:iam::123...456:role/{role}",
  },
  clientConfig: {
    region: "us-west-2",
    profile: "profile_name",
  },
});

const credentials = await provider();

output:

DEBUG: @aws-sdk/credential-providers - fromTemporaryCredentials (STS)
DEBUG: @aws-sdk/credential-providers - fromTemporaryCredentials STS client init with options.clientConfig.region=us-west-2, AWS SDK default credentials, STS default requestHandler.

when I run this I get credentials successfully.

[jandppw]

@kuhe Thanks for the response.

Mocking of @aws-sdk/client-sts needs to be transferred to @aws-sdk/nested-clients/sts, if your test does this.

I do not really understand what you are saying here. If you assume there is something mocked in this test: there isn’t.

There are new versions since yesterday. @aws-sdk/credential-providers@3.743.0 has the issue.

Thank you for the suggestion to add logger: console. I didn’t know about that.

Here is the complete code, simplified to zoom in on the issue:

…/common/awsV3ClientConfig.js:

const { fromTemporaryCredentials } = require('@aws-sdk/credential-providers')
const { v4: uuidv4 } = require('uuid')
const { STSClient, GetCallerIdentityCommand } = require('@aws-sdk/client-sts')

/**
 * @param {string} region
 * @param {string} profile
 * @returns {Promise<string>}
 */
async function accountId(region, profile) {
  console.info('Using AWS credentials from `~/.aws/credentials to determine account id`')
  /** @type {STSClient | undefined } */
  const stsClient = new STSClient({ region, profile })
  const command = new GetCallerIdentityCommand({})
  const response = await stsClient.send(command)
  const accountId = response.Account
  console.info('  AWS Account ID retrieved') // do not log accountId

  return accountId
}

/**
 * @param {string} region
 * @param {string} profile
 * @param {string} rolePathAndName
 * @returns {Promise<AWSv3ClientConfig>}
 */
async function clientConfig(region, profile, rolePathAndName) {
  console.info('Using AWS credentials from `~/.aws/credentials` to assume role.')
  const id = await accountId(region, profile)
  const roleSessionName = `XXXXXX-${uuidv4()}`
  /** @type {import('@aws-sdk/credential-providers').FromTemporaryCredentialsOptions} */
  const fromTemporaryCredentialsOptions = {
    logger: console, // NOTE: added as suggested by @kuhe
    params: {
      RoleArn: `arn:aws:iam::${id}:role/${rolePathAndName}`,
      // Member must have length less than or equal to 64
      RoleSessionName: roleSessionName,
      DurationSeconds: 1800
    },
    clientConfig: { region, profile }
  }
  const credentialsProvider = fromTemporaryCredentials(fromTemporaryCredentialsOptions)
  const credentials = await credentialsProvider()
  console.info(`  AWS credentials for role ${rolePathAndName} received (session name: ${roleSessionName}). Assuming.`)

  return { region, credentials }
}

module.exports = {
  clientConfig
}

…/test/unit/00.common/awsV3ClientConfig.test.js:

const { clientConfig } = require('../../../common/awsV3ClientConfig')
const { profile, automatedTestRoleNameAndPath } = require('../../../common/developerInfo')

const region = 'eu-west-1'

describe('awsV3ClientConfig, function () {
  describe('clientConfig', function () {
    it('works', async function () {
      const result = await clientConfig(region, profile, automatedTestRoleNameAndPath)
      console.log(result)
      result.should.be.an.Object()
    })
    …
  })
})

The output when using @aws-sdk/credential-providers@3.729.0 is:

Using AWS credentials from `~/.aws/credentials` to assume role.
Using AWS credentials from `~/.aws/credentials to determine account id`
  AWS Account ID retrieved
@aws-sdk/credential-providers - fromTemporaryCredentials (STS)
  AWS credentials for role automated-test/automated-test-XXXXXX received (session name: XXXXXX-f89c3d0a-ebae-47a1-bfc0-7da67adab1a5). Assuming.
{
  region: 'eu-west-1',
  credentials: {
    accessKeyId: 'AXXXXXXXXXXXXXXXXXXXX',
    secretAccessKey: 'cBXXXXXXXXXXXXXXXXXXXX',
    sessionToken: 'IQXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXFA==',
    expiration: 2025-02-07T09:43:13.000Z,
    credentialScope: undefined
  }
}

The output when using @aws-sdk/credential-providers@3.743.0 is:

Using AWS credentials from `~/.aws/credentials` to assume role.
Using AWS credentials from `~/.aws/credentials to determine account id`
  AWS Account ID retrieved
@aws-sdk/credential-providers - fromTemporaryCredentials (STS)
@aws-sdk/credential-providers - fromTemporaryCredentials STS client init with options.clientConfig.region=eu-west-1, AWS SDK default credentials, STS default requestHandler.
endpoints Initial EndpointParams: {
  "UseGlobalEndpoint": false,
  "UseFIPS": false,
  "Region": "eu-west-1",
  "UseDualStack": false
}
endpoints evaluateCondition: booleanEquals($UseGlobalEndpoint, true) = false
endpoints evaluateCondition: isSet($Endpoint) = false
endpoints evaluateCondition: isSet($Region) = true
endpoints evaluateCondition: aws.partition($Region) = {
  "dnsSuffix": "amazonaws.com",
  "dualStackDnsSuffix": "api.aws",
  "implicitGlobalRegion": "us-east-1",
  "name": "aws",
  "supportsDualStack": true,
  "supportsFIPS": true,
  "description": "Europe (Ireland)"
}
endpoints assign: PartitionResult := {
  "dnsSuffix": "amazonaws.com",
  "dualStackDnsSuffix": "api.aws",
  "implicitGlobalRegion": "us-east-1",
  "name": "aws",
  "supportsDualStack": true,
  "supportsFIPS": true,
  "description": "Europe (Ireland)"
}
endpoints evaluateCondition: booleanEquals($UseFIPS, true) = false
endpoints evaluateCondition: booleanEquals($UseFIPS, true) = false
endpoints evaluateCondition: booleanEquals($UseDualStack, true) = false
endpoints evaluateCondition: stringEquals($Region, aws-global) = false
endpoints Resolving endpoint from template: {
  "url": "https://sts.{Region}.{PartitionResult#dnsSuffix}",
  "properties": {},
  "headers": {}
}
endpoints Resolved endpoint: {
  "headers": {},
  "properties": {},
  "url": "https://sts.eu-west-1.amazonaws.com/"
}
@smithy/property-provider -> Not found in config files w/ profile [<PROFILE>]: CONFIG_RETRY_MODE
@smithy/property-provider -> Not found in config files w/ profile [<PROFILE>]: CONFIG_MAX_ATTEMPTS
{
  clientName: 'STSClient',
  commandName: 'AssumeRoleCommand',
  input: {
    RoleArn: 'arn:aws:iam::XXXXXXXXXXXXX:role/automated-test/automated-test-XXXXXXX',
    RoleSessionName: 'XXXXXX-767fdf62-8b1f-4094-981e-285b55fa1125',
    DurationSeconds: 1800
  },
  error: InvalidClientTokenId: The security token included in the request is invalid.
      at throwDefaultError (…/node_modules/@smithy/smithy-client/dist-cjs/index.js:867:20)
      at …/node_modules/@smithy/smithy-client/dist-cjs/index.js:876:5
      at de_CommandError (…/node_modules/@aws-sdk/nested-clients/dist-cjs/submodules/sts/index.js:299:14)
      at process.processTicksAndRejections (node:internal/process/task_queues:105:5)
      at async …/node_modules/@smithy/middleware-serde/dist-cjs/index.js:35:20
      at async …/node_modules/@smithy/core/dist-cjs/index.js:167:18
      at async …/node_modules/@smithy/middleware-retry/dist-cjs/index.js:321:38
      at async …/node_modules/@aws-sdk/middleware-logger/dist-cjs/index.js:33:22
      at async …/node_modules/@aws-sdk/credential-providers/dist-cjs/fromTemporaryCredentials.base.js:108:33
      at async clientConfig (…/common/awsV3ClientConfig.js:49:23)
      at async Context.<anonymous> (…/test/unit/00.common/awsV3ClientConfig.test.js:9:22) {
    '$fault': 'client',
    '$metadata': {
      httpStatusCode: 403,
      requestId: '058110da-de42-4413-975a-100a1582c15f',
      extendedRequestId: undefined,
      cfId: undefined,
      attempts: 1,
      totalRetryDelay: 0
    },
    Type: 'Sender',
    Code: 'InvalidClientTokenId'
  },
  metadata: {
    httpStatusCode: 403,
    requestId: '058110da-de42-4413-975a-100a1582c15f',
    extendedRequestId: undefined,
    cfId: undefined,
    attempts: 1,
    totalRetryDelay: 0
  }
}

InvalidClientTokenId: The security token included in the request is invalid.
    at throwDefaultError (node_modules/@smithy/smithy-client/dist-cjs/index.js:867:20)
    at …/node_modules/@smithy/smithy-client/dist-cjs/index.js:876:5
    at de_CommandError (node_modules/@aws-sdk/nested-clients/dist-cjs/submodules/sts/index.js:299:14)
    at process.processTicksAndRejections (node:internal/process/task_queues:105:5)
    at async …/node_modules/@smithy/middleware-serde/dist-cjs/index.js:35:20
    at async …/node_modules/@smithy/core/dist-cjs/index.js:167:18
    at async …/node_modules/@smithy/middleware-retry/dist-cjs/index.js:321:38
    at async …/node_modules/@aws-sdk/middleware-logger/dist-cjs/index.js:33:22
    at async …/node_modules/@aws-sdk/credential-providers/dist-cjs/fromTemporaryCredentials.base.js:108:33
    at async clientConfig (common/awsV3ClientConfig.js:49:23)
    at async Context.<anonymous> (test/unit/00.common/awsV3ClientConfig.test.js:9:22)

Note that getting the account id with accountId(region, profile) with the same credentials works.

The output when using @aws-sdk/credential-providers@3.730.0 is:

Using AWS credentials from `~/.aws/credentials` to assume role.
Using AWS credentials from `~/.aws/credentials to determine account id`
  AWS Account ID retrieved
@aws-sdk/credential-providers - fromTemporaryCredentials (STS)

Error: `credentials` is missing
    at credentialsProvider (node_modules/@aws-sdk/credential-providers/node_modules/@aws-sdk/core/dist-cjs/submodules/httpAuthSchemes/index.js:211:15)
    at boundCredentialsProvider (node_modules/@aws-sdk/credential-providers/node_modules/@aws-sdk/core/dist-cjs/submodules/httpAuthSchemes/index.js:215:71)
    at …/node_modules/@smithy/core/dist-cjs/index.js:85:23
    at process.processTicksAndRejections (node:internal/process/task_queues:105:5)
    at async …/node_modules/@aws-sdk/credential-providers/node_modules/@aws-sdk/middleware-logger/dist-cjs/index.js:33:22
    at async …/node_modules/@aws-sdk/credential-providers/dist-cjs/fromTemporaryCredentials.js:50:33
    at async clientConfig (common/awsV3ClientConfig.js:49:23)
    at async Context.<anonymous> (test/unit/00.common/awsV3ClientConfig.test.js:9:22)

[kuhe]

We usually need a more specific reproduction sample repository for issues with this many variables that are unseen, such as ENV and credentials file(s).

But, I have a theory as to what is happening. It is related to a fix in behavior for fromTemporaryCredentials, and interacts with the value you have set for region in two places: the clientConfig object and your configuration files.

In v3.729.0, the region set in the profile takes precedence and in the latest version of the SDK the clientConfig.region takes higher precedence. The latter is intended as the correct way to set the region, because the field clientConfig is intended as the highest precedence when configuring credential clients that are not instantiated by the user.

Your code-level configured region appears to be eu-west-1. Please compare with what region is set in the configuration file of the profile being used, and ensure that your AWS account enabled the intended regions and permission levels are set in those regions.