SDK Credential Provider in v3 errors out

[nagmesh]

Pre-Migration Checklist

• I've read the Migration Guide.
• I've reviewed the upgrading notes and major version differences mentioned in UPGRADING.md.
• I've checked AWS Forums and StackOverflow for similar migration issues.

Which JavaScript Runtime is this issue in?

Node.js (includes AWS Lambda)

AWS Lambda Usage

• Yes, my application is running on AWS Lambda.
• No, my application is not running on AWS Lambda.

Describe the Migration Issue

The solution I'm working on gets credential with assume role and provides accessKey, session token, secret key. When a similar account comes to get credentials it checks the cache and takes credentials if its there or adds credentials to it.

With SDK V3 we are testing to use @aws-sdk/credential-providers. When a test is run across 5 accounts and 3 regions everything looks fine. However, when a test is run with 250 accounts across 10 regions it throws an error ECONNREFUSED.

Current code snippet

New code

Code Comparison

Current code snippet

New code:

async function sdkCredentials(
  command: string,
  partition: string,
  stage: string,
  configDirPath?: string,
  accountId?: string,
  region?: string,
): Promise<SDKv3CompatibleCredentialProvider> {
  const currentAccount = await getCurrentAccountId(partition, getGlobalRegion(partition));

  // If we're already in the target account, use environment credentials
  if (currentAccount === accountId || !accountId) {
    logger.debug(`Using environment credentials for current account ${accountId}`);
    return fromNodeProviderChain();
  }

  // Otherwise, we need cross-account credentials
  return crossAccountCredentails(command, partition, stage, configDirPath, accountId, region);
}
function crossAccountCredentails(
  command: string,
  partition: string,
  stage: string,
  configDirPath?: string,
  accountId?: string,
  region?: string,
) {
  const assumeRoleName = setPluginAssumeRoleName(command, configDirPath, stage);

  if (!assumeRoleName) {
    throw new Error('Assume role name is required for credential configuration');
  }

  if (!accountId) {
    throw new Error('Account ID is required for credential configuration');
  }

  // External pipeline scenario - chain through management account
  if (
    process.env['MANAGEMENT_ACCOUNT_ID'] &&
    process.env['MANAGEMENT_ACCOUNT_ROLE_NAME'] &&
    accountId! !== process.env['MANAGEMENT_ACCOUNT_ID']!
  ) {
    logger.debug(
      `Using chained credentials: Management Account ${process.env['MANAGEMENT_ACCOUNT_ID']} -> Target Account ${accountId} with role ${assumeRoleName}`,
    );
    return fromTemporaryCredentials({
      params: {
        RoleArn: `arn:${partition}:iam::${accountId}:role/${assumeRoleName}`,
        RoleSessionName: 'cdk-toolkit-session',
      },
      masterCredentials: fromTemporaryCredentials({
        params: {
          RoleArn: `arn:${partition}:iam::${process.env['MANAGEMENT_ACCOUNT_ID']}:role/${process.env['MANAGEMENT_ACCOUNT_ROLE_NAME']}`,
          RoleSessionName: 'lza-external-pipeline-session',
        },
      }),
      clientConfig: { retryStrategy: setRetryStrategy(), region: region ?? getGlobalRegion(partition) },
    });
  } else if (
    process.env['MANAGEMENT_ACCOUNT_ID'] &&
    process.env['MANAGEMENT_ACCOUNT_ROLE_NAME'] &&
    accountId! === process.env['MANAGEMENT_ACCOUNT_ID']!
  ) {
    logger.debug(
      `Using environment credentials for account ${accountId} with role ${process.env['MANAGEMENT_ACCOUNT_ROLE_NAME']}`,
    );
    return fromTemporaryCredentials({
      params: {
        RoleArn: `arn:${partition}:iam::${process.env['MANAGEMENT_ACCOUNT_ID']}:role/${process.env['MANAGEMENT_ACCOUNT_ROLE_NAME']}`,
        RoleSessionName: 'lza-external-pipeline-session',
      },
      clientConfig: { retryStrategy: setRetryStrategy(), region: region ?? getGlobalRegion(partition) },
    });
  }

  // Direct assume role to target account
  logger.debug(`Using assume role credentials for account ${accountId} with role ${assumeRoleName}`);
  return fromTemporaryCredentials({
    params: {
      RoleArn: `arn:${partition}:iam::${accountId}:role/${assumeRoleName}`,
      RoleSessionName: 'cdk-toolkit-session',
    },
    clientConfig: { retryStrategy: setRetryStrategy(), region: region ?? getGlobalRegion(partition) },
  });
}

export function setRetryStrategy() {
  const numberOfRetries = Number(process.env['ACCELERATOR_SDK_MAX_ATTEMPTS'] ?? 800);
  return new ConfiguredRetryStrategy(numberOfRetries, (attempt: number) => 100 + attempt * 1000);
}

Typical run of these command is something like

for (const region of enabledRegions) {
      for (const account of nonManagementAccounts) {
        const accountId = accountsConfig.getAccountId(account.name);
        logger.info(`Executing ${toolkitProps.stage} for ${account.name} account in ${region} region.`);
        promises.push(
          AcceleratorToolkit.execute({
            accountId,
            region,
            ...toolkitProps,
          }),
        );
        if (promises.length >= maxStacks) {
          await Promise.all(promises);
          promises.length = 0;
        }
      }

Observed Differences/Errors

Older sdkv2 code works and fails sometimes with security token expired (non-standard partitions) intermittently. Retry normally works.

New code fails with

Cannot assume role for 3600 seconds: Error: getaddrinfo ENOTFOUND sts.us-west-2.amazonaws.com
Assuming role arn:aws:iam::xx:role/AWSControlTowerExecution for 3600 seconds
2025-09-02 19:10:25.405 | error | toolkit | Deployment of AWSAccelerator-ResourcePolicyEnforcementStack-xx-us-west-2 failed: connect ECONNREFUSED 52.94.177.163:443

Could not assume role in target account using current credentials (which are for account xx) connect ECONNREFUSED 44.248.100.60:443 .

Additional Context

I would prefer if we can use native sdkv3 credential helper and deprecate those packages.

[aBurmeseDev]

Hi @nagmesh - thanks for reaching out.

As you mentioned in the description, it sounds like the culprit could be the number of cross-account operations causing the system to run out of sockets. Here are a couple of suggestions I'd try:

• stagger the outgoing traffic instead of executing all operations simultaneously
• use an STS client directly to assume roles with a higher socket allocation, rather than relying on the credential provider. The credential provider may be performing unnecessary operations when handling multiple accounts

Best
John

[mrgrain]

Probably related issue: #5236

[github-actions]

This issue has not received a response in 1 week. If you still think there is a problem, please leave a comment to avoid the issue from automatically closing.