S3 HeadBucketCommand does not correctly request endpoints for cross-region buckets

[rickyrattlesnake]

Checkboxes for prior research

• I've gone through Developer Guide and API reference
• I've checked AWS Forums and StackOverflow.
• I've searched for previous similar issues and didn't find any solution.

Describe the bug

Using the @aws-sdk/client-s3 to fire a HeadBucketCommand for a bucket in the us-east-1 when the client has been configured for the af-south-1 region will fail with a 400 response code.

The same command using the aws cli aws s3api head-bucket works as expected.

Regression Issue

• Select this option if this issue appears to be a regression.

SDK version number

@aws-sdk/client-s3@3.879.0

Which JavaScript Runtime is this issue in?

Node.js

Details of the browser/Node.js/ReactNative version

v22.15.0

Reproduction Steps

1. Create a bucket in us-east-1 with ListBucket permissions for the user performing the headbucket request.
2. Make sure the af-south-1 region is enabled for the account.
3. Create a script to run the head bucket command using the SDK from the af-south-1. Run the script using a user or role with access to that bucket.

import { S3Client, HeadBucketCommand } from "@aws-sdk/client-s3";

async function main() {
	const client = new S3Client({
		region: "af-south-1",
		followRegionRedirects: true,
		logger: console
	});
	const bucketName = "test-ricky-config-bucket-us-east-1";
	try {
		const command = new HeadBucketCommand({ Bucket: bucketName });
		const response = await client.send(command);
		console.log("Bucket exists:", response);
	} catch (error) {
		console.error("Error calling HeadBucketCommand:", error);
	}
}

main();

4. Run the same command with the same credentials using the aws cli

aws s3api head-bucket --region af-south-1 --bucket test-ricky-config-bucket-us-east-1 --debug

Observed Behavior

The HeadBucket command using the AWS SDK fails with a 400 error code when it is expected to succeed.

Here are some debug logs for that call.

@smithy/property-provider -> Not found in ENV: NODE_USE_ARN_REGION_ENV_NAME
@smithy/property-provider -> Not found in config files w/ profile [sandbox]: NODE_USE_ARN_REGION_INI_NAME
@smithy/property-provider -> Not found in ENV: NODE_DISABLE_S, _EXPRESS_SESSION_AUTH_ENV_NAME
@smithy/property-provider -> Not found in config files w/ profile [sandbox]: NODE_DISABLE_S, _EXPRESS_SESSION_AUTH_INI_NAME
@smithy/property-provider -> Not found in ENV: ENV_USE_FIPS_ENDPOINT
@smithy/property-provider -> Not found in config files w/ profile [sandbox]: CONFIG_USE_FIPS_ENDPOINT
@smithy/property-provider -> Not found in ENV: ENV_USE_DUALSTACK_ENDPOINT
@smithy/property-provider -> Not found in config files w/ profile [sandbox]: CONFIG_USE_DUALSTACK_ENDPOINT
@smithy/property-provider -> Not found in ENV: NODE_AUTH_SCHEME_PREFERENCE_ENV_KEY
@smithy/property-provider -> Not found in config files w/ profile [sandbox]: NODE_AUTH_SCHEME_PREFERENCE_CONFIG_KEY
@smithy/property-provider -> AWS_PROFILE is set, skipping fromEnv provider.
@aws-sdk/credential-provider-node - defaultProvider::fromSSO
@smithy/property-provider -> Skipping SSO provider in default chain (inputs do not include SSO fields).
@aws-sdk/credential-provider-node - defaultProvider::fromIni
@aws-sdk/credential-provider-ini - fromIni
@aws-sdk/credential-provider-ini - resolveStaticCredentials
@smithy/property-provider -> Not found in ENV: ENV_RETRY_MODE
@smithy/property-provider -> Not found in config files w/ profile [sandbox]: CONFIG_RETRY_MODE
@smithy/property-provider -> Not found in ENV: UA_APP_ID_ENV_NAME
@smithy/property-provider -> Not found in config files w/ profile [sandbox]: UA_APP_ID_INI_NAME, UA_APP_ID_INI_NAME_DEPRECATED
@smithy/property-provider -> Not found in ENV: ENV_MAX_ATTEMPTS
@smithy/property-provider -> Not found in config files w/ profile [sandbox]: CONFIG_MAX_ATTEMPTS
{
  clientName: 'S3Client',
  commandName: 'HeadBucketCommand',
  input: { Bucket: 'test-ricky-config-bucket-us-east-1' },
  error: 400: UnknownError
      at throwDefaultError (/Users/rickyratnayake/projects/service-cloud-assets/node_modules/@smithy/smithy-client/dist-cjs/index.js:425:20)
      at /Users/rickyratnayake/projects/service-cloud-assets/node_modules/@smithy/smithy-client/dist-cjs/index.js:434:5
      at de_CommandError (/Users/rickyratnayake/projects/service-cloud-assets/node_modules/@aws-sdk/client-s3/dist-cjs/index.js:5202:14)
      at processTicksAndRejections (node:internal/process/task_queues:105:5)
      at async /Users/rickyratnayake/projects/service-cloud-assets/node_modules/@smithy/middleware-serde/dist-cjs/index.js:36:20
      at async /Users/rickyratnayake/projects/service-cloud-assets/node_modules/@aws-sdk/middleware-sdk-s3/dist-cjs/index.js:484:18
      at async /Users/rickyratnayake/projects/service-cloud-assets/node_modules/@smithy/middleware-retry/dist-cjs/index.js:320:38
      at async /Users/rickyratnayake/projects/service-cloud-assets/node_modules/@aws-sdk/middleware-sdk-s3/dist-cjs/index.js:110:22
      at async /Users/rickyratnayake/projects/service-cloud-assets/node_modules/@aws-sdk/middleware-sdk-s3/dist-cjs/index.js:137:14
      at async /Users/rickyratnayake/projects/service-cloud-assets/node_modules/@aws-sdk/client-s3/node_modules/@aws-sdk/middleware-logger/dist-cjs/index.js:33:22 {
    '$fault': 'client',
    '$metadata': {
      httpStatusCode: 400,
      requestId: 'W184XJRNEM61JPRR',
      extendedRequestId: 'F5GmjnVv44QT966F7AT42CMOXnpMsV4RVd8WTBaNFE+0hp6evRWQVH/qkIpbARDYpry8PwMrSIx8pu0GU/gY5hQxq65zHi3aP5JO4G4LC0k=',
      cfId: undefined,
      attempts: 1,
      totalRetryDelay: 0
    }
  },
  metadata: {
    httpStatusCode: 400,
    requestId: 'W184XJRNEM61JPRR',
    extendedRequestId: 'F5GmjnVv44QT966F7AT42CMOXnpMsV4RVd8WTBaNFE+0hp6evRWQVH/qkIpbARDYpry8PwMrSIx8pu0GU/gY5hQxq65zHi3aP5JO4G4LC0k=',
    cfId: undefined,
    attempts: 1,
    totalRetryDelay: 0
  }
}
Error calling HeadBucketCommand: 400: UnknownError
    at throwDefaultError (/Users/rickyratnayake/projects/service-cloud-assets/node_modules/@smithy/smithy-client/dist-cjs/index.js:425:20)
    at /Users/rickyratnayake/projects/service-cloud-assets/node_modules/@smithy/smithy-client/dist-cjs/index.js:434:5
    at de_CommandError (/Users/rickyratnayake/projects/service-cloud-assets/node_modules/@aws-sdk/client-s3/dist-cjs/index.js:5202:14)
    at processTicksAndRejections (node:internal/process/task_queues:105:5)
    at async /Users/rickyratnayake/projects/service-cloud-assets/node_modules/@smithy/middleware-serde/dist-cjs/index.js:36:20
    at async /Users/rickyratnayake/projects/service-cloud-assets/node_modules/@aws-sdk/middleware-sdk-s3/dist-cjs/index.js:484:18
    at async /Users/rickyratnayake/projects/service-cloud-assets/node_modules/@smithy/middleware-retry/dist-cjs/index.js:320:38
    at async /Users/rickyratnayake/projects/service-cloud-assets/node_modules/@aws-sdk/middleware-sdk-s3/dist-cjs/index.js:110:22
    at async /Users/rickyratnayake/projects/service-cloud-assets/node_modules/@aws-sdk/middleware-sdk-s3/dist-cjs/index.js:137:14
    at async /Users/rickyratnayake/projects/service-cloud-assets/node_modules/@aws-sdk/client-s3/node_modules/@aws-sdk/middleware-logger/dist-cjs/index.js:33:22 {
  '$fault': 'client',
  '$metadata': {
    httpStatusCode: 400,
    requestId: 'W184XJRNEM61JPRR',
    extendedRequestId: 'F5GmjnVv44QT966F7AT42CMOXnpMsV4RVd8WTBaNFE+0hp6evRWQVH/qkIpbARDYpry8PwMrSIx8pu0GU/gY5hQxq65zHi3aP5JO4G4LC0k=',
    cfId: undefined,
    attempts: 1,
    totalRetryDelay: 0
  }
}

The AWS CLI command works as expected.

Here are some debug logs for the cli call

2025-09-03 13:43:35,327 - MainThread - awscli.clidriver - DEBUG - CLI version: aws-cli/2.28.1 Python/3.13.5 Darwin/24.5.0 source/arm64
2025-09-03 13:43:35,327 - MainThread - awscli.clidriver - DEBUG - Arguments entered to CLI: ['s3api', 'head-bucket', '--region', 'af-south-1', '--bucket', 'test-ricky-config-bucket-us-east-1', '--debug']
2025-09-03 13:43:35,396 - MainThread - botocore.hooks - DEBUG - Event building-command-table.main: calling handler <function add_s3 at 0x104a8b1a0>
2025-09-03 13:43:35,396 - MainThread - botocore.hooks - DEBUG - Event building-command-table.main: calling handler <function add_ddb at 0x104860540>
2025-09-03 13:43:35,396 - MainThread - botocore.hooks - DEBUG - Event building-command-table.main: calling handler <bound method BasicCommand.add_command of <class 'awscli.customizations.configure.configure.ConfigureCommand'>>
2025-09-03 13:43:35,396 - MainThread - botocore.hooks - DEBUG - Event building-command-table.main: calling handler <function change_name at 0x1047b05e0>
2025-09-03 13:43:35,396 - MainThread - botocore.hooks - DEBUG - Event building-command-table.main: calling handler <function change_name at 0x1047b1760>
2025-09-03 13:43:35,396 - MainThread - botocore.hooks - DEBUG - Event building-command-table.main: calling handler <function alias_opsworks_cm at 0x1049d5080>
2025-09-03 13:43:35,396 - MainThread - botocore.hooks - DEBUG - Event building-command-table.main: calling handler <function add_history_commands at 0x1049a9300>
2025-09-03 13:43:35,396 - MainThread - botocore.hooks - DEBUG - Event building-command-table.main: calling handler <bound method BasicCommand.add_command of <class 'awscli.customizations.devcommands.CLIDevCommand'>>
2025-09-03 13:43:35,396 - MainThread - botocore.hooks - DEBUG - Event building-command-table.main: calling handler <function add_waiters at 0x104ac07c0>
2025-09-03 13:43:35,396 - MainThread - botocore.hooks - DEBUG - Event building-command-table.main: calling handler <bound method AliasSubCommandInjector.on_building_command_table of <awscli.alias.AliasSubCommandInjector object at 0x104ab2f90>>
2025-09-03 13:43:35,397 - MainThread - botocore.loaders - DEBUG - Loading JSON file: /opt/homebrew/Cellar/awscli/2.28.1/libexec/lib/python3.13/site-packages/awscli/data/cli.json
2025-09-03 13:43:35,397 - MainThread - botocore.hooks - DEBUG - Event top-level-args-parsed: calling handler <function resolve_types at 0x104984180>
2025-09-03 13:43:35,397 - MainThread - botocore.hooks - DEBUG - Event top-level-args-parsed: calling handler <function no_sign_request at 0x1049844a0>
2025-09-03 13:43:35,397 - MainThread - botocore.hooks - DEBUG - Event top-level-args-parsed: calling handler <function resolve_verify_ssl at 0x104984400>
2025-09-03 13:43:35,397 - MainThread - botocore.hooks - DEBUG - Event top-level-args-parsed: calling handler <function resolve_cli_read_timeout at 0x1049845e0>
2025-09-03 13:43:35,397 - MainThread - botocore.hooks - DEBUG - Event top-level-args-parsed: calling handler <function resolve_cli_connect_timeout at 0x104984540>
2025-09-03 13:43:35,398 - MainThread - botocore.hooks - DEBUG - Event top-level-args-parsed: calling handler <built-in method update of dict object at 0x104c3cd80>
2025-09-03 13:43:35,398 - MainThread - botocore.session - DEBUG - Setting config variable for region to 'af-south-1'
2025-09-03 13:43:35,398 - MainThread - awscli.clidriver - DEBUG - CLI version: aws-cli/2.28.1 Python/3.13.5 Darwin/24.5.0 source/arm64
2025-09-03 13:43:35,398 - MainThread - awscli.clidriver - DEBUG - Arguments entered to CLI: ['s3api', 'head-bucket', '--region', 'af-south-1', '--bucket', 'test-ricky-config-bucket-us-east-1', '--debug']
2025-09-03 13:43:35,398 - MainThread - botocore.hooks - DEBUG - Event session-initialized: calling handler <function add_timestamp_parser at 0x104a96e80>
2025-09-03 13:43:35,398 - MainThread - botocore.hooks - DEBUG - Event session-initialized: calling handler <function register_uri_param_handler at 0x103864f40>
2025-09-03 13:43:35,398 - MainThread - botocore.hooks - DEBUG - Event session-initialized: calling handler <function add_binary_formatter at 0x1045bad40>
2025-09-03 13:43:35,398 - MainThread - botocore.hooks - DEBUG - Event session-initialized: calling handler <function no_pager_handler at 0x10455c5e0>
2025-09-03 13:43:35,398 - MainThread - botocore.hooks - DEBUG - Event session-initialized: calling handler <function inject_assume_role_provider_cache at 0x1045ba020>
2025-09-03 13:43:35,401 - MainThread - botocore.utils - DEBUG - IMDS ENDPOINT: http://169.254.169.254/
2025-09-03 13:43:35,406 - MainThread - botocore.hooks - DEBUG - Event session-initialized: calling handler <function attach_history_handler at 0x104987a60>
2025-09-03 13:43:35,406 - MainThread - botocore.hooks - DEBUG - Event session-initialized: calling handler <function inject_json_file_cache at 0x104845da0>
2025-09-03 13:43:35,433 - MainThread - botocore.loaders - DEBUG - Loading JSON file: /opt/homebrew/Cellar/awscli/2.28.1/libexec/lib/python3.13/site-packages/awscli/botocore/data/s3/2006-03-01/service-2.json
2025-09-03 13:43:35,437 - MainThread - botocore.loaders - DEBUG - Loading JSON file: /opt/homebrew/Cellar/awscli/2.28.1/libexec/lib/python3.13/site-packages/awscli/botocore/data/s3/2006-03-01/service-2.sdk-extras.json
2025-09-03 13:43:35,439 - MainThread - botocore.hooks - DEBUG - Event building-command-table.s3api: calling handler <function add_waiters at 0x104ac07c0>
2025-09-03 13:43:35,456 - MainThread - botocore.loaders - DEBUG - Loading JSON file: /opt/homebrew/Cellar/awscli/2.28.1/libexec/lib/python3.13/site-packages/awscli/botocore/data/s3/2006-03-01/waiters-2.json
2025-09-03 13:43:35,456 - MainThread - botocore.hooks - DEBUG - Event building-command-table.s3api: calling handler <bound method AliasSubCommandInjector.on_building_command_table of <awscli.alias.AliasSubCommandInjector object at 0x104ab2f90>>
2025-09-03 13:43:35,457 - MainThread - awscli.clidriver - DEBUG - OrderedDict({'bucket': <awscli.arguments.CLIArgument object at 0x104c978c0>, 'expected-bucket-owner': <awscli.arguments.CLIArgument object at 0x104c8e990>})
2025-09-03 13:43:35,457 - MainThread - botocore.hooks - DEBUG - Event building-argument-table.s3api.head-bucket: calling handler <function add_streaming_output_arg at 0x104a963e0>
2025-09-03 13:43:35,457 - MainThread - botocore.hooks - DEBUG - Event building-argument-table.s3api.head-bucket: calling handler <function add_cli_input_json at 0x1045bb880>
2025-09-03 13:43:35,457 - MainThread - botocore.hooks - DEBUG - Event building-argument-table.s3api.head-bucket: calling handler <function add_cli_input_yaml at 0x1045c8680>
2025-09-03 13:43:35,457 - MainThread - botocore.hooks - DEBUG - Event building-argument-table.s3api.head-bucket: calling handler <function unify_paging_params at 0x104860fe0>
2025-09-03 13:43:35,474 - MainThread - botocore.loaders - DEBUG - Loading JSON file: /opt/homebrew/Cellar/awscli/2.28.1/libexec/lib/python3.13/site-packages/awscli/botocore/data/s3/2006-03-01/paginators-1.json
2025-09-03 13:43:35,474 - MainThread - botocore.loaders - DEBUG - Loading JSON file: /opt/homebrew/Cellar/awscli/2.28.1/libexec/lib/python3.13/site-packages/awscli/botocore/data/s3/2006-03-01/paginators-1.sdk-extras.json
2025-09-03 13:43:35,474 - MainThread - botocore.hooks - DEBUG - Event building-argument-table.s3api.head-bucket: calling handler <function add_generate_skeleton at 0x1049728e0>
2025-09-03 13:43:35,474 - MainThread - botocore.hooks - DEBUG - Event before-building-argument-table-parser.s3api.head-bucket: calling handler <bound method OverrideRequiredArgsArgument.override_required_args of <awscli.customizations.cliinput.CliInputJSONArgument object at 0x104c97b60>>
2025-09-03 13:43:35,474 - MainThread - botocore.hooks - DEBUG - Event before-building-argument-table-parser.s3api.head-bucket: calling handler <bound method OverrideRequiredArgsArgument.override_required_args of <awscli.customizations.cliinput.CliInputYAMLArgument object at 0x104c97cb0>>
2025-09-03 13:43:35,474 - MainThread - botocore.hooks - DEBUG - Event before-building-argument-table-parser.s3api.head-bucket: calling handler <bound method GenerateCliSkeletonArgument.override_required_args of <awscli.customizations.generatecliskeleton.GenerateCliSkeletonArgument object at 0x104c97e00>>
2025-09-03 13:43:35,474 - MainThread - botocore.hooks - DEBUG - Event building-command-table.s3api_head-bucket: calling handler <function add_waiters at 0x104ac07c0>
2025-09-03 13:43:35,474 - MainThread - botocore.hooks - DEBUG - Event building-command-table.s3api_head-bucket: calling handler <bound method AliasSubCommandInjector.on_building_command_table of <awscli.alias.AliasSubCommandInjector object at 0x104ab2f90>>
2025-09-03 13:43:35,474 - MainThread - botocore.hooks - DEBUG - Event load-cli-arg.s3.head-bucket.bucket: calling handler <awscli.paramfile.URIArgumentHandler object at 0x104ab3a10>
2025-09-03 13:43:35,474 - MainThread - botocore.hooks - DEBUG - Event process-cli-arg.s3.head-bucket: calling handler <awscli.argprocess.ParamShorthandParser object at 0x104ab12b0>
2025-09-03 13:43:35,474 - MainThread - awscli.arguments - DEBUG - Unpacked value of 'test-ricky-config-bucket-us-east-1' for parameter "bucket": 'test-ricky-config-bucket-us-east-1'
2025-09-03 13:43:35,474 - MainThread - botocore.hooks - DEBUG - Event load-cli-arg.s3.head-bucket.expected-bucket-owner: calling handler <awscli.paramfile.URIArgumentHandler object at 0x104ab3a10>
2025-09-03 13:43:35,474 - MainThread - botocore.hooks - DEBUG - Event load-cli-arg.s3.head-bucket.cli-input-json: calling handler <awscli.paramfile.URIArgumentHandler object at 0x104ab3a10>
2025-09-03 13:43:35,474 - MainThread - botocore.hooks - DEBUG - Event load-cli-arg.s3.head-bucket.cli-input-yaml: calling handler <awscli.paramfile.URIArgumentHandler object at 0x104ab3a10>
2025-09-03 13:43:35,474 - MainThread - botocore.hooks - DEBUG - Event load-cli-arg.s3.head-bucket.generate-cli-skeleton: calling handler <awscli.paramfile.URIArgumentHandler object at 0x104ab3a10>
2025-09-03 13:43:35,475 - MainThread - botocore.hooks - DEBUG - Event calling-command.s3api.head-bucket: calling handler <bound method CliInputArgument.add_to_call_parameters of <awscli.customizations.cliinput.CliInputJSONArgument object at 0x104c97b60>>
2025-09-03 13:43:35,475 - MainThread - botocore.hooks - DEBUG - Event calling-command.s3api.head-bucket: calling handler <bound method CliInputArgument.add_to_call_parameters of <awscli.customizations.cliinput.CliInputYAMLArgument object at 0x104c97cb0>>
2025-09-03 13:43:35,475 - MainThread - botocore.hooks - DEBUG - Event calling-command.s3api.head-bucket: calling handler <bound method GenerateCliSkeletonArgument.generate_skeleton of <awscli.customizations.generatecliskeleton.GenerateCliSkeletonArgument object at 0x104c97e00>>
2025-09-03 13:43:35,475 - MainThread - botocore.credentials - DEBUG - Looking for credentials via: env
2025-09-03 13:43:35,475 - MainThread - botocore.credentials - DEBUG - Looking for credentials via: assume-role
2025-09-03 13:43:35,475 - MainThread - botocore.credentials - DEBUG - Looking for credentials via: assume-role-with-web-identity
2025-09-03 13:43:35,475 - MainThread - botocore.credentials - DEBUG - Looking for credentials via: sso
2025-09-03 13:43:35,475 - MainThread - botocore.credentials - DEBUG - Looking for credentials via: shared-credentials-file
2025-09-03 13:43:35,475 - MainThread - botocore.credentials - INFO - Found credentials in shared credentials file: ~/.aws/credentials
2025-09-03 13:43:35,476 - MainThread - botocore.loaders - DEBUG - Loading JSON file: /opt/homebrew/Cellar/awscli/2.28.1/libexec/lib/python3.13/site-packages/awscli/botocore/data/endpoints.json
2025-09-03 13:43:35,483 - MainThread - botocore.hooks - DEBUG - Event choose-service-name: calling handler <function handle_service_name_alias at 0x10354ce00>
2025-09-03 13:43:35,504 - MainThread - botocore.loaders - DEBUG - Loading JSON file: /opt/homebrew/Cellar/awscli/2.28.1/libexec/lib/python3.13/site-packages/awscli/botocore/data/s3/2006-03-01/endpoint-rule-set-1.json
2025-09-03 13:43:35,506 - MainThread - botocore.loaders - DEBUG - Loading JSON file: /opt/homebrew/Cellar/awscli/2.28.1/libexec/lib/python3.13/site-packages/awscli/botocore/data/partitions.json
2025-09-03 13:43:35,507 - MainThread - botocore.hooks - DEBUG - Event creating-client-class.s3: calling handler <function add_generate_presigned_post at 0x103290cc0>
2025-09-03 13:43:35,507 - MainThread - botocore.hooks - DEBUG - Event creating-client-class.s3: calling handler <function add_generate_presigned_url at 0x103290a40>
2025-09-03 13:43:35,507 - MainThread - botocore.configprovider - DEBUG - Looking for endpoint for s3 via: environment_service
2025-09-03 13:43:35,507 - MainThread - botocore.configprovider - DEBUG - Looking for endpoint for s3 via: environment_global
2025-09-03 13:43:35,507 - MainThread - botocore.configprovider - DEBUG - Looking for endpoint for s3 via: config_service
2025-09-03 13:43:35,507 - MainThread - botocore.configprovider - DEBUG - Looking for endpoint for s3 via: config_global
2025-09-03 13:43:35,507 - MainThread - botocore.configprovider - DEBUG - No configured endpoint found.
2025-09-03 13:43:35,509 - MainThread - botocore.endpoint - DEBUG - Setting s3 timeout as (60, 60)
2025-09-03 13:43:35,509 - MainThread - botocore.utils - DEBUG - Registering S3 region redirector handler
2025-09-03 13:43:35,510 - MainThread - botocore.utils - DEBUG - Registering S3Express Identity Resolver
2025-09-03 13:43:35,510 - MainThread - botocore.hooks - DEBUG - Event provide-client-params.s3.HeadBucket: calling handler <function base64_decode_input_blobs at 0x1045bade0>
2025-09-03 13:43:35,510 - MainThread - botocore.hooks - DEBUG - Event before-parameter-build.s3.HeadBucket: calling handler <function validate_bucket_name at 0x10354d3a0>
2025-09-03 13:43:35,510 - MainThread - botocore.hooks - DEBUG - Event before-parameter-build.s3.HeadBucket: calling handler <function remove_bucket_from_url_paths_from_model at 0x10354f560>
2025-09-03 13:43:35,510 - MainThread - botocore.hooks - DEBUG - Event before-parameter-build.s3.HeadBucket: calling handler <bound method S3RegionRedirectorv2.annotate_request_context of <botocore.utils.S3RegionRedirectorv2 object at 0x10503f620>>
2025-09-03 13:43:35,510 - MainThread - botocore.hooks - DEBUG - Event before-parameter-build.s3.HeadBucket: calling handler <bound method S3ExpressIdentityResolver.inject_signing_cache_key of <botocore.utils.S3ExpressIdentityResolver object at 0x10503f770>>
2025-09-03 13:43:35,510 - MainThread - botocore.hooks - DEBUG - Event before-parameter-build.s3.HeadBucket: calling handler <function generate_idempotent_uuid at 0x10354d1c0>
2025-09-03 13:43:35,510 - MainThread - botocore.hooks - DEBUG - Event before-parameter-build.s3.HeadBucket: calling handler <function _handle_request_validation_mode_member at 0x10354fba0>
2025-09-03 13:43:35,510 - MainThread - botocore.hooks - DEBUG - Event before-endpoint-resolution.s3: calling handler <function customize_endpoint_resolver_builtins at 0x10354f740>
2025-09-03 13:43:35,510 - MainThread - botocore.hooks - DEBUG - Event before-endpoint-resolution.s3: calling handler <bound method S3RegionRedirectorv2.redirect_from_cache of <botocore.utils.S3RegionRedirectorv2 object at 0x10503f620>>
2025-09-03 13:43:35,510 - MainThread - botocore.regions - DEBUG - Calling endpoint provider with parameters: {'Bucket': 'test-ricky-config-bucket-us-east-1', 'Region': 'af-south-1', 'UseFIPS': False, 'UseDualStack': False, 'ForcePathStyle': False, 'Accelerate': False, 'UseGlobalEndpoint': False, 'DisableMultiRegionAccessPoints': False, 'UseArnRegion': True}
2025-09-03 13:43:35,510 - MainThread - botocore.regions - DEBUG - Endpoint provider result: https://test-ricky-config-bucket-us-east-1.s3.af-south-1.amazonaws.com
2025-09-03 13:43:35,510 - MainThread - botocore.regions - DEBUG - Selecting from endpoint provider's list of auth schemes: "sigv4". User selected auth scheme is: "None"
2025-09-03 13:43:35,510 - MainThread - botocore.regions - DEBUG - Selected auth type "v4" as "v4" with signing context params: {'region': 'af-south-1', 'signing_name': 's3', 'disableDoubleEncoding': True}
2025-09-03 13:43:35,510 - MainThread - botocore.hooks - DEBUG - Event before-call.s3.HeadBucket: calling handler <function add_expect_header at 0x10354d760>
2025-09-03 13:43:35,510 - MainThread - botocore.hooks - DEBUG - Event before-call.s3.HeadBucket: calling handler <bound method S3ExpressIdentityResolver.apply_signing_cache_key of <botocore.utils.S3ExpressIdentityResolver object at 0x10503f770>>
2025-09-03 13:43:35,510 - MainThread - botocore.hooks - DEBUG - Event before-call.s3.HeadBucket: calling handler <function inject_api_version_header_if_needed at 0x10354eca0>
2025-09-03 13:43:35,510 - MainThread - botocore.endpoint - DEBUG - Making request for OperationModel(name=HeadBucket) with params: {'url_path': '', 'query_string': {}, 'method': 'HEAD', 'headers': {'User-Agent': 'aws-cli/2.28.1 md/awscrt#0.26.1 ua/2.1 os/macos#24.5.0 md/arch#arm64 lang/python#3.13.5 md/pyimpl#CPython m/E,Z,b cfg/retry-mode#standard md/installer#source md/prompt#off md/command#s3api.head-bucket'}, 'body': b'', 'auth_path': '/test-ricky-config-bucket-us-east-1/', 'url': 'https://test-ricky-config-bucket-us-east-1.s3.af-south-1.amazonaws.com/', 'context': {'client_region': 'af-south-1', 'client_config': <botocore.config.Config object at 0x104c8f250>, 'has_streaming_input': False, 'auth_type': 'v4', 'unsigned_payload': None, 'auth_options': ['aws.auth#sigv4'], 's3_redirect': {'redirected': False, 'bucket': 'test-ricky-config-bucket-us-east-1', 'params': {'Bucket': 'test-ricky-config-bucket-us-east-1'}}, 'S3Express': {'bucket_name': 'test-ricky-config-bucket-us-east-1'}, 'signing': {'region': 'af-south-1', 'signing_name': 's3', 'disableDoubleEncoding': True}, 'endpoint_properties': {'authSchemes': [{'disableDoubleEncoding': True, 'name': 'sigv4', 'signingName': 's3', 'signingRegion': 'af-south-1'}]}}}
2025-09-03 13:43:35,510 - MainThread - botocore.hooks - DEBUG - Event request-created.s3.HeadBucket: calling handler <bound method RequestSigner.handler of <botocore.signers.RequestSigner object at 0x10503cc20>>
2025-09-03 13:43:35,510 - MainThread - botocore.hooks - DEBUG - Event choose-signer.s3.HeadBucket: calling handler <function set_operation_specific_signer at 0x10354cfe0>
2025-09-03 13:43:35,510 - MainThread - botocore.hooks - DEBUG - Event before-sign.s3.HeadBucket: calling handler <function remove_arn_from_signing_path at 0x10354f6a0>
2025-09-03 13:43:35,510 - MainThread - botocore.hooks - DEBUG - Event before-sign.s3.HeadBucket: calling handler <function _set_extra_headers_for_unsigned_request at 0x10354fe20>
2025-09-03 13:43:35,510 - MainThread - botocore.hooks - DEBUG - Event before-sign.s3.HeadBucket: calling handler <bound method S3ExpressIdentityResolver.resolve_s3express_identity of <botocore.utils.S3ExpressIdentityResolver object at 0x10503f770>>
2025-09-03 13:43:35,511 - MainThread - botocore.auth - DEBUG - Calculating signature using v4 auth.
2025-09-03 13:43:35,511 - MainThread - botocore.auth - DEBUG - CanonicalRequest:
HEAD
/

host:test-ricky-config-bucket-us-east-1.s3.af-south-1.amazonaws.com
x-amz-content-sha256:e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
x-amz-date:20250903T034335Z
x-amz-security-token:"<redacted>"

host;x-amz-content-sha256;x-amz-date;x-amz-security-token
e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
2025-09-03 13:43:35,511 - MainThread - botocore.auth - DEBUG - StringToSign:
AWS4-HMAC-SHA256
20250903T034335Z
20250903/af-south-1/s3/aws4_request
c8177ce9c7eb106d748972accb41b8f4fc92e41c8b4f48180765da39a355a705
2025-09-03 13:43:35,511 - MainThread - botocore.auth - DEBUG - Signature:
"<redacted>"
2025-09-03 13:43:35,511 - MainThread - botocore.hooks - DEBUG - Event request-created.s3.HeadBucket: calling handler <bound method UserAgentString.rebuild_and_replace_user_agent_handler of <botocore.useragent.UserAgentString object at 0x105864690>>
2025-09-03 13:43:35,511 - MainThread - botocore.endpoint - DEBUG - Sending http request: <AWSPreparedRequest stream_output=False, method=HEAD, url=https://test-ricky-config-bucket-us-east-1.s3.af-south-1.amazonaws.com/, headers={'User-Agent': b'aws-cli/2.28.1 md/awscrt#0.26.1 ua/2.1 os/macos#24.5.0 md/arch#arm64 lang/python#3.13.5 md/pyimpl#CPython m/E,Z,b cfg/retry-mode#standard md/installer#source md/prompt#off md/command#s3api.head-bucket', 'X-Amz-Date': b'20250903T034335Z', 'X-Amz-Security-Token': b'"<redacted>"', 'X-Amz-Content-SHA256': b'e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855', 'Authorization': b'AWS4-HMAC-SHA256 Credential=ASIA3Z6DBA73JGQCRW7R/20250903/af-south-1/s3/aws4_request, SignedHeaders=host;x-amz-content-sha256;x-amz-date;x-amz-security-token, Signature="<redacted>"'}>
2025-09-03 13:43:35,511 - MainThread - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): test-ricky-config-bucket-us-east-1.s3.af-south-1.amazonaws.com:443
2025-09-03 13:43:36,723 - MainThread - urllib3.connectionpool - DEBUG - https://test-ricky-config-bucket-us-east-1.s3.af-south-1.amazonaws.com:443 "HEAD / HTTP/1.1" 400 0
2025-09-03 13:43:36,723 - MainThread - botocore.hooks - DEBUG - Event before-parse.s3.HeadBucket: calling handler <function _handle_200_error at 0x10354f9c0>
2025-09-03 13:43:36,723 - MainThread - botocore.hooks - DEBUG - Event before-parse.s3.HeadBucket: calling handler <function handle_expires_header at 0x10354f7e0>
2025-09-03 13:43:36,723 - MainThread - botocore.parsers - DEBUG - Response headers: {'x-amz-bucket-region': 'us-east-1', 'x-amz-request-id': '6N3CFWV5PTX5KMKQ', 'x-amz-id-2': 'IvPsNWbV62oQCY3Of0GgOqwU/9OJ4aqG9cIKQUQOtNU48cqd1ALfuoJ0EJpxrjYia2wASSzD5A19ePSL7gk1EYeCUCDk791U', 'Content-Type': 'application/xml', 'Transfer-Encoding': 'chunked', 'Date': 'Wed, 03 Sep 2025 03:43:36 GMT', 'Connection': 'close', 'Server': 'AmazonS3'}
2025-09-03 13:43:36,723 - MainThread - botocore.parsers - DEBUG - Response body:
b''
2025-09-03 13:43:36,726 - MainThread - botocore.hooks - DEBUG - Event needs-retry.s3.HeadBucket: calling handler <function _update_status_code at 0x10354fb00>
2025-09-03 13:43:36,726 - MainThread - botocore.hooks - DEBUG - Event needs-retry.s3.HeadBucket: calling handler <bound method RetryHandler.needs_retry of <botocore.retries.standard.RetryHandler object at 0x10503f4d0>>
2025-09-03 13:43:36,726 - MainThread - botocore.retries.standard - DEBUG - Not retrying request.
2025-09-03 13:43:36,726 - MainThread - botocore.hooks - DEBUG - Event needs-retry.s3.HeadBucket: calling handler <bound method S3RegionRedirectorv2.redirect_from_error of <botocore.utils.S3RegionRedirectorv2 object at 0x10503f620>>
2025-09-03 13:43:36,726 - MainThread - botocore.utils - DEBUG - S3 client configured for region af-south-1 but the bucket test-ricky-config-bucket-us-east-1 is in region us-east-1; Please configure the proper region to avoid multiple unnecessary redirects and signing attempts.
2025-09-03 13:43:36,727 - MainThread - botocore.hooks - DEBUG - Event before-endpoint-resolution.s3: calling handler <function customize_endpoint_resolver_builtins at 0x10354f740>
2025-09-03 13:43:36,727 - MainThread - botocore.hooks - DEBUG - Event before-endpoint-resolution.s3: calling handler <bound method S3RegionRedirectorv2.redirect_from_cache of <botocore.utils.S3RegionRedirectorv2 object at 0x10503f620>>
2025-09-03 13:43:36,727 - MainThread - botocore.regions - DEBUG - Calling endpoint provider with parameters: {'Bucket': 'test-ricky-config-bucket-us-east-1', 'Region': 'us-east-1', 'UseFIPS': False, 'UseDualStack': False, 'ForcePathStyle': False, 'Accelerate': False, 'UseGlobalEndpoint': False, 'DisableMultiRegionAccessPoints': False, 'UseArnRegion': True}
2025-09-03 13:43:36,727 - MainThread - botocore.regions - DEBUG - Endpoint provider result: https://test-ricky-config-bucket-us-east-1.s3.us-east-1.amazonaws.com
2025-09-03 13:43:36,727 - MainThread - botocore.utils - DEBUG - Updating URI from https://test-ricky-config-bucket-us-east-1.s3.af-south-1.amazonaws.com/ to https://test-ricky-config-bucket-us-east-1.s3.us-east-1.amazonaws.com/
2025-09-03 13:43:36,727 - MainThread - botocore.regions - DEBUG - Selecting from endpoint provider's list of auth schemes: "sigv4". User selected auth scheme is: "None"
2025-09-03 13:43:36,728 - MainThread - botocore.regions - DEBUG - Selected auth type "v4" as "v4" with signing context params: {'region': 'us-east-1', 'signing_name': 's3', 'disableDoubleEncoding': True}
2025-09-03 13:43:36,728 - MainThread - botocore.endpoint - DEBUG - Response received to retry, sleeping for 0 seconds
2025-09-03 13:43:36,728 - MainThread - botocore.hooks - DEBUG - Event request-created.s3.HeadBucket: calling handler <bound method RequestSigner.handler of <botocore.signers.RequestSigner object at 0x10503cc20>>
2025-09-03 13:43:36,728 - MainThread - botocore.hooks - DEBUG - Event choose-signer.s3.HeadBucket: calling handler <function set_operation_specific_signer at 0x10354cfe0>
2025-09-03 13:43:36,728 - MainThread - botocore.hooks - DEBUG - Event before-sign.s3.HeadBucket: calling handler <function remove_arn_from_signing_path at 0x10354f6a0>
2025-09-03 13:43:36,728 - MainThread - botocore.hooks - DEBUG - Event before-sign.s3.HeadBucket: calling handler <function _set_extra_headers_for_unsigned_request at 0x10354fe20>
2025-09-03 13:43:36,728 - MainThread - botocore.hooks - DEBUG - Event before-sign.s3.HeadBucket: calling handler <bound method S3ExpressIdentityResolver.resolve_s3express_identity of <botocore.utils.S3ExpressIdentityResolver object at 0x10503f770>>
2025-09-03 13:43:36,728 - MainThread - botocore.auth - DEBUG - Calculating signature using v4 auth.
2025-09-03 13:43:36,728 - MainThread - botocore.auth - DEBUG - CanonicalRequest:
HEAD
/

host:test-ricky-config-bucket-us-east-1.s3.us-east-1.amazonaws.com
x-amz-content-sha256:e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
x-amz-date:20250903T034336Z
x-amz-security-token:"<redacted>"

host;x-amz-content-sha256;x-amz-date;x-amz-security-token
e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
2025-09-03 13:43:36,728 - MainThread - botocore.auth - DEBUG - StringToSign:
AWS4-HMAC-SHA256
20250903T034336Z
20250903/us-east-1/s3/aws4_request
6794c7423fa533cba4ee9fca6cb2a9b41c780b7465223b6b852228c1a61b64b5
2025-09-03 13:43:36,728 - MainThread - botocore.auth - DEBUG - Signature:
37a17fe527f5c288fdde54d46f706142a75511d34fed20f739cf24653dd285a2
2025-09-03 13:43:36,729 - MainThread - botocore.hooks - DEBUG - Event request-created.s3.HeadBucket: calling handler <bound method UserAgentString.rebuild_and_replace_user_agent_handler of <botocore.useragent.UserAgentString object at 0x105864690>>
2025-09-03 13:43:36,729 - MainThread - botocore.endpoint - DEBUG - Sending http request: <AWSPreparedRequest stream_output=False, method=HEAD, url=https://test-ricky-config-bucket-us-east-1.s3.us-east-1.amazonaws.com/, headers={'User-Agent': b'aws-cli/2.28.1 md/awscrt#0.26.1 ua/2.1 os/macos#24.5.0 md/arch#arm64 lang/python#3.13.5 md/pyimpl#CPython m/E,Z,b cfg/retry-mode#standard md/installer#source md/prompt#off md/command#s3api.head-bucket', 'X-Amz-Date': b'20250903T034336Z', 'X-Amz-Security-Token': b'"<redacted>"', 'X-Amz-Content-SHA256': b'e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855', 'Authorization': b'AWS4-HMAC-SHA256 Credential=ASIA3Z6DBA73JGQCRW7R/20250903/us-east-1/s3/aws4_request, SignedHeaders=host;x-amz-content-sha256;x-amz-date;x-amz-security-token, Signature=37a17fe527f5c288fdde54d46f706142a75511d34fed20f739cf24653dd285a2'}>
2025-09-03 13:43:36,729 - MainThread - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): test-ricky-config-bucket-us-east-1.s3.us-east-1.amazonaws.com:443
2025-09-03 13:43:37,623 - MainThread - urllib3.connectionpool - DEBUG - https://test-ricky-config-bucket-us-east-1.s3.us-east-1.amazonaws.com:443 "HEAD / HTTP/1.1" 200 0
2025-09-03 13:43:37,623 - MainThread - botocore.hooks - DEBUG - Event before-parse.s3.HeadBucket: calling handler <function _handle_200_error at 0x10354f9c0>
2025-09-03 13:43:37,623 - MainThread - botocore.hooks - DEBUG - Event before-parse.s3.HeadBucket: calling handler <function handle_expires_header at 0x10354f7e0>
2025-09-03 13:43:37,623 - MainThread - botocore.parsers - DEBUG - Response headers: {'x-amz-id-2': 'mUPsOPOlLOx0gjTstuRKFWk4V+FUT2/62/Og4NY1xauPZUOcEgVaNrXGayBiGmfwh3oDgIroDNg=', 'x-amz-request-id': 'YD2RYKJ08YNRFX4G', 'Date': 'Wed, 03 Sep 2025 03:43:38 GMT', 'x-amz-bucket-region': 'us-east-1', 'x-amz-access-point-alias': 'false', 'Content-Type': 'application/xml', 'Transfer-Encoding': 'chunked', 'Server': 'AmazonS3'}
2025-09-03 13:43:37,623 - MainThread - botocore.parsers - DEBUG - Response body:
b''
2025-09-03 13:43:37,624 - MainThread - botocore.hooks - DEBUG - Event needs-retry.s3.HeadBucket: calling handler <function _update_status_code at 0x10354fb00>
2025-09-03 13:43:37,624 - MainThread - botocore.hooks - DEBUG - Event needs-retry.s3.HeadBucket: calling handler <bound method RetryHandler.needs_retry of <botocore.retries.standard.RetryHandler object at 0x10503f4d0>>
2025-09-03 13:43:37,624 - MainThread - botocore.retries.standard - DEBUG - Not retrying request.
2025-09-03 13:43:37,624 - MainThread - botocore.hooks - DEBUG - Event needs-retry.s3.HeadBucket: calling handler <bound method S3RegionRedirectorv2.redirect_from_error of <botocore.utils.S3RegionRedirectorv2 object at 0x10503f620>>
2025-09-03 13:43:37,624 - MainThread - botocore.utils - DEBUG - S3 request was previously redirected, not redirecting.
2025-09-03 13:43:37,624 - MainThread - botocore.hooks - DEBUG - Event after-call.s3.HeadBucket: calling handler <function enhance_error_msg at 0x104a8b600>
2025-09-03 13:43:37,624 - MainThread - botocore.hooks - DEBUG - Event after-call.s3.HeadBucket: calling handler <bound method RetryQuotaChecker.release_retry_quota of <botocore.retries.standard.RetryQuotaChecker object at 0x10503e3c0>>
2025-09-03 13:43:37,625 - MainThread - awscli.formatter - DEBUG - RequestId: YD2RYKJ08YNRFX4G
{
    "BucketRegion": "us-east-1",
    "AccessPointAlias": false
}

Expected Behavior

The HeadBucket command using the AWS SDK fails with a 400 error code when it is expected to succeed.

Possible Solution

No response

Additional Information/Context

No response

[kuhe]

I'm checking with my team to see whether we will support this. While we strive to have feature parity between all AWS SDKs and the AWS CLI, there are cases like this where a certain feature is not necessarily a recommended usage pattern and support is not extended.

---

S3 is returning 400 for this request because the region is newer. For other combinations of older regions, such as us-east-1 to us-west-2, S3 returns 301 and the JS SDK follows the redirect. The returning of 400 signals that perhaps they don't really want callers to make cross-region requests of this nature.

In general we recommend discovering the bucket region before making a regional request if you don't know it ahead of time:

import { S3Client, HeadBucketCommand, GetBucketLocationCommand } from "@aws-sdk/client-s3";

const sharedConfig = {};
const s3Clients = {};

function getS3ClientForRegion(region) {
  const client =
    s3Clients[region] ??
    (s3Clients[region] = new S3Client({
      ...sharedConfig,
      region,
    }));
  return client;
}

const bucketLocation = await s3.send(
  new GetBucketLocationCommand({
    Bucket: "bucket in another region",
  })
);
const regionalS3 = await getS3ClientForRegion(bucketLocation.LocationConstraint);

await regionalS3
  .send(
    new HeadBucketCommand({
      Bucket: "bucket in another region",
    })
  );

[rickyrattlesnake]

Thanks for the reply, @kuhe. I'm currently using GetBucketLocationCommand as a workaround, which works in all regions with a cross-region bucket , but it presumes I have permissions for that bucket which we can't rely on for my use case. The docs specifically mention using HeadBucketCommand and that the former is no longer best practice.

What also confused me was that the aws cli (which I believe is from the Python SDK) seems to work for this case, while the JavaScript SDK doesn't.

I attached a proxy to the SDK script, and also the cli to inspect the HTTP Request and Response sent by both clients.

It seems the AWS CLI:

• First makes a request to HEAD https://test-ricky-config-bucket-us-east-1.s3.af-south-1.amazonaws.com/ which responds with 400 but with a Response Header x-amz-bucket-region | us-east-1
• Makes a second request to HEAD https://test-ricky-config-bucket-us-east-1.s3.us-east-1.amazonaws.com/ which responds with 200

It seems the AWS SDK script:

• Makes only one request to HEAD https://test-ricky-config-bucket-us-east-1.s3.af-south-1.amazonaws.com/ which responds with 400 but with a Response Header x-amz-bucket-region | us-east-1

This validates what you were saying earlier. I guess its up to how the client interprets retrying a 400 error.

In the regionRedirectMiddleware it seems we're handling this error when the http status code is 400 but we're also checking a error name (which won't be parsed on a HEAD response - see docs) so I suppose this handling was for other commands that fail with 400 status codes.

Boto-core handles this as a special case here, which I believe is why the CLI performs the redirect.

[kuhe]

After discussing this with my team, I believe we should support this redirect case. It would be unusual for other redirects to work and this very specific case with newer regions not to also work.

[smilkuri]

A fix has been merged and is available in @aws-sdk/middleware-sdk-s3 version 3.883.0
https://www.npmjs.com/package/@aws-sdk/middleware-sdk-s3/v/3.883.0