Method to set up Amazon Bedrock and Claude 3.7 Sonnet in LiteLLM

[hongbo-miao]

Just share the method allowing LiteLLM to access Amazon Bedrock Claude 3.7 Sonnet.

In Amazon Bedrock, Claude 3.7 Sonnet only supports cross-region inference as of today.

Based on https://docs.aws.amazon.com/bedrock/latest/userguide/inference-profiles-support.html, cross-region inference allows you to manage unplanned traffic bursts by utilizing compute across different AWS regions. With cross-region inference, you can distribute traffic across multiple AWS Regions. So cross-region is actually a good thing, and you see what models support cross-region inference in that link.

From the link, we can see Claude 3.7 Sonnet exists in other US regions us-east-1, us-east-2, us-west-2:

Assuming you want to access the Claude 3.7 Sonnet in these US regions (us-east-1, us-east-2, us-west-2) from us-west-2, you could set up like

IAM

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Action": [
                "bedrock:InvokeModel",
                "bedrock:InvokeModelWithResponseStream"
            ],
            "Effect": "Allow",
            "Resource": [
                "arn:aws:bedrock:us-west-2:{yourAwsAccountId}:inference-profile/us.anthropic.claude-3-7-sonnet-20250219-v1:0",
                "arn:aws:bedrock:*::foundation-model/anthropic.claude-3-7-sonnet-20250219-v1:0"
            ]
        }
    ]
}

LiteLLM config.yaml

model_list:
  - model_name: claude-3-7-sonnet
    litellm_params:
      model: bedrock/us.anthropic.claude-3-7-sonnet-20250219-v1:0
      aws_region_name: us-west-2

Full Deployment Code

The full deployment code can be found here

• https://github.com/hongbo-miao/hongbomiao.com/blob/a18f3ef7ba7034180af4c22bc362d1628456fbc8/cloud-infrastructure/terraform/modules/kubernetes/hm_litellm_iam_role/main.tf
• https://github.com/hongbo-miao/hongbomiao.com/blob/a18f3ef7ba7034180af4c22bc362d1628456fbc8/kubernetes/argo-cd/applications/production-hm/litellm/kubernetes-manifests/hm-litellm-config-map.yaml

Hopefully it helps some people save some time ☺️