Skip to content

Commit fc2ec52

Browse files
authored
update amazon bedrock configuration example (#1116)
- updated Amazon Bedrock model example to Sonnet 3.7 - Added clarifying note about `AWS_REGION` config setting - Added clarifying note about IMDSv2 `HttpPutResponseHopLimit` instance metadata config
1 parent 548a414 commit fc2ec52

File tree

1 file changed

+20
-14
lines changed

1 file changed

+20
-14
lines changed

docs/cody/enterprise/model-config-examples.mdx

Lines changed: 20 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -695,34 +695,34 @@ In the configuration above,
695695
"displayName": "Anthropic models through AWS Bedrock",
696696
"serverSideConfig": {
697697
"type": "awsBedrock",
698-
"accessToken": "<base64 encoded IAM username:password",
698+
"accessToken": "<ACCESS_KEY_ID>:<SECRET_ACCESS_KEY>",
699699
"endpoint": "<VPC Endpoint URL>",
700700
"region": "us-west-2"
701701
}
702702
}
703703
],
704704
"modelOverrides": [
705705
{
706-
"modelRef": "aws-bedrock::2024-02-29::claude-3-sonnet",
707-
"displayName": "Claude 3 Sonnet",
708-
"modelName": "claude-3-sonnet",
706+
"modelRef": "aws-bedrock::2025-02-19::claude-3-7-sonnet",
707+
"displayName": "Claude 3.7 Sonnet",
708+
"modelName": "anthropic.claude-3-7-sonnet-20250219-v1:0",
709709
"serverSideConfig": {
710710
"type": "awsBedrockProvisionedThroughput",
711711
"arn": "<ARN>" // e.g., arn:aws:bedrock:us-west-2:537452198621:provisioned-model/57z3lgkt1cx2
712712
},
713713
"contextWindow": {
714-
"maxInputTokens": 16000,
715-
"maxOutputTokens": 4000
714+
"maxInputTokens": 132000,
715+
"maxOutputTokens": 8192
716716
},
717717
"capabilities": ["chat", "autocomplete"],
718718
"category": "balanced",
719719
"status": "stable"
720720
},
721721
],
722722
"defaultModels": {
723-
"chat": "aws-bedrock::2024-02-29::claude-3-sonnet",
724-
"codeCompletion": "aws-bedrock::2024-02-29::claude-3-sonnet",
725-
"fastChat": "aws-bedrock::2024-02-29::claude-3-sonnet"
723+
"chat": "aws-bedrock::2025-02-19::claude-3-7-sonnet",
724+
"codeCompletion": "aws-bedrock::2025-02-19::claude-3-7-sonnet",
725+
"fastChat": "aws-bedrock::2025-02-19::claude-3-7-sonnet"
726726
},
727727
}
728728
```
@@ -731,7 +731,7 @@ In the configuration described above,
731731

732732
- Set up a provider override for Amazon Bedrock, routing requests for this provider directly to the specified endpoint, bypassing Cody Gateway
733733
- Add the `"aws-bedrock::2024-02-29::claude-3-sonnet"` model, which is used for all Cody features. We do not add other models for simplicity, as adding multiple models is already covered in the examples above
734-
- Note: Since the model in the example uses provisioned throughput, specify the ARN in the `serverSideConfig.arn` field of the model override.
734+
- Since the model in the example uses [Amazon Bedrock provisioned throughput](https://docs.aws.amazon.com/bedrock/latest/userguide/prov-throughput.html), specify the ARN in the `serverSideConfig.arn` field of the model override.
735735

736736
Provider override `serverSideConfig` fields:
737737

@@ -740,18 +740,24 @@ Provider override `serverSideConfig` fields:
740740
| `type` | Must be `"awsBedrock"`. |
741741
| `accessToken` | Leave empty to rely on instance role bindings or other AWS configurations in the frontend service. Use `<ACCESS_KEY_ID>:<SECRET_ACCESS_KEY>` for direct credential configuration, or `<ACCESS_KEY_ID>:<SECRET_ACCESS_KEY>:<SESSION_TOKEN>` if a session token is also required. |
742742
| `endpoint` | For pay-as-you-go, set it to an AWS region code (e.g., `us-west-2`) when using a public Amazon Bedrock endpoint. For provisioned throughput, set it to the provisioned VPC endpoint for the bedrock-runtime API (e.g., `https://vpce-0a10b2345cd67e89f-abc0defg.bedrock-runtime.us-west-2.vpce.amazonaws.com`). |
743-
| `region` | The region to use when configuring API clients. This is necessary because the 'frontend' binary container cannot access environment variables from the host OS. |
743+
| `region` | The region to use when configuring API clients. The `AWS_REGION` Environment variable must also be configured in the `sourcegraph-frontend` container to match. |
744744

745745
Provisioned throughput for Amazon Bedrock models can be configured using the `"awsBedrockProvisionedThroughput"` server-side configuration type. Refer to the [Model Overrides](/cody/enterprise/model-configuration#model-overrides) section for more details.
746746

747+
<Callout type="note">
748+
If using [IAM roles for EC2 / instance role binding](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/iam-roles-for-amazon-ec2.html),
749+
you may need to increase the [HttpPutResponseHopLimit
750+
](https://docs.aws.amazon.com/AWSEC2/latest/APIReference/API_InstanceMetadataOptionsRequest.html#:~:text=HttpPutResponseHopLimit) instance metadata option to a higher value (e.g., 2) to ensure that the metadata service can be accessed from the frontend container running in the EC2 instance. See [here](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/configuring-IMDS-existing-instances.html) for instructions.
751+
</Callout>
752+
747753
<Callout type="warning">
748754
We only recommend configuring AWS Bedrock to use an accessToken for
749755
authentication. Specifying no accessToken (e.g. to use [IAM roles for EC2 /
750756
instance role
751757
binding](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/iam-roles-for-amazon-ec2.html))
752-
is not currently recommended (there is a known performance bug with this
753-
method which will prevent autocomplete from working correctly. (internal
754-
issue: PRIME-662)
758+
is not currently recommended. There is a known performance bug with this
759+
method which will prevent autocomplete from working correctly (internal
760+
issue: CORE-819)
755761
</Callout>
756762

757763
</Accordion>

0 commit comments

Comments
 (0)