Skip to content

Conversation

jmcphers
Copy link
Collaborator

@jmcphers jmcphers commented Oct 1, 2025

Adds support for new Anthropic models on Bedrock (Claude 4.5 Sonnet & Claude 4.1 Opus).

Also removes Claude 3.5 as it's soon to be deprecated: https://aws.amazon.com/blogs/machine-learning/migrate-from-anthropics-claude-3-5-sonnet-to-claude-4-sonnet-on-amazon-bedrock/

Addresses #9643, #9339

@jmcphers jmcphers requested a review from timtmok October 1, 2025 00:11
Copy link

github-actions bot commented Oct 1, 2025

E2E Tests 🚀
This PR will run tests tagged with: @:critical

readme  valid tags

@jmcphers jmcphers marked this pull request as draft October 1, 2025 00:16
@jmcphers
Copy link
Collaborator Author

jmcphers commented Oct 1, 2025

It looks like Bedrock doesn't yet allow us to query the new models directly.

image

Instead you need to create an inference profile. It's not clear whether this is something we need to adapt to in the API or whether this is just a restriction on very new models to help gate/balance access.

@timtmok
Copy link
Contributor

timtmok commented Oct 1, 2025

I get the same error after checking that the model is accessible in AWS. I also saw there's an inference profile set for Claude 4.5 that was recently added. I think it's a configuration problem in AWS.

image

@timtmok
Copy link
Contributor

timtmok commented Oct 15, 2025

So it does work when using the ARN that's set in the inference profile. All that had to be done is use the ARN in place of the model id. The Bedrock SDK should be able to query for available models and get the ARN. I haven't looked into when it's necessary to use the ARN as well as how to query if the model is accessible. My model queries so far have returned far more models that I would expect (looks to be all models from various providers).

image

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants