[Question]: how to use AWS Bedrock with AWS profiles auth

[PavelPolyakov]

What is your question?

Anybody can help me to run LibreChat against with AWS Bedrock against AWS profile setup?

More Details

In the documentation it says

You can also omit the access keys in order to use the default AWS credentials chain but you must set the default region

This is what I did in my .env file:

BEDROCK_AWS_DEFAULT_REGION=us-east-1 # A default region must be provided

In our setup I do not have AWS auth env variables (BEDROCK_AWS_ACCESS_KEY_ID, BEDROCK_AWS_SECRET_ACCESS_KEY).

All I have is a profile definition in ~/.aws/config.

[profile bedrock]
sso_session    = bedrock
sso_account_id = %REDACTED%
sso_role_name  = %REDACTED%

I run LibreChat with AWS_PROFILE=bedrock docker-compose up, when I try to talk to the AWS Bedrock model I have the next error:

LibreChat         | 2025-01-27T08:05:42.266Z error: [api/server/controllers/agents/client.js #sendCompletion] Unhandled error type Could not load credentials from any providers
LibreChat         | 2025-01-27T08:05:42.266Z error: [handleAbortError] AI response error; aborting request: Could not load credentials from any providers

Thus it seems it can not authenticate itself against AWS Bedrock.

What is the main subject of your question?

Other

Screenshots

No response

Code of Conduct

• I agree to follow this project's Code of Conduct

[danny-avila]

Docker must have access to the volumes and be mapped to the same locations as your aws config

[PavelPolyakov]

Thank you, I did the next adjustments to docker-compose.yml:

environment:
      - AWS_PROFILE=bedrock
volumes:
      - ~/.aws:/root/.aws:ro

and now I can chat with some models

However I observed, that for the models with "Cross region-inference"

I can not chat. I see this answer in the UI:

And this in the logs:

Invocation of model ID anthropic.claude-3-5-sonnet-20241022-v2:0 with on-demand throughput isn’t supported. Retry your request with the ID or ARN of an inference profile that contains this model.

Is there a way to configure LibreChat to use an ID or ARN?

[danny-avila]

No, that error message is misleading. See this discussion: #4571 (comment)

[PavelPolyakov]

Perfect, thank you for your help. I have it working now!
Thanks a lot 🙇