Skip to content

Fix AccessDeniedException when using AmazonBedrockModel in async evaluations#2471

Open
Fizza-Mukhtar wants to merge 2 commits intoconfident-ai:mainfrom
Fizza-Mukhtar:fix/amazon-bedrock-async-session
Open

Fix AccessDeniedException when using AmazonBedrockModel in async evaluations#2471
Fizza-Mukhtar wants to merge 2 commits intoconfident-ai:mainfrom
Fizza-Mukhtar:fix/amazon-bedrock-async-session

Conversation

@Fizza-Mukhtar
Copy link

Bug Fix: AmazonBedrockModel fails in async evaluation context

This PR fixes an issue where AmazonBedrockModel raises an
AccessDeniedException when used inside DeepEval’s async evaluation
pipeline, despite valid AWS credentials and confirmed Bedrock access.

Problem

  • AmazonBedrockModel worked with direct boto3 calls
  • Failed only inside DeepEval async evaluations
  • Error: AccessDeniedException during Converse calls
  • Root cause: async session / credential handling was not aligned with
    aiobotocore best practices in concurrent contexts

Solution

  • Ensure aiobotocore session handling is async-safe
  • Avoid credential loss during concurrent async evaluations
  • Keep behavior unchanged for sync execution paths

Testing

  • Verified behavior with mocked aiobotocore async sessions
  • Confirmed no regression for non-Bedrock models

This brings Bedrock behavior in line with other async-supported LLM providers in DeepEval.
Fixes #1916

@vercel
Copy link

vercel bot commented Feb 3, 2026

@Fizza-Mukhtar is attempting to deploy a commit to the Confident AI Team on Vercel.

A member of the Team first needs to authorize it.

@greptile-apps
Copy link
Contributor

greptile-apps bot commented Feb 3, 2026

PR author is not in the allowed authors list.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

AccessDeniedException with AmazonBedrockModel in async evaluation context

1 participant