Add bedrock contract tests. #227
Merged
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The PR is a follow up on bedrock service support PR:
#209
We add contract tests for following Bedrock services that covers all resource attributes we newly support:
GetGuardrail
GetAgent
,GetDataSource
,GetKnowledgeBase
InvokeModel
InvokeAgent
Upgrade
botocore
andboto3
to the latest version1.34.143
so that to support Bedrock services API calls.Upgrade
localstack/localstack
image to the latest version3.5.0
so resolve the SQS API call issue usinglocalstack/localstack:2.0.1
with new version ofboto3
: localstack/localstack#9610Contract test limitation:
The contract tests in current repo is using LocalStackContainer to serve AWS SDK service calls. But it doesn’t has bedrock related service support (This is the full service list it support.). In this case, no matter which bedrock API we call in contract test, the response will always be 4XX.
As a workaround, we inject
inject_200_success
andinject_500_error
directly into the API call to make sure we receive http response with expected status code and attributes._assert_semantic_conventions_span_attributes
function change:In _assert_semantic_conventions_span_attributes function it is checking the input
service
equals torpc.service
, however, we pass the input service with "remote_service", where there is mismatch for example for Bedrock Agent Runtime: we haverpc_service="Bedrock Agent Runtime", remote_service="AWS::Bedrock".
Thus we change to use
rpc_service
if it is provided by:By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.