Skip to content

Conversation

@majanjua-amzn
Copy link
Contributor

Background

As part of our next release, we would like to support Bedrock, Bedrock Agent, Bedrock Runtime, and Bedrock Agent Runtime attributes in our span attributes. As such, we need to add a patch to our implementation to support these services as ServiceExtensions. This PR adds ServiceExtensions for all the listed services and provides unit tests to ensure they are functioning appropriately, as well as manual test evidence showing the attributes appear as expected.

This PR is based heavily on this one made in our aws-observability/aws-otel-python-instrumentation repo.

Description of Changes

General

  • Added Bedrock, Bedrock Agent, Bedrock Runtime, and Bedrock Agent Runtime clients to our devDependencies for unit testing.
  • Ran npm install to generate a new package-lock.json file. This removed many references to licenses; unsure if this is intended. Will follow up offline with those who might know why this happened.

Bedrock

  • Extract guardrailId from API response, and add into aws.bedrock.guardrail.id span attribute.

Bedrock Agent

  • Logic to ensure only one of the following attributes is added based on the incoming operation.
  • Extract agentId from both API request and response, and add into aws.bedrock.agent.id span attribute.
  • Extract knowledgeBaseId from API request, and add into aws.bedrock.knowledge_base.id" span attribute.
  • Extract dataSourceId from both API request and response,, and add into aws.bedrock.data_source.id span attribute.

Bedrock Agent Runtime

  • Extract agentId from API request, and add into aws.bedrock.agent.id span attribute.
  • Extract knowledgeBaseId from API request, and add into aws.bedrock.knowledgebase.id span attribute.

According to the Python PR, there should only be one of these attributes available for each possible API call.

Bedrock Runtime

gen_ai.request.model
gen_ai.system

Populate RemoteResourceType and RemoteResourceIdentifier

The RemoteResourceType and RemoteResourceIdentifier attributes are populated as follows:

"RemoteResourceType": "AWS::Bedrock::Agent"
"RemoteResourceIdentifier": "<agent_id>"

"RemoteResourceType": "AWS::Bedrock::DataSource"
"RemoteResourceIdentifier": "<datasource_id>"

"RemoteResourceType": "AWS::Bedrock::Guardrail"
"RemoteResourceIdentifier": "<guardrail_id>"

"RemoteResourceType": "AWS::Bedrock::KnowledgeBase"
"RemoteResourceIdentifier": "<knowledgeBase_id>"

"RemoteResourceType": "AWS::Bedrock::Model"
"RemoteResourceIdentifier": "<modelId>"

Testing

For this PR, I modified our NodeJS sample application in our Application Signals test framework to add APIs that result in the following traces and metrics for each of the listed use cases.

Traces

Bedrock Agent with Agent ID present (using ListAgentVersions):
Screenshot 2024-09-20 at 6 47 01 PM

Bedrock Agent with Knowledge Base ID present (using GetKnowledgeBase):
Screenshot 2024-09-20 at 6 49 10 PM

Bedrock Agent with Data Source ID present (using GetDataSource):
Screenshot 2024-09-20 at 6 50 33 PM

Bedrock Agent Runtime with Knowledge Base ID present (using Retrieve):
Screenshot 2024-09-20 at 6 51 20 PM

Bedrock with Guardrail ID present (using GetGuardrail):
Screenshot 2024-09-20 at 6 52 01 PM

Bedrock Runtime with Gen AI attributes present (using InvokeModel):
Screenshot 2024-09-20 at 7 12 45 PM

Metrics

Bedrock Agent with Agent ID present (using ListAgentVersions):
Screenshot 2024-09-20 at 6 56 13 PM

Bedrock Agent with Knowledge Base ID present (using GetKnowledgeBase):
Screenshot 2024-09-20 at 6 59 10 PM

Bedrock Agent with Data Source ID present (using GetDataSource):
Screenshot 2024-09-20 at 6 59 40 PM

Bedrock Agent Runtime with Knowledge Base ID present (using Retrieve):
Screenshot 2024-09-20 at 7 00 28 PM

Bedrock with Guardrail ID present (using GetGuardrail):
Screenshot 2024-09-20 at 7 00 57 PM

Bedrock Runtime with Gen AI attributes present (using InvokeModel):
Screenshot 2024-09-20 at 7 13 12 PM

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

@majanjua-amzn majanjua-amzn added the enhancement New feature or request label Sep 21, 2024
@majanjua-amzn majanjua-amzn self-assigned this Sep 21, 2024
@majanjua-amzn majanjua-amzn requested a review from a team as a code owner September 21, 2024 02:28
Copy link
Contributor

@jj22ee jj22ee left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm!

@majanjua-amzn majanjua-amzn merged commit a600740 into main Sep 25, 2024
8 checks passed
@majanjua-amzn majanjua-amzn deleted the bedrock-support branch September 25, 2024 01:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants