feat: Add botocore instrumentation extension for Bedrock AgentCore services #490

liustve · 2025-10-10T22:43:49Z

Description of changes:

Patches botocore instrumentor to add span attributes for Bedrock AgentCore control and data plane calls.

Added _BedrockAgentCoreExtension to extract telemetry attributes from API parameters
Registered extensions for bedrock-agentcore (data plane) and bedrock-agentcore-control (control plane)
Added attribute constants for runtime, browser, code interpreter, gateway, memory, and identity resources

New semantic conventions based off this PR:
open-telemetry/semantic-conventions#2886

Manually tested verified span attributes examples:

{
    "name": "Bedrock AgentCore.InvokeAgentRuntime",
    "context": {
        "trace_id": "0x68e9815d69c6335ba2be2c3a0dc9460d",
        "span_id": "0x7b65fdc962dfe773",
        "trace_state": "[]"
    },
    "kind": "SpanKind.CLIENT",
    "parent_id": null,
    "start_time": "2025-10-10T21:57:49.492869Z",
    "end_time": "2025-10-10T21:57:50.070664Z",
    "status": {
        "status_code": "UNSET"
    },
    "attributes": {
        "rpc.system": "aws-api",
        "rpc.service": "Bedrock AgentCore",
        "rpc.method": "InvokeAgentRuntime",
        "aws.region": "us-east-1",
        "server.address": "bedrock-agentcore.us-east-1.amazonaws.com",
        "server.port": 443,
        "aws.auth.region": "us-east-1",
        "aws.auth.account.access_key": "",
        "aws.bedrock.agentcore.runtime.arn": "arn:aws:bedrock-agentcore:us-east-1:571600841604:runtime/completeAgent-w8slyU6q5M",
        "aws.request_id": "a4483186-85ba-4879-b83e-99562da78376",
        "retry_attempts": 0,
        "http.status_code": 200
    },
    "events": [],
    "links": [],
    "resource": {
        "attributes": {
            "telemetry.sdk.language": "python",
            "telemetry.sdk.name": "opentelemetry",
            "telemetry.sdk.version": "1.33.1",
            "service.name": "unknown_service",
            "telemetry.auto.version": "0.12.1.dev0-aws",
            "aws.local.service": "UnknownService"
        },
        "schema_url": ""
    }
}

{
    "name": "Bedrock AgentCore.InvokeCodeInterpreter",
    "context": {
        "trace_id": "0x68e98228e901936ec8fe3d76839e3da3",
        "span_id": "0x1751acb67a5b3073",
        "trace_state": "[]"
    },
    "kind": "SpanKind.CLIENT",
    "parent_id": null,
    "start_time": "2025-10-10T22:01:12.522286Z",
    "end_time": "2025-10-10T22:01:12.691952Z",
    "status": {
        "status_code": "UNSET"
    },
    "attributes": {
        "rpc.system": "aws-api",
        "rpc.service": "Bedrock AgentCore",
        "rpc.method": "InvokeCodeInterpreter",
        "aws.region": "us-east-1",
        "server.address": "bedrock-agentcore.us-east-1.amazonaws.com",
        "server.port": 443,
        "aws.auth.region": "us-east-1",
        "gen_ai.code_interpreter.id": "agentCodeInterpreter-m9Mvuwkg6j",
        "aws.request_id": "b9fd2327-edc4-413d-89d7-715ae836ec64",
        "retry_attempts": 0,
        "http.status_code": 200
    },
    "events": [],
    "links": [],
    "resource": {
        "attributes": {
            "telemetry.sdk.language": "python",
            "telemetry.sdk.name": "opentelemetry",
            "telemetry.sdk.version": "1.33.1",
            "service.name": "unknown_service",
            "telemetry.auto.version": "0.12.1.dev0-aws",
            "aws.local.service": "UnknownService"
        },
        "schema_url": ""
    }
}

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

aws-opentelemetry-distro/src/amazon/opentelemetry/distro/patches/_bedrock_agentcore_patches.py

*Description of changes:* Follow up to: #490 1. Normalizes `AWS::Bedrock AgentCore` and `AWS::Bedrock AgentCore Control` from botocore instrumentation to `AWS::BedrockAgentCore` 2. Add Resource and CFN attributes for Bedrock AgentCore support as following: a. **Runtime**: Extract `agentRuntimeId` or parse from `agentRuntimeArn` and add into `aws.remote.resource.identifier` span attribute. Set `aws.remote.resource.type` to `AWS::BedrockAgentCore::Runtime`. b. **Runtime Endpoint**: Extract endpoint ID from `agentRuntimeEndpointArn` and add into `aws.remote.resource.identifier` span attribute, but use full ARN as `aws.cloudformation.primary.identifier`. Set `aws.remote.resource.type` to `AWS::BedrockAgentCore::RuntimeEndpoint`. c. **Browser**: Extract `browserIdentifier` or parse from `browserArn`, and add into `aws.remote.resource.identifier` span attribute. Set `aws.remote.resource.type` to `AWS::BedrockAgentCore::Browser` (for `aws.browser.v1`) or `AWS::BedrockAgentCore::BrowserCustom` (for custom browsers). d. **Gateway**: Extract `gatewayId` or parse from `gatewayArn`, and add into `aws.remote.resource.identifier` span attribute. Set `aws.remote.resource.type` to `AWS::BedrockAgentCore::Gateway`. e. **Gateway Target**: Extract `targetId` from span attributes and add into `aws.remote.resource.identifier` span attribute. Set `aws.remote.resource.type` to `AWS::BedrockAgentCore::GatewayTarget`. Takes priority over Gateway when present. f. **Code Interpreter**: Extract `codeInterpreterIdentifier` or parse from `codeInterpreterArn`, and add into `aws.remote.resource.identifier` span attribute. Set `aws.remote.resource.type` to `AWS::BedrockAgentCore::CodeInterpreter` (for `aws.codeinterpreter.v1`) or `AWS::BedrockAgentCore::CodeInterpreterCustom` (for custom interpreters). g. **Memory**: Extract `memoryId` or parse from `memoryArn` and add into `aws.remote.resource.identifier` span attribute, but use full ARN as `aws.cloudformation.primary.identifier` when available. Set `aws.remote.resource.type` to `AWS::BedrockAgentCore::Memory`. h. **Identity**: Extract provider ID from credential provider ARNs for OAuth2 and API Key credential providers and add into `aws.remote.resource.identifier` span attribute. Set `aws.remote.resource.type` to `AWS::BedrockAgentCore::OAuth2CredentialProvider` or `AWS::BedrockAgentCore::APIKeyCredentialProvider`. **Testing** 1. Contract tests added to validate E2E behavior of new Application Signals attributes. 2. Unit tests added to validate logic flow is as expected. 3. Manual testing to verify the new spans: ``` { "name": "Bedrock AgentCore.InvokeAgentRuntime", "context": { "trace_id": "0x68f7e9d8113e41fe74ca4bedf2aee364", "span_id": "0xff15834f0aa966ea", "trace_state": "[]" }, "kind": "SpanKind.CLIENT", "parent_id": null, "start_time": "2025-10-21T20:15:20.875227Z", "end_time": "2025-10-21T20:15:21.827053Z", "status": { "status_code": "UNSET" }, "attributes": { "rpc.system": "aws-api", "rpc.service": "Bedrock AgentCore", "rpc.method": "InvokeAgentRuntime", "aws.region": "us-west-2", "server.address": "bedrock-agentcore.us-west-2.amazonaws.com", "server.port": 443, "aws.auth.region": "us-west-2", "aws.auth.account.access_key": "", "aws.bedrock.agentcore.runtime.arn": "arn:aws:bedrock-agentcore:us-west-2:571600841604:runtime/app-B7cysl8fTW", "aws.local.operation": "InternalOperation", "aws.local.service": "test-genesis", "aws.remote.service": "AWS::BedrockAgentCore", "aws.remote.operation": "InvokeAgentRuntime", "aws.remote.resource.type": "AWS::BedrockAgentCore::Runtime", "aws.remote.resource.identifier": "app-B7cysl8fTW", "aws.remote.resource.cfn.primary.identifier": "app-B7cysl8fTW", "aws.remote.resource.account.access_key": "ASIAYKFQQVOCK7JJYIFI", "aws.remote.resource.region": "us-west-2", "aws.span.kind": "LOCAL_ROOT" }, "events": [], "links": [], "resource": { "attributes": { "telemetry.sdk.language": "python", "telemetry.sdk.name": "opentelemetry", "telemetry.sdk.version": "1.33.1", "service.name": "test-genesis", "telemetry.auto.version": "0.12.1.dev0-aws", "aws.local.service": "test-genesis" }, "schema_url": "" } } ``` ``` { "name": "Bedrock AgentCore.CreateEvent", "context": { "trace_id": "0x68f811f6b5b57a276e378106ac9f7bca", "span_id": "0xc60262df93b1c44f", "trace_state": "[]" }, "kind": "SpanKind.CLIENT", "parent_id": "0x98a57a740465b917", "start_time": "2025-10-21T23:06:36.435203Z", "end_time": "2025-10-21T23:06:37.069681Z", "status": { "status_code": "UNSET" }, "attributes": { "rpc.system": "aws-api", "rpc.service": "Bedrock AgentCore", "rpc.method": "CreateEvent", "aws.region": "us-west-2", "server.address": "bedrock-agentcore.us-west-2.amazonaws.com", "server.port": 443, "aws.auth.region": "us-west-2", "aws.auth.account.access_key": "", "gen_ai.memory.id": "app_mem-ZPNcI839KD", "aws.local.operation": "POST /invocations", "aws.request_id": "0eabdad1-6ab4-4efb-98c5-ef1b9bbe028a", "retry_attempts": 1, "http.status_code": 201, "aws.local.service": "test-genesis", "aws.remote.service": "AWS::BedrockAgentCore", "aws.remote.operation": "CreateEvent", "aws.remote.resource.type": "AWS::BedrockAgentCore::Memory", "aws.remote.resource.identifier": "app_mem-ZPNcI839KD", "aws.remote.resource.cfn.primary.identifier": "app_mem-ZPNcI839KD", "aws.remote.resource.account.access_key": "", "aws.remote.resource.region": "us-west-2", "aws.span.kind": "CLIENT" }, "events": [], "links": [], "resource": { "attributes": { "telemetry.sdk.language": "python", "telemetry.sdk.name": "opentelemetry", "telemetry.sdk.version": "1.33.1", "service.name": "test-genesis", "telemetry.auto.version": "0.12.1.dev0-aws", "aws.local.service": "test-genesis" }, "schema_url": "" } } ``` By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice. --------- Co-authored-by: ADOT Patch workflow <[email protected]>

add new bedrock agentcore patching and attributes

be08bab

liustve requested a review from a team as a code owner October 10, 2025 22:43

ADOT Patch workflow and others added 5 commits October 10, 2025 15:45

add changelog

98d567a

Merge branch 'main' into bedrock-agentcore-resource-support

52744ef

lint fix

17716ac

fix arn names

25f91ac

fix arn namse in test

9e39da9

wangzlei reviewed Oct 15, 2025

View reviewed changes

aws-opentelemetry-distro/src/amazon/opentelemetry/distro/patches/_bedrock_agentcore_patches.py Show resolved Hide resolved

wangzlei approved these changes Oct 15, 2025

View reviewed changes

Merge branch 'main' into bedrock-agentcore-resource-support

9c06e50

mxiamxia reviewed Oct 15, 2025

View reviewed changes

aws-opentelemetry-distro/src/amazon/opentelemetry/distro/patches/_bedrock_agentcore_patches.py Show resolved Hide resolved

liustve merged commit 7745ce0 into main Oct 16, 2025
16 checks passed

liustve deleted the bedrock-agentcore-resource-support branch October 16, 2025 01:16

liustve mentioned this pull request Oct 16, 2025

feat: Add Resource and CFN Attributes for Bedrock AgentCore spans #495

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

feat: Add botocore instrumentation extension for Bedrock AgentCore services #490

feat: Add botocore instrumentation extension for Bedrock AgentCore services #490

Uh oh!

liustve commented Oct 10, 2025 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

feat: Add botocore instrumentation extension for Bedrock AgentCore services #490

feat: Add botocore instrumentation extension for Bedrock AgentCore services #490

Uh oh!

Conversation

liustve commented Oct 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

liustve commented Oct 10, 2025 •

edited

Loading