Automatically generate AWS SDK field / attribute mappings

Is your feature request related to a problem? Please describe.
Currently AWS SDK field to span attribute mappings are hard coded for DynamoDB. It would be good to have those generated automatically from a YAML file for all AWS SDK request types.

Describe the solution you'd like
Spec: https://github.com/open-telemetry/opentelemetry-specification/pull/1422/files
Existing prop generator: https://github.com/open-telemetry/opentelemetry-java/blob/main/buildscripts/semantic-convention/generate.sh

Describe alternatives you've considered

• hardcoding

Additional context
Add any other context or screenshots about the feature request here.

@anuraaga FYI :)

[eric-spence-code]

@kubawach, I would be interested in working on this one.

For the auto-generation portion of this, I am thinking of targeting the following types of files. For example:

v1: https://github.com/aws/aws-sdk-java/blob/a73f09cf6f74b5a3385f2ce2622f63bd26f2e80f/aws-java-sdk-models/src/main/resources/models/sqs-2012-11-05-model.json

v2: https://github.com/aws/aws-sdk-java-v2/blob/ee3d58396f580ab5393612f65c9ef55cb08d56cc/services/sqs/src/main/resources/codegen-resources/service-2.json

I am interested if you have any thoughts on not instrumenting specific fields like the message in an SQS SendMessage or the Dynamodb item in a PutItem request? I would assume some deny list in the code-gen file, but curious if you had any thoughts on it or prior art you were thinking about following. Also curious about your opinions on adding hints for the code generator on messaging spec on SQS or DB spec on Dynamodb or aurora?

[pingleig]

For context of anyone finding this issue, the hard coded parameters are

opentelemetry-java-instrumentation/instrumentation/aws-sdk/aws-sdk-2.2/library/src/main/java/io/opentelemetry/instrumentation/awssdk/v2_2/AwsSdkRequest.java

Lines 22 to 122 in 1c1ca91

	/**
	* Temporary solution - maps only DynamoDB attributes. Final solution should be generated from AWS
	* SDK automatically
	* (https://github.com/open-telemetry/opentelemetry-java-instrumentation/issues/2291).
	*/
	// We match the actual name in the AWS SDK for better consistency with it and possible future
	// autogeneration.
	@SuppressWarnings("MemberName")
	enum AwsSdkRequest {
	// generic requests
	DynamoDbRequest(DYNAMODB, "DynamoDbRequest"),
	S3Request(S3, "S3Request"),
	SnsRequest(SNS, "SnsRequest"),
	SqsRequest(SQS, "SqsRequest"),
	KinesisRequest(KINESIS, "KinesisRequest"),
	// specific requests
	BatchGetItem(
	DYNAMODB,
	"BatchGetItemRequest",
	request("aws.dynamodb.table_names", "RequestItems"),
	response("aws.dynamodb.consumed_capacity", "ConsumedCapacity")),
	BatchWriteItem(
	DYNAMODB,
	"BatchWriteItemRequest",
	request("aws.dynamodb.table_names", "RequestItems"),
	response("aws.dynamodb.consumed_capacity", "ConsumedCapacity"),
	response("aws.dynamodb.item_collection_metrics", "ItemCollectionMetrics")),
	CreateTable(
	DYNAMODB,
	"CreateTableRequest",
	request("aws.dynamodb.global_secondary_indexes", "GlobalSecondaryIndexes"),
	request("aws.dynamodb.local_secondary_indexes", "LocalSecondaryIndexes"),
	request(
	"aws.dynamodb.provisioned_throughput.read_capacity_units",
	"ProvisionedThroughput.ReadCapacityUnits"),
	request(
	"aws.dynamodb.provisioned_throughput.write_capacity_units",
	"ProvisionedThroughput.WriteCapacityUnits")),
	DeleteItem(
	DYNAMODB,
	"DeleteItemRequest",
	response("aws.dynamodb.consumed_capacity", "ConsumedCapacity"),
	response("aws.dynamodb.item_collection_metrics", "ItemCollectionMetrics")),
	GetItem(
	DYNAMODB,
	"GetItemRequest",
	request("aws.dynamodb.projection_expression", "ProjectionExpression"),
	response("aws.dynamodb.consumed_capacity", "ConsumedCapacity"),
	request("aws.dynamodb.consistent_read", "ConsistentRead")),
	ListTables(
	DYNAMODB,
	"ListTablesRequest",
	request("aws.dynamodb.exclusive_start_table_name", "ExclusiveStartTableName"),
	response("aws.dynamodb.table_count", "TableNames"),
	request("aws.dynamodb.limit", "Limit")),
	PutItem(
	DYNAMODB,
	"PutItemRequest",
	response("aws.dynamodb.consumed_capacity", "ConsumedCapacity"),
	response("aws.dynamodb.item_collection_metrics", "ItemCollectionMetrics")),
	Query(
	DYNAMODB,
	"QueryRequest",
	request("aws.dynamodb.attributes_to_get", "AttributesToGet"),
	request("aws.dynamodb.consistent_read", "ConsistentRead"),
	request("aws.dynamodb.index_name", "IndexName"),
	request("aws.dynamodb.limit", "Limit"),
	request("aws.dynamodb.projection_expression", "ProjectionExpression"),
	request("aws.dynamodb.scan_index_forward", "ScanIndexForward"),
	request("aws.dynamodb.select", "Select"),
	response("aws.dynamodb.consumed_capacity", "ConsumedCapacity")),
	Scan(
	DYNAMODB,
	"ScanRequest",
	request("aws.dynamodb.attributes_to_get", "AttributesToGet"),
	request("aws.dynamodb.consistent_read", "ConsistentRead"),
	request("aws.dynamodb.index_name", "IndexName"),
	request("aws.dynamodb.limit", "Limit"),
	request("aws.dynamodb.projection_expression", "ProjectionExpression"),
	request("aws.dynamodb.segment", "Segment"),
	request("aws.dynamodb.select", "Select"),
	request("aws.dynamodb.total_segments", "TotalSegments"),
	response("aws.dynamodb.consumed_capacity", "ConsumedCapacity"),
	response("aws.dynamodb.count", "Count"),
	response("aws.dynamodb.scanned_count", "ScannedCount")),
	UpdateItem(
	DYNAMODB,
	"UpdateItemRequest",
	response("aws.dynamodb.consumed_capacity", "ConsumedCapacity"),
	response("aws.dynamodb.item_collection_metrics", "ItemCollectionMetrics")),
	UpdateTable(
	DYNAMODB,
	"UpdateTableRequest",
	request("aws.dynamodb.attribute_definitions", "AttributeDefinitions"),
	request("aws.dynamodb.global_secondary_index_updates", "GlobalSecondaryIndexUpdates"),
	request(
	"aws.dynamodb.provisioned_throughput.read_capacity_units",
	"ProvisionedThroughput.ReadCapacityUnits"),
	request(
	"aws.dynamodb.provisioned_throughput.write_capacity_units",
	"ProvisionedThroughput.WriteCapacityUnits"));

In X-Ray's Java SDK, AWS SDK attributes extractor are also hard coded but using JSON files such as https://github.com/aws/aws-xray-sdk-java/blob/8161976ab0495a1b3d9a1d87a0bbe08fcc75dcc9/aws-xray-recorder-sdk-aws-sdk/src/main/resources/com/amazonaws/xray/handlers/DefaultOperationParameterWhitelist.json#L3 and used in https://github.com/aws/aws-xray-sdk-java/blob/3c15d4030e0381bef9445c2fa2ce78aad184d10e/aws-xray-recorder-sdk-aws-sdk-v2/src/main/java/com/amazonaws/xray/interceptors/TracingInterceptor.java#L143

[trask]

Closing, can revisit later if anyone has renewed interest.