OTLP: determine target data stream based on attributes and scope name #133903
Conversation
|
Pinging @elastic/es-storage-engine (Team:StorageEngine) |
elasticsearchmachine
added
external-contributor
.../src/main/java/org/elasticsearch/xpack/oteldata/otlp/datapoint/DataPointGroupingContext.java
Outdated
Show resolved
Hide resolved
...gin/otel-data/src/main/java/org/elasticsearch/xpack/oteldata/otlp/datapoint/TargetIndex.java
Outdated
Show resolved
Hide resolved
...gin/otel-data/src/main/java/org/elasticsearch/xpack/oteldata/otlp/datapoint/TargetIndex.java
Show resolved
Hide resolved
...gin/otel-data/src/main/java/org/elasticsearch/xpack/oteldata/otlp/datapoint/TargetIndex.java
Show resolved
Hide resolved
...l-data/src/main/java/org/elasticsearch/xpack/oteldata/otlp/tsid/AttributeListTsidFunnel.java
Outdated
Show resolved
Hide resolved
...gin/otel-data/src/main/java/org/elasticsearch/xpack/oteldata/otlp/datapoint/TargetIndex.java
Outdated
Show resolved
Hide resolved
| if (isPopulated()) { | ||
| return; | ||
| } | ||
| for (int i = 0, size = attributes.size(); i < size; i++) { |
There was a problem hiding this comment.
I wonder if these should be ordered e.g. lexicographically, so that naming is deterministic in the presence of multiple matching attributes.
There was a problem hiding this comment.
Sorting may be pretty expensive. But I see what you mean. If both elasticsearch.index and data_stream.* attributes are set, it depends on the ordering. We can probably fix that by defining a precedence and only checking for isPopulated in the beginning of the method.
There was a problem hiding this comment.
Wouldn't this still use the first match on DATA_STREAM_DATASET or DATA_STREAM_NAMESPACE? These are user-provided, so the list order is arbitrary? Maybe a unittest will help here.
There was a problem hiding this comment.
I'm not sure I understand. The DATA_STREAM_DATASET and DATA_STREAM_NAMESPACE attributes populate different properties. The order doesn't matter as we're iterating over all attributes.
...tel-data/src/test/java/org/elasticsearch/xpack/oteldata/otlp/datapoint/TargetIndexTests.java
Outdated
Show resolved
Hide resolved
|
Let's update the title and description to replace |
...ta/src/main/java/org/elasticsearch/xpack/oteldata/otlp/docbuilder/MetricDocumentBuilder.java
Outdated
Show resolved
Hide resolved
...ta/src/main/java/org/elasticsearch/xpack/oteldata/otlp/docbuilder/MetricDocumentBuilder.java
Outdated
Show resolved
Hide resolved
felixbarny
changed the title
Properly determines the target index for metric data points based on the following attributes:
data_stream.datasetdata_stream.namespaceelasticsearch.indexThis mimics the "dynamic routing" behavior of the OTel collector's
elasticsearchexporterPart of #133057