Skip to content

time_series_dimension should be ignored for non-TSDB indices #140882

@tonyghiani

Description

@tonyghiani

📓 Summary

OTel logs fail to validate when users modify fields that have been marked with time_series_dimension: true in the built-in component templates. The shared OTel component template applies time series dimension settings to resource.attributes and scope.attributes for both logs and metrics, but this causes a "shadowing" error when fields are modified in logs indices.

When a dynamic field like resource.attributes.service.name gets concretized to a keyword field during schema processing, Elasticsearch rejects it because it conflicts with the parent's time_series_dimension: true marker.

Image

✔️ Acceptance Criteria

  • time_series_dimension settings are ignored or not applied to non-TSDB indices (logs, events, etc.)
  • Users can modify OTel log fields without encountering mapper_parsing_exception for shadowing violations on time_series_dimension
  • Metrics indices continue to correctly apply time_series_dimension to resource.attributes as routing dimensions

🤓 Implementation Details

The OTel templates need differentiation between logs and metrics:

  • Current state: Both logs-otel@mappings and metrics-otel@mappings share the same otel@mappings component template which marks attributes with time_series_dimension: true
  • Expected behavior: The dimension definition should only apply to time series data streams
  • Solution approach: Ensure that Elasticsearch ignores time_series_dimension constraints for indices that are not in TSDB mode, or create separate component templates for logs vs metrics that respect their respective index modes

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions