-
Notifications
You must be signed in to change notification settings - Fork 2.3k
Description
Summary
The OTLP handler in Thanos Receive does not export created timestamps (_created metrics) even though theotlptranslator supports this functionality via the ExportCreatedMetric setting. This causes start_time_unix_nano from OTLP cumulative counters to be discarded, preventing proper counter reset detection and rate calculations.
Problem Details
The settings initialization in the function convertToPrometheusFormat() (File: handler_otlp.go) is missing ExportCreatedMetric:
settings := otlptranslator.Settings{
AddMetricSuffixes: true,
DisableTargetInfo: !h.options.OtlpEnableTargetInfo,
PromoteResourceAttributes: h.options.OtlpResourceAttributes,
// ExportCreatedMetric is not set (defaults to false)
}Expected behavior:
When an OTLP cumulative counter with start_time_unix_nano is ingested, Thanos should create a corresponding _created metric (e.g., my_counter_total_created) containing the start timestamp.
Actual behavior:
The start_time_unix_nano field is silently discarded. Only the main counter metric is stored.
Impact
Without created timestamps:
- Counter resets cannot be detected properly
- Rate calculations may be incorrect
Proposed Solution
Enable created timestamp export in pkg/receive/handler_otlp.go:
settings := otlptranslator.Settings{
AddMetricSuffixes: true,
DisableTargetInfo: !h.options.OtlpEnableTargetInfo,
PromoteResourceAttributes: h.options.OtlpResourceAttributes,
ExportCreatedMetric: true, // ADD THIS LINE
}This leverages existing functionality in the otlptranslator package without requiring additional code changes.