Skip to content

Bug in resizing buckets of a native histogram #2270

Description

@pchks


It should be int index = (keys[i] > 0 ? keys[i] + 1 : keys[i]) / 2. (See https://github.com/prometheus/client_golang/blob/832cec72badf6ff6ad3bd5fed4338e4e27fd646b/prometheus/histogram.go#L1119, noting that Java handles negative integer division in the same way as Go.)

Because of this bug, when doubleBucketWidth() happens, counters of some buckets are merged into wrong new buckets (more specifically, the counter of a bucket with a negative even index is merged into a new bucket that is to the right of the correct one). This causes histogram_count(rate(my_native_histogram[1m])) (RPS calculation) to glitch with a weird spike, as Prometheus may treat such random change in counters as a reset. On the other hand, histogram_quantile(..., my_native_histogram) probably is not affected significantly (i.e. painfully visibly), as (some of) counters are merely shifted to adjacent buckets, and adjacent buckets have quite close boundaries, e.g. x1.044 for schema=4, I guess.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions