Skip to content
Merged
Changes from 2 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 20 additions & 0 deletions solutions/observability/apm/apm-server/tail-based-sampling.md
Original file line number Diff line number Diff line change
Expand Up @@ -180,6 +180,26 @@ This metric tracks the total number of events dropped by the tail-based sampler.

This is a counter metric so, should be visualized with `counter_rate`.

### `apm-server.sampling.tail.events.failed_writes` [sampling-tail-monitoring-events-failed-writes-ref]

This metric tracks the total number of events that failed to be written to the tail-based sampling storage. Failed writes typically occur when the storage limit is reached or when there are issues with the local sampling database.

The value of this metric should be 0 if tail-based sampling is functioning properly. If it is consistently increasing, check for misconfigured [storage limit](#sampling-tail-storage_limit-ref).

This is a counter metric so, should be visualized with `counter_rate`.

### `apm-server.sampling.tail.events.sampled` [sampling-tail-monitoring-events-sampled-ref]

This metric tracks the total number of events that were sampled (kept) by the tail-based sampler after applying the configured policies and were selected for indexing. This includes all events that belong to traces that matched tail-based sampling policies.

This is a counter metric so, should be visualized with `counter_rate`.

### `apm-server.sampling.tail.events.head_unsampled` [sampling-tail-monitoring-events-head-unsampled-ref]

This metric tracks the total number of events that were already unsampled by head-based sampling before reaching the tail-based sampler. These events are processed by the tail-based sampler but are not stored or indexed because they were already filtered out by head-based sampling decisions.

This is a counter metric so, should be visualized with `counter_rate`.

### `apm-server.sampling.tail.storage.lsm_size` [sampling-tail-monitoring-storage-lsm-size-ref]

This metric tracks the storage size of the log-structured merge trees used by the tail-based sampling database in bytes. Starting in version 9.0.0, this metric is effectively equal to the total storage size used by the database. This is the most crucial metric to track storage requirements for tail-based sampler, especially for big deployments with large distributed traces. Deployments using tail-based sampling extensively should set up alerts and monitoring on this metric.
Expand Down
Loading