From 03961cc819586e639a18d2e80efb3ba6cd571778 Mon Sep 17 00:00:00 2001 From: Isaac Flores Date: Fri, 27 Jun 2025 14:04:11 -0700 Subject: [PATCH 1/3] [apm]: Document sampling.tail.discard_on_write_failure config --- .../observability/apm/configure/sampling.asciidoc | 14 ++++++++++++++ 1 file changed, 14 insertions(+) diff --git a/docs/en/observability/apm/configure/sampling.asciidoc b/docs/en/observability/apm/configure/sampling.asciidoc index c98f0564e7..85cd8d8942 100644 --- a/docs/en/observability/apm/configure/sampling.asciidoc +++ b/docs/en/observability/apm/configure/sampling.asciidoc @@ -76,6 +76,20 @@ Default: `3GB`. (text) | Fleet-managed | `Storage limit` |==== +[float] +[id="sampling-tail-discard_on_write_failure-{input-type}"] +=== Discard On Write Failure +Defines the indexing behavior when trace events fail to be written to storage (for example, when the storage limit is reached). +When set to `false`, traces bypass sampling and are always indexed, which significantly increases the indexing load. +When set to `true`, traces are discarded, causing data loss which can result in broken traces. + +Default: `false`. (bool) + +|==== +| APM Server binary | `sampling.tail.discard_on_write_failure` +| Fleet-managed (version 8.19+) | `Discard On Write Failure` +|==== + // end::tbs-top[] [float] From 13637d41663858fab8f8798faf3501fb7a816a1d Mon Sep 17 00:00:00 2001 From: Isaac Flores Date: Tue, 1 Jul 2025 14:15:59 -0700 Subject: [PATCH 2/3] [apm]: Add changes to storage limit behavior based on the Discard On Write Failure config --- docs/en/observability/apm/configure/sampling.asciidoc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/en/observability/apm/configure/sampling.asciidoc b/docs/en/observability/apm/configure/sampling.asciidoc index 85cd8d8942..e997dea1fd 100644 --- a/docs/en/observability/apm/configure/sampling.asciidoc +++ b/docs/en/observability/apm/configure/sampling.asciidoc @@ -67,7 +67,7 @@ This final policy is used to catch remaining trace events that don't match a str === Storage limit The amount of storage space allocated for trace events matching tail sampling policies. Caution: Setting this limit higher than the allowed space may cause APM Server to become unhealthy. -If the configured storage limit is insufficient, it logs "configured storage limit reached". The event will bypass sampling and will always be indexed when storage limit is reached. +If the configured storage limit is insufficient, it logs "configured storage limit reached". When the storage limit is reached, the event will be indexed or discarded based on the <> configuration. Default: `3GB`. (text) From 1b981f27f94ee8ae7e93ae5df381c84ab63b7d18 Mon Sep 17 00:00:00 2001 From: Isaac Flores <34590010+isaacaflores2@users.noreply.github.com> Date: Mon, 7 Jul 2025 14:46:18 -0700 Subject: [PATCH 3/3] remove 8.19 version since it is not needed Co-authored-by: Carson Ip --- docs/en/observability/apm/configure/sampling.asciidoc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/en/observability/apm/configure/sampling.asciidoc b/docs/en/observability/apm/configure/sampling.asciidoc index e997dea1fd..60b90536fb 100644 --- a/docs/en/observability/apm/configure/sampling.asciidoc +++ b/docs/en/observability/apm/configure/sampling.asciidoc @@ -87,7 +87,7 @@ Default: `false`. (bool) |==== | APM Server binary | `sampling.tail.discard_on_write_failure` -| Fleet-managed (version 8.19+) | `Discard On Write Failure` +| Fleet-managed | `Discard On Write Failure` |==== // end::tbs-top[]