apm: Document sampling.tail.discard_on_write_failure config

Author: isaacaflores2

reference/apm/cloud/apm-settings.md

@@ -53,6 +53,10 @@ If a setting is not supported by {{ech}}, you will get an error message when you
 Some settings that could break your cluster if set incorrectly are blocklisted. The following settings are generally safe in cloud environments. For detailed information about APM settings, check the [APM documentation](/solutions/observability/apm/configure-apm-server.md).
 ::::
 
+### Version 9.1+ [ec_version_9_1]
+
+`apm-server.sampling.tail.discard_on_write_failure`
+:   Defines the indexing behavior when trace events fail to be written to storage (e.g. when the storage limit is reached). When set to `false`, traces will be indexed, significantly increasing the indexing load. When set to `true`, traces will be discarded, there will be data loss potentially resulting in broken traces. The default is `false`. 
 
 ### Version 8.0+ [ec_version_8_0_3]
 

solutions/observability/apm/configure-apm-server.md

@@ -77,6 +77,11 @@ If a setting is not supported on {{ecloud}}, you will get an error message when
 Some settings that could break your cluster if set incorrectly are blocklisted. The following settings are generally safe in cloud environments. For detailed information about APM settings, check the [APM documentation](/solutions/observability/apm/configure-apm-server.md).
 ::::
 
+### Version 9.1+ [ec_version_9_1]
+
+`apm-server.sampling.tail.discard_on_write_failure`
+:   Defines the indexing behavior when trace events fail to be written to storage (e.g. when the storage limit is reached). When set to `false`, traces will be indexed, significantly increasing the indexing load. When set to `true`, traces will be discarded, there will be data loss potentially resulting in broken traces. The default is `false`.
+
 ### Version 8.0+ [ec_version_8_0_3]
 
 This stack version removes support for some previously supported settings. These are all of the supported settings for this version:

solutions/observability/apm/tail-based-sampling.md

@@ -85,6 +85,18 @@ Policies map trace events to a sample rate. Each policy must specify a sample ra
 | APM Server binary | `sampling.tail.policies` |
 | Fleet-managed | `Policies` |
 
+### Discard On Write Failure [sampling-tail-discard-on-write-failure-ref]
+
+Defines the indexing behavior when trace events fail to be written to storage (e.g. when the storage limit is reached). When set to `false`, traces will be indexed, significantly increasing the indexing load. When set to `true`, traces will be discarded, there will be data loss potentially resulting in broken traces.
+
+Default: `false`. (bool)
+
+|                              |                                          |
+|------------------------------|------------------------------------------|
+| APM Server binary            | `sampling.tail.discard_on_write_failure` |
+| Fleet-managed (version 9.1+) | `Discard On Write Failure`               |
+
+
 ### Storage limit [sampling-tail-storage_limit-ref]
 
 The amount of storage space allocated for trace events matching tail sampling policies. Caution: Setting this limit higher than the allowed space may cause APM Server to become unhealthy.
@@ -93,7 +105,7 @@ A value of `0GB` (or equivalent) does not set a concrete limit, but rather allow
 
 If this is not desired, a concrete `GB` value can be set for the maximum amount of disk used for tail-based sampling.
 
-If the configured storage limit is insufficient, it logs "configured limit reached". The event will bypass sampling and will always be indexed when storage limit is reached.
+If the configured storage limit is insufficient, it logs "configured limit reached". When the storage limit is reached, the event will be indexed or discarded based on the [Discard On Write Failure](#sampling-tail-discard-on-write-failure-ref) configuration.
 
 Default: `0GB`. (text)