images

Author: shainaraskas

docs/reference/data-streams/failure-store-recipes.asciidoc

@@ -310,23 +310,31 @@ Since failure stores can be searched just like a normal data stream, we can use
 If you want to use KQL or Lucene query types, you should first create a data view for your failure store data.
 If you plan to use {esql} or the Query DSL query types, this step is not required.
 Navigate to the data view page in Kibana and add a new data view. Set the index pattern to your failure store using the selector syntax.
+
 image::images/data-streams/failure_store_alerting_create_data_view.png[create a data view using the failure store syntax in the index name]
 
 ===== Step 2: Create new rule
 Navigate to Management / Alerts and Insights / Rules. Create a new rule. Choose the {es} query option.
+
 image::images/data-streams/failure_store_alerting_create_rule.png[create a new alerting rule and select the elasticsearch query option]
 
 ===== Step 3: Pick your query type
 Choose which query type you wish to use
 For KQL/Lucene queries, reference the data view that contains your failure store.
+
 image::images/data-streams/failure_store_alerting_kql.png[use the data view created in the previous step as the input to the kql query]
+
 For Query DSL queries, use the `::failures` suffix on your data stream name.
+
 image::images/data-streams/failure_store_alerting_dsl.png[use the ::failures suffix in the data stream name in the query dsl]
+
 For {esql} queries, use the `::failures` suffix on your data stream name in the `FROM` command.
+
 image::images/data-streams/failure_store_alerting_esql.png[use the ::failures suffix in the data stream name in the from command]
 
 ===== Step 4: Test
 Configure schedule, actions, and details of the alert before saving the rule.
+
 image::images/data-streams/failure_store_alerting_finish.png[complete the rule configuration and save it]
 
 [[data-remediation]]

docs/reference/data-streams/failure-store.asciidoc

@@ -15,12 +15,14 @@ On this page, you'll learn how to set up, use, and manage a failure store, as we
 For examples of how to use failure stores to identify and fix errors in ingest pipelines and your data, refer to <<failure-store-recipes,Using failure stores to address ingestion issues>>.
 
 [discrete]
-=== Set up a data stream failure store [[set-up-failure-store]]
+[[set-up-failure-store]]
+=== Set up a data stream failure store 
 
 Each data stream has its own failure store that can be enabled to accept failed documents. By default, this failure store is disabled and any ingestion problems are raised in the response to write operations.
 
 [discrete]
-==== Set up for new data streams [[set-up-failure-store-new]]
+[[set-up-failure-store-new]]
+==== Set up for new data streams 
 
 You can specify in a data stream's <<index-templates,index templates>> if it should enable the failure store when it is first created.
 
@@ -52,7 +54,8 @@ PUT _index_template/my-index-template
 After a matching data stream is created, its failure store will be enabled.
 
 [discrete]
-==== Set up for existing data streams [[set-up-failure-store-existing]]
+[[set-up-failure-store-existing]]
+==== Set up for existing data streams 
 
 Enabling the failure store via <<index-templates,index templates>> can only affect data streams that are newly created. Existing data streams that use a template are not affected by changes to the template's `data_stream_options` field.
 To modify an existing data stream's options, use the https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-indices-put-data-stream-options[Update data stream options API]:
@@ -82,7 +85,8 @@ PUT _data_stream/my-datastream-existing/_options
 <1> Redirecting failed documents into the failure store will now be disabled.
 
 [discrete]
-==== Enable failure store via cluster setting [[set-up-failure-store-cluster-setting]]
+[[set-up-failure-store-cluster-setting]]
+==== Enable failure store via cluster setting 
 
 If you have a large number of existing data streams you may want to enable their failure stores in one place. Instead of updating each of their options individually, set `data_streams.failure_store.enabled` to a list of index patterns in the https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-cluster-put-settings[cluster settings]. Any data streams that match one of these patterns will operate with their failure store enabled.
 
@@ -111,14 +115,16 @@ PUT _data_stream/my-datastream-1/_options
 <1> The failure store for `my-datastream-1` is disabled even though it matches `my-datastream-*`. The data stream options override the cluster setting.
 
 [discrete]
-=== Using a failure store [[use-failure-store]]
+[[use-failure-store]]
+=== Using a failure store
 
 The failure store is meant to ease the burden of detecting and handling failures when ingesting data to {es}. Clients are less likely to encounter unrecoverable failures when writing documents, and developers are more easily able to troubleshoot faulty pipelines and mappings.
 
 For examples of how to use failure stores to identify and fix errors in ingest pipelines and your data, refer to <<failure-store-recipes,Using failure stores to address ingestion issues>>.
 
 [discrete]
-==== Failure redirection [[use-failure-store-redirect]]
+[[use-failure-store-redirect]]
+==== Failure redirection 
 
 Once a failure store is enabled for a data stream it will begin redirecting documents that fail due to common ingestion problems instead of returning errors in write operations. Clients are notified in a non-intrusive way when a document is redirected to the failure store.
 
@@ -277,7 +283,8 @@ If the document was redirected to a data stream's failure store but that failed
 <5> The response status is `400 Bad Request` due to the original mapping problem.
 
 [discrete]
-==== Searching failures [[use-failure-store-searching]]
+[[use-failure-store-searching]]
+==== Searching failures 
 
 Once you have accumulated some failures, the failure store can be searched much like a regular data stream.
 
@@ -415,7 +422,8 @@ Because the `document.source` field is unmapped, it is absent from the ES|SQL an
 ====
 
 [discrete]
-==== Failure document structure [[use-failure-store-document]]
+[[use-failure-store-document]]
+==== Failure document structure 
 Failure documents have a uniform structure that is handled internally by {es}.
 `@timestamp`:: (`date`) The timestamp at which the document encountered a failure in {es}.
 `document`:: (`object`) The document at time of failure. If the document failed in an ingest pipeline, then the document will be the unprocessed version of the document as it arrived in the original indexing request. If the document failed due to a mapping issue, then the document will be as it was after any ingest pipelines were applied to it.
@@ -681,10 +689,14 @@ Caused by: j.l.IllegalArgumentException: For input string: "this field is invali
 The `document` field attempts to show the effective input to whichever process led to the failure occurring. This gives you all the information you need to reproduce the problem.
 
 [discrete]
-=== Manage a data stream's failure store [[manage-failure-store]]
+[[manage-failure-store]]
+=== Manage a data stream's failure store 
 Failure data can accumulate in a data stream over time. To help manage this accumulation, most administrative operations that can be done on a data stream can be applied to the data stream's failure store.
 
-==== Failure store rollover [[manage-failure-store-rollover]]
+[discrete]
+[[manage-failure-store-rollover]]
+==== Failure store rollover 
+
 A data stream treats its failure store much like a secondary set of <<backing-indices,backing indices>>. Multiple dedicated hidden indices serve search requests for the failure store, while one index acts as the current write index. You can use the https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-indices-rollover[rollover] API to rollover the failure store. Much like the regular indices in a data stream, a new write index will be created in the failure store to accept new failure documents.
 
 [source,console]
@@ -707,7 +719,8 @@ POST my-datastream::failures/_rollover
 ----
 
 [discrete]
-==== Failure store lifecycle [[manage-failure-store-lifecycle]]
+[[manage-failure-store-lifecycle]]
+==== Failure store lifecycle 
 
 Failure stores have their retention managed using an internal <<data-stream-lifecycle,data stream lifecycle>>. A thirty day (30d) retention is applied to failure store data. You can view the active lifecycle for a failure store index by calling the https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-indices-get-data-stream[get data stream API]:
 
@@ -808,7 +821,8 @@ PUT _data_stream/my-datastream/_options
 <2> Set only this data stream's failure store retention to ten days.
 
 [discrete]
-==== Add and remove from failure store [[manage-failure-store-indices]]
+[[manage-failure-store-indices]]
+==== Add and remove from failure store 
 
 Failure stores support adding and removing indices from them using the https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-indices-modify-data-stream[Update data streams] API.