dangling-indices-move

Author: shainaraskas

docs/reference/commands/node-tool.asciidoc

@@ -309,7 +309,7 @@ Unsafe cluster bootstrapping is only possible if there is at least one
 surviving master-eligible node. If there are no remaining master-eligible nodes
 then the cluster metadata is completely lost. However, the individual data
 nodes also contain a copy of the index metadata corresponding with their shards. This sometimes allows a new cluster to import these shards as
-<<dangling-indices,dangling indices>>. You can sometimes
+<<dangling-indices-api,dangling indicesi>>. You can sometimes
 recover some indices after the loss of all main-eligible nodes in a cluster
 by creating a new cluster and then using the `elasticsearch-node
 detach-cluster` command to move any surviving nodes into this new cluster. Once the new cluster is fully formed,

docs/reference/data-streams/downsampling.asciidoc

@@ -72,6 +72,45 @@ the granularity of `cold` archival data to monthly or less.
 .Downsampled metrics series
 image::images/data-streams/time-series-downsampled.png[align="center"]
 
+[discrete]
+[[downsample-api-process]]
+==== The downsampling process
+
+The downsampling operation traverses the source TSDS index and performs the
+following steps:
+
+. Creates a new document for each value of the `_tsid` field and each
+`@timestamp` value, rounded to the `fixed_interval` defined in the downsample
+configuration.
+. For each new document, copies all <<time-series-dimension,time
+series dimensions>> from the source index to the target index. Dimensions in a
+TSDS are constant, so this is done only once per bucket.
+. For each <<time-series-metric,time series metric>> field, computes aggregations
+for all documents in the bucket. Depending on the metric type of each metric
+field a different set of pre-aggregated results is stored:
+
+** `gauge`: The `min`, `max`, `sum`, and `value_count` are stored; `value_count`
+is stored as type `aggregate_metric_double`.
+** `counter`: The `last_value` is stored.
+. For all other fields, the most recent value is copied to the target index.
+
+[discrete]
+[[downsample-api-mappings]]
+==== Source and target index field mappings
+
+Fields in the target, downsampled index are created based on fields in the
+original source index, as follows:
+
+. All fields mapped with the `time-series-dimension` parameter are created in
+the target downsample index with the same mapping as in the source index.
+. All fields mapped with the `time_series_metric` parameter are created
+in the target downsample index with the same mapping as in the source
+index. An exception is that for fields mapped as `time_series_metric: gauge`
+the field type is changed to `aggregate_metric_double`.
+. All other fields that are neither dimensions nor metrics (that is, label
+fields), are created in the target downsample index with the same mapping
+that they had in the source index.
+
 [discrete]
 [[running-downsampling]]
 === Running downsampling on time series data

docs/reference/indices/downsample-data-stream.asciidoc

@@ -75,6 +75,8 @@ DELETE _index_template/*
 ////
 // end::downsample-example[]
 
+Check the <<downsampling,Downsampling>> documentation for an overview, details about the downsampling process, and examples of running downsampling manually and as part of an ILM policy.
+
 [[downsample-api-request]]
 ==== {api-request-title}
 
@@ -115,44 +117,4 @@ to aggregate the original time series index. For example, `60m` produces a
 document for each 60 minute (hourly) interval. This follows standard time
 formatting syntax as used elsewhere in {es}.
 +
-NOTE: Smaller, more granular intervals take up proportionally more space.
-
-[[downsample-api-process]]
-==== The downsampling process
-
-The downsampling operation traverses the source TSDS index and performs the
-following steps:
-
-. Creates a new document for each value of the `_tsid` field and each
-`@timestamp` value, rounded to the `fixed_interval` defined in the downsample
-configuration.
-. For each new document, copies all <<time-series-dimension,time
-series dimensions>> from the source index to the target index. Dimensions in a
-TSDS are constant, so this is done only once per bucket.
-. For each <<time-series-metric,time series metric>> field, computes aggregations
-for all documents in the bucket. Depending on the metric type of each metric
-field a different set of pre-aggregated results is stored:
-
-** `gauge`: The `min`, `max`, `sum`, and `value_count` are stored; `value_count`
-is stored as type `aggregate_metric_double`.
-** `counter`: The `last_value` is stored.
-. For all other fields, the most recent value is copied to the target index.
-
-[[downsample-api-mappings]]
-==== Source and target index field mappings
-
-Fields in the target, downsampled index are created based on fields in the
-original source index, as follows:
-
-. All fields mapped with the `time-series-dimension` parameter are created in
-the target downsample index with the same mapping as in the source index.
-. All fields mapped with the `time_series_metric` parameter are created
-in the target downsample index with the same mapping as in the source
-index. An exception is that for fields mapped as `time_series_metric: gauge`
-the field type is changed to `aggregate_metric_double`.
-. All other fields that are neither dimensions nor metrics (that is, label
-fields), are created in the target downsample index with the same mapping
-that they had in the source index.
-
-Check the <<downsampling,Downsampling>> documentation for an overview and
-examples of running downsampling manually and as part of an ILM policy.
+NOTE: Smaller, more granular intervals take up proportionally more space.

docs/reference/modules/gateway.asciidoc

@@ -36,17 +36,4 @@ These settings can be configured in `elasticsearch.yml` as follows:
 gateway.expected_data_nodes: 3
 gateway.recover_after_time: 600s
 gateway.recover_after_data_nodes: 3
---------------------------------------------------
-
-[[dangling-indices]]
-==== Dangling indices
-
-When a node joins the cluster, if it finds any shards stored in its local
-data directory that do not already exist in the cluster, it will consider
-those shards to belong to a "dangling" index. You can list, import or
-delete dangling indices using the <<dangling-indices-api,Dangling indices
-API>>.
-
-NOTE: The API cannot offer any guarantees as to whether the imported data
-truly represents the latest state of the data when the index was still part
-of the cluster.
+--------------------------------------------------

docs/reference/modules/indices/fielddata.asciidoc

@@ -5,10 +5,6 @@ The field data cache contains <<fielddata-mapping-param, field data>> and <<eage
 which are both used to support aggregations on certain field types.
 Since these are on-heap data structures, it is important to monitor the cache's use.
 
-[discrete]
-[[fielddata-sizing]]
-==== Cache size
-
 The entries in the cache are expensive to build, so the default behavior is
 to keep the cache loaded in memory. The default cache size is unlimited,
 causing the cache to grow until it reaches the limit set by the <<fielddata-circuit-breaker, field data circuit breaker>>. This behavior can be configured.
@@ -20,16 +16,12 @@ at the cost of rebuilding the cache as needed.
 If the circuit breaker limit is reached, further requests that increase the cache
 size will be prevented. In this case you should manually <<indices-clearcache, clear the cache>>.
 
+TIP: You can monitor memory usage for field data as well as the field data circuit
+breaker using
+the <<cluster-nodes-stats,nodes stats API>> or the <<cat-fielddata,cat fielddata API>>.
+
 `indices.fielddata.cache.size`::
 (<<static-cluster-setting,Static>>)
 The max size of the field data cache, eg `38%` of node heap space, or an
 absolute value, eg `12GB`. Defaults to unbounded. If you choose to set it,
-it should be smaller than <<fielddata-circuit-breaker>> limit.
-
-[discrete]
-[[fielddata-monitoring]]
-==== Monitoring field data
-
-You can monitor memory usage for field data as well as the field data circuit
-breaker using
-the <<cluster-nodes-stats,nodes stats API>> or the <<cat-fielddata,cat fielddata API>>.
+it should be smaller than <<fielddata-circuit-breaker>> limit.

docs/reference/modules/node.asciidoc

@@ -420,7 +420,7 @@ Similarly, each master-eligible node maintains the following data on disk:
 
 Each node checks the contents of its data path at startup. If it discovers
 unexpected data then it will refuse to start. This is to avoid importing
-unwanted <<dangling-indices,dangling indices>> which can lead
+unwanted <<dangling-indices-api,dangling indices>> which can lead
 to a red cluster health. To be more precise, nodes without the `data` role will
 refuse to start if they find any shard data on disk at startup, and nodes
 without both the `master` and `data` roles will refuse to start if they have any

docs/reference/monitoring/index.asciidoc

@@ -9,6 +9,7 @@ performance of your {es} cluster.
 
 * <<monitoring-overview>>
 * <<how-monitoring-works>>
+* <<configuring-logging>>
 * <<monitoring-production>>
 * <<configuring-elastic-agent>>
 * <<configuring-metricbeat>>
@@ -23,6 +24,8 @@ include::overview.asciidoc[]
 
 include::how-monitoring-works.asciidoc[]
 
+include::setup/logging-config.asciidoc[]
+
 include::production.asciidoc[]
 
 include::configuring-elastic-agent.asciidoc[]

docs/reference/redirects.asciidoc

@@ -1215,7 +1215,7 @@ See <<search-search>>.
 [role="exclude",id="modules-gateway-dangling-indices"]
 === Dangling indices
 
-See <<dangling-indices>>.
+See <<dangling-indices-api>>.
 
 [role="exclude",id="shards-allocation"]
 === Cluster-level shard allocation

docs/reference/setup.asciidoc

@@ -65,8 +65,6 @@ include::settings/license-settings.asciidoc[]
 
 include::modules/gateway.asciidoc[]
 
-include::setup/logging-config.asciidoc[]
-
 include::settings/ml-settings.asciidoc[]
 
 include::settings/inference-settings.asciidoc[]