elastic · shainaraskas · Nov 28, 2024 · Nov 29, 2024 · Nov 29, 2024 · Dec 2, 2024
diff --git a/docs/reference/ccr/index.asciidoc b/docs/reference/ccr/index.asciidoc
@@ -1,6 +1,7 @@
 [role="xpack"]
 [[xpack-ccr]]
 == {ccr-cap}
+
 With {ccr}, you can replicate indices across clusters to:
 
 * Continue handling search requests in the event of a datacenter outage

diff --git a/docs/reference/data-management.asciidoc b/docs/reference/data-management.asciidoc
@@ -4,27 +4,8 @@
 
 [partintro]
 --
-The data you store in {es} generally falls into one of two categories:
 
-* *Content*: a collection of items you want to search, such as a catalog of products
-* *Time series data*: a stream of continuously-generated timestamped data, such as log entries
-
-*Content* might be frequently updated,
-but the value of the content remains relatively constant over time.
-You want to be able to retrieve items quickly regardless of how old they are.
-
-*Time series data* keeps accumulating over time, so you need strategies for
-balancing the value of the data against the cost of storing it.
-As it ages, it tends to become less important and less-frequently accessed,
-so you can move it to less expensive, less performant hardware.
-For your oldest data, what matters is that you have access to the data.
-It's ok if queries take longer to complete.
-
-To help you manage your data, {es} offers you the following options:
-
-* <<index-lifecycle-management, {ilm-cap}>>
-* <<data-stream-lifecycle, Data stream lifecycle>>
-* {curator-ref-current}/about.html[Elastic Curator]
+include::{es-ref-dir}/lifecycle-options.asciidoc[]
 
 **{ilm-init}** can be used to manage both indices and data streams. It allows you to do the following:
 
@@ -40,17 +21,17 @@ for your older indices while reducing operating costs and maintaining search per
 
 * Define the retention period of your data. The retention period is the minimum time your data will be stored in {es}.
 Data older than this period can be deleted by {es} at a later time.
-* Improve the performance of your data stream by performing background operations that will optimise the way your data stream is stored.
+* Improve the performance of your data stream by performing background operations that will optimize the way your data stream is stored.
 
 **Elastic Curator** is a tool that allows you to manage your indices and snapshots using user-defined filters and predefined actions. If ILM provides the functionality to manage your index lifecycle, and you have at least a Basic license, consider using ILM in place of Curator. Many stack components make use of ILM by default. {curator-ref-current}/ilm.html[Learn more].
 
-NOTE: <<xpack-rollup,Data rollup>> is a deprecated {es} feature that allows you to manage the amount of data that is stored in your cluster, similar to the downsampling functionality of {ilm-init} and data stream lifecycle. This feature should not be used for new deployments.
+NOTE: <<xpack-rollup,data rollup>> is a deprecated Elasticsearch feature that allows you to manage the amount of data that is stored in your cluster, similar to the downsampling functionality of {ilm-init} and data stream lifecycle. This feature should not be used for new deployments.
 
 [TIP]
 ====
 {ilm-init} is not available on {es-serverless}.
 
-In an {ecloud} or self-managed environment, ILM lets you automatically transition indices through data tiers according to your performance needs and retention requirements. This allows you to balance hardware costs with performance. {es-serverless} eliminates this complexity by optimizing your cluster performance for you.
+In an {cloud} or self-managed environment, ILM lets you automatically transition indices through data tiers according to your performance needs and retention requirements. This allows you to balance hardware costs with performance. {es-serverless} eliminates this complexity by optimizing your cluster performance for you.
 
 Data stream lifecycle is an optimized lifecycle tool that lets you focus on the most common lifecycle management needs, without unnecessary hardware-centric concepts like data tiers.
 ====

diff --git a/docs/reference/data-store-architecture.asciidoc b/docs/reference/data-store-architecture.asciidoc
@@ -9,16 +9,10 @@ from any node.
 The topics in this section provides information about the architecture of {es} and how it stores and retrieves data: 
 
 * <<nodes-shards,Nodes and shards>>: Learn about the basic building blocks of an {es} cluster, including nodes, shards, primaries, and replicas.
-* <<node-roles-overview,Node roles>>: Learn about the different roles that nodes can have in an {es} cluster.
 * <<docs-replication,Reading and writing documents>>: Learn how {es} replicates read and write operations across shards and shard copies.
 * <<shard-allocation-relocation-recovery,Shard allocation, relocation, and recovery>>: Learn how {es} allocates and balances shards across nodes.
-** <<shard-allocation-awareness,Shard allocation awareness>>: Learn how to use custom node attributes to distribute shards across different racks or availability zones.
-* <<shard-request-cache,Shard request cache>>: Learn how {es} caches search requests to improve performance.
 --
 
 include::nodes-shards.asciidoc[]
-include::node-roles.asciidoc[]
 include::docs/data-replication.asciidoc[leveloffset=-1]
 include::modules/shard-ops.asciidoc[]
-include::modules/cluster/allocation_awareness.asciidoc[leveloffset=+1]
-include::shard-request-cache.asciidoc[leveloffset=-1]
diff --git a/docs/reference/docs.asciidoc b/docs/reference/docs.asciidoc
@@ -1,12 +1,6 @@
 [[docs]]
 == Document APIs
 
-.New API reference
-[sidebar]
---
-For the most up-to-date API details, refer to {api-es}/group/endpoint-document[Document APIs].
---
-
 This section describes the following CRUD APIs:
 
 .Single document APIs

diff --git a/docs/reference/high-availability-overview.asciidoc b/docs/reference/high-availability-overview.asciidoc
@@ -0,0 +1,20 @@
+Your data is important to you. Keeping it safe and available is important to Elastic. Sometimes your cluster may experience hardware failure or a power loss. To help you plan for this, {es} offers a number of features to achieve high availability despite failures. Depending on your deployment type, you might need to provision servers in different zones or configure external repositories to meet your organization's availability needs.
+
+* *<<high-availability-cluster-design,Design for resilience>>* 
++
+Distributed systems like Elasticsearch are designed to keep working even if some of their components have failed. An Elasticsearch cluster can continue operating normally if some of its nodes are unavailable or disconnected, as long as there are enough well-connected nodes to take over the unavailable node's responsibilities.
++
+If you're designing a smaller cluster, you might focus on making your cluster resilient to single-node failures. Designers of larger clusters must also consider cases where multiple nodes fail at the same time.
+// need to improve connections to ECE, EC hosted, ECK pod/zone docs in the child topics
+* *<<xpack-ccr,Cross-cluster replication>>*
++
+To effectively distribute read and write operations across nodes, the nodes in a cluster need good, reliable connections to each other. To provide better connections, you typically co-locate the nodes in the same data center or nearby data centers.
++
+Co-locating nodes in a single location exposes you to the risk of a single outage taking your entire cluster offline. To maintain high availability, you can prepare a second cluster that can take over in case of disaster by implementing {ccr} (CCR).
++
+CCR provides a way to automatically synchronize indices from a leader cluster to a follower cluster. This cluster could be in a different data center or even a different content from the leader cluster. If the primary cluster fails, the secondary cluster can take over.
++
+TIP: You can also use CCR to create secondary clusters to serve read requests in geo-proximity to your users.
+* *<<snapshot-restore,Snapshots>>* 
++
+Take snapshots of your cluster that can be restored in case of failure.
diff --git a/docs/reference/high-availability.asciidoc b/docs/reference/high-availability.asciidoc
@@ -3,28 +3,7 @@
 
 [partintro]
 --
-Your data is important to you. Keeping it safe and available is important to Elastic. Sometimes your cluster may experience hardware failure or a power loss. To help you plan for this, {es} offers a number of features to achieve high availability despite failures. Depending on your deployment type, you might need to provision servers in different zones or configure external repositories to meet your organization's availability needs.
-
-* *<<high-availability-cluster-design,Design for resilience>>* 
-+
-Distributed systems like Elasticsearch are designed to keep working even if some of their components have failed. An Elasticsearch cluster can continue operating normally if some of its nodes are unavailable or disconnected, as long as there are enough well-connected nodes to take over the unavailable node's responsibilities.
-+
-If you're designing a smaller cluster, you might focus on making your cluster resilient to single-node failures. Designers of larger clusters must also consider cases where multiple nodes fail at the same time.
-// need to improve connections to ECE, EC hosted, ECK pod/zone docs in the child topics
-
-* *<<xpack-ccr,Cross-cluster replication>>*
-+
-To effectively distribute read and write operations across nodes, the nodes in a cluster need good, reliable connections to each other. To provide better connections, you typically co-locate the nodes in the same data center or nearby data centers.
-+
-Co-locating nodes in a single location exposes you to the risk of a single outage taking your entire cluster offline. To maintain high availability, you can prepare a second cluster that can take over in case of disaster by implementing {ccr} (CCR).
-+
-CCR provides a way to automatically synchronize indices from a leader cluster to a follower cluster. This cluster could be in a different data center or even a different content from the leader cluster. If the primary cluster fails, the secondary cluster can take over.
-+
-TIP: You can also use CCR to create secondary clusters to serve read requests in geo-proximity to your users.
-
-* *<<snapshot-restore,Snapshots>>* 
-+
-Take snapshots of your cluster that can be restored in case of failure.
+include::{es-ref-dir}/high-availability-overview.asciidoc[]
 --
 
 include::high-availability/cluster-design.asciidoc[]
diff --git a/docs/reference/index.asciidoc b/docs/reference/index.asciidoc
@@ -60,21 +60,25 @@ include::geospatial-analysis.asciidoc[]
 
 include::watcher/index.asciidoc[]
 
-// cluster management
-
-include::monitoring/index.asciidoc[]
+// production tasks
 
-include::security/index.asciidoc[]
+include::production.asciidoc[]
 
-// production tasks
+include::how-to.asciidoc[]
 
 include::high-availability.asciidoc[]
 
-include::how-to.asciidoc[]
+include::snapshot-restore/index.asciidoc[]
+
+include::ccr/index.asciidoc[leveloffset=-1]
 
 include::autoscaling/index.asciidoc[]
 
-include::snapshot-restore/index.asciidoc[]
+// cluster management
+
+include::security/index.asciidoc[]
+
+include::monitoring/index.asciidoc[]
 
 include::ccr/index.asciidoc[leveloffset=-1]
 

diff --git a/docs/reference/intro.asciidoc b/docs/reference/intro.asciidoc
@@ -374,88 +374,3 @@ Does not yet support full-text search.
 | N/A
 
 |===
-
-// New html page
-[[scalability]]
-=== Get ready for production
-
-Many teams rely on {es} to run their key services. To keep these services running, you can design your {es} deployment
-to keep {es} available, even in case of large-scale outages. To keep it running fast, you also can design your
-deployment to be responsive to production workloads.
-
-{es} is built to be always available and to scale with your needs. It does this using a distributed architecture.
-By distributing your cluster, you can keep Elastic online and responsive to requests.
-
-In case of failure, {es} offers tools for cross-cluster replication and cluster snapshots that can
-help you fall back or recover quickly. You can also use cross-cluster replication to serve requests based on the
-geographic location of your users and your resources.
-
-{es} also offers security and monitoring tools to help you keep your cluster highly available.
-
-[discrete]
-[[use-multiple-nodes-shards]]
-==== Use multiple nodes and shards
-
-When you move to production, you need to introduce multiple nodes and shards to your cluster. Nodes and shards are what make {es} distributed and scalable. The size and number of these nodes and shards depends on your data, your use case, and your budget.
-
-These concepts aren't essential if you're just getting started. How you <<elasticsearch-intro-deploy,deploy {es}>> in production determines what you need to know:
-
-* *Self-managed {es}*: You are responsible for setting up and managing nodes, clusters, shards, and replicas. This includes
-managing the underlying infrastructure, scaling, and ensuring high availability through failover and backup strategies.
-* *Elastic Cloud*: Elastic can autoscale resources in response to workload changes. Choose from different deployment types
-to apply sensible defaults for your use case. A basic understanding of nodes, shards, and replicas is still important.
-* *Elastic Cloud Serverless*: You don't need to worry about nodes, shards, or replicas. These resources are 100% automated
-on the serverless platform, which is designed to scale with your workload.
-
-Learn more about <<nodes-shards,nodes and shards>>.
-
-[discrete]
-[[ccr-disaster-recovery-geo-proximity]]
-==== CCR for disaster recovery and geo-proximity
-
-To effectively distribute read and write operations across nodes, the nodes in a cluster need good, reliable connections
-to each other. To provide better connections, you typically co-locate the nodes in the same data center or nearby data centers.
-
-Co-locating nodes in a single location exposes you to the risk of a single outage taking your entire cluster offline. To
-maintain high availability, you can prepare a second cluster that can take over in case of disaster by implementing
-cross-cluster replication (CCR).
-
-CCR provides a way to automatically synchronize indices from your primary cluster to a secondary remote cluster that
-can serve as a hot backup. If the primary cluster fails, the secondary cluster can take over.
-
-You can also use CCR to create secondary clusters to serve read requests in geo-proximity to your users.
-
-Learn more about <<xpack-ccr,cross-cluster replication>> and about <<high-availability-cluster-design,designing for resilience>>.
-
-[TIP]
-====
-You can also take <<snapshot-restore,snapshots>> of your cluster that can be restored in case of failure.
-====
-
-[discrete]
-[[security-and-monitoring]]
-==== Security and monitoring
-
-As with any enterprise system, you need tools to secure, manage, and monitor your {es} clusters. Security,
-monitoring, and administrative features that are integrated into {es} enable you to use {kibana-ref}/introduction.html[Kibana] as a
-control center for managing a cluster.
-
-<<secure-cluster,Learn about securing an {es} cluster>>.
-
-<<monitor-elasticsearch-cluster,Learn about monitoring your cluster>>.
-
-[discrete]
-[[cluster-design]]
-==== Cluster design
-
-{es} offers many options that allow you to configure your cluster to meet your organization’s goals, requirements,
-and restrictions. You can review the following guides to learn how to tune your cluster to meet your needs:
-
-* <<high-availability-cluster-design,Designing for resilience>>
-* <<tune-for-indexing-speed,Tune for indexing speed>>
-* <<tune-for-search-speed,Tune for search speed>>
-* <<tune-for-disk-usage,Tune for disk usage>>
-* <<use-elasticsearch-for-time-series-data,Tune for time series data>>
-
-Many {es} options come with different performance considerations and trade-offs. The best way to determine the
-optimal configuration for your use case is through https://www.elastic.co/elasticon/conf/2016/sf/quantitative-cluster-sizing[testing with your own data and queries].
diff --git a/docs/reference/lifecycle-options.asciidoc b/docs/reference/lifecycle-options.asciidoc
@@ -0,0 +1,21 @@
+The data you store in {es} generally falls into one of two categories:
+
+* *Content*: a collection of items you want to search, such as a catalog of products
+* *Time series data*: a stream of continuously-generated timestamped data, such as log entries
+
+*Content* might be frequently updated,
+but the value of the content remains relatively constant over time.
+You want to be able to retrieve items quickly regardless of how old they are.
+
+*Time series data* keeps accumulating over time, so you need strategies for
+balancing the value of the data against the cost of storing it.
+As it ages, it tends to become less important and less-frequently accessed,
+so you can move it to less expensive, less performant hardware.
+For your oldest data, what matters is that you have access to the data.
+It's ok if queries take longer to complete.
+
+To help you manage your data, {es} offers you the following options:
+
+* <<index-lifecycle-management, {ilm-cap}>>
+* <<data-stream-lifecycle, Data stream lifecycle>>
+* {curator-ref-current}/about.html[Elastic Curator]
diff --git a/docs/reference/nodes-shards.asciidoc b/docs/reference/nodes-shards.asciidoc
@@ -4,14 +4,19 @@
 [NOTE]
 ====
 Nodes and shards are what make {es} distributed and scalable.
+
 These concepts aren't essential if you're just getting started. How you <<elasticsearch-intro-deploy,deploy {es}>> in production determines what you need to know:
 
-* *Self-managed {es}*: You are responsible for setting up and managing nodes, clusters, shards, and replicas. This includes managing the underlying infrastructure, scaling, and ensuring high availability through failover and backup strategies.
-* *Elastic Cloud*: Elastic can autoscale resources in response to workload changes. Choose from different deployment types to apply sensible defaults for your use case. A basic understanding of nodes, shards, and replicas is still important.
-* *Elastic Cloud Serverless*: You don't need to worry about nodes, shards, or replicas. These resources are 100% automated on the serverless platform, which is designed to scale with your workload.
+* *Self-managed {es}*: You are responsible for setting up and managing nodes, clusters, shards, and replicas. This includes
+managing the underlying infrastructure, scaling, and ensuring high availability through failover and backup strategies.
+* *Elastic Cloud*: Elastic can autoscale resources in response to workload changes. Choose from different deployment types
+to apply sensible defaults for your use case. A basic understanding of nodes, shards, and replicas is still important.
+* *Elastic Cloud Serverless*: You don't need to worry about nodes, shards, or replicas. These resources are 100% automated
+on the serverless platform, which is designed to scale with your workload.
 ====
 
-You can add servers (_nodes_) to a cluster to increase capacity, and {es} automatically distributes your data and query load across all of the available nodes.
+You can add servers (_nodes_) to a cluster to increase capacity, and {es} automatically distributes your data and query load
+across all of the available nodes.
 
 Elastic is able to distribute your data across nodes by subdividing an index into _shards_. Each index in {es} is a grouping
 of one or more physical shards, where each shard is a self-contained Lucene index containing a subset of the documents in
@@ -40,4 +45,4 @@ with multiple shards to increase performance and resilience.
 
 * To learn about optimizing the number and size of shards in your cluster, refer to <<size-your-shards,Size your shards>>. 
 * To learn about how read and write operations are replicated across shards and shard copies, refer to <<docs-replication,Reading and writing documents>>.
-* To adjust how shards are allocated and balanced across nodes, refer to <<shard-allocation-relocation-recovery,Shard allocation, relocation, and recovery>>.
+* To adjust how shards are allocated and balanced across nodes, refer to <<shard-allocation-relocation-recovery,Shard allocation, relocation, and recovery>>.