From b3ef7a04d757f4f08e1badd2f260c5f800127ccd Mon Sep 17 00:00:00 2001 From: David Kilfoyle Date: Wed, 19 Feb 2025 09:50:34 -0500 Subject: [PATCH 1/2] Small fixes to 'data streams' section --- manage-data/data-store/index-types.md | 89 --------------------------- manage-data/toc.yml | 3 +- 2 files changed, 1 insertion(+), 91 deletions(-) delete mode 100644 manage-data/data-store/index-types.md diff --git a/manage-data/data-store/index-types.md b/manage-data/data-store/index-types.md deleted file mode 100644 index 5b2a3a4051..0000000000 --- a/manage-data/data-store/index-types.md +++ /dev/null @@ -1,89 +0,0 @@ ---- -mapped_pages: - - https://www.elastic.co/guide/en/serverless/current/index-management.html ---- - -# Index types [index-management] - -This content applies to: [![Elasticsearch](../../images/serverless-es-badge.svg "")](../../solutions/search.md) [![Observability](../../images/serverless-obs-badge.svg "")](../../solutions/observability.md) [![Security](../../images/serverless-sec-badge.svg "")](../../solutions/security/elastic-security-serverless.md) - -Elastic’s index management features are an easy, convenient way to manage your cluster’s indices, data streams, index templates, and enrich policies. Practicing good index management ensures your data is stored correctly and in the most cost-effective way possible. - - -## Manage indices [index-management-manage-indices] - -Go to **{{project-settings}} → {{manage-app}} → {{index-manage-app}}**: - -:::{image} ../../images/serverless-index-management-indices.png -:alt: {{index-manage-app}} UI -:class: screenshot -::: - -The **{{index-manage-app}}** page contains an overview of your indices. - -* To show details or perform operations, such as delete, click the index name. To perform operations on multiple indices, select their checkboxes and then open the **Manage** menu. -* To filter the list of indices, use the search bar. -* To drill down into the index mappings, settings, and statistics, click an index name. From this view, you can navigate to **Discover** to further explore the documents in the index. - - -## Manage data streams [index-management-manage-data-streams] - -Investigate your data streams and address lifecycle management needs in the **Data Streams** view. - -In {{es-serverless}}, indices matching the `logs-*-*` pattern use the logsDB index mode by default. The logsDB index mode creates a [logs data stream](data-streams/logs-data-stream.md). - -The value in the **Indices** column indicates the number of backing indices. Click this number to drill down into details. - -A value in the data retention column indicates that the data stream is managed by a data stream lifecycle policy. - -This value is the time period for which your data is guaranteed to be stored. Data older than this period can be deleted by {{es}} at a later time. - -:::{image} ../../images/serverless-management-data-stream.png -:alt: Data stream details -:class: screenshot -::: - -To view information about the stream’s backing indices, click the number in the **Indices** column. - -* To view more information about a data stream, such as its generation or its current index lifecycle policy, click the stream’s name. From this view, you can navigate to **Discover** to further explore data within the data stream. -* [preview] To modify the data retention value, select an index, open the **Manage** menu, and click **Edit data retention**. - - -## Manage index templates [index-management-manage-index-templates] - -Create, edit, clone, and delete your index templates in the **Index Templates** view. Changes made to an index template do not affect existing indices. - -:::{image} ../../images/serverless-index-management-index-templates.png -:alt: Index templates -:class: screenshot -::: - -The default **logs** template uses the logsDB index mode to create a [logs data stream](data-streams/logs-data-stream.md). - -If you don’t have any templates, you can create one using the **Create template** wizard. - - -## Manage enrich policies [index-management-manage-enrich-policies] - -Use the **Enrich Policies** view to add data from your existing indices to incoming documents during ingest. An [enrich policy](../ingest/transform-enrich/data-enrichment.md) contains: - -* The policy type that determines how the policy matches the enrich data to incoming documents -* The source indices that store enrich data as documents -* The fields from the source indices used to match incoming documents -* The enrich fields containing enrich data from the source indices that you want to add to incoming documents -* An optional query. - -:::{image} ../../images/serverless-management-enrich-policies.png -:alt: Enrich policies -:class: screenshot -::: - -When creating an enrich policy, the UI walks you through the configuration setup and selecting the fields. Before you can use the policy with an enrich processor, you must execute the policy. - -When executed, an enrich policy uses enrich data from the policy’s source indices to create a streamlined system index called the enrich index. The policy uses this index to match and enrich incoming documents. - -Check out these examples: - -* [Example: Enrich your data based on geolocation](../ingest/transform-enrich/example-enrich-data-based-on-geolocation.md) -* [Example: Enrich your data based on exact values](../ingest/transform-enrich/example-enrich-data-based-on-exact-values.md) -* [Example: Enrich your data by matching a value to a range](../ingest/transform-enrich/example-enrich-data-by-matching-value-to-range.md) diff --git a/manage-data/toc.yml b/manage-data/toc.yml index f25d1871be..4a056ec2de 100644 --- a/manage-data/toc.yml +++ b/manage-data/toc.yml @@ -4,7 +4,6 @@ toc: - file: data-store.md children: - file: data-store/index-basics.md - - file: data-store/index-types.md - file: data-store/data-streams.md children: - file: data-store/data-streams/set-up-data-stream.md @@ -19,7 +18,7 @@ toc: - file: data-store/data-streams/run-downsampling-manually.md - file: data-store/data-streams/run-downsampling-using-data-stream-lifecycle.md - file: data-store/data-streams/reindex-tsds.md - - file: data-store/data-streams/logs-data-stream.md + - file: data-store/data-streams/logs-data-stream.md - file: data-store/mapping.md children: - file: data-store/mapping/dynamic-mapping.md From 8d38d22875a7e0b606f1dda8a34bd224fc1930be Mon Sep 17 00:00:00 2001 From: David Kilfoyle Date: Wed, 19 Feb 2025 12:48:50 -0500 Subject: [PATCH 2/2] Add mention of data streams in 'ES data store' topic --- manage-data/data-store.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/manage-data/data-store.md b/manage-data/data-store.md index dafaed4e5b..28ff3fa345 100644 --- a/manage-data/data-store.md +++ b/manage-data/data-store.md @@ -11,4 +11,6 @@ The documentation in this section details how {{es}} works as a _data store_ sta Then, learn how these documents and the fields they contain are stored and indexed in [Mapping](/manage-data/data-store/mapping.md), and how unstructured text is converted into a structured format that’s optimized for search in [Text analysis](/manage-data/data-store/text-analysis.md). -You can also read more about working with {{es}} as a data store including how to use [index templates](/manage-data/data-store/templates.md) to tell {{es}} how to configure an index when it is created, how to use [aliases](/manage-data/data-store/aliases.md) to point to multiple indices, and how to use the [command line to manage data](/manage-data/data-store/manage-data-from-the-command-line.md) stored in {{es}}. \ No newline at end of file +You can also read more about working with {{es}} as a data store including how to use [index templates](/manage-data/data-store/templates.md) to tell {{es}} how to configure an index when it is created, how to use [aliases](/manage-data/data-store/aliases.md) to point to multiple indices, and how to use the [command line to manage data](/manage-data/data-store/manage-data-from-the-command-line.md) stored in {{es}}. + +If your use case involves working with continuous streams of time series data, you may consider using a [data stream](./data-store/data-streams.md). These are optimally suited for storing append-only data. The data can be accessed through a single, named resource, while it is stored in a series of hidden, auto-generated backing indices.