[Docs] Add Index basics page (#464)

wajihaparvez · web-flow · commit ab5bd09286d0 · 2025-02-18T20:59:47.000Z
Closes [#369](elastic/docs-projects#369) Preview [here](https://docs-v3-preview.elastic.dev/elastic/docs-content/pull/464/manage-data/data-store/index-basics) ### Summary What I changed: - Combined content from the three mapped pages - Altered the heading structure to make sense of the added content and improve readability i.e.: divided the page into **H2: Index components** and **H2: Index management** + changed all other headings to H3 and moved them under the relevant H2 - Drafted a new **Manage component templates** subsection to be consistent with the UI/screenshots - Cut down extra descriptive content from subsections to limit this page to Index _basics_ only, but linked out to relevant pages to compensate. The additions to the Mapping page are from one of the raw migrated files that I think made more sense there than on Index basics - Deleted links to Enriching your data examples from the **Manage enrich policies** subsection because I don't think that's relevant to this page I'll delete the code comments after this is reviewed. @colleenmcginnis @kilfoyle let me know what you think of these decisions. I'm open to all insight/suggestions 😄
diff --git a/images/serverless-management-component-templates.png b/images/serverless-management-component-templates.png
diff --git a/manage-data/data-store/index-basics.md b/manage-data/data-store/index-basics.md
@@ -7,18 +7,144 @@ mapped_urls:
 
 # Index basics
 
-% What needs to be done: Refine
+This content applies to: [![Elasticsearch](/images/serverless-es-badge.svg "")](/solutions/search.md) [![Observability](/images/serverless-obs-badge.svg "")](/solutions/observability.md) [![Security](/images/serverless-sec-badge.svg "")](/solutions/security/elastic-security-serverless.md)
 
-% GitHub issue: docs-projects#369
+An index is a fundamental unit of storage in {{es}}. It is a collection of documents uniquely identified by a name or an [alias](/manage-data/data-store/aliases.md). This unique name is important because it’s used to target the index in search queries and other operations.
 
-% Scope notes: Combine content from linked sources.
+::::{tip} 
+A closely related concept is a [data stream](/manage-data/data-store/index-types/data-streams.md). This index abstraction is optimized for append-only timestamped data, and is made up of hidden, auto-generated backing indices. If you’re working with timestamped data, we recommend the [Elastic Observability](https://www.elastic.co/guide/en/observability/current) solution for additional tools and optimized content.
+::::
 
-% Use migrated content from existing pages that map to this page:
+## Index components
 
-% - [ ] ./raw-migrated-files/elasticsearch/elasticsearch-reference/documents-indices.md
-% - [ ] ./raw-migrated-files/elasticsearch/elasticsearch-reference/index-mgmt.md
-% - [ ] ./raw-migrated-files/docs-content/serverless/index-management.md
+An index is made up of the following components.
+
+### Documents [elasticsearch-intro-documents-fields] 
+
+{{es}} serializes and stores data in the form of JSON documents. A document is a set of fields, which are key-value pairs that contain your data. Each document has a unique ID, which you can create or have {{es}} auto-generate.
+
+A simple {{es}} document might look like this:
+
+```js
+{
+  "_index": "my-first-elasticsearch-index",
+  "_id": "DyFpo5EBxE8fzbb95DOa",
+  "_version": 1,
+  "_seq_no": 0,
+  "_primary_term": 1,
+  "found": true,
+  "_source": {
+    "email": "john@smith.com",
+    "first_name": "John",
+    "last_name": "Smith",
+    "info": {
+      "bio": "Eco-warrior and defender of the weak",
+      "age": 25,
+      "interests": [
+        "dolphins",
+        "whales"
+      ]
+    },
+    "join_date": "2024/05/01"
+  }
+}
+```
+
+### Metadata fields [elasticsearch-intro-documents-fields-data-metadata] 
+
+An indexed document contains data and metadata. [Metadata fields](https://www.elastic.co/guide/en/elasticsearch/reference/current/mapping-fields.html) are system fields that store information about the documents. In {{es}}, metadata fields are prefixed with an underscore. For example, the following fields are metadata fields:
+
+* `_index`: The name of the index where the document is stored.
+* `_id`: The document’s ID. IDs must be unique per index.
+
+
+### Mappings and data types [elasticsearch-intro-documents-fields-mappings] 
+
+Each index has a [mapping](/manage-data/data-store/mapping.md) or schema for how the fields in your documents are indexed. A mapping defines the [data type](https://www.elastic.co/guide/en/elasticsearch/reference/current/mapping-types.html) for each field, how the field should be indexed, and how it should be stored.
+
+## Index management
+
+Elastic's index management features are an easy, convenient way to manage your cluster's indices, data streams, index templates, and enrich policies. Practicing good index management ensures your data is stored correctly and in the most cost-effective way possible.
+
+Go to **{{project-settings}} → {{manage-app}} → {{index-manage-app}}**.
+
+### Manage indices
+
+Investigate your indices and perform operations from the **Indices** view.
+
+:::{image} /images/serverless-index-management-indices.png
+:alt: Index Management indices
+:class: screenshot
+:::
+
+* To show details and perform operations, click the index name. To perform operations on multiple indices, select their checkboxes and then open the **Manage** menu. For more information on managing indices, refer to [Index APIs](https://www.elastic.co/guide/en/elasticsearch/reference/current/indices.html).
+* To filter the list of indices, use the search bar or click a badge. Badges indicate if an index is a [follower index](https://www.elastic.co/guide/en/elasticsearch/reference/current/ccr-put-follow.html), a [rollup index](https://www.elastic.co/guide/en/elasticsearch/reference/current/rollup-get-rollup-index-caps.html), or [frozen](https://www.elastic.co/guide/en/elasticsearch/reference/current/unfreeze-index-api.html).
+* To drill down into the index [mappings](/manage-data/data-store/mapping.md), [settings](https://www.elastic.co/guide/en/elasticsearch/reference/current/index-modules.html#index-modules-settings), and statistics, click an index name. From this view, you can navigate to **Discover** to further explore the documents in the index.
+* To create new indices, use the **Create index** wizard.
+
+### Manage data streams
+
+A [data stream](/manage-data/data-store/index-types/data-streams.md) lets you store append-only time series data across multiple indices while giving you a single named resource for requests. 
+
+Investigate your data streams and address lifecycle management needs in the **Data Streams** view.
+
+:::{image} /images/serverless-management-data-stream.png
+:alt: Data stream details
+:class: screenshot
+:::
+
+In {{es-serverless}}, indices matching the `logs-*-*` pattern use the logsDB index mode by default. The logsDB index mode creates a [logs data stream](https://www.elastic.co/guide/en/elasticsearch/reference/master/logs-data-stream.html). 
+
+* To view information about the stream's backing indices, click the number in the **Indices** column.
+* A value in the **Data retention** column indicates that the data stream is managed by a data stream lifecycle policy. This value is the time period for which your data is guaranteed to be stored. Data older than this period can be deleted by {{es}} at a later time. 
+* To modify the data retention value, select an index, open the **Manage**  menu, and click **Edit data retention**.
+* To view more information about a data stream, such as its generation or its current index lifecycle policy, click the stream's name. From this view, you can navigate to **Discover** to further explore data within the data stream.
+
+### Manage index templates
+
+An [index template](/manage-data/data-store/templates.md) is a way to tell {{es}} how to configure an index when it is created. 
+
+Create, edit, clone, and delete your index templates in the **Index Templates** view. Changes made to an index template do not affect existing indices.
+
+:::{image} /images/serverless-index-management-index-templates.png
+:alt: Index templates
+:class: screenshot
+:::
+
+* To show details and perform operations, click the template name.
+* To view more information about the component templates within an index template, click the value in the **Component templates** column. 
+* Values in the **Content** column indicate whether a template contains index mappings, settings, and aliases.
+* To create new index templates, use the **Create template** wizard.
+
+### Manage component templates
+
+[Component templates](/manage-data/data-store/templates.md) are reusable building blocks that configure mappings, settings, and aliases.
+
+Create, edit, clone, and delete your component templates in the **Component Templates** view.
+
+:::{image} /images/serverless-management-component-templates.png
+:alt: Component templates
+:class: screenshot
+:::
+
+* To show details and perform operations, click the template name.
+* To create new component templates, use the **Create component template** wizard.
+
+### Manage enrich policies
+
+An [enrich policy](https://www.elastic.co/guide/en/elasticsearch/reference/current/ingest-enriching-data.html#enrich-policy) is a set of configuration options used to add the right enrich data to the right incoming documents.
+
+Add data from your existing indices to incoming documents using the **Enrich Policies** view.
+
+:::{image} /images/serverless-management-enrich-policies.png
+:alt: Enrich policies
+:class: screenshot
+:::
+
+* To show details click the policy name.
+* To perform operations, click the policy name or use the buttons in the **Actions** column.
+* To create new policies, use the **Create enrich policy** wizard.
+
+You must execute a new enrich policy before you can use it with an enrich processor. When executed, an enrich policy uses enrich data from the policy's source indices to create a streamlined system index called the enrich index. The policy uses this index to match and enrich incoming documents.
 
-% Internal links rely on the following IDs being on this page (e.g. as a heading ID, paragraph ID, etc):
 
-$$$elasticsearch-intro-documents-fields-mappings$$$
diff --git a/manage-data/data-store/mapping.md b/manage-data/data-store/mapping.md
@@ -34,21 +34,21 @@ Mapping is the process of defining how a document and the fields it contains are
 
 Each document is a collection of fields, which each have their own [data type](https://www.elastic.co/guide/en/elasticsearch/reference/current/mapping-types.html). When mapping your data, you create a mapping definition, which contains a list of fields that are pertinent to the document. A mapping definition also includes [metadata fields](https://www.elastic.co/guide/en/elasticsearch/reference/current/mapping-fields.html), like the `_source` field, which customize how a document’s associated metadata is handled.
 
-Depending on where you are in your data journey, use **dynamic mapping** and **explicit mapping** to define your data. For example, you can explicitly map fields where you don’t want to use the defaults, or to gain greater control over which fields are created. Then you can allow {{es}} to dynamically map other fields.
+Depending on where you are in your data journey, use *dynamic mapping* and *explicit mapping* to define your data. For example, you can explicitly map fields where you don’t want to use the defaults, or to gain greater control over which fields are created. Then you can allow {{es}} to dynamically map other fields. Using a combination of dynamic and explicit mapping on the same index is especially useful when you have a mix of known and unknown fields in your data.
 
 ::::{note}
 Before 7.0.0, the mapping definition included a type name. {{es}} 7.0.0 and later no longer accept a default mapping. [Removal of mapping types](/manage-data/data-store/mapping/removal-of-mapping-types.md) provides more information.
 ::::
 
 ## Dynamic mapping [mapping-dynamic] 
 
-When you use [dynamic mapping](/manage-data/data-store/mapping/dynamic-mapping.md), {{es}} automatically attempts to detect the data type of fields in your documents. This allows you to get started quickly by just adding data to an index. If you index additional documents with new fields, {{es}} will add these fields automatically. You can add fields to the top-level mapping, and to inner [`object`](https://www.elastic.co/guide/en/elasticsearch/reference/current/object.html) and [`nested`](https://www.elastic.co/guide/en/elasticsearch/reference/current/nested.html) fields.
+When you use [dynamic mapping](/manage-data/data-store/mapping/dynamic-mapping.md), {{es}} automatically detects the data types of fields in your documents and creates mappings for you. If you index additional documents with new fields, {{es}} will add these fields automatically. You can add fields to the top-level mapping, and to inner [`object`](https://www.elastic.co/guide/en/elasticsearch/reference/current/object.html) and [`nested`](https://www.elastic.co/guide/en/elasticsearch/reference/current/nested.html) fields. Dynamic mapping helps you get started quickly, but might yield suboptimal results for your specific use case due to automatic field type inference.
 
-Use [dynamic templates](/manage-data/data-store/mapping/dynamic-templates.md) to define custom mappings that are applied to dynamically added fields based on the matching condition.
+Use [dynamic templates](/manage-data/data-store/mapping/dynamic-templates.md) to define custom mappings that are applied to dynamically added fields based on the matching condition. 
 
 ## Explicit mapping [mapping-explicit] 
 
-Use [explicit mapping](/manage-data/data-store/mapping/explicit-mapping.md) to define exactly how data types are mapped to fields, customized to your specific use case.
+Use [explicit mapping](/manage-data/data-store/mapping/explicit-mapping.md) to define mappings by specifying data types for each field. This is recommended for production use cases, because you have full control over how your data is indexed to suit your specific use case.
 
 Defining your own mappings enables you to:
 
