Merge branch 'main' into ppf2-slow-tokenizers

Author: bmorelli25

deploy-manage/cloud-organization/billing/elasticsearch-billing-dimensions.md

@@ -27,9 +27,9 @@ For detailed {{es-serverless}} project rates, see the [{{es-serverless}} pricing
 
 {{es}} uses three VCU types:
 
-* **Indexing:** The VCUs used to index incoming documents.
-* **Search:** The VCUs used to return search results, with the latency and queries per second (QPS) you require.
-* **Machine learning:** The VCUs used to perform inference, NLP tasks, and other ML activities.
+* **Indexing:** The VCUs used to index incoming documents. Indexing VCUs account for compute resources consumed for ingestion. This is based on ingestion rate, and amount of data ingested at any given time. Transforms and ingest pipelines also contribute to ingest VCU consumption. 
+* **Search:** The VCUs used to return search results, with the latency and queries per second (QPS) you require. Search VCUs are calculated as a factor of the compute resources needed to run search queries, search throughput and latency. Search VCUs are not charged per search request, but instead are a factor of the compute resources that scale up and down based on amount of searchable data, search load (QPS) and performance (latency and availability). 
+* **Machine learning:** The VCUs used to perform inference, NLP tasks, and other ML activities. ML VCUs are a factor of the models deployed, and number of ML operations such as inference for search and ingest. ML VCUs are typically consumed for generating embeddings during ingestion, and during semantic search or reranking.  
 * **Tokens:** The Elastic Managed LLM is charged per 1Mn Input and Output tokens. The LLM powers all AI Search features such as Playground and AI Assistant for Search, and is enabled by default.
 
 

deploy-manage/upgrade/orchestrator/upgrade-cloud-on-k8s.md

@@ -101,8 +101,11 @@ This will update the ECK installation to the latest binary and update the CRDs a
 Upgrading the operator results in a one-time update to existing managed resources in the cluster. This potentially triggers a rolling restart of pods by Kubernetes to apply those changes. The following list contains the ECK operator versions that would cause a rolling restart after they have been installed.
 
 ```
-1.6, 1.9, 2.0, 2.1, 2.2, 2.4, 2.5, 2.6, 2.7, 2.8, 2.14
+1.6, 1.9, 2.0, 2.1, 2.2, 2.4, 2.5, 2.6, 2.7, 2.8, 2.14, 3.1 <1>
 ```
+
+1. The restart when upgrading to version 3.1 happens only for applications using [stack monitoring](/deploy-manage/monitor/stack-monitoring/eck-stack-monitoring.md).
+
 ::::{note}
 Stepping over one of these versions, for example, upgrading ECK from 2.6 to 2.9, still triggers a rolling restart.
 ::::

explore-analyze/ai-assistant.md

@@ -12,31 +12,33 @@ applies_to:
 products:
   - id: kibana
   - id: observability
+  - id: security
   - id: cloud-serverless
 ---
 
 # AI assistant
 
 $$$token-limits$$$
 
-**AI Assistant** is a chat-based interactive tool to help you with a variety of tasks related to Elasticsearch and Kibana, including:
+**AI Assistant** is a chat-based interactive tool that can help with a variety of tasks throughout Elasticsearch and Kibana, including:
 
 - **Constructing queries**: Assists you in building queries to search and analyze your data, including converting queries from other languages to [ES|QL](query-filter/languages/esql.md).
 - **Indexing data**: Guides you on how to index data into Elasticsearch.
 - **Using APIs**: Calls Elasticsearch APIs on your behalf if you need specific operations performed.
 - **Generating sample data**: Helps you create sample data for testing and development purposes.
 - **Visualizing and analyzing data**: Assists you in creating visualizations and analyzing your data using Kibana.
 - **Troubleshooting**: Explains errors, messages, and suggests remediation.
+- **Investigating threats**: Helps analyze potential attacks, provides threat research, and assists with alert triage.
 
-AI Assistant requires specific privileges and a generative AI connector.
-
-% Check [Configure AI Assistant](../deploy-manage/) for more details on how to enable and configure it.
-
-The capabilities and ways to interact with AI Assistant can differ for each solution. Find more information in the respective solution docs:
+AI Assistant requires specific privileges and a large language model (LLM) connector. The capabilities and ways to interact with AI Assistant can differ for each solution. To learn more about how it works in each solution, refer to:
 
 - [{{obs-ai-assistant}}](../solutions/observability/observability-ai-assistant.md)
 - [AI Assistant for Security](../solutions/security/ai/ai-assistant.md)
 
+To learn more about configuring LLM connectors, refer to:
+
+- [Enable LLM access](../solutions/security/ai/set-up-connectors-for-large-language-models-llm.md)
+
 ## Prompt best practices [rag-for-esql]
 Elastic AI Assistant allows you to take full advantage of the Elastic platform to improve your operations. It can help you write an ES|QL query for a particular use case, or answer general questions about how to use the platform. Its ability to assist you depends on the specificity and detail of your questions. The more context and detail you provide, the more tailored and useful its responses will be.
 

explore-analyze/query-filter.md

@@ -26,8 +26,7 @@ You’ll use a combination of an API endpoint and a query language to interact w
 
 - A number of [tools](/explore-analyze/query-filter/tools.md) are available for you to save, debug, and optimize your queries.
 
-% todo: update link to the best target
-If you're just getting started with Elasticsearch, try the hands-on [API quickstart](/solutions/search/elasticsearch-basics-quickstart.md) to learn how to add data and run basic searches using Query DSL and the `_search` endpoint.
+If you're just getting started with {{es}}, try the hands-on [](/solutions/search/get-started/index-basics.md) to learn how to add data and run basic searches using Query DSL and the `_search` endpoint.
 
 ## Filtering
 

manage-data/images/elasticsearch-reference-lifecycle-disabled.png

@@ -10,15 +10,42 @@ products:
 
 # Tutorial: Update existing data stream [tutorial-manage-existing-data-stream]
 
-To update the lifecycle of an existing data stream you do the following actions:
+To update the lifecycle of an existing data stream, perform the following actions:
 
 1. [Set a data stream’s lifecycle](#set-lifecycle)
 2. [Remove lifecycle for a data stream](#delete-lifecycle)
 
 
 ## Set a data stream’s lifecycle [set-lifecycle] 
 
-To add or to change the retention period of your data stream you can use the [PUT lifecycle API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-indices-put-data-lifecycle).
+To add or to change the retention period of your data stream you can use the **Index Management** tools in {{kib}} or the {{es}} [lifecycle API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-indices-put-data-lifecycle).
+
+
+:::::{tab-set}
+:group: kibana-api
+:::{tab-item} {{kib}}
+:sync: kibana
+
+To change the data retention settings for a data stream:
+
+1. Go to **Stack Management > Index Management** and open the **Data Streams** tab.
+1. Use the search tool to find the data stream you're looking for.
+1. Select the data stream to view its details.
+1. In the data stream details pane, select **Manage > Edit data retention** to adjust the settings. You can do any of the following:
+
+    - Select how long to retain your data, in days, hours, minutes, or seconds.
+    - Choose to **Keep data indefinitely**, so that your data will not be deleted. Your data stream is still managed but the data will never be deleted. Managing a time series data stream such as for logs or metrics enables {{es}} to better store your data even if you do not use a retention period.
+    - Disable **Enable data retention** to turn off data stream lifecycle management for your data stream.
+
+    Note that if the data stream is already managed by [{{ilm-init}}](/manage-data/lifecycle/index-lifecycle-management.md), to edit the data retention settings you must edit the associated {{ilm-init}} policy.
+
+
+:::
+
+:::{tab-item} API
+:sync: api
+
+To change the data retention settings for a data stream:
 
 * You can set infinite retention period, meaning that your data should never be deleted. For example:
 
@@ -27,7 +54,7 @@ To add or to change the retention period of your data stream you can use the [PU
     { } <1>
     ```
 
-    1. An empty payload means that your data stream is still managed but the data will never be deleted. Managing a time series data stream such as logs or metrics enables {{es}} to better store your data even if you do not use a retention period.
+    1. An empty payload means that your data stream is still managed but the data will never be deleted. Managing a time series data stream such as for logs or metrics enables {{es}} to better store your data even if you do not use a retention period.
 
 * Or you can set the retention period of your choice. For example:
 
@@ -39,9 +66,32 @@ To add or to change the retention period of your data stream you can use the [PU
     ```
 
     1. The retention period of this data stream is set to 30 days. This means that {{es}} is allowed to delete data that is older than 30 days at its own discretion.
+:::
+:::::
+
+The changes in the lifecycle are applied on all backing indices of the data stream. 
+
+You can see the effect of the change in {{kib}} or using the {{es}} [explain API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-indices-explain-data-lifecycle):
+
+:::::{tab-set}
+:group: kibana-api
+:::{tab-item} {{kib}}
+:sync: kibana
+To check the data retention settings for a data stream:
+
+1. Go to **Stack Management > Index Management** and open the **Data Streams** tab.
+1. Use the search tool to find the data stream you're looking for.
+1. Select the data stream to view its details. The flyout shows the data retention settings for the data stream. Note that if the data stream is currently managed by an [{{ilm-init}} policy](/manage-data/lifecycle/index-lifecycle-management.md), the **Effective data retention** may differ from the retention value that you've set in the data stream, as indicated by the **Data retention**.
+
+  :::{image} /manage-data/images/elasticsearch-reference-lifecycle-status.png
+  :alt: Index lifecycle status page
+  :width: 500px
+  :::
 
+:::{tab-item} API
+:sync: api
 
-The changes in the lifecycle are applied on all backing indices of the data stream. You can see the effect of the change via the [explain API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-indices-explain-data-lifecycle):
+To check the data retention settings for a data stream:
 
 ```console
 GET .ds-my-data-stream-*/_lifecycle/explain
@@ -90,16 +140,48 @@ The response will look like:
 9. The time that will be used to determine when it’s safe to delete this index and all its data.
 10. The data retention for this index as well is at least 30 days, as it was recently updated.
 
-
+:::
+:::::
 
 ## Remove lifecycle for a data stream [delete-lifecycle] 
 
-To remove the lifecycle of a data stream you can use the [delete lifecycle API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-indices-delete-data-lifecycle). As consequence, the maintenance operations that were applied by the lifecycle will no longer be applied to the data stream and all its backing indices. For example:
+To remove the lifecycle of a data stream you can use the **Index Management** tools in {{kib}} or the {{es}} [delete lifecycle API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-indices-delete-data-lifecycle). 
+
+
+:::::{tab-set}
+:group: kibana-api
+:::{tab-item} {{kib}}
+:sync: kibana
+
+To remove a data stream's lifecycle:
+
+1. Go to **Stack Management > Index Management** and open the **Data Streams** tab.
+1. Use the search tool to find the data stream you're looking for.
+1. Select the data stream to view its details.
+1. In the data stream details pane, select **Manage > Edit data retention**.
+1. Turn off the **Enable data retention** option and save your changes. The maintenance operations that were applied by the lifecycle will no longer be applied to the data stream and all of its backing indices.
+
+    You can confirm your changes by reopening the data stream pane. The **Effective data retention** will show a **Disabled** status.
+
+  ::::{image} /manage-data/images/elasticsearch-reference-lifecycle-disabled.png
+  :alt: Index lifecycle status is disabled
+  :width: 500px
+  ::::
+
+
+:::
+
+:::{tab-item} API
+:sync: api
+
+To remove a data stream's lifecycle:
 
 ```console
 DELETE _data_stream/my-data-stream/_lifecycle
 ```
 
+After running the API request, the maintenance operations that were applied by the lifecycle will no longer be applied to the data stream and all of its backing indices.
+
 You can then use the [explain API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-indices-explain-data-lifecycle) again to see that the indices are no longer managed.
 
 ```console
@@ -125,5 +207,5 @@ GET .ds-my-data-stream-*/_lifecycle/explain
 2. Indication that the index is not managed by the data stream lifecycle.
 3. The name of another backing index.
 4. Indication that the index is not managed by the data stream lifecycle.
-
-
+:::
+:::::

manage-data/images/elasticsearch-reference-lifecycle-status.png

@@ -308,3 +308,5 @@ redirects:
   'solutions/security/configure-elastic-defend/enable-access-for-macos-monterey.md': 'solutions/security/configure-elastic-defend/enable-access-for-macos.md'
   'solutions/security/configure-elastic-defend/enable-access-for-macos-ventura-higher.md': 'solutions/security/configure-elastic-defend/enable-access-for-macos.md'
 
+# Related to https://github.com/elastic/docs-content/pull/2245/
+  'solutions/search/elasticsearch-basics-quickstart.md': 'solutions/search/get-started/index-basics.md'

manage-data/lifecycle/data-stream/tutorial-update-existing-data-stream.md

@@ -336,7 +336,7 @@ Adapt and change the suggested manifests and deployment strategy to your needs,
           automountServiceAccountToken: false
           containers:
           - name: elastic-agent
-            image: docker.elastic.co/beats/elastic-agent:{{version.stack}}
+            image: docker.elastic.co/elastic-agent:{{version.stack}}
             env:
               - name: FLEET_SERVER_ENABLE
                 value: "true"
@@ -422,7 +422,7 @@ Adapt and change the suggested manifests and deployment strategy to your needs,
           automountServiceAccountToken: false
           containers:
           - name: elastic-agent
-            image: docker.elastic.co/beats/elastic-agent:{{version.stack}}
+            image: docker.elastic.co/elastic-agent:{{version.stack}}
             env:
               - name: FLEET_SERVER_ENABLE
                 value: "true"