Merge branch 'main' into release-9.0.5

Author: beiske

deploy-manage/_snippets/autoops-cc-ech-faq.md

@@ -1,4 +1,6 @@
 :::{dropdown} Can I use Cloud Connect to connect my {{ech}} clusters to AutoOps?
 
 This path is not supported. Currently, we only support using Cloud Connect to connect ECE, ECK, and self-managed clusters to AutoOps.
+
+For {{ech}} clusters, AutoOps is set up and enabled automatically in all supported [regions](/deploy-manage/monitor/autoops/ec-autoops-regions.md), and can be [accessed](/deploy-manage/monitor/autoops/ec-autoops-how-to-access.md) from the deployment overview page. 
 :::

deploy-manage/cloud-organization/billing/elasticsearch-billing-dimensions.md

@@ -30,6 +30,7 @@ For detailed {{es-serverless}} project rates, see the [{{es-serverless}} pricing
 * **Indexing:** The VCUs used to index incoming documents.
 * **Search:** The VCUs used to return search results, with the latency and queries per second (QPS) you require.
 * **Machine learning:** The VCUs used to perform inference, NLP tasks, and other ML activities.
+* **Tokens:** The Elastic Managed LLM is charged per 1Mn Input and Output tokens. The LLM powers all AI Search features such as Playground and AI Assistant for Search, and is enabled by default.
 
 
 ## Data storage and billing [elasticsearch-billing-information-about-the-search-ai-lake-dimension-gb]
@@ -43,10 +44,6 @@ You can control costs using the following strategies:
 
 * **Search Power setting:** [Search Power](../../deploy/elastic-cloud/project-settings.md#elasticsearch-manage-project-search-power-settings) controls the speed of searches against your data. With Search Power, you can improve search performance by adding more resources for querying, or you can reduce provisioned resources to cut costs.
 * **Search boost window**: By limiting the number of days of [time series data](../../../solutions/search/ingest-for-search.md#elasticsearch-ingest-time-series-data) that are available for caching, you can reduce the number of search VCUs required.
-* **Machine learning trained model autoscaling:** Configure your trained model deployment to allow it to scale down to zero allocations when there are no active inference requests:
-
-    * When starting or updating a trained model deployment, [Enable adaptive resources](../../autoscaling/trained-model-autoscaling.md#enabling-autoscaling-in-kibana-adaptive-resources) and set the VCU usage level to **Low**.
-    * When using the inference API for {{es}} or ELSER, [enable `adaptive_allocations`](../../autoscaling/trained-model-autoscaling.md#enabling-autoscaling-through-apis-adaptive-allocations).
 
 * **Indexing Strategies:** Consider your indexing strategies and how they might impact overall VCU usage and costs:
 

deploy-manage/monitor/autoops.md

@@ -12,8 +12,6 @@ products:
 
 AutoOps diagnoses issues in {{es}} by analyzing hundreds of metrics, providing root-cause analysis and accurate resolution paths. With AutoOps, customers can prevent and resolve issues, cut down administration time, and optimize resource utilization.
 
-AutoOps is currently only available for [{{ech}} deployments](/deploy-manage/deploy/elastic-cloud/cloud-hosted.md).
-
 :::{image} /deploy-manage/images/cloud-autoops-overview-page.png
 :alt: The Overview page
 :::

deploy-manage/monitor/autoops/cc-autoops-as-cloud-connected.md

@@ -17,7 +17,8 @@ Cloud Connect enables users of ECE, ECK, and self-managed clusters to use {{eclo
 
 In this section, you'll find the following information:
 
-* How to [connect your cluster to AutoOps](/deploy-manage/monitor/autoops/cc-connect-self-managed-to-autoops.md)
+* How to [connect your self-managed cluster to AutoOps](/deploy-manage/monitor/autoops/cc-connect-self-managed-to-autoops.md)
+* How to [connect your local development cluster to AutoOps](/deploy-manage/monitor/autoops/cc-connect-local-dev-to-autoops.md)
 * How to [manage users of your connected clusters](/deploy-manage/monitor/autoops/cc-manage-users.md)
 * [FAQs](/deploy-manage/monitor/autoops/cc-cloud-connect-autoops-faq.md) about AutoOps for your clusters
 * A [troubleshooting guide](/deploy-manage/monitor/autoops/cc-cloud-connect-autoops-troubleshooting.md) to help you with any issues you may encounter

deploy-manage/monitor/autoops/cc-connect-local-dev-to-autoops.md

@@ -0,0 +1,59 @@
+---
+applies_to:
+  deployment:
+    self:
+navigation_title: Connect your local development cluster
+---
+
+# Connect your local development cluster to AutoOps
+
+If you have an {{es}} cluster set up for local development or testing, you can connect it to AutoOps using Docker.
+
+## Prerequisites
+
+Ensure your system meets the following requirements before proceeding:
+
+* You have set up [{{es}} for local development](/deploy-manage/deploy/self-managed/local-development-installation-quickstart.md).
+* You have installed [Docker Desktop](https://www.docker.com/products/docker-desktop).
+* You have an {{ecloud}} account with the [Organization owner role](/deploy-manage/monitor/autoops/cc-manage-users.md#assign-roles).
+
+## Connect your local development cluster to AutoOps
+
+Complete the following steps to connect your local development cluster to AutoOps.
+
+1. Run the following command in your terminal to open the `/etc/hosts` file in a text editor with administrator privileges:
+
+  ```sh
+  vim /etc/hosts
+  ```
+2. On a new line in the `/etc/hosts` file, add an entry to map the {{es}} cluster URL to the IP address representing the local development cluster.
+
+    The entry should be formatted as `127.0.0.1 {{hostname}}`.
+3. Save the changes.
+4. In your terminal, run the following command to reload the hostname service:
+    * For Linux: 
+    ```sh
+    /bin/systemctl restart systemd-hostnamed
+    ```
+    * For macOS: 
+    ```sh
+    sudo dscacheutil -flushcache
+    ```
+5. Follow the instructions to [Connect to AutoOps](/deploy-manage/monitor/autoops/cc-connect-self-managed-to-autoops.md#connect-to-autoops) with the following differences:
+    * In the [Select installation method](/deploy-manage/monitor/autoops/cc-connect-self-managed-to-autoops.md#select-installation-method) step, select **Docker**.
+    * In the [Configure agent](/deploy-manage/monitor/autoops/cc-connect-self-managed-to-autoops.md#configure-agent) step, when prompted to enter your **{{es}} endpoint URL**, enter the name of your local development cluster or enter the following:
+        ```sh
+        http://localhost:9200
+        ```
+    * In the [Install agent](/deploy-manage/monitor/autoops/cc-connect-self-managed-to-autoops.md#install-agent) step, paste the command into the text editor and replace `docker run -d \` with:
+        ```sh
+        docker run -d --network host \
+        ```
+        
+        This replacement is also required if your cluster is running on macOS.
+      
+After completing all the steps, you can [Access AutoOps](/deploy-manage/monitor/autoops/cc-connect-self-managed-to-autoops.md#access-autoops).
+
+
+
+

deploy-manage/monitor/autoops/cc-connect-self-managed-to-autoops.md

@@ -4,15 +4,19 @@ applies_to:
     self:
     ece:
     eck:
-navigation_title: Connect your cluster
+navigation_title: Connect your self-managed cluster
 ---
 
-# Connect your cluster to AutoOps
+# Connect your self-managed cluster to AutoOps
 
 To use AutoOps with your ECE, ECK, or self-managed cluster, you first need to create an {{ecloud}} account or log in to your existing account. An installation wizard will then guide you through the steps of installing {{agent}} to send metrics from your cluster to AutoOps in {{ecloud}}.  
 
 Complete the steps in the following subsections to connect your cluster to AutoOps. The connection process takes about 10 minutes.
 
+:::{note}
+If you have an {{es}} cluster set up for local development or testing, you can connect it to AutoOps using Docker. Refer to [](/deploy-manage/monitor/autoops/cc-connect-local-dev-to-autoops.md).
+:::
+
 ## Prerequisites
 
 Ensure your system meets the following requirements before proceeding:
@@ -21,7 +25,7 @@ Ensure your system meets the following requirements before proceeding:
 * Your cluster is on an [Enterprise self-managed license](https://www.elastic.co/subscriptions) or an active self-managed [trial](https://cloud.elastic.co/registration).
 * The agent you install for the connection is allowed to send metrics to {{ecloud}}.
 
-## Connect to AutoOps
+## Connect to AutoOps (private preview) [connect-to-autoops]
 
 :::{note}
 :::{include} /deploy-manage/monitor/_snippets/single-cloud-org.md
@@ -86,8 +90,8 @@ This is the first step of the installation wizard. Your cluster ships metrics to
 
 Select one of the following methods to install {{agent}}:
 
-* Kubernetes
-* Docker
+* **Kubernetes**
+* **Docker**
 <!-- Not applicable for private preview
 * Linux
 * Windows
@@ -208,7 +212,8 @@ The wizard will generate an installation command based on your configuration. De
 
 * Kubernetes
     * YAML
-    * Helm Chart
+    <!-- Not applicable for private preview 
+    * Helm Chart -->
 * Docker
     * Docker
     * Docker compose
@@ -226,18 +231,18 @@ Complete the following steps to run the command:
 1. Copy the command. 
 2. Paste it into a text editor and update the placeholder values in the following environment variables:
 
-| Environment variable | Description |
-| --- | --- |
-| `AUTOOPS_OTEL_URL` | The {{ecloud}} URL to which {{agent}} ships data. The URL is generated based on the CSP and region you pick. <br> This URL shouldn't be edited. |
-| `AUTOOPS_ES_URL` | The URL {{agent}} uses to communicate with {{es}}. |
-| `ELASTICSEARCH_READ_API_KEY` | The API key for API key authentication to access the cluster. It combines the `${id}:${api_key}` values. <br> This variable shouldn't be used with `ELASTICSEARCH_READ_USERNAME` and `ELASTICSEARCH_READ_PASSWORD`. |
-| `ELASTICSEARCH_READ_USERNAME` | The username for basic authentication to access the cluster. <br> This variable should be used with `ELASTICSEARCH_READ_PASSWORD`. |
-| `ELASTICSEARCH_READ_PASSWORD` | The password for basic authentication to access the cluster. <br> This variable should be used with `ELASTICSEARCH_READ_USERNAME`. |
-| `ELASTIC_CLOUD_CONNECTED_MODE_API_KEY` | The {{ecloud}} API Key used to register the cluster. <br> This key shouldn't be edited. |
-| `AUTOOPS_TEMP_RESOURCE_ID` | The temporary ID for the current installation wizard. |
-
-4. Run the command from the machine where you want to install the agent. 
-5. Return to the wizard and select **I have run the command**.
+    | Environment variable | Description |
+    | --- | --- |
+    | `AUTOOPS_OTEL_URL` | The {{ecloud}} URL to which {{agent}} ships data. The URL is generated based on the CSP and region you pick. <br> This URL shouldn't be edited. |
+    | `AUTOOPS_ES_URL` | The URL {{agent}} uses to communicate with {{es}}. |
+    | `ELASTICSEARCH_READ_API_KEY` | The API key for API key authentication to access the cluster. It combines the `${id}:${api_key}` values. <br> This variable shouldn't be used with `ELASTICSEARCH_READ_USERNAME` and `ELASTICSEARCH_READ_PASSWORD`. |
+    | `ELASTICSEARCH_READ_USERNAME` | The username for basic authentication to access the cluster. <br> This variable should be used with `ELASTICSEARCH_READ_PASSWORD`. |
+    | `ELASTICSEARCH_READ_PASSWORD` | The password for basic authentication to access the cluster. <br> This variable should be used with `ELASTICSEARCH_READ_USERNAME`. |
+    | `ELASTIC_CLOUD_CONNECTED_MODE_API_KEY` | The {{ecloud}} API Key used to register the cluster. <br> This key shouldn't be edited. |
+    | `AUTOOPS_TEMP_RESOURCE_ID` | The temporary ID for the current installation wizard. |
+
+3. Run the command from the machine where you want to install the agent. 
+4. Return to the wizard and select **I have run the command**.
 
 It might take a few minutes for your cluster details to be validated and the first metrics to be shipped to AutoOps.
 

deploy-manage/toc.yml

@@ -676,6 +676,7 @@ toc:
           - file: monitor/autoops/ec-autoops-faq.md
           - hidden: monitor/autoops/cc-autoops-as-cloud-connected.md
           - hidden: monitor/autoops/cc-connect-self-managed-to-autoops.md
+          - hidden: monitor/autoops/cc-connect-local-dev-to-autoops.md
           - hidden: monitor/autoops/cc-manage-users.md
           - hidden: monitor/autoops/cc-cloud-connect-autoops-faq.md
           - hidden: monitor/autoops/cc-cloud-connect-autoops-troubleshooting.md

deploy-manage/users-roles/cluster-or-deployment-auth/file-based.md

@@ -215,6 +215,10 @@ stringData:
   roles: kibana_admin,ingest_admin  # optional, not part of kubernetes.io/basic-auth
 ```
 
+::::{tip}
+To create custom roles that can be referenced in this list refer to [](/deploy-manage/users-roles/cluster-or-deployment-auth/defining-roles.md#roles-management-file).
+::::
+
 You can make this file available to {{eck}} by adding it as a file realm secret:
 
 ```yaml

explore-analyze/alerts-cases/alerts/alerting-setup.md

@@ -51,7 +51,11 @@ Likewise, you can customize the **Rules Settings** sub-feature privileges relate
 
 To create a rule that uses the [Cases connector](kibana://reference/connectors-kibana/cases-action-type.md), you must also have `All` privileges for the **Cases** feature.
 
-The rule type also affects the privileges that are required. For example, to create or edit {{ml}} rules, you must have `all` privileges for the **Analytics > {{ml-app}}** feature. For {{stack-monitor-app}} rules, you must have the `monitoring_user` role. For {{observability}} rules, you must have `all` privileges for the appropriate {{observability}} features. For Security rules, refer to [Detections prerequisites and requirements](../../../solutions/security/detect-and-alert/detections-requirements.md).
+The rule type also affects the privileges that are required to create and edit rules. For example:
+* For {{ml}} rules, you must have `all` privileges for the **Analytics > {{ml-app}}** feature.
+* For {{stack-monitor-app}} rules, you must have the `monitoring_user` role.
+* For most {{observability}} rules, you must have `all` privileges for the appropriate {{observability}} features. However, for a custom threshold rule, you only need the `stack alerts` privilege.
+* For Security rules, refer to [Detections prerequisites and requirements](../../../solutions/security/detect-and-alert/detections-requirements.md).
 
 ::::
 

explore-analyze/query-filter/languages/querydsl.md

@@ -11,12 +11,6 @@ products:
 
 # Query DSL
 
-$$$filter-context$$$
-
-$$$query-dsl-allow-expensive-queries$$$
-
-$$$relevance-scores$$$
-
 ## What's Query DSL? [search-analyze-query-dsl]
 
 **Query DSL** is a full-featured JSON-style query language that enables complex searching, filtering, and aggregations. It is the original and most powerful query language for {{es}} today.
@@ -65,21 +59,21 @@ $$$query-dsl-allow-expensive-queries$$$
 
 **Allow expensive queries**: Certain types of queries will generally execute slowly due to the way they are implemented, which can affect the stability of the cluster. Those queries can be categorized as follows:
 
-  - Queries that need to do linear scans to identify matches:
+ - Queries that need to do linear scans to identify matches:
 
     - [`script` queries](elasticsearch://reference/query-languages/query-dsl/query-dsl-script-query.md)
     - queries on [numeric](elasticsearch://reference/elasticsearch/mapping-reference/number.md), [date](elasticsearch://reference/elasticsearch/mapping-reference/date.md), [boolean](elasticsearch://reference/elasticsearch/mapping-reference/boolean.md), [ip](elasticsearch://reference/elasticsearch/mapping-reference/ip.md), [geo_point](elasticsearch://reference/elasticsearch/mapping-reference/geo-point.md) or [keyword](elasticsearch://reference/elasticsearch/mapping-reference/keyword.md) fields that are not indexed but have [doc values](elasticsearch://reference/elasticsearch/mapping-reference/doc-values.md) enabled
 
-  - Queries that have a high up-front cost:
+ - Queries that have a high up-front cost:
 
     - [`fuzzy` queries](elasticsearch://reference/query-languages/query-dsl/query-dsl-fuzzy-query.md) (except on [`wildcard`](elasticsearch://reference/elasticsearch/mapping-reference/keyword.md#wildcard-field-type) fields)
     - [`regexp` queries](elasticsearch://reference/query-languages/query-dsl/query-dsl-regexp-query.md) (except on [`wildcard`](elasticsearch://reference/elasticsearch/mapping-reference/keyword.md#wildcard-field-type) fields)
     - [`prefix` queries](elasticsearch://reference/query-languages/query-dsl/query-dsl-prefix-query.md)  (except on [`wildcard`](elasticsearch://reference/elasticsearch/mapping-reference/keyword.md#wildcard-field-type) fields or those without [`index_prefixes`](elasticsearch://reference/elasticsearch/mapping-reference/index-prefixes.md))
     - [`wildcard` queries](elasticsearch://reference/query-languages/query-dsl/query-dsl-wildcard-query.md) (except on [`wildcard`](elasticsearch://reference/elasticsearch/mapping-reference/keyword.md#wildcard-field-type) fields)
     - [`range` queries](elasticsearch://reference/query-languages/query-dsl/query-dsl-range-query.md) on [`text`](elasticsearch://reference/elasticsearch/mapping-reference/text.md) and [`keyword`](elasticsearch://reference/elasticsearch/mapping-reference/keyword.md) fields
 
-  - [Joining queries](elasticsearch://reference/query-languages/query-dsl/joining-queries.md)
-  - Queries that may have a high per-document cost:
+ - [Joining queries](elasticsearch://reference/query-languages/query-dsl/joining-queries.md)
+ - Queries that may have a high per-document cost:
 
     - [`script_score` queries](elasticsearch://reference/query-languages/query-dsl/query-dsl-script-score-query.md)
     - [`percolate` queries](elasticsearch://reference/query-languages/query-dsl/query-dsl-percolate-query.md)