Merge branch 'main' into bootstrap-entitlements-for-testing

Author: prdoyle

branches.json

@@ -4,6 +4,9 @@
     {
       "branch": "main"
     },
+    {
+      "branch": "9.1"
+    },
     {
       "branch": "9.0"
     },

docs/changelog/129738.yaml

@@ -0,0 +1,5 @@
+pr: 129738
+summary: Watch SSL files instead of directories
+area: TLS
+type: bug
+issues: []

docs/changelog/129884.yaml

@@ -0,0 +1,5 @@
+pr: 129884
+summary: Move to the Cohere V2 API for new inference endpoints
+area: Machine Learning
+type: enhancement
+issues: []

docs/changelog/129962.yaml

@@ -0,0 +1,5 @@
+pr: 129962
+summary: Simplified Linear & RRF Retrievers - Return error on empty fields param
+area: Search
+type: bug
+issues: []

docs/extend/creating-classic-plugins.md

@@ -136,13 +136,12 @@ org.example.module: # or 'ALL-UNNAMED' if the plugin is non-modular
 Allows code to access the filesystem, to read or write paths as specified by the entitlement's fields. The filesystem of the OS hosting {{es}} may contain sensitive files, for example credentials. Some files are meant to be always accessible to {{es}}, but plugins can not access them directly: {{es}} enforces that certain files can only be read by its core code, while some other files can not be read or written at all. A plugin is always granted `read` access to the {{es}} config directory and `read_write` access to the temp directory; if the plugin requires to read, write or access additional files or directories, it must specify them via this entitlement.
 
 It is possible to specify 3 different types of file entitlement:
-  - `path` to specify an absolute path
-  - `relative_path` to specify a relative path. The path will be resolved via the `relative_to` field, which is used to qualify the relative path. It can be a specific {{es}} directory (`config` or `data`), or to the user home directory (`home`) (the home of the user running {{es}})
-  - `relative_path` to specify a path resolved via the `relative_to` field, which can have the following values:
-    - `config`: the {{es}} [config directory](https://www.elastic.co/guide/en/elasticsearch/reference/current/settings.html#config-files-location)
-    - `data`: the {{es}} [data directory](https://www.elastic.co/guide/en/elasticsearch/reference/current/path-settings-overview.html)
-    - `home`: the home directory of the user running {{es}}
-  - `path_setting` to specify a path defined via an {{es}} setting. The path can be absolute or relative; in the latter case, the path will be resolved using the `basedir_if_relative` path (which can assume the same values as `relative_to`)
+1. `path` to specify an absolute path
+2. `relative_path` to specify a relative path. Use the `relative_to` field to qualify the relative path. `relative_to` accepts the following options:
+   - `config`: the {{es}} [config directory](https://www.elastic.co/guide/en/elasticsearch/reference/current/settings.html#config-files-location)
+   - `data`: the {{es}} [data directory](https://www.elastic.co/guide/en/elasticsearch/reference/current/path-settings-overview.html)
+   - `home`: the home directory of the user running {{es}}
+3. `path_setting` to specify a path defined via an {{es}} setting. The path can be absolute or relative; in the latter case, the path will be resolved using the `basedir_if_relative` path (which can assume the same values as `relative_to`)
 
 Each of the 3 types has some additional fields:
 - `mode` (required): can be either `read` or `read_write`

docs/reference/query-languages/esql/_snippets/commands/layout/change_point.md

@@ -4,12 +4,10 @@
 The `CHANGE_POINT` command requires a [platinum license](https://www.elastic.co/subscriptions).
 :::
 
-::::{warning}
-This functionality is in technical preview and may be
-changed or removed in a future release. Elastic will work to fix any
-issues, but features in technical preview are not subject to the support
-SLA of official GA features.
-::::
+```yaml {applies_to}
+serverless: preview
+stack: preview 9.1.0
+```
 
 `CHANGE_POINT` detects spikes, dips, and change points in a metric.
 

docs/reference/query-languages/esql/_snippets/commands/layout/lookup-join.md

@@ -1,11 +1,8 @@
 ## `LOOKUP JOIN` [esql-lookup-join]
 
-::::{warning}
-This functionality is in technical preview and may be
-changed or removed in a future release. Elastic will work to fix any
-issues, but features in technical preview are not subject to the support
-SLA of official GA features.
-::::
+```yaml {applies_to}
+stack: preview 9.0.0, ga 9.1.0
+```
 
 `LOOKUP JOIN` enables you to add data from another index, AKA a 'lookup'
 index, to your {{esql}} query results, simplifying data enrichment

docs/reference/query-languages/esql/_snippets/commands/layout/sample.md

@@ -1,7 +1,8 @@
 ## `SAMPLE` [esql-sample]
 
 ```yaml {applies_to}
-stack: preview 9.1
+serverless: preview
+stack: preview 9.1.0
 ```
 
 The `SAMPLE` command samples a fraction of the table rows.

docs/reference/query-languages/esql/_snippets/lists/processing-commands.md

@@ -6,7 +6,7 @@
 * [`GROK`](../../commands/processing-commands.md#esql-grok)
 * [`KEEP`](../../commands/processing-commands.md#esql-keep)
 * [`LIMIT`](../../commands/processing-commands.md#esql-limit)
-* [preview] [`LOOKUP JOIN`](../../commands/processing-commands.md#esql-lookup-join)
+* [`LOOKUP JOIN`](../../commands/processing-commands.md#esql-lookup-join)
 * [preview] [`MV_EXPAND`](../../commands/processing-commands.md#esql-mv_expand)
 * [`RENAME`](../../commands/processing-commands.md#esql-rename)
 * [preview] [`SAMPLE`](../../commands/processing-commands.md#esql-sample)

docs/reference/query-languages/esql/esql-metadata-fields.md

@@ -6,24 +6,40 @@ mapped_pages:
 
 # {{esql}} metadata fields [esql-metadata-fields]
 
+{{esql}} can access [metadata fields](/reference/elasticsearch/mapping-reference/document-metadata-fields.md).
 
-{{esql}} can access [metadata fields](/reference/elasticsearch/mapping-reference/document-metadata-fields.md). The currently supported ones are:
-
-* [`_index`](/reference/elasticsearch/mapping-reference/mapping-index-field.md): the index to which the document belongs. The field is of the type [keyword](/reference/elasticsearch/mapping-reference/keyword.md).
-* [`_id`](/reference/elasticsearch/mapping-reference/mapping-id-field.md): the source document’s ID. The field is of the type [keyword](/reference/elasticsearch/mapping-reference/keyword.md).
-* `_version`: the source document’s version. The field is of the type [long](/reference/elasticsearch/mapping-reference/number.md).
-* [`_ignored`](/reference/elasticsearch/mapping-reference/mapping-ignored-field.md): the ignored source document fields. The field is of the type [keyword](/reference/elasticsearch/mapping-reference/keyword.md).
-* `_score`: when enabled, the final score assigned to each row matching an ES|QL query. Scoring will be updated when using [full text search functions](/reference/query-languages/esql/functions-operators/search-functions.md).
-
-To enable the access to these fields, the [`FROM`](/reference/query-languages/esql/commands/source-commands.md#esql-from) source command needs to be provided with a dedicated directive:
+To access these fields, use the `METADATA` directive with the [`FROM`](/reference/query-languages/esql/commands/source-commands.md#esql-from) source command. For example:
 
 ```esql
 FROM index METADATA _index, _id
 ```
 
-Metadata fields are only available if the source of the data is an index. Consequently, `FROM` is the only source commands that supports the `METADATA` directive.
+## Available metadata fields
+
+The following metadata fields are available in {{esql}}:
+
+| Metadata field | Type | Description |
+|---------------|------|-------------|
+| [`_id`](/reference/elasticsearch/mapping-reference/mapping-id-field.md) | [keyword](/reference/elasticsearch/mapping-reference/keyword.md) | Unique document ID. |
+| [`_ignored`](/reference/elasticsearch/mapping-reference/mapping-ignored-field.md) | [keyword](/reference/elasticsearch/mapping-reference/keyword.md) | Names every field in a document that was ignored when the document was indexed. |
+| [`_index`](/reference/elasticsearch/mapping-reference/mapping-index-field.md) | [keyword](/reference/elasticsearch/mapping-reference/keyword.md) | Index name. |
+| `_index_mode` | [keyword](/reference/elasticsearch/mapping-reference/keyword.md) | [Index mode](/reference/elasticsearch/index-settings/index-modules.md#index-mode-setting). For example: `standard`, `lookup`, or `logsdb`. |
+| `_score` | [`float`](/reference/elasticsearch/mapping-reference/number.md) | Query relevance score (when enabled). Scores are updated when using [full text search functions](/reference/query-languages/esql/functions-operators/search-functions.md). |
+| [`_source`](/reference/elasticsearch/mapping-reference/mapping-source-field.md) | Special `_source` type | Original JSON document body passed at index time (or a reconstructed version if [synthetic `_source`](/reference/elasticsearch/mapping-reference/mapping-source-field.md#synthetic-source) is enabled). |
+| `_version` | [`long`](/reference/elasticsearch/mapping-reference/number.md) | Document version number |
+
+## Usage and limitations
+
+- Metadata fields are only available when the data source is an index
+- The `_source` type is not supported by functions
+- Only the `FROM` command supports the `METADATA` directive
+- Once enabled, metadata fields work like regular index fields
+
+## Examples
 
-Once enabled, these fields will be available to subsequent processing commands, just like other index fields:
+### Basic metadata usage
+
+Once enabled, metadata fields are available to subsequent processing commands, just like other index fields:
 
 ```esql
 FROM ul_logs, apps METADATA _index, _version
@@ -40,6 +56,8 @@ FROM ul_logs, apps METADATA _index, _version
 | 14 | apps | 1 | apps_14 |
 | 14 | ul_logs | 1 | ul_logs_14 |
 
+### Metadata fields and aggregations
+
 Similar to index fields, once an aggregation is performed, a metadata field will no longer be accessible to subsequent commands, unless used as a grouping field:
 
 ```esql
@@ -51,3 +69,15 @@ FROM employees METADATA _index, _id
 | --- | --- |
 | 10100 | employees |
 
+### Sort results by search score
+
+```esql
+FROM products METADATA _score
+| WHERE MATCH(description, "wireless headphones")
+| SORT _score DESC
+| KEEP name, description, _score
+```
+
+:::{tip}
+Refer to [{{esql}} for search](docs-content://solutions/search/esql-for-search.md#esql-for-search-scoring) for more information on relevance scoring and how to use `_score` in your queries.
+:::