diff --git a/docs/reference/query-languages/esql/esql-metadata-fields.md b/docs/reference/query-languages/esql/esql-metadata-fields.md index d58a927ae4bc7..0dd5f5db941f6 100644 --- a/docs/reference/query-languages/esql/esql-metadata-fields.md +++ b/docs/reference/query-languages/esql/esql-metadata-fields.md @@ -6,24 +6,40 @@ mapped_pages: # {{esql}} metadata fields [esql-metadata-fields] +{{esql}} can access [metadata fields](/reference/elasticsearch/mapping-reference/document-metadata-fields.md). -{{esql}} can access [metadata fields](/reference/elasticsearch/mapping-reference/document-metadata-fields.md). The currently supported ones are: - -* [`_index`](/reference/elasticsearch/mapping-reference/mapping-index-field.md): the index to which the document belongs. The field is of the type [keyword](/reference/elasticsearch/mapping-reference/keyword.md). -* [`_id`](/reference/elasticsearch/mapping-reference/mapping-id-field.md): the source document’s ID. The field is of the type [keyword](/reference/elasticsearch/mapping-reference/keyword.md). -* `_version`: the source document’s version. The field is of the type [long](/reference/elasticsearch/mapping-reference/number.md). -* [`_ignored`](/reference/elasticsearch/mapping-reference/mapping-ignored-field.md): the ignored source document fields. The field is of the type [keyword](/reference/elasticsearch/mapping-reference/keyword.md). -* `_score`: when enabled, the final score assigned to each row matching an ES|QL query. Scoring will be updated when using [full text search functions](/reference/query-languages/esql/functions-operators/search-functions.md). - -To enable the access to these fields, the [`FROM`](/reference/query-languages/esql/commands/source-commands.md#esql-from) source command needs to be provided with a dedicated directive: +To access these fields, use the `METADATA` directive with the [`FROM`](/reference/query-languages/esql/commands/source-commands.md#esql-from) source command. For example: ```esql FROM index METADATA _index, _id ``` -Metadata fields are only available if the source of the data is an index. Consequently, `FROM` is the only source commands that supports the `METADATA` directive. +## Available metadata fields + +The following metadata fields are available in {{esql}}: + +| Metadata field | Type | Description | +|---------------|------|-------------| +| [`_id`](/reference/elasticsearch/mapping-reference/mapping-id-field.md) | [keyword](/reference/elasticsearch/mapping-reference/keyword.md) | Unique document ID. | +| [`_ignored`](/reference/elasticsearch/mapping-reference/mapping-ignored-field.md) | [keyword](/reference/elasticsearch/mapping-reference/keyword.md) | Names every field in a document that was ignored when the document was indexed. | +| [`_index`](/reference/elasticsearch/mapping-reference/mapping-index-field.md) | [keyword](/reference/elasticsearch/mapping-reference/keyword.md) | Index name. | +| `_index_mode` | [keyword](/reference/elasticsearch/mapping-reference/keyword.md) | [Index mode](/reference/elasticsearch/index-settings/index-modules.md#index-mode-setting). For example: `standard`, `lookup`, or `logsdb`. | +| `_score` | [`float`](/reference/elasticsearch/mapping-reference/number.md) | Query relevance score (when enabled). Scores are updated when using [full text search functions](/reference/query-languages/esql/functions-operators/search-functions.md). | +| [`_source`](/reference/elasticsearch/mapping-reference/mapping-source-field.md) | Special `_source` type | Original JSON document body passed at index time (or a reconstructed version if [synthetic `_source`](/reference/elasticsearch/mapping-reference/mapping-source-field.md#synthetic-source) is enabled). | +| `_version` | [`long`](/reference/elasticsearch/mapping-reference/number.md) | Document version number | + +## Usage and limitations + +- Metadata fields are only available when the data source is an index +- The `_source` type is not supported by functions +- Only the `FROM` command supports the `METADATA` directive +- Once enabled, metadata fields work like regular index fields + +## Examples -Once enabled, these fields will be available to subsequent processing commands, just like other index fields: +### Basic metadata usage + +Once enabled, metadata fields are available to subsequent processing commands, just like other index fields: ```esql FROM ul_logs, apps METADATA _index, _version @@ -40,6 +56,8 @@ FROM ul_logs, apps METADATA _index, _version | 14 | apps | 1 | apps_14 | | 14 | ul_logs | 1 | ul_logs_14 | +### Metadata fields and aggregations + Similar to index fields, once an aggregation is performed, a metadata field will no longer be accessible to subsequent commands, unless used as a grouping field: ```esql @@ -51,3 +69,15 @@ FROM employees METADATA _index, _id | --- | --- | | 10100 | employees | +### Sort results by search score + +```esql +FROM products METADATA _score +| WHERE MATCH(description, "wireless headphones") +| SORT _score DESC +| KEEP name, description, _score +``` + +:::{tip} +Refer to [{{esql}} for search](docs-content://solutions/search/esql-for-search.md#esql-for-search-scoring) for more information on relevance scoring and how to use `_score` in your queries. +::: \ No newline at end of file diff --git a/docs/reference/query-languages/esql/functions-operators/search-functions.md b/docs/reference/query-languages/esql/functions-operators/search-functions.md index 44207fbf30ce7..bf31cce7df3ca 100644 --- a/docs/reference/query-languages/esql/functions-operators/search-functions.md +++ b/docs/reference/query-languages/esql/functions-operators/search-functions.md @@ -6,13 +6,19 @@ mapped_pages: # {{esql}} Search functions [esql-search-functions] +:::{tip} +Get started with {{esql}} for search use cases with +our [hands-on tutorial](docs-content://solutions/search/esql-search-tutorial.md). + +For a high-level overview of search functionalities in {{esql}}, and to learn about relevance scoring, refer to [{{esql}} for search](docs-content://solutions/search/esql-for-search.md#esql-for-search-scoring). +::: + +{{esql}} provides a set of functions for performing searching on text fields. + Use these functions for [full-text search](docs-content://solutions/search/full-text.md) and [semantic search](docs-content://solutions/search/semantic-search/semantic-search-semantic-text.md). -Get started with {{esql}} for search use cases with -our [hands-on tutorial](docs-content://solutions/search/esql-search-tutorial.md). - Full text functions can be used to match [multivalued fields](/reference/query-languages/esql/esql-multivalued-fields.md). A multivalued field that contains a value that matches a full text query is @@ -20,7 +26,7 @@ considered to match the query. Full text functions are significantly more performant for text search use cases on large data sets than using pattern matching or regular expressions with -`LIKE` or `RLIKE` +`LIKE` or `RLIKE`. See [full text search limitations](/reference/query-languages/esql/limitations.md#esql-limitations-full-text-search) for information on the limitations of full text search.