Skip to content

Commit 5bc429e

Browse files
alex-spiesomricohenn
authored andcommitted
ESQL: Add more details on ENRICH vs. LOOKUP JOIN to docs (elastic#125487)
* Add more details on ENRICH vs. LOOKUP JOIN * Move example, fix syntax formatting
1 parent daccc10 commit 5bc429e

File tree

3 files changed

+17
-15
lines changed

3 files changed

+17
-15
lines changed

docs/reference/query-languages/esql/esql-commands.md

Lines changed: 9 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -677,17 +677,11 @@ This functionality is in technical preview and may be changed or removed in a fu
677677

678678
**Syntax**
679679

680-
```
680+
```esql
681681
FROM <source_index>
682682
| LOOKUP JOIN <lookup_index> ON <field_name>
683683
```
684684

685-
```esql
686-
FROM firewall_logs
687-
| LOOKUP JOIN threat_list ON source.IP
688-
| WHERE threat_level IS NOT NULL
689-
```
690-
691685
**Parameters**
692686

693687
`<lookup_index>`
@@ -717,6 +711,14 @@ FROM firewall_logs
717711
| LOOKUP JOIN threat_list ON source.IP
718712
```
719713

714+
To filter only for those rows that have a matching `threat_list` entry, use `WHERE ... IS NOT NULL` with a field from the lookup index:
715+
716+
```esql
717+
FROM firewall_logs
718+
| LOOKUP JOIN threat_list ON source.IP
719+
| WHERE threat_level IS NOT NULL
720+
```
721+
720722
**Host metadata correlation**: This query pulls in environment or ownership details for each host to correlate with your metrics data.
721723

722724
```esql

docs/reference/query-languages/esql/esql-enrich-data.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -19,10 +19,10 @@ For example, you can use `ENRICH` to:
1919

2020
* Enrichment data doesn't change frequently
2121
* You can accept index-time overhead
22-
* You are working with structured enrichment patterns
2322
* You can accept having multiple matches combined into multi-values
2423
* You can accept being limited to predefined match fields
25-
* `ENRICH` has a simplified security model. There are no restirctions to specific enrich policies or document and field level security.
24+
* You do not need fine-grained security: There are no restrictions to specific enrich policies or document and field level security.
25+
* You want to match using ranges or spatial relations
2626

2727
### How the `ENRICH` command works [esql-how-enrich-works]
2828

docs/reference/query-languages/esql/esql-lookup-join.md

Lines changed: 6 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -20,11 +20,11 @@ For example, you can use `LOOKUP JOIN` to:
2020

2121
* Your enrichment data changes frequently
2222
* You want to avoid index-time processing
23-
* You're working with regular indices
24-
* You need to preserve distinct matches
23+
* You want SQL-like behavior, so that multiple matches result in multiple rows
2524
* You need to match on any field in a lookup index
2625
* You use document or field level security
27-
* You want to restrict users to a specific lookup indices that they can you
26+
* You want to restrict users to use only specific lookup indices
27+
* You do not need to match using ranges or spatial relations
2828

2929
## How the `LOOKUP JOIN` command works [esql-how-lookup-join-works]
3030

@@ -108,7 +108,7 @@ FROM employees
108108
To use `LOOKUP JOIN`, the following requirements must be met:
109109

110110
* **Compatible data types**: The join key and join field in the lookup index must have compatible data types. This means:
111-
* The data types must either be identical or be internally represented as the same type in Elasticsearch's type system
111+
* The data types must either be identical or be internally represented as the same type in {esql}
112112
* Numeric types follow these compatibility rules:
113113
* `short` and `byte` are compatible with `integer` (all represented as `int`)
114114
* `float`, `half_float`, and `scaled_float` are compatible with `double` (all represented as `double`)
@@ -120,9 +120,9 @@ For a complete list of supported data types and their internal representations,
120120

121121
The following are the current limitations with `LOOKUP JOIN`
122122

123-
* `LOOKUP JOIN` will be successful if the join field in the lookup index is a `KEYWORD` type. If the main index's join field is `TEXT` type, it must have an exact `.keyword` subfield that can be matched with the lookup index's `KEYWORD` field.
124123
* Indices in [lookup](/reference/elasticsearch/index-settings/index-modules.md#index-mode-setting) mode are always single-sharded.
125124
* Cross cluster search is unsupported. Both source and lookup indices must be local.
125+
* Currently, only matching on equality is supported.
126126
* `LOOKUP JOIN` can only use a single match field and a single index. Wildcards, aliases, datemath, and datastreams are not supported.
127-
* The name of the match field in `LOOKUP JOIN lu_idx ON match_field` must match an existing field in the query. This may require renames or evals to achieve.
127+
* The name of the match field in `LOOKUP JOIN lu_idx ON match_field` must match an existing field in the query. This may require `RENAME`s or `EVAL`s to achieve.
128128
* The query will circuit break if there are too many matching documents in the lookup index, or if the documents are too large. More precisely, `LOOKUP JOIN` works in batches of, normally, about 10,000 rows; a large amount of heap space is needed if the matching documents from the lookup index for a batch are multiple megabytes or larger. This is roughly the same as for `ENRICH`.

0 commit comments

Comments
 (0)