Skip to content

Commit d8c77e3

Browse files
committed
lookup join first draft
1 parent 113f0c1 commit d8c77e3

File tree

4 files changed

+84
-0
lines changed

4 files changed

+84
-0
lines changed

docs/images/esql-lookup-join.png

15.6 KB
Loading

docs/reference/query-languages/esql/esql-commands.md

Lines changed: 45 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -39,6 +39,7 @@ An {{esql}} source command produces a table, typically with data from {{es}}. An
3939
* [`GROK`](#esql-grok)
4040
* [`KEEP`](#esql-keep)
4141
* [`LIMIT`](#esql-limit)
42+
* [`LOOKUP JOIN`](#esql-lookup-join)
4243
* [preview] [`MV_EXPAND`](#esql-mv_expand)
4344
* [`RENAME`](#esql-rename)
4445
* [`SORT`](#esql-sort)
@@ -663,6 +664,50 @@ FROM employees
663664
| LIMIT 5
664665
```
665666

667+
## `LOOKUP JOIN` [esql-lookup-join]
668+
669+
`LOOKUP JOIN` is useful for any scenario where you need to pull in information from a lookup index to streamline data enrichment and analysis.
670+
671+
**Syntax**
672+
673+
```esql
674+
FROM firewall_logs
675+
| LOOKUP JOIN threat_list ON source.IP
676+
```
677+
678+
**Parameters**
679+
680+
TBD
681+
682+
**Description**
683+
684+
TBD
685+
686+
**Examples**
687+
688+
TBD
689+
690+
```esql
691+
FROM firewall_logs
692+
| LOOKUP JOIN threat_list ON source.IP
693+
```
694+
695+
696+
```esql
697+
FROM system_metrics
698+
| LOOKUP JOIN host_inventory ON host.name
699+
| LOOKUP JOIN employees ON host.name
700+
```
701+
702+
TBD
703+
704+
```esql
705+
FROM app_logs
706+
| LOOKUP JOIN service_owners ON service_id
707+
```
708+
709+
710+
In case of name collisions, the newly created columns will override existing columns.
666711

667712
## `MV_EXPAND` [esql-mv_expand]
668713

Lines changed: 38 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,38 @@
1+
---
2+
navigation_title: "LOOKUP JOIN"
3+
mapped_pages:
4+
- https://www.elastic.co/guide/en/elasticsearch/reference/current/esql-enrich-data.html
5+
---
6+
7+
# LOOKUP JOIN [esql-lookup-join]
8+
9+
The {{esql}} [`LOOKUP join`](/reference/query-languages/esql/esql-commands.md#esql-lookup-index) processing command combines, at query-time, data from one or more source indexes with field-value combinations found in an input table.
10+
11+
For example, you can use `LOOKUP JOIN` to:
12+
13+
* Pull in environment or ownership details for each host to enrich your metrics data
14+
* Quickly see if any source IPs match known malicious addresses.
15+
* Tag logs with the owning team or escalation info for faster triage and incident response.
16+
17+
18+
### How the `LOOKUP JOIN` command works [esql-how-lookup-join-works]
19+
20+
The `LOOKUP JOIN` command adds new columns to a table, with data from {{es}} indices. It requires a few special components:
21+
22+
:::{image} ../../../images/esql-lookup-join.png
23+
:alt: esql lookup join
24+
:::
25+
26+
27+
$$$esql-source-index$$$
28+
29+
Source index
30+
: An index which stores enrich data that the `LOOKUP` command can add to input tables. You can create and manage these indices just like a regular {{es}} index. You can use multiple source indices in an enrich policy. You also can use the same source index in multiple enrich policies.
31+
32+
33+
### Prerequisites [esql-enrich-prereqs]
34+
35+
To use `LOOKUP JOIN`, you must have:
36+
37+
* Data types of join key and join field in the lookup index need to generally be the same - up to widening of data types, where e.g. `short,byte` are considered equal to `integer`. Also, text fields can be used on the left hand side if and only if there is an exact subfield whose name is suffixed with `.keyword`.
38+

docs/reference/toc.yml

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -513,6 +513,7 @@ toc:
513513
- file: query-languages/esql/esql-multivalued-fields.md
514514
- file: query-languages/esql/esql-process-data-with-dissect-grok.md
515515
- file: query-languages/esql/esql-enrich-data.md
516+
- file: query-languages/esql/esql-lookup-join.md
516517
- file: query-languages/esql/esql-implicit-casting.md
517518
- file: query-languages/esql/esql-time-spans.md
518519
- file: query-languages/esql/limitations.md

0 commit comments

Comments
 (0)