ESQL: Enable Lookup Join on Expression Tech Preview (elastic#134952)

Enable Lookup Join on Expression Tech Preview

Author: julian-elastic

docs/changelog/134952.yaml

@@ -0,0 +1,11 @@
+pr: 134952
+summary: Add support for expressions with LOOKUP JOIN in tech preview
+area: ES|QL
+type: enhancement
+issues: [ ]
+highlight:
+  title: Add support for expressions with LOOKUP JOIN in tech preview
+  body: |-
+    Enable Lookup Join on Expression Tech Preview
+    FROM index1 | LOOKUP JOIN lookup_index on left_field1 > right_field1 AND left_field2 <= right_field2
+  notable: true

docs/reference/query-languages/esql/_snippets/commands/layout/lookup-join.md

@@ -19,17 +19,22 @@ FROM <source_index>
 FROM <source_index>
 | LOOKUP JOIN <lookup_index> ON <field_name1>, <field_name2>, <field_name3>
 ```
+```esql
+FROM <source_index>
+| LOOKUP JOIN <lookup_index> ON <left_field1> >= <lookup_field1> AND <left_field2> == <lookup_field2>
+```
 
 **Parameters**
 
 `<lookup_index>`
 :   The name of the lookup index. This must be a specific index name - wildcards, aliases, and remote cluster references are not supported. Indices used for lookups must be configured with the [`lookup` index mode](/reference/elasticsearch/index-settings/index-modules.md#index-mode-setting).
 
-`<field_name>` or `<field_name1>, <field_name2>, <field_name3>`
-:   The field(s) to join on. Can be either:
-  * A single field name
-  * A comma-separated list of field names {applies_to}`stack: ga 9.2`
-:   These fields must exist in both your current query results and in the lookup index. If the fields contains multi-valued entries, those entries will not match anything (the added fields will contain `null` for those rows).
+`<field_name>` or `<field_name1>, <field_name2>, <field_name3>` or `<left_field1> >= <lookup_field1> AND <left_field2> == <lookup_field2>`
+:   The join condition. Can be one of the following:
+   * A single field name
+   * A comma-separated list of field names {applies_to}`stack: ga 9.2`
+   * An expression with one or more join conditions linked by `AND`. Each condition compares a field from the left index with a field from the lookup index using [binary operators](/reference/query-languages/esql/functions-operators/operators.md#esql-binary-operators) (`==`, `>=`, `<=`, `>`, `<`, `!=`). Each field name in the join condition must exist in only one of the indexes. Use RENAME to resolve naming conflicts. {applies_to}`stack: preview 9.2` {applies_to}`serverless: preview`
+:   If using join on a single field or a field list, the fields used must exist in both your current query results and in the lookup index. If the fields contains multi-valued entries, those entries will not match anything (the added fields will contain `null` for those rows).
 
 
 **Description**

docs/reference/query-languages/esql/esql-lookup-join.md

@@ -34,13 +34,15 @@ The `LOOKUP JOIN` command adds fields from the lookup index as new columns to yo
 
 The command requires two parameters:
 * The name of the lookup index (which must have the `lookup` [`index.mode setting`](/reference/elasticsearch/index-settings/index-modules.md#index-mode-setting))
-* The field(s) to join on. Can be either:
-  * A single field name
-  * A comma-separated list of field names {applies_to}`stack: ga 9.2`
+* The join condition. Can be one of the following:
+   * A single field name
+   * A comma-separated list of field names {applies_to}`stack: ga 9.2`
+   * An expression with one or more join conditions linked by `AND`. Each condition compares a field from the left index with a field from the lookup index using [binary operators](/reference/query-languages/esql/functions-operators/operators.md#esql-binary-operators) (`==`, `>=`, `<=`, `>`, `<`, `!=`). Each field name in the join condition must exist in only one of the indexes. Use RENAME to resolve naming conflicts. {applies_to}`stack: preview 9.2` {applies_to}`serverless: preview`
 
 ```esql
 LOOKUP JOIN <lookup_index> ON <field_name>  # Join on a single field
 LOOKUP JOIN <lookup_index> ON <field_name1>, <field_name2>, <field_name3>  # Join on multiple fields
+LOOKUP JOIN <lookup_index> ON <left_field1> >= <lookup_field1> AND <left_field2> == <lookup_field2>  # Join on expression
 ```
 
 :::{image} ../images/esql-lookup-join.png

x-pack/plugin/esql/src/main/java/org/elasticsearch/xpack/esql/action/EsqlCapabilities.java

@@ -1478,7 +1478,7 @@ public enum Cap {
         /**
          * Allow lookup join on boolean expressions
          */
-        LOOKUP_JOIN_ON_BOOLEAN_EXPRESSION(Build.current().isSnapshot()),
+        LOOKUP_JOIN_ON_BOOLEAN_EXPRESSION,
 
         /**
          * FORK with remote indices

x-pack/plugin/esql/src/test/java/org/elasticsearch/xpack/esql/parser/StatementParserTests.java

@@ -3250,13 +3250,13 @@ public void testValidJoinPatternFieldJoin() {
         assertThat(joinType.coreJoin().joinName(), equalTo("LEFT OUTER"));
     }
 
-    public void testExpressionJoinNonSnapshotBuild() {
-        assumeFalse("LOOKUP JOIN is not yet in non-snapshot builds", Build.current().isSnapshot());
-        expectThrows(
-            ParsingException.class,
-            startsWith("line 1:31: JOIN ON clause only supports fields at the moment."),
-            () -> statement("FROM test | LOOKUP JOIN test2 ON left_field >= right_field")
-        );
+    /**
+     * Verify that both in snapshot and in release build the feature is enabled and the parsing works
+     * without checking for the capability
+     */
+    public void testExpressionJoinEnabled() {
+        var plan = statement("FROM test | LOOKUP JOIN test2 ON left_field >= right_field");
+        var join = as(plan, LookupJoin.class);
     }
 
     public void testValidJoinPatternExpressionJoin() {
@@ -3290,7 +3290,6 @@ public void testValidJoinPatternExpressionJoin() {
             }
         }
 
-        // add a check that the feature is disabled on non-snaphsot build
         String query = "FROM " + basePattern + " | LOOKUP JOIN " + joinPattern + " ON " + onExpressionString;
         var plan = statement(query);