Skip to content

Support types table in lookup join docs #130410

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 14 commits into from
Jul 3, 2025
Merged

Support types table in lookup join docs #130410

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
14 commits
Select commit Hold shift + click to select a range
854d292
Support types table in lookup join docs
craigtaverner Jul 1, 2025
bad33c1
Don't show a results column in the join types
craigtaverner Jul 1, 2025
a4af15e
Merge remote-tracking branch 'origin/main' into esql_lookup_join_type…
craigtaverner Jul 2, 2025
edf5047
Newline changes after running tests
craigtaverner Jul 2, 2025
8e470d7
Fix compile error after merging main
craigtaverner Jul 2, 2025
f928275
Update docs/reference/query-languages/esql/esql-lookup-join.md
craigtaverner Jul 2, 2025
4620139
Merge branch 'main' into esql_lookup_join_types_table
craigtaverner Jul 2, 2025
138e489
Merge branch 'main' into esql_lookup_join_types_table
craigtaverner Jul 2, 2025
78e10d8
Make LOOKUP JOIN types table more compact
craigtaverner Jul 2, 2025
8e554e9
Merge branch 'main' into esql_lookup_join_types_table
craigtaverner Jul 2, 2025
2092a23
Update docs/reference/query-languages/esql/esql-lookup-join.md
craigtaverner Jul 2, 2025
724d3d4
Code review suggestions
craigtaverner Jul 2, 2025
6949a31
Merge branch 'esql_lookup_join_types_table' of github.com:craigtavern…
craigtaverner Jul 2, 2025
33401a7
Merge branch 'main' into esql_lookup_join_types_table
craigtaverner Jul 2, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
86 changes: 15 additions & 71 deletions docs/reference/query-languages/esql/_snippets/commands/types/lookup-join.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,76 +2,20 @@

**Supported types**

| main | join |
| field from the left index | field from the lookup index |
| --- | --- |
| boolean | boolean |
| byte | byte |
| byte | double |
| byte | float |
| byte | half_float |
| byte | integer |
| byte | long |
| byte | scaled_float |
| byte | short |
| date | date |
| date_nanos | date_nanos |
| double | byte |
| double | double |
| double | float |
| double | half_float |
| double | integer |
| double | long |
| double | scaled_float |
| double | short |
| float | byte |
| float | double |
| float | float |
| float | half_float |
| float | integer |
| float | long |
| float | scaled_float |
| float | short |
| half_float | byte |
| half_float | double |
| half_float | float |
| half_float | half_float |
| half_float | integer |
| half_float | long |
| half_float | scaled_float |
| half_float | short |
| integer | byte |
| integer | double |
| integer | float |
| integer | half_float |
| integer | integer |
| integer | long |
| integer | scaled_float |
| integer | short |
| ip | ip |
| keyword | keyword |
| long | byte |
| long | double |
| long | float |
| long | half_float |
| long | integer |
| long | long |
| long | scaled_float |
| long | short |
| scaled_float | byte |
| scaled_float | double |
| scaled_float | float |
| scaled_float | half_float |
| scaled_float | integer |
| scaled_float | long |
| scaled_float | scaled_float |
| scaled_float | short |
| short | byte |
| short | double |
| short | float |
| short | half_float |
| short | integer |
| short | long |
| short | scaled_float |
| short | short |
| text | keyword |
| boolean | boolean |
| byte | half_float, float, double, scaled_float, byte, short, integer, long |
| date | date |
| date_nanos | date_nanos |
| double | half_float, float, double, scaled_float, byte, short, integer, long |
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, much better now in this format!

| float | half_float, float, double, scaled_float, byte, short, integer, long |
| half_float | half_float, float, double, scaled_float, byte, short, integer, long |
| integer | half_float, float, double, scaled_float, byte, short, integer, long |
| ip | ip |
| keyword | keyword |
| long | half_float, float, double, scaled_float, byte, short, integer, long |
| scaled_float | half_float, float, double, scaled_float, byte, short, integer, long |
| short | half_float, float, double, scaled_float, byte, short, integer, long |
| text | keyword |

4 changes: 2 additions & 2 deletions ...l/src/internalClusterTest/java/org/elasticsearch/xpack/esql/action/LookupJoinTypesIT.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -768,8 +768,8 @@ private boolean isValidDataType(DataType dataType) {

private static void saveJoinTypes(Supplier<Map<List<DataType>, DataType>> signatures) throws Exception {
ArrayList<EsqlFunctionRegistry.ArgSignature> args = new ArrayList<>();
args.add(new EsqlFunctionRegistry.ArgSignature("main", null, "Field from the main index", false, false));
args.add(new EsqlFunctionRegistry.ArgSignature("join", null, "Field from the join index", false, false));
args.add(new EsqlFunctionRegistry.ArgSignature("field from the left index", null, null, false, false));
args.add(new EsqlFunctionRegistry.ArgSignature("field from the lookup index", null, null, false, false));
DocsV3Support.CommandsDocsSupport docs = new DocsV3Support.CommandsDocsSupport(
"lookup-join",
LookupJoinTypesIT.class,
Expand Down
44 changes: 43 additions & 1 deletion ...in/esql/src/test/java/org/elasticsearch/xpack/esql/expression/function/DocsV3Support.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -64,6 +64,7 @@
import java.util.Map;
import java.util.Objects;
import java.util.Optional;
import java.util.TreeMap;
import java.util.function.Function;
import java.util.function.Supplier;
import java.util.regex.Matcher;
Expand Down Expand Up @@ -311,7 +312,7 @@ public License.OperationMode invoke(List<DataType> fieldTypes) throws Exception
protected final String name;
protected final FunctionDefinition definition;
protected final Logger logger;
private final Supplier<Map<List<DataType>, DataType>> signatures;
protected final Supplier<Map<List<DataType>, DataType>> signatures;
private TempFileWriter tempFileWriter;
private final LicenseRequirementChecker licenseChecker;

Expand Down Expand Up @@ -930,6 +931,47 @@ void renderKibanaCommandDefinition() throws Exception {
writeToTempKibanaDir("definition", "json", rendered);
}
}

@Override
void renderTypes(String name, List<EsqlFunctionRegistry.ArgSignature> args) throws IOException {
assert args.size() == 2;
StringBuilder header = new StringBuilder("| ");
StringBuilder separator = new StringBuilder("| ");
List<String> argNames = args.stream().map(EsqlFunctionRegistry.ArgSignature::name).toList();
for (String arg : argNames) {
header.append(arg).append(" | ");
separator.append("---").append(" | ");
}

Map<String, List<String>> compactedTable = new TreeMap<>();
for (Map.Entry<List<DataType>, DataType> sig : this.signatures.get().entrySet()) {
if (shouldHideSignature(sig.getKey(), sig.getValue())) {
continue;
}
String mainType = sig.getKey().getFirst().esNameIfPossible();
String secondaryType = sig.getKey().get(1).esNameIfPossible();
List<String> secondaryTypes = compactedTable.computeIfAbsent(mainType, (k) -> new ArrayList<>());
secondaryTypes.add(secondaryType);
}

List<String> table = new ArrayList<>();
for (Map.Entry<String, List<String>> sig : compactedTable.entrySet()) {
String row = "| " + sig.getKey() + " | " + String.join(", ", sig.getValue()) + " |";
table.add(row);
}
Collections.sort(table);
if (table.isEmpty()) {
logger.info("Warning: No table of types generated for [{}]", name);
return;
}

String rendered = DOCS_WARNING + """
**Supported types**

""" + header + "\n" + separator + "\n" + String.join("\n", table) + "\n\n";
logger.info("Writing function types for [{}]:\n{}", name, rendered);
writeToTempSnippetsDir("types", rendered);
}
}

protected String buildFunctionSignatureSvg() throws IOException {
Expand Down
Loading