[9.1] [NL2ESQL] Fix tool going into endless loop for ES|QL LOOKUP JOIN (elastic#231217) (elastic#231654)

# Backport

This will backport the following commits from `main` to `9.1`:
- [[NL2ESQL] Fix tool going into endless loop for ES|QL LOOKUP JOIN
(elastic#231217)](elastic#231217)

<!--- Backport version: 10.0.1 -->

### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sorenlouv/backport)

<!--BACKPORT [{"author":{"name":"Quynh Nguyen
(Quinn)","email":"[email protected]"},"sourceCommit":{"committedDate":"2025-08-12T21:02:42Z","message":"[NL2ESQL]
Fix tool going into endless loop for ES|QL LOOKUP JOIN (elastic#231217)\n\n##
Summary\n\nThis PR fixes elastic#229979, where a behavior with assistant going
into an\nendless loops with whenever it tries to find documentation for
ES|QL\nlookup join. This PR is part 1 of 2, which fixes an issue with
the\nLOOKUP JOIN command couldn't be found. We'll have another PR follow
up\nwhich will add a limit to how much a tool/function can be called
to\nprevent problems like this from
happening.\n\nTracing:\n\nhttps://35-187-109-62.sslip.io/projects/UHJvamVjdDoxMTAy/spans/666d7a485eaa2984ae20e180d2f7a2e4?selectedSpanNodeId=U3Bhbjo2MTg5NDI%3D\n\n\nhttps://35-187-109-62.sslip.io/projects/UHJvamVjdDoxMTAy/traces/879e6ccb74927318ee675710e5cde890?selected\n\n###
Checklist\n\nCheck the PR satisfies following conditions. \n\nReviewers
should verify this PR satisfies this list as well.\n\n- [ ] Any text
added follows [EUI's
writing\nguidelines](https://elastic.github.io/eui/#/guidelines/writing),
uses\nsentence case text and includes
[i18n\nsupport](https://github.com/elastic/kibana/blob/main/src/platform/packages/shared/kbn-i18n/README.md)\n-
[
]\n[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)\nwas
added for features that require explanation or tutorials\n- [ ] [Unit or
functional\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\nwere
updated or added to match the most common scenarios\n- [ ] If a plugin
configuration key changed, check if it needs to be\nallowlisted in the
cloud and added to the
[docker\nlist](https://github.com/elastic/kibana/blob/main/src/dev/build/tasks/os_packages/docker_generator/resources/base/bin/kibana-docker)\n-
[ ] This was checked for breaking HTTP API changes, and any
breaking\nchanges have been approved by the breaking-change committee.
The\n`release_note:breaking` label should be applied in these
situations.\n- [ ] [Flaky
Test\nRunner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1)
was\nused on any tests changed\n- [ ] The PR description includes the
appropriate Release Notes section,\nand the correct `release_note:*`
label is applied per
the\n[guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)\n-
[ ] Review the
[backport\nguidelines](https://docs.google.com/document/d/1VyN5k91e5OVumlc0Gb9RPa3h1ewuPE705nRtioPiTvY/edit?usp=sharing)\nand
apply applicable `backport:*` labels.\n\n### Identify risks\n\nDoes this
PR introduce any risks? For example, consider risks like hard\nto test
bugs, performance regression, potential of data loss.\n\nDescribe the
risk, its severity, and mitigation for each identified\nrisk. Invite
stakeholders and evaluate how to proceed before merging.\n\n- [ ] [See
some
risk\nexamples](https://github.com/elastic/kibana/blob/main/RISK_MATRIX.mdx)\n-
[ ] ...\n\nCo-authored-by: Elastic Machine
<[email protected]>","sha":"b93f08ac49c95f89bd72d64a9f4c083cd21a01f6","branchLabelMapping":{"^v9.2.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["bug","release_note:fix","ci:project-deploy-observability","backport:version","v9.2.0","v9.0.5","v9.1.2","v8.19.2"],"title":"[NL2ESQL]
Fix tool going into endless loop for ES|QL LOOKUP
JOIN","number":231217,"url":"https://github.com/elastic/kibana/pull/231217","mergeCommit":{"message":"[NL2ESQL]
Fix tool going into endless loop for ES|QL LOOKUP JOIN (elastic#231217)\n\n##
Summary\n\nThis PR fixes elastic#229979, where a behavior with assistant going
into an\nendless loops with whenever it tries to find documentation for
ES|QL\nlookup join. This PR is part 1 of 2, which fixes an issue with
the\nLOOKUP JOIN command couldn't be found. We'll have another PR follow
up\nwhich will add a limit to how much a tool/function can be called
to\nprevent problems like this from
happening.\n\nTracing:\n\nhttps://35-187-109-62.sslip.io/projects/UHJvamVjdDoxMTAy/spans/666d7a485eaa2984ae20e180d2f7a2e4?selectedSpanNodeId=U3Bhbjo2MTg5NDI%3D\n\n\nhttps://35-187-109-62.sslip.io/projects/UHJvamVjdDoxMTAy/traces/879e6ccb74927318ee675710e5cde890?selected\n\n###
Checklist\n\nCheck the PR satisfies following conditions. \n\nReviewers
should verify this PR satisfies this list as well.\n\n- [ ] Any text
added follows [EUI's
writing\nguidelines](https://elastic.github.io/eui/#/guidelines/writing),
uses\nsentence case text and includes
[i18n\nsupport](https://github.com/elastic/kibana/blob/main/src/platform/packages/shared/kbn-i18n/README.md)\n-
[
]\n[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)\nwas
added for features that require explanation or tutorials\n- [ ] [Unit or
functional\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\nwere
updated or added to match the most common scenarios\n- [ ] If a plugin
configuration key changed, check if it needs to be\nallowlisted in the
cloud and added to the
[docker\nlist](https://github.com/elastic/kibana/blob/main/src/dev/build/tasks/os_packages/docker_generator/resources/base/bin/kibana-docker)\n-
[ ] This was checked for breaking HTTP API changes, and any
breaking\nchanges have been approved by the breaking-change committee.
The\n`release_note:breaking` label should be applied in these
situations.\n- [ ] [Flaky
Test\nRunner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1)
was\nused on any tests changed\n- [ ] The PR description includes the
appropriate Release Notes section,\nand the correct `release_note:*`
label is applied per
the\n[guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)\n-
[ ] Review the
[backport\nguidelines](https://docs.google.com/document/d/1VyN5k91e5OVumlc0Gb9RPa3h1ewuPE705nRtioPiTvY/edit?usp=sharing)\nand
apply applicable `backport:*` labels.\n\n### Identify risks\n\nDoes this
PR introduce any risks? For example, consider risks like hard\nto test
bugs, performance regression, potential of data loss.\n\nDescribe the
risk, its severity, and mitigation for each identified\nrisk. Invite
stakeholders and evaluate how to proceed before merging.\n\n- [ ] [See
some
risk\nexamples](https://github.com/elastic/kibana/blob/main/RISK_MATRIX.mdx)\n-
[ ] ...\n\nCo-authored-by: Elastic Machine
<[email protected]>","sha":"b93f08ac49c95f89bd72d64a9f4c083cd21a01f6"}},"sourceBranch":"main","suggestedTargetBranches":["9.0","9.1"],"targetPullRequestStates":[{"branch":"main","label":"v9.2.0","branchLabelMappingKey":"^v9.2.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/231217","number":231217,"mergeCommit":{"message":"[NL2ESQL]
Fix tool going into endless loop for ES|QL LOOKUP JOIN (elastic#231217)\n\n##
Summary\n\nThis PR fixes elastic#229979, where a behavior with assistant going
into an\nendless loops with whenever it tries to find documentation for
ES|QL\nlookup join. This PR is part 1 of 2, which fixes an issue with
the\nLOOKUP JOIN command couldn't be found. We'll have another PR follow
up\nwhich will add a limit to how much a tool/function can be called
to\nprevent problems like this from
happening.\n\nTracing:\n\nhttps://35-187-109-62.sslip.io/projects/UHJvamVjdDoxMTAy/spans/666d7a485eaa2984ae20e180d2f7a2e4?selectedSpanNodeId=U3Bhbjo2MTg5NDI%3D\n\n\nhttps://35-187-109-62.sslip.io/projects/UHJvamVjdDoxMTAy/traces/879e6ccb74927318ee675710e5cde890?selected\n\n###
Checklist\n\nCheck the PR satisfies following conditions. \n\nReviewers
should verify this PR satisfies this list as well.\n\n- [ ] Any text
added follows [EUI's
writing\nguidelines](https://elastic.github.io/eui/#/guidelines/writing),
uses\nsentence case text and includes
[i18n\nsupport](https://github.com/elastic/kibana/blob/main/src/platform/packages/shared/kbn-i18n/README.md)\n-
[
]\n[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)\nwas
added for features that require explanation or tutorials\n- [ ] [Unit or
functional\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\nwere
updated or added to match the most common scenarios\n- [ ] If a plugin
configuration key changed, check if it needs to be\nallowlisted in the
cloud and added to the
[docker\nlist](https://github.com/elastic/kibana/blob/main/src/dev/build/tasks/os_packages/docker_generator/resources/base/bin/kibana-docker)\n-
[ ] This was checked for breaking HTTP API changes, and any
breaking\nchanges have been approved by the breaking-change committee.
The\n`release_note:breaking` label should be applied in these
situations.\n- [ ] [Flaky
Test\nRunner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1)
was\nused on any tests changed\n- [ ] The PR description includes the
appropriate Release Notes section,\nand the correct `release_note:*`
label is applied per
the\n[guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)\n-
[ ] Review the
[backport\nguidelines](https://docs.google.com/document/d/1VyN5k91e5OVumlc0Gb9RPa3h1ewuPE705nRtioPiTvY/edit?usp=sharing)\nand
apply applicable `backport:*` labels.\n\n### Identify risks\n\nDoes this
PR introduce any risks? For example, consider risks like hard\nto test
bugs, performance regression, potential of data loss.\n\nDescribe the
risk, its severity, and mitigation for each identified\nrisk. Invite
stakeholders and evaluate how to proceed before merging.\n\n- [ ] [See
some
risk\nexamples](https://github.com/elastic/kibana/blob/main/RISK_MATRIX.mdx)\n-
[ ] ...\n\nCo-authored-by: Elastic Machine
<[email protected]>","sha":"b93f08ac49c95f89bd72d64a9f4c083cd21a01f6"}},{"branch":"9.0","label":"v9.0.5","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"9.1","label":"v9.1.2","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"8.19","label":"v8.19.2","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"url":"https://github.com/elastic/kibana/pull/231550","number":231550,"state":"OPEN"}]}]
BACKPORT-->

Co-authored-by: Elastic Machine <[email protected]>

Author: qn895

x-pack/platform/plugins/shared/inference/server/tasks/nl_to_esql/doc_base/aliases.ts

@@ -12,7 +12,7 @@
 const aliases: Record<string, string[]> = {
   STATS: ['STATS_BY', 'BY', 'STATS...BY', 'STATS ... BY'],
   OPERATORS: ['LIKE', 'RLIKE', 'IN'],
-  JOIN: ['LOOKUP JOIN'],
+  LOOKUP_JOIN: ['LOOKUPJOIN'],
 };
 
 const getAliasMap = () => {

x-pack/platform/plugins/shared/inference/server/tasks/nl_to_esql/doc_base/esql_doc_base.ts

@@ -43,8 +43,7 @@ export class EsqlDocumentBase {
     }: GetDocsOptions = {}
   ) {
     const keywords = rawKeywords.map((raw) => {
-      // LOOKUP JOIN  has space so we want to retain as is
-      let keyword = raw.toLowerCase().includes('join') ? raw : format(raw);
+      let keyword = format(raw);
       if (resolveAliases) {
         keyword = tryResolveAlias(keyword);
       }

x-pack/solutions/observability/plugins/observability_ai_assistant_app/scripts/evaluation/scenarios/esql/index.spec.ts

@@ -28,7 +28,7 @@ async function evaluateEsqlQuery({
   const evaluation = await chatClient.evaluate(conversation, [
     ...(expected
       ? [
-          `Returns a ES|QL query that is functionally equivalent to:      
+          `Returns a ES|QL query that is functionally equivalent to:
       ${expected}. It's OK if column names are slightly different, as long as the expected end result is the same.`,
         ]
       : []),