Skip to content

Conversation

@ioanatia
Copy link
Contributor

Shows how subqueries can be supported through the same mechanism as FORK.

I wanted to explore how we can enhance FROM such that it's able to receive multiple source subqueries.
In this prototype, we introduce a source command called FROMM (yes, it should just be FROM, but found it easier to add a new one for prototyping purposes).

Besides the grammar changes, we needed to modify the LogicalPlanBuilder:

    @Override
    public LogicalPlan visitFrommCommand(EsqlBaseParser.FrommCommandContext ctx) {

        List<LogicalPlan> subPlans = ctx.fromSubQuery().stream().map(subquery -> plan(subquery.query())).toList();
        return new Fork(source(ctx), subPlans, List.of());
    }

This was all that was needed to make the following examples work.

Supporting multiple rows:

FROMM (ROW a = [1,2,3], b = 1)
      (ROW a = [4, 5, 6], b = 2)
      (ROW a = [7, 8, 9], b = 3)
| MV_EXPAND a

results in:

       a       |       b       
---------------+---------------
1              |1              
2              |1              
3              |1              
4              |2              
5              |2              
6              |2              
7              |3              
8              |3              
9              |3              

Multiple FROM source subqueries:

FROMM
  (
   FROM movies
   | WHERE Title:"Shakespeare"
   | KEEP Title
  )
  (
    FROM movies METADATA _score
    | WHERE Title:"Harry Potter"
    | SORT _score DESC
    | KEEP Title, _score
    | LIMIT 3
  )
  (
    FROM movies
    | STATS c = count(*)
  )
| EVAL foo = "bar"

results in:

                 Title                  |     _score      |       c       |      foo      
----------------------------------------+-----------------+---------------+---------------
null                                    |null             |4051           |bar            
Shakespeare in Love                     |null             |null           |bar            
Harry Potter and the Sorcerer's Stone   |8.198508262634277|null           |bar            
Harry Potter and the Goblet of Fire     |7.45938777923584 |null           |bar            
Harry Potter and the Prisoner of Azkaban|7.45938777923584 |null           |bar            

Combine ROW with FROM:

FROMM (ROW a = [1,2,3], b = 1)
      (FROM movies | WHERE Title:"Shakespeare" | KEEP Title)
| MV_EXPAND a

results:

       a       |       b       |       Title       
---------------+---------------+-------------------
1              |1              |null               
2              |1              |null               
3              |1              |null               
null           |null           |Shakespeare in Love

What does not work:

  • adding a FORK command when you multiple source subqueries - that's because a single FORK is allowed and FROMM is using FORK underneath
  • having source subqueries that use multiple index patterns - this requires a bit of work, but seems doable

@ioanatia ioanatia added >non-issue :Analytics/ES|QL AKA ESQL Team:Search - Relevance The Search organization Search Relevance team labels Jun 19, 2025
@ioanatia ioanatia closed this Jun 24, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

:Analytics/ES|QL AKA ESQL >non-issue Team:Search - Relevance The Search organization Search Relevance team v9.1.0

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants