ES|QL: INLINE STATS leads to "can't build page out of released blocks"

[luigidellaquila]

With CSV dataset

from dense_vector,airport_city_bound* 
| eval  kMvNZVQSH = -4048759096317256968, GzAYNHaHuIS = 4262770411405329967, EwPpQFmOPoB = "lkdgnbTrBT", `abbrev` = "LiXI" 
| keep `city`, `kMvNZVQSH`, GzAYNHaHuIS, city_boundary, EwPpQFmOPoB, `city_boundary`, `id`, kMvNZVQSH, id, airport 
| limit 6888 
| keep `kMvNZVQSH`, EwPpQFmOPoB, city, `id`, kMvNZVQSH 
| stats  `EwPpQFmOPoB` = min(id), `EwPpQFmOPoB` = count(*), kZiJPhNZ = count(id), ErGhdKhdv = count_distinct(city), EwPpQFmOPoB = min(id) | eval  ErGhdKhdv = null, fHSyxniFCS = -710891486, ErGhdKhdv = -1104488048, zdxdQdVIyFh = -1579062572, `kZiJPhNZ` = null, SzmOySyUh = null, eWVVRiwP = 1092261898855405071, nUxVhGhcpkiV = true, RvqGioBAAzYs = 169356242 
| eval  `kZiJPhNZ` = "gWJHDTcDMjRVSnlzSX", lxLgaagX = 4700531997851493340, `kZiJPhNZ` = false 
| keep `zdxdQdVIyFh` 
| inline stats  `zdxdQdVIyFh` = count(*), WbWVJnYJ = count(*), zdxdQdVIyFh = max(zdxdQdVIyFh), uKjfeKTfaH = max(zdxdQdVIyFh), QpovVxjMWSjh = min(zdxdQdVIyFh)

"type": "illegal_argument_exception",
"reason": "can't build page out of released blocks but [ConstantNullBlock[positions=1]] was released"

Caused by: java.lang.IllegalArgumentException: can't build page out of released blocks but [ConstantNullBlock[positions=1]] was released
	at org.elasticsearch.compute.data.Page.<init>(Page.java:76)
	at org.elasticsearch.compute.data.Page.<init>(Page.java:53)
	at org.elasticsearch.compute.operator.LocalSourceOperator.lambda$new$2(LocalSourceOperator.java:59)
	at org.elasticsearch.compute.operator.LocalSourceOperator.getOutput(LocalSourceOperator.java:79)
	at org.elasticsearch.compute.operator.Driver.runSingleLoopIteration(Driver.java:272)
	at org.elasticsearch.compute.operator.Driver.run(Driver.java:186)
	at org.elasticsearch.compute.operator.Driver$1.doRun(Driver.java:420)
	at org.elasticsearch.server@9.2.0-SNAPSHOT/org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:27)

[elasticsearchmachine]

Pinging @elastic/es-analytical-engine (Team:Analytics)

[luigidellaquila]

cc @astefan maybe you could be interested

[bpintea]

Just a quick look into this: it reproduces with:

from employees
| stats m = count(emp_no) 
| eval c = 0
| keep c 
| inline stats c = count(*)

and likely has the same cause as seen before: accessing twice the blocks in a LocalRelation:

[2025-10-08T10:48:16,738][DEBUG][o.e.x.e.s.EsqlSession    ] [runTask-0] Optimized logicalPlan plan:
Limit[1000[INTEGER],false]
\_InlineJoin[LEFT,[],[]]
  |_EsqlProject[[c{r}#1238]]
  | \_Eval[[0[INTEGER] AS c#1238]]
  |   \_LocalRelation[[m{r}#1236],Page{blocks=[ConstantNullBlock[positions=1]]}]
  \_Aggregate[[],[COUNT(*[KEYWORD],true[BOOLEAN]) AS c#1241]]
    \_StubRelation[[c{r}#1238]]
[2025-10-08T10:48:16,739][DEBUG][o.e.x.e.s.EsqlSession    ] [runTask-0] Executing subplan:
Aggregate[[],[COUNT(*[KEYWORD],true[BOOLEAN]) AS c#1241]]
\_EsqlProject[[c{r}#1238]]
  \_Eval[[0[INTEGER] AS c#1238]]
    \_LocalRelation[[m{r}#1236],Page{blocks=[ConstantNullBlock[positions=1]]}]
[...]
[2025-10-08T10:48:16,741][DEBUG][o.e.x.e.s.EsqlSession    ] [runTask-0] Executing final plan:
Limit[1000[INTEGER],false]
\_Eval[[1[LONG] AS c#1241]]
  \_EsqlProject[[c{r}#1238]]
    \_Eval[[0[INTEGER] AS c#1238]]
      \_LocalRelation[[m{r}#1236],Page{blocks=[ConstantNullBlock[positions=1]]}]

[astefan]

from colors,airports_web,boo*
| rename scalerank as language_code
| lookup join languages_lookup on language_code
| keep `ratings`, `rgb_vector`
| mv_expand ratings
| keep `ratings`, `rgb_vector`
| inline stats  rycaPYFzpAId = absent(ratings), WkdxfjwuR = count(ratings) + avg(ratings)
| rename ratings AS `WkdxfjwuR`
| inline stats  WkdxfjwuR = sum(WkdxfjwuR) + median(WkdxfjwuR), sjivSBvV = count(*)
| sort sjivSBvV NULLS FIRST
| where false OR  NOT true OR false OR  NOT false
| where false OR false AND false AND  NOT true AND false
| stats  rycaPYFzpAId = median(sjivSBvV) + count(sjivSBvV) + sum(sjivSBvV), NUiLBJXQoMXd = present(sjivSBvV)
| mv_expand `NUiLBJXQoMXd`
| inline stats  NUiLBJXQoMXd = count(rycaPYFzpAId)

java.lang.IllegalStateException: can't release already released object [DoubleArrayBlock[positions=1, mvOrdering=UNORDERED, vector=DoubleArrayVector[positions=1, values=[0.0]]]]
        at org.elasticsearch.base@9.3.0-SNAPSHOT/org.elasticsearch.core.Releasables.closeExpectNoException(Releasables.java:64)
        at org.elasticsearch.compute.data.Page.releaseBlocks(Page.java:245)
        at org.elasticsearch.compute.operator.AbstractPageMappingOperator.lambda$close$0(AbstractPageMappingOperator.java:107)
        at org.elasticsearch.base@9.3.0-SNAPSHOT/org.elasticsearch.core.Releasables.close(Releasables.java:39)
        at org.elasticsearch.base@9.3.0-SNAPSHOT/org.elasticsearch.core.Releasables.closeExpectNoException(Releasables.java:72)
        at org.elasticsearch.compute.operator.AbstractPageMappingOperator.close(AbstractPageMappingOperator.java:107)
        at org.elasticsearch.compute.operator.EvalOperator.lambda$close$1(EvalOperator.java:64)
        at org.elasticsearch.base@9.3.0-SNAPSHOT/org.elasticsearch.core.Releasables.close(Releasables.java:39)
        at org.elasticsearch.base@9.3.0-SNAPSHOT/org.elasticsearch.core.Releasables.close(Releasables.java:48)
        at org.elasticsearch.base@9.3.0-SNAPSHOT/org.elasticsearch.core.Releasables.closeExpectNoException(Releasables.java:62)
        at org.elasticsearch.compute.operator.EvalOperator.close(EvalOperator.java:61)
        at org.elasticsearch.base@9.3.0-SNAPSHOT/org.elasticsearch.core.Releasables.close(Releasables.java:39)
        at org.elasticsearch.base@9.3.0-SNAPSHOT/org.elasticsearch.core.Releasables.closeWhileHandlingException(Releasables.java:83)
        at org.elasticsearch.compute.operator.Driver.drainAndCloseOperators(Driver.java:396)
        at org.elasticsearch.compute.operator.Driver$1.onFailure(Driver.java:439)
        at org.elasticsearch.server@9.3.0-SNAPSHOT/org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:29)
        at org.elasticsearch.compute.operator.DriverScheduler$1.doRun(DriverScheduler.java:57)```

[astefan]

GenerativeIT > test FAILED
    java.lang.AssertionError: query: from app_log* | FORK  (where false OR  NOT true) (where true) (where true) (where false OR true) (where false) (where true | mv_expand @timestamp) | WHERE _fork == "fork2" | DROP _fork| rename service_id as name_str | lookup join multi_column_joinable_lookup on name_str | inline stats  `is_active_bool` = count(id_int) + max(id_int), siofAMlpoHvh = max(id_int) + max(id_int) + max(id_int), `message` = present(other1), `id_int` = max(id_int) | stats  `message` = count(is_active_bool), id_int = median(is_active_bool), `siofAMlpoHvh` = count(*), `id_int` = max(id_int) | eval  NUtENNcs = message, message = "zaYFcCTy", id_int = null, VizcTbjMnBlQ = siofAMlpoHvh, siofAMlpoHvh = siofAMlpoHvh + siofAMlpoHvh + siofAMlpoHvh, gPzLgKdLfZ = null, `message` = 1099438720, sdsrveQOoi = -2373351299779434224, hpxNweCm = false, PwyOmHforRKI = -2230544247357512169 | stats  VizcTbjMnBlQ = count(*), siofAMlpoHvh = median(sdsrveQOoi) + max(sdsrveQOoi), DUuKdsGgU = avg(PwyOmHforRKI) + count(PwyOmHforRKI), `gPzLgKdLfZ` = present(sdsrveQOoi + VizcTbjMnBlQ + PwyOmHforRKI), `siofAMlpoHvh` = count(sdsrveQOoi) + max(sdsrveQOoi) | limit 61 | inline stats  gPzLgKdLfZ = sum(VizcTbjMnBlQ), `siofAMlpoHvh` = count(*), zvgqSiVGqli = max(VizcTbjMnBlQ), WFjgBzYRXwa = median(VizcTbjMnBlQ) + count(VizcTbjMnBlQ) + avg(VizcTbjMnBlQ) | keep `VizcTbjMnBlQ` | inline stats  wXPJguff = median(VizcTbjMnBlQ) + avg(VizcTbjMnBlQ) + max(VizcTbjMnBlQ)
    exception: method [POST], host [http://[::1]:52974], URI [/_query?format=cbor&pretty=true&error_trace=true], status line [HTTP/1.1 500 Internal Server Error]
    Warnings: [No limit defined, adding default limit of [1000], Line 1:890: Field 'siofAMlpoHvh' shadowed by field at line 1:1062, Line 1:488: Field 'id_int' shadowed by field at line 1:548]
    ┬┐eerror┬┐jroot_cause┬ƒ┬┐dtypewillegal_state_exceptionfreasonx↑can't read released pagekstack_trace⌂y☼┬£org.elasticsearch.ElasticsearchException$1: can't read released page

    Caused by: java.lang.IllegalStateException: can't read released page
        at org.elasticsearch.compute.data.Page.getBlock(Page.java:143)
        at org.elasticsearch.compute.aggregation.CountAggregatorFunction.addRawInput(CountAggregatorFunction.java:86)
        at org.elasticsearch.compute.aggregation.Aggregator.processPage(Aggregator.java:43)
        at org.elasticsearch.compute.operator.AggregationOperator.addInput(AggregationOperator.java:111)
        at org.elasticsearch.compute.operator.Driver.runSingleLoopIteration(Driver.java:286)
        at org.elasticsearch.compute.operator.Driver.run(Driver.java:186)
        at org.elasticsearch.compute.operator.Driver$1.doRun(Driver.java:420)
        at org.elasticsearch.server@9.3.0-SNAPSHOT/org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:27)
        at org.elasticsearch.compute.operator.DriverScheduler$1.doRun(DriverScheduler.java:57)
        ... 7 more
    ├┐├┐├┐dtypewillegal_state_exceptionfreasonx↑can't read released pagekstack_tracey♠┬èjava.lang.IllegalStateException: can't read released page
        at org.elasticsearch.compute.data.Page.getBlock(Page.java:143)
        at org.elasticsearch.compute.aggregation.CountAggregatorFunction.addRawInput(CountAggregatorFunction.java:86)
        at org.elasticsearch.compute.aggregation.Aggregator.processPage(Aggregator.java:43)
        at org.elasticsearch.compute.operator.AggregationOperator.addInput(AggregationOperator.java:111)
        at org.elasticsearch.compute.operator.Driver.runSingleLoopIteration(Driver.java:286)
        at org.elasticsearch.compute.operator.Driver.run(Driver.java:186)
        at org.elasticsearch.compute.operator.Driver$1.doRun(Driver.java:420)
        at org.elasticsearch.server@9.3.0-SNAPSHOT/org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:27)
        at org.elasticsearch.compute.operator.DriverScheduler$1.doRun(DriverScheduler.java:57)
        at org.elasticsearch.server@9.3.0-SNAPSHOT/org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:27)
        at org.elasticsearch.server@9.3.0-SNAPSHOT/org.elasticsearch.common.util.concurrent.TimedRunnable.doRun(TimedRunnable.java:35)
        at org.elasticsearch.server@9.3.0-SNAPSHOT/org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:1067)
        at org.elasticsearch.server@9.3.0-SNAPSHOT/org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:27)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1090)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:614)
        at java.base/java.lang.Thread.run(Thread.java:1474)

[astefan]

from d*,*| rename scalerank as other2| rename author.keyword as name_str| rename intersects as is_active_bool| rename year as id_int| rename status as other1 | lookup join multi_column_joinable_lookup on other2, name_str, is_active_bool, id_int, other1 | rename street AS `huZQQIKVli`, `network.eth0.currently_connected_clients` AS is_rehired | eval  GhMzAVXivNI = message.raw, VDJKNsopeKQq = substring(threat_level, 1, 3), xeUVJNra = length(huZQQIKVli), kOQjzWgZNhwI = false | where false AND  NOT false AND true AND  NOT false | CHANGE_POINT salary_change.int ON birth_date AS UpYfyPcYFkes, EwnNgKIrDEKS | sort owner.name DESC NULLS LAST, message.raw, country.keyword DESC NULLS LAST, UpYfyPcYFkes ASC NULLS FIRST, name_str ASC NULLS FIRST, num, event_duration DESC NULLS LAST, extra2 DESC, languages.byte ASC NULLS FIRST, client.ip NULLS LAST, ip1, collection NULLS FIRST, city.country.name DESC, destination.IP DESC NULLS LAST, host.version DESC, hex_code ASC NULLS LAST, language.code ASC NULLS LAST, language.name ASC, details DESC, language_name DESC NULLS FIRST, event ASC, salary_change.keyword DESC NULLS FIRST, lk ASC | stats  CLnBmEGor = sum(height.double) + sum(height.double) + avg(height.double), JLSxMYld = count(extra2) + max(extra2) + avg(extra2), `network.total_bytes_out` = min(language_code_integer), `version` = values(env), contains = min(languages.short) + min(languages.short) | mv_expand `CLnBmEGor` | dissect version "%{JLSxMYld} %{LTMJrburyt}" | enrich languages_policy on LTMJrburyt | inline stats  QyCBayGHtAF = count(CLnBmEGor), yUMkeLezSN = present(contains + network.total_bytes_out), `JLSxMYld` = max(contains), GYbfUvbf = count(*)

can't release already released object [DoubleArrayBlock[positions=1, mvOrdering=UNORDERED, vector=DoubleArrayVector[positions=1, values=[0.0]]]]
        at org.elasticsearch.base@9.3.0-SNAPSHOT/org.elasticsearch.core.Releasables.closeExpectNoException(Releasables.java:64)
        at org.elasticsearch.compute.data.Page.releaseBlocks(Page.java:245)
        at org.elasticsearch.compute.operator.AbstractPageMappingOperator.lambda$close$0(AbstractPageMappingOperator.java:107)
        at org.elasticsearch.base@9.3.0-SNAPSHOT/org.elasticsearch.core.Releasables.close(Releasables.java:39)
        at org.elasticsearch.base@9.3.0-SNAPSHOT/org.elasticsearch.core.Releasables.closeExpectNoException(Releasables.java:72)
        at org.elasticsearch.compute.operator.AbstractPageMappingOperator.close(AbstractPageMappingOperator.java:107)
        at org.elasticsearch.compute.operator.EvalOperator.lambda$close$1(EvalOperator.java:64)
        at org.elasticsearch.base@9.3.0-SNAPSHOT/org.elasticsearch.core.Releasables.close(Releasables.java:39)
        at org.elasticsearch.base@9.3.0-SNAPSHOT/org.elasticsearch.core.Releasables.close(Releasables.java:48)
        at org.elasticsearch.base@9.3.0-SNAPSHOT/org.elasticsearch.core.Releasables.closeExpectNoException(Releasables.java:62)
        at org.elasticsearch.compute.operator.EvalOperator.close(EvalOperator.java:61)
        at org.elasticsearch.base@9.3.0-SNAPSHOT/org.elasticsearch.core.Releasables.close(Releasables.java:39)
        at org.elasticsearch.base@9.3.0-SNAPSHOT/org.elasticsearch.core.Releasables.closeWhileHandlingException(Releasables.java:83)
        at org.elasticsearch.compute.operator.Driver.drainAndCloseOperators(Driver.java:396)
        at org.elasticsearch.compute.operator.Driver$1.onFailure(Driver.java:439)
        at org.elasticsearch.server@9.3.0-SNAPSHOT/org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:29)
        at org.elasticsearch.compute.operator.DriverScheduler$1.doRun(DriverScheduler.java:57)
        at org.elasticsearch.server@9.3.0-SNAPSHOT/org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:27)
        at org.elasticsearch.server@9.3.0-SNAPSHOT/org.elasticsearch.common.util.concurrent.TimedRunnable.doRun(TimedRunnable.java:35)
        at org.elasticsearch.server@9.3.0-SNAPSHOT/org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:1067)
        at org.elasticsearch.server@9.3.0-SNAPSHOT/org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:27)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1090)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:614)
        at java.base/java.lang.Thread.run(Thread.java:1474)
Caused by: java.lang.IllegalStateException: can't release already released object [DoubleArrayBlock[positions=1, mvOrdering=UNORDERED, vector=DoubleArrayVector[positions=1, values=[0.0]]]]
        at org.elasticsearch.compute.data.AbstractNonThreadSafeRefCounted.decRef(AbstractNonThreadSafeRefCounted.java:40)
        at org.elasticsearch.compute.data.AbstractNonThreadSafeRefCounted.close(AbstractNonThreadSafeRefCounted.java:59)
        at org.elasticsearch.base@9.3.0-SNAPSHOT/org.elasticsearch.core.Releasables.close(Releasables.java:39)
        at org.elasticsearch.base@9.3.0-SNAPSHOT/org.elasticsearch.core.Releasables.close(Releasables.java:48)
        at org.elasticsearch.base@9.3.0-SNAPSHOT/org.elasticsearch.core.Releasables.closeExpectNoException(Releasables.java:62)