-
Notifications
You must be signed in to change notification settings - Fork 2.7k
Description
Checks
- I have checked that this issue has not already been reported.
- I have confirmed this bug exists on the latest version of Polars.
Reproducible example
import polars as pl
print("=== Slow path: empty list becomes a phantom scalar result ===")
s = pl.Series([[1, 2], [], [3, 4]])
print("sum got: ", s.list.eval(pl.element().sum()).to_list())
print("sum expect:", [[3], [], [7]])
print("max got: ", s.list.eval(pl.element().max()).to_list())
print("max expect:", [[2], [], [4]])
print("mean got: ", s.list.eval(pl.element().mean()).to_list())
print("mean expect:", [[1.5], [], [3.5]])
print("\n=== Fast path: adjacent row corruption ===")
s2 = pl.Series([[1, 2], [], [3]])
print("sum got: ", s2.list.eval(pl.element().sum()).to_list())
print("sum expect:", [[3], [], [3]])
print("\n=== Elementwise control (works correctly) ===")
print("got: ", s.list.eval(pl.element() * 2).to_list())
print("expect:", [[2, 4], [], [6, 8]])Log output
=== Slow path: empty list becomes a phantom scalar result ===
sum got: [[3], [0], [7]]
sum expect: [[3], [], [7]]
max got: [[2], [None], [4]]
max expect: [[2], [], [4]]
mean got: [[1.5], [None], [3.5]]
mean expect: [[1.5], [], [3.5]]
=== Fast path: adjacent row corruption ===
sum got: [[3, 0], [], [3]]
sum expect: [[3], [], [3]]
=== Elementwise control (works correctly) ===
got: [[2, 4], [], [6, 8]]
expect: [[2, 4], [], [6, 8]]Issue description
list.eval with scalar aggregation expressions can silently return incorrect results when at least one valid empty inner list is present. Two failure modes exist. If the scalar output length happens to equal the original flattened inner-element count, the fast path can reuse the original list offsets with the new aggregated values, causing row-boundary corruption (for example [[3,0],[],[3]] instead of [[3],[],[3]]). Otherwise, the slow path converts each scalar result into a one-element list, so the empty inner list incorrectly contributes a phantom singleton (for example [[3],[0],[7]] instead of [[3],[],[7]]).
This is caused because evaluate_on_list_chunked builds groups directly from list offsets for every valid outer row, so valid empty inner lists become zero-length groups and are still evaluated. The scalar result produced for that empty group is then either reinterpreted under the original offsets in the fast path or emitted as a singleton list in the slow path.
Expected behavior
A valid empty inner list should remain empty in the output list structure. It should not produce a phantom scalar singleton and should not affect adjacent rows
Installed versions
Details
--------Version info---------
Polars: 1.38.1
Index type: UInt32
Platform: Linux-6.6.87.2-microsoft-standard-WSL2-x86_64-with-glibc2.39
Python: 3.12.12 | packaged by conda-forge | (main, Oct 22 2025, 23:25:55) [GCC 14.3.0]
Runtime: rt32
----Optional dependencies----
Azure CLI <not installed>
adbc_driver_manager 1.10.0
altair 6.0.0
azure.identity 1.25.2
boto3 1.42.49
cloudpickle 3.1.2
connectorx 0.4.5
deltalake 1.4.2
fastexcel 0.19.0
fsspec 2026.2.0
gevent 25.9.1
google.auth 2.48.0
great_tables 0.20.0
matplotlib 3.10.8
numpy 2.4.2
openpyxl 3.1.5
pandas 3.0.1
polars_cloud 0.5.0
pyarrow 23.0.1
pydantic 2.12.5
pyiceberg 0.11.0
sqlalchemy 2.0.46
torch <not installed>
xlsx2csv 0.8.6
xlsxwriter 3.2.9
Metadata
Metadata
Assignees
Labels
Type
Projects
Status