Skip to content

Unable to run groupby, map after shuffle_to_chunks #11212

@joshua-gould

Description

@joshua-gould

What happened?

ValueError: Array chunk size or shape is unknown. Possible solution with x.compute_chunk_sizes()

What did you expect to happen?

Able to successfully iterate over groups and apply a function

Minimal Complete Verifiable Example

# /// script
# requires-python = ">=3.11"
# dependencies = [
#   "xarray[complete]@git+https://github.com/pydata/xarray.git@main",
# ]
# ///
#
# This script automatically imports the development branch of xarray to check for issues.
# Please delete this header if you have _not_ tested this script with `uv run`!

import xarray as xr
xr.show_versions()
import dask.array as da
import numpy as np
import xarray as xr

data = da.ones((583753, 184), chunks=(18243, 12))

y = np.random.choice(np.array([1, 2, 3, 4, 5]), size=data.shape[0], replace=True)
d = xr.DataArray(data=data, coords={"y": y}, dims=["y", "x"])
d = d.groupby("y").shuffle_to_chunks()
d.data = d.data.compute_chunk_sizes()
d.groupby("y").map(lambda x: x)[
    0, 0].compute()  # ValueError: Array chunk size or shape is unknown. Possible solution with x.compute_chunk_sizes()

Steps to reproduce

No response

MVCE confirmation

  • Minimal example — the example is as focused as reasonably possible to demonstrate the underlying issue in xarray.
  • Complete example — the example is self-contained, including all data and the text of any traceback.
  • Verifiable example — the example copy & pastes into an IPython prompt or Binder notebook, returning the result.
  • New issue — a search of GitHub Issues suggests this is not a duplicate.
  • Recent environment — the issue occurs with the latest version of xarray and its dependencies.

Relevant log output

Anything else we need to know?

No response

Environment

Details

INSTALLED VERSIONS

commit: None
python: 3.12.12 | packaged by conda-forge | (main, Jan 27 2026, 00:01:15) [Clang 19.1.7 ]
python-bits: 64
OS: Darwin
OS-release: 25.3.0
machine: arm64
processor: arm
byteorder: little
LC_ALL: None
LANG: C.UTF-8
LOCALE: ('C', 'UTF-8')
libhdf5: 1.14.6
libnetcdf: None
xarray: 2026.2.0
pandas: 2.3.3
numpy: 1.26.4
scipy: 1.16.3
netCDF4: None
pydap: None
h5netcdf: 1.8.1
h5py: 3.15.1
zarr: 2.18.7
cftime: None
nc_time_axis: None
iris: None
bottleneck: None
dask: 2025.11.0
distributed: 2025.11.0
matplotlib: 3.10.8
cartopy: None
seaborn: 0.13.2
numbagg: None
fsspec: 2026.2.0
cupy: None
pint: 0.25.2
sparse: None
flox: 0.11.2
numpy_groupies: 0.11.3
setuptools: 82.0.0
pip: 26.0.1
conda: None
pytest: 9.0.2
mypy: None
IPython: 9.10.0
sphinx: None

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugneeds triageIssue that has not been reviewed by xarray team member

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions