to_zarr() appending uses the first chunk size as the maximum chunk size

[lbesnard]

Hi,

I’m seeing some behaviour around chunking when appending with append_dim="TIME" in ds.to_zarr() that I can’t fully explain.

1) I create a new Zarr dataset from a single NetCDF file with the chunking strategy:

{'TIME': (1,), 'LATITUDE': (74,), 'LONGITUDE': (102,)}

After creation, the store reports:
ItemsView(Frozen({'TIME': (1,), 'LATITUDE': (74,), 'LONGITUDE': (102,)}))

2) Append

Next, I append 3000 NetCDF files.
My input dataset for the append has this chunking:
{'TIME': (100, 100, 100, 100, 100, 100, 100, 100, 100, 100, 100, 100, 100, 100, 100, 100, 100, 100, 100, 100, 100, 100, 100, 100, 100, 100, 100, 100, 100, 100), 'LATITUDE': (74,), 'LONGITUDE': (102,)}

append call:

ds.to_zarr(
    store,
    mode="a-",
    append_dim="TIME",
    safe_chunks=False,
    align_chunks=True,
    write_empty_chunks=True,
    consolidated=True,
    compute=True,
)

3) Result after append

When I check the updated Zarr store, the chunking is:
ItemsView(Frozen({'TIME': (1, 1, ......., 1), 'LATITUDE': (74,), 'LONGITUDE': (102,)}))

It’s all (1,) chunks along TIME.

I had expected something like:
ItemsView(Frozen({'TIME': (1, 100,100, ....) ...

My questions:

• Is chunking along the append dimension fixed by the first write in Zarr v2?
• Is there any way to allow larger chunks along the append dimension after initial creation?
• If this is intended, can you confirm the reasoning? It seems to use the first chunk as the maximum value to use.

Environment:

• Zarr v2
• xarray from this commit: align_chunks not working for datasets #10516