Writing a dataset from a dask array?

[Becheler]

Hi! That looks like a great lib! 👏🏽 :

I would like to read a large dataset, perform some interpolation on the data and the mask, then write the masked array back to another dataset.

The way I've been doing that is the following:

# Read dataset
x_array = rioxarray.open_rasterio(inputFile, masked=True, chunks = (1,'auto','auto'))
# Create a view
src_array = x_array.to_masked_array(copy=False)

# ...
# some operations to produce arr_interpolate and mask_interpolated from scr_array ...
# ...

# Create the dask masked array
interpolated = da.ma.masked_array(arr_interpolated, mask_interpolated)

# Fill the mask with nodata value for writing dataset
filled = da.ma.filled(interpolated, fill_value=source.nodata)

# I want to push that back to the writing step
filled.compute()
new_meta = source.meta
with rasterio.open(outputFile, "w", **new_meta) as dst:
     dst.write(filled)

The problem is that the array filled is unlikely to hold on memory, so I don't think I want to call filled.compute(). Therefore I would like to use something like:

filled = da.ma.filled(interpolated, fill_value=source.nodata)
new_meta = source.meta
with rioxarray.open_rasterio(inputFile, "w", **new_meta) as dst:
       dst.write(filled)

But it fails with:

Traceback (most recent call last):
  File "../../../src/crumbs/interpolate.py", line 192, in <module>
    main(sys.argv[1:])
  File "../../../src/crumbs/interpolate.py", line 185, in main
    return temporal_interpolation(
  File "../../../src/crumbs/interpolate.py", line 162, in temporal_interpolation
    dst.write(filled)
  File "/home/becheler/dev/virtual_environments/crumbs-env/lib/python3.8/site-packages/xarray/core/common.py", line 239, in __getattr__
    raise AttributeError(
AttributeError: 'DataArray' object has no attribute 'write'

How are we supposed to perform writing without having to call the dask.compute() method? I read somewhere that rioxarray does not directly support dask arrays? Am I supposed to convert the dask array?
Thank you 👍🏽

[snowman2]

Have you looked at these examples?

• https://corteva.github.io/rioxarray/stable/examples/dask_read_write.html
• https://corteva.github.io/rioxarray/stable/examples/read-locks.html

[Becheler]

Yes I have read them! 😃 But it is still unclear how I should call the writing part?
Am I supposed to call the following?

filled.rio.to_raster(outputFile, lock=threading.Lock())

[snowman2]

Yes, that is correct.

[Becheler]

Wonderful! Thank you so much! Can I ask why the expected interface (below) can not apply?

with rioxarray.open_rasterio(inputFile, "w", **new_meta) as dst:
       dst.write(filled)

[snowman2]

Glad to hear that it worked 👍

The format you have presented uses rasterio syntax and differs from the usage patterns of xarray|rioxarray. open_rasterio is a read-only method for loading your data into an xarray object. It's write counterpart is to_raster.

with rioxarray.open_rasterio(...) as xds:
    xds.rio.to_raster(...)

This is the same pattern used by xarray:

with xarray.open_dataset(...) as xds:
    xds.to_netcdf(...)

[Becheler]

thank you!