Make the creation of `.idx` files an opt-in feature, rather than opt-out

[etienneschalk]

Is your feature request related to a problem? Please describe.

It is common to open grib files via xarray. Sometimes, the format (netCDF or GRIB) is not know in advance, and one wants to relay on a vanilla xr.open_dataset() function call without backend kwarg.

Cf this issue: #126 (comment)

In my usage, I observed that I consistently disable the creation of .idx files. However, maybe my own usage does not represent the majority, so I open this issue also to see with other users if they want to use or not these .idx files.

My proposal is to make it the default behaviour. My other argument, rather than preference, is that when opening a file, we don't expect other files to be created. A "read" function that actually also writes is surprising, and can often leads to problems when working with read-only data sources.

Other arguments:

GRIB index file

By default cfgrib saves the index of the GRIB file to disk appending .idx to the GRIB file name. Index files are an experimental and completely optional feature, feel free to remove them and try again in case of problems. Index files saving can be disable passing adding indexpath='' to the backend_kwargs keyword argument.

Source: https://github.com/ecmwf/cfgrib?tab=readme-ov-file#grib-index-file

We can see here that this default feature is described as experiemental and optional in the README, so this adds in favor of making this feature opt-in rather than opt-out.

I am open to contribute and make the change myself if needed.

Describe the solution you'd like

Disable creation of indexes by default.

Example code: (no specificity to open GRIB files, will work with NetCDF and GRIB files)

    concat_xds = xr.open_mfdataset(list(output_paths.values())).compute()

Describe alternatives you've considered

Continue to specify manually the backend kwargs for xarray:

Example code: (need to manually distinguish the GRIB case from others)

    if suffix == ".grib":
        backend_kwargs = {"indexpath" :""}
    else:
        backend_kwargs = {}

    concat_xds = xr.open_mfdataset(list(output_paths.values()), backend_kwargs=backend_kwargs).compute()

Additional context

No response

Organisation

No response