segfault while closing netcdf files #10783
Description
What happened?
I encounter random segfaults when running the xbout test suite with python3.14rc3.
I tried switching the backend from netcdf4 to h5netcdf, but the issue remains.
I assume this is because netcdf4 and h5netcdf are not thread safe. Python did remove the GIL in 3.14, thus previously parallelism was automatically limited, while it is not anymore the case.
I checked several backtraces, and they all have one thread in the __del__ / close and one in __getitem__ (I think I also saw __setitem__, but cannot find that now)
Note that the other thread with __getitem__ in the posted example is currently waiting for the lock.
What did you expect to happen?
No errors, like with python3.13
Minimal Complete Verifiable Example
Steps to reproduce
git clone https://github.com/boutproject/xbout
cd xbout
pytest -s -k TestSave
Repeat the last step until you get a segfault.
I have only tested this on multi-core systems. Without any additional constraints is should crash within a few iterations.
MVCE confirmation
- Minimal example — the example is as focused as reasonably possible to demonstrate the underlying issue in xarray.
- Complete example — the example is self-contained, including all data and the text of any traceback.
- Verifiable example — the example copy & pastes into an IPython prompt or Binder notebook, returning the result.
- New issue — a search of GitHub Issues suggests this is not a duplicate.
- Recent environment — the issue occurs with the latest version of xarray and its dependencies.
Relevant log output
build09596507.log-
build09596507.log-Thread 0x00007f00d988b6c0 [ThreadPoolExecu] (most recent call first):
build09596507.log- File "/usr/lib/python3.14/site-packages/xarray/backends/file_manager.py", line 234 in close
build09596507.log- File "/usr/lib/python3.14/site-packages/xarray/backends/file_manager.py", line 250 in __del__
build09596507.log- File "/usr/lib/python3.14/site-packages/dask/_task_spec.py", line 996 in __getitem__
build09596507.log- File "/usr/lib/python3.14/site-packages/dask/order.py", line 215 in sort_key
--
build09596507.log-
build09596507.log-Thread 0x00007f00da08c6c0 [ThreadPoolExecu] (most recent call first):
build09596507.log- File "/usr/lib/python3.14/site-packages/xarray/backends/locks.py", line 72 in __enter__
build09596507.log- File "/usr/lib/python3.14/site-packages/xarray/backends/h5netcdf_.py", line 69 in _getitem
build09596507.log- File "/usr/lib/python3.14/site-packages/xarray/core/indexing.py", line 1129 in explicit_indexing_adapter
build09596507.log- File "/usr/lib/python3.14/site-packages/xarray/backends/h5netcdf_.py", line 64 in __getitem__
build09596507.log- File "/usr/lib/python3.14/site-packages/xarray/core/indexing.py", line 737 in get_duck_array
build09596507.log- File "/usr/lib/python3.14/site-packages/xarray/core/indexing.py", line 897 in get_duck_array
--
build09596507.log- File "/usr/lib/python3.14/site-packages/h5netcdf/core.py", line 491 in _get_padding
build09596507.log- File "/usr/lib/python3.14/site-packages/h5netcdf/core.py", line 538 in __getitem__
build09596507.log- File "/usr/lib/python3.14/site-packages/xarray/backends/h5netcdf_.py", line 71 in _getitem
build09596507.log- File "/usr/lib/python3.14/site-packages/xarray/core/indexing.py", line 1129 in explicit_indexing_adapter
build09596507.log- File "/usr/lib/python3.14/site-packages/xarray/backends/h5netcdf_.py", line 64 in __getitem__
build09596507.log- File "/usr/lib/python3.14/site-packages/xarray/core/indexing.py", line 737 in get_duck_array
build09596507.log- File "/usr/lib/python3.14/site-packages/xarray/core/indexing.py", line 897 in get_duck_arrayAnything else we need to know?
This can also be reproduced in a clean mock environment, e.g. in fedora copr
Environment
The backtraces are from copr, but I can also reproduce this locally with:
INSTALLED VERSIONS
commit: 3bb00ad
python: 3.14.0rc3 (main, Sep 18 2025, 00:00:00) [GCC 15.2.1 20250808 (Red Hat 15.2.1-1)]
python-bits: 64
OS: Linux
OS-release:
machine: x86_64
processor:
byteorder: little
LC_ALL: None
LANG: en_US.utf8
LOCALE: ('en_US', 'UTF-8')
libhdf5: 1.14.6
libnetcdf: 4.9.3
xarray: 2025.9.1.dev3+g53da83298.d20250923
pandas: 2.2.3
numpy: 2.3.3
scipy: 1.15.3
netCDF4: 1.7.2
pydap: None
h5netcdf: 1.6.4
h5py: 3.13.0
zarr: None
cftime: 1.6.4
nc_time_axis: None
iris: None
bottleneck: 1.5.0
dask: 2025.9.0
distributed: 2025.9.0
matplotlib: 3.10.6
cartopy: None
seaborn: None
numbagg: None
fsspec: 2025.9.0
cupy: None
pint: None
sparse: None
flox: None
numpy_groupies: None
setuptools: 78.1.1
pip: 25.2
conda: None
pytest: 8.3.5
mypy: 1.18.1
IPython: 8.37.0
sphinx: 8.2.3