Update hdf5 to 1.14.6; rebuild h5py

[MMesch]

Template B: Checklist for updating a package

• [ ] ⚠️ Bump build number if the version remains unchanged. Version changed.
• Or reset build number to 0 if updating the package to a newer version

---

PR Formatting

• PR title follows format: Add [package-name] or Update [package-name] to [version]
• PR description includes:
  • Version being added/updated
  • Any special build considerations or patches applied

Package Details

• Package Name: hdf5
• Version: 1.14.6

Build Notes

• This updated was initiated by a hard failure in netcdf when writing files (see Rebuild netcdf4; Rebuild libnetcdf; fix runtime failure due to missing listxattr #5951)
• It's a small update to 1.14.6. Upstream is already at 2.x since Nov 2025. Another update may be needed soon.

There are two patches here, both quite simple but the second difficult to find even with Opus 4.7 assistance:

• the simple one 0001-fenv-fe-invalid-guard.patch guards FE_INVALID use in H5Tinit_float.c which emscripten does not declare. This patch is already upstream in hdf5 2.x and can probably be dropped once we update further
  https://github.com/HDFGroup/hdf5/blob/f8844a64c639f7ba33592a7948d4adacacb2d451/src/H5Tinit_float.c#L614-L617
• the more complex one, patches/0002-drop-soversion.patch: skips VERSION/SOVERSION under EMSCRIPTEN so libhdf5_hl.so records bare "libhdf5.so" as its NEEDED entry. Otherwise it records "libhdf5.so.310.5.1" while other packages link with -lhdf5 and record "libhdf5.so". The emscripten loader seems to dedupe by string identity (not realpath) and ends up loading libhdf5.so twice. One visible symptom of this was that the hdf5 metadata cache errors when calls cross between those two loaded instances, something which blocks simple file reading and also affects netcdf4 transitively. We discovered dual loading by consequently tracing back the error to hdf5 (a metadata ring error), then outputting memory addresses and realizing that there were two addresses where there should only be one at some point, causing the actual failure. Note that this is a hot fix and a proper solution that also works for other libraries would be nicer to have. We added a dimension-scale round-trip test to h5py to test the failure path associated with this patch.

[DerThorsten]

2. libhdf5_hl.so records NEEDED libhdf5.so.310.5.1

@MMesch isnt that the culprint / root-problem
I would expect that one never explicitly links against the versioned so, just the unversioned so (ie libhdf5.so)
the unversioned so ist then just a symlink to the versioned so, that way one can swap out the symlink and point to a newer version -- but I dont have much expertise on that subject

[MMesch]

Here is what I did from memory:

• xarray.open_dataset(...) → netCDF4.Dataset → nc_open → H5DSis_scale → Dynamic linking error: cannot resolve symbol H5DSis_scale triggered the bug in the browser on notebook.link
• I then switched to the emscripten-forge/recipes repo and added a test on netcdf4 to reproduce. I used H5Eprint() to see more which pointed to some (obscure) metadata issue.
• I tried to simply update hdf5 which is the update you can see here. Without success.
• I started chatting with Opus 4.7 and it found the preload which seeded the idea that this is unusual but I had no idea and still don't fully understand the implications of it.
• Now the heart of it: I then asked Opus 4.7 to inject print statements all over the place the error happened in the metadata cache (not the exact prompt but you get the gist). The error itself was quite obscure, some stuff getting pushed, popped and then suddenly failure. After quite some iteration, it ended up printing &H5CX_head_g , which is defined still in v2.x here: https://github.com/HDFGroup/hdf5/blob/4b1ffa68a84065ca758845f020b21f93f8e396fa/src/H5CX.c#L230 . It is the "Pointer to head of context stack" and there should only be a single one per instance loaded. If there are several, you inevitably get an issue if you pop / push stuff thinking you have one but actually have two stacks. I think stuff that went through hdf5_hl was routed to one (because of the extra SONAME), and stuff that went directly through hdf5 went through another. So printing this address showed two outputs with two different addresses made the issue pretty clear.
• Opus4.7 then told me how to dump the NEEDED strings and they were different. That hinted that this is the problem, and the easiest fix was to just drop the SONAME from one of them. The h5py test worked afterwards. And I continued with netcdf4

In summary, the easiest way I know how to see it is to add the print statement for this stack head address with a hot patch at build time and then run the test. Or you dump the NEEDED strings directly but I wouldn't have guessed without the other info that a difference would be a problem.

Does this help? I realize it's not that easy to reproduce but I don't have a better way right now.

[MMesch]

I would expect that one never explicitly links against the versioned so, just the unversioned so (ie libhdf5.so)
the unversioned so ist then just a symlink to the versioned so, that way one can swap out the symlink and point to a newer version -- but I dont have much expertise on that subject

What you say makes sense to me but I frankly also don't have that much expertise on it. I also don't know anyone who has, and maybe we need to research it ourselves.

[MMesch]

regarding dumping the needed strings, I still have the cache on my computer and this is the command (using nix here to pull in the wabt package):

nix shell nixpkgs#wabt -c wasm-objdump -x /home/xxxx/.cache/rattler/cache/pkgs/hdf5-1.14.6-ha7c90dc_5/lib/libhdf5_hl.so | grep -A2 needed_dynlibs
 - needed_dynlibs[1]:
  - libhdf5.so
  

nix shell nixpkgs#wabt -c wasm-objdump -x /home/xxxx/.cache/rattler/cache/pkgs/hdf5-1.12.3-h8d20eea_0/lib/libhdf5_hl.so | grep -A2 needed_dynlibs
 - needed_dynlibs[1]:
  - libhdf5.so.200.3.0

[MMesch]

@DerThorsten , I just tried to make a local reproducer of this on small dummy files ... and I can't reproduce. In other words, emscripten does not dedup simply by string but in a more elaborate way locally. I didn't understand yet why and what the difference to this PR is. Will look deeper into it tomorrow.

[MMesch]

https://github.com/MMesch/wasm-soname-dedup-test minimal reproducer above. Roughly the story likely goes like this:

• when you import h5py you import hdf5 and hdf5_hl. hdf5_hl itself then registers hdf5 with a sonamed NEEDED string, but h5py imported it without soname (libhdf5.so.310.5.1 vs libhdf5.so, 310 is the interface number for 1.14.6). This is a first divergence from the native case where all NEEDED strings are substituted by major version at link time, it seems. Both would end up as libhdf5.so.310 in this case.
• Emscripten's LDSO tracks each loaded .so in loadedLibsByName (keyed by NEEDED string). It adds two entries for libhdf5 because there are two names. You can think of each entry as a javascript object with exports, memory address and some module-wide data.
• Separately, LDSO builds a flat GOT table keyed by symbol name. After instantiating a .so, LDSO adds its "exports" into the GOT. If a symbol already exists (because a library was loaded twice), the first name wins and later ones are logged as EXISTING SYMBOL and dropped. So the second libhdf5 exists in memory, but every call resolves through the GOT to the first, hiding the double-load for externally visible symbols. In addition, libhdf5_hl itself is loaded only once and its exports also enter the GOT. Its own calls into libhdf5 also route through the GOT and land in the same instance. So far so good, all goes to one instance.
• But for modules loaded via dlopen (the path CPython uses for Cython extension .sos), LDSO doesn't use the global GOT to satisfy function-call imports. It builds a per-instantiation import scope from the dlopen'd module's own NEEDED chain. So each of h5py's Cython .sos (NEEDED = libhdf5.so) has its H5CX_push import bound to instance B; libhdf5_hl (NEEDED = libhdf5.so.310.5.1) has its bound to instance A. Different consumers of the same HDF5 API reach different libhdf5 instances. Dangerous, and additional caveat is that even under dlopen the GOT table still handles pointer values (function pointers, extern data addresses) with first-wins dedup, so &foo_global from B reads A's variable.
• H5CX_head_g (the head pointer of HDF5's API-context stack, in H5CX.c) is a file-scope static, so it never enters the GOT. Nothing outside its own unit can access it. Each libhdf5 instance therefore has its own private H5CX_head_g. A code path that reached libhdf5 through h5py's Cython imports (instance B, per-NEEDED) pushes onto B's H5CX_head_g; a code path that reached it through libhdf5_hl (instance A) reads A's H5CX_head_g unexpectedly gets empty, because the push landed in B. The metadata cache sees the tag mismatch and reports "ring type mismatch."
• Other consumers like netcdf4 can run into the same problem.

[MMesch]

I did a simple scan of all latest .so on emscripten-forge-4x for NEEDED divergence (not only hdf but general) @DerThorsten . This is what came back:

libhdf5 has two NEEDED variants in the wild:

• libhdf5.so.200.3.0 (SOVERSION-versioned) — emitted by hdf5's own libhdf5_hl.so.200.1.1 and by matio's libmatio.so.1.5.30.
• libhdf5.so (bare) — emitted by every h5py Cython .so and by netCDF4's _netCDF4.cpython-*.so.

It seems that maybe matio is affected but nothing else.