Durin does not build on CentOS 7 / x86_64

[jcbollinger]

I attempted to build Durin on 64-bit CentOS 7, using the HDF5 provided by that distribution. I expected that I would get a link error if the platform HDF5 had not been built with appropriate switches, but the build did not make it to that point.

Following the build instructions, I

1. downloaded the 2019v1 source tarball from GitHub,
2. unpacked the tarball and changed to the resulting directory, and
3. executed make.

Here is the full make output:

mkdir -p ./build
h5cc -Wall -g -O2 -fpic -I./src -I./bslz4/src -std=c89 -c src/plugin.c -o build/plugin.o
mkdir -p ./build
h5cc -Wall -g -O2 -fpic -I./src -I./bslz4/src -std=c89 -c src/file.c -o build/file.o
src/file.c: In function ‘get_frame_from_chunk’:
src/file.c:207:2: warning: implicit declaration of function ‘H5Dget_chunk_storage_size’ [-Wimplicit-function-declaration]
  if (H5Dget_chunk_storage_size(d_id, c_offset, &c_bytes) < 0) {
  ^
src/file.c:230:2: warning: implicit declaration of function ‘H5DOread_chunk’ [-Wimplicit-function-declaration]
  if (H5DOread_chunk(d_id, H5P_DEFAULT, c_offset, &c_filter_mask, c_buffer) < 0) {
  ^
mkdir -p ./build
h5cc -Wall -g -O2 -fpic -I./src -I./bslz4/src -std=c89 -c src/err.c -o build/err.o
mkdir -p ./build
h5cc -Wall -g -O2 -fpic -I./src -I./bslz4/src -std=c89 -c src/filters.c -o build/filters.o
In file included from ./bslz4/src/bitshuffle.h:32:0,
                 from src/filters.c:9:
./bslz4/src/bitshuffle_core.h:41:31: error: conflicting types for ‘int64_t’
   typedef long long           int64_t;
                               ^
In file included from /usr/include/stdlib.h:314:0,
                 from ./bslz4/src/bitshuffle.h:31,
                 from src/filters.c:9:
/usr/include/sys/types.h:197:1: note: previous declaration of ‘int64_t’ was here
 __intN_t (64, __DI__);
 ^
make: *** [build/filters.o] Error 1

This arises from the Bitshuffle sources making some incorrect assumptions and taking unsafe liberties with reserved identifiers, but the issue can be sidestepped in Durin's top-level Makefile by changing the CFLAGS to specify -std=c99 instead of -std=c89. On distributions that provide more recent compilers, it would likely be sufficient to just remove the -std=c89 option altogether.

[graeme-winter]

On RHEL7.9 it compiles as is, which is interesting - however works equally well with the change suggested to use -std=c99. Re: recent compilers, yes, the compilers on RHEL7 are still astonishingly old (4.8) so I'll just move to c99 rather than removing completely.

What HDF5 version do you get my default? You'll probably be wanting at least 1.10.(something)

[graeme-winter]

I could do with making a release with this in before closing... however this reminds me there are some other outstanding issues I should probably look at. @jcbollinger are you happy to build from a clone of the repo?

[jcbollinger]

Interesting indeed. The system on which I hit this has not yet been updated to release 7.9 (it is still on 7.8), but I wouldn't usually expect that to make a difference to this sort of thing.

This distro's default HDF5 is version 1.8(.12), so I'll need to look for problems related to that, but I haven't recognized such issues in other bitshuffle-using software, such as CBFlib.

[jcbollinger]

I would be willing to build from a clone of the repo if that helps you out. Otherwise, I am happy to patch the source locally, and that was my original plan.

[graeme-winter]

It's probably a good fix so not too worried. Personally I would suggest building against 1.10 HDF5 (even if not in system) as "places" are probably using that shortly (e.g. Diamond)

What is your use case here? Just for info, to help some more

[jcbollinger]

I'm managing Linux workstations for a Structural Biology department. We get data from several sources, mostly in the U.S., including two beamlines at Brookhaven that have Eigers. It seems likely that we will see more Eigers in the future. Some of our key users are big proponents of XDS and XDS-GUI, and that constellation led me here.

[jcbollinger]

I think that the HDF version issue will go away for us, because we are starting to migrate to CentOS 8, which provides HDF 1.10 as its standard. Until that's complete, though, we are (re)processing at least some of our data on CentOS 7.

[graeme-winter]

No problem - if you are processing data collected with the DECTRIS file writer then I think you should be fine with 1.8 HDF5 for the foreseeable. That being said, there are also binaries on the release page which would probably suit your needs already.

I'll add "making a new release" to my to-do list once I have squashed some of the other bugs... if it is worth doing, should do properly.