update checktestdata

[chistopher]

TLDR: the committed executable was compiled for specific hardware. The new executable was compiled in the docker image and should hopefully be more portable.

When running the docker container on MacOS I got an illegal instruction in checktestdata.
My laptop is arm64 and the image is amd64 which is emulated by docker.
Some gdb debugging revealed that the executable here was compiled on hardware that has a bit more instruction sets available than the MacOS emulation supports.

I built this dockerfile

FROM ragnargrootkoerkamp/bapctools
RUN pacman -Sy --noconfirm make boost gmp automake autoconf
RUN git clone https://github.com/DOMjudge/checktestdata.git /opt/checktestdata
WORKDIR /opt/checktestdata
RUN git switch release
RUN ./bootstrap
RUN make

and then copied the resulting executable from /opt/checktestdata/checktestdata to my local machine.

[RagnarGrootKoerkamp]

Hmm, we may want to do a quick check if to see if there are performances regressions from this. (I'm assuming it's some avx2 instructions that you don't have?) I'm not sure how much checktestdata depends on those. Not much is my assumption, in which case it's fine to merge this, but it's already kinda slow for many cases, so would be annoying for make it, say, 2x slower for everyone who does have x86_64-v2 support.

[mzuenni]

On WSL there now is this error:

checktestdata: error while loading shared libraries: libgmpxx.so.4: cannot open shared object file: No such file or directory

[RagnarGrootKoerkamp]

Ah yes, most likely the binary that is currently in the repo is one with static linking that I build manually at some point.

[mzuenni]

yea i think all ressources should be baked in/not shared objects

[chistopher]

I recompiled it with a statically linked gmp. Under arch pacman does not ship static libs and I would've had to compile gmp (and it's dependency tree) from source.
Instead I compiled it in an ubuntu image which ships static libs via apt.

My dockerfile to compile this executable looks like this.

FROM ubuntu:latest

RUN apt update && apt install -y git make g++ libboost-dev libgmp-dev autotools-dev automake
RUN git clone https://github.com/DOMjudge/checktestdata.git /opt/checktestdata
WORKDIR /opt/checktestdata
RUN git switch release
RUN sed -i 's/configure/configure --enable-static-linking/' bootstrap 
RUN ./bootstrap
RUN make

You can run it yourself if you don't like to take random executables from strangers 😉

It now links agains this

        libstdc++.so.6 => /usr/lib/libstdc++.so.6 (0x00007fffff47c000)
        libgcc_s.so.1 => /usr/lib/libgcc_s.so.1 (0x00007fffff44d000)
        libc.so.6 => /usr/lib/libc.so.6 (0x00007fffff25d000)
        libm.so.6 => /usr/lib/libm.so.6 (0x00007fffff165000)
        /lib64/ld-linux-x86-64.so.2 => /usr/lib64/ld-linux-x86-64.so.2 (0x00007ffffffc7000)

instead of this

        libgmpxx.so.4 => /usr/lib/libgmpxx.so.4 (0x00007fffff741000)
        libgmp.so.10 => /usr/lib/libgmp.so.10 (0x00007fffff699000)
        libstdc++.so.6 => /usr/lib/libstdc++.so.6 (0x00007fffff405000)
        libm.so.6 => /usr/lib/libm.so.6 (0x00007fffff30d000)
        libgcc_s.so.1 => /usr/lib/libgcc_s.so.1 (0x00007fffff2e0000)
        libc.so.6 => /usr/lib/libc.so.6 (0x00007fffff0f0000)
        /lib64/ld-linux-x86-64.so.2 => /usr/lib64/ld-linux-x86-64.so.2 (0x00007ffffffc7000)

[RagnarGrootKoerkamp]

The current binary doesn't link against anything though, making it more forward and backward compatible. My system does have the same lib versions as what you write, but system updates could break this.

Did you try the command from the readme instead? Probably it's sufficient to add -march=x86_64 or so? (Interestingly there isn't even a -march=native in there, so not sure how you even got non-standard instructions in the binary.)

[chistopher]

I did not see that readme yet. A first try failed because it could not find static libs for the dependencies. Probably I need some more packages.

I'm a bit busy in the coming weeks but I'll come back to this eventually.

[chistopher]

Interestingly there isn't even a -march=native in there, so not sure how you even got non-standard instructions in the binary

The instructions that make it crash for me are in the gmp lib and depend on how it was compiled.

[mpsijm]

A first try failed because it could not find static libs for the dependencies. Probably I need some more packages.

Additionally, there are more compilation instructions (including the necessary apt packages) listed at: https://github.com/DOMjudge/checktestdata