build-rr.sh fails to detects Xen headers

[roberto-torre]

Hello,
I am trying to run a unikernel via rumprun on Xen. I'm using a Ubuntu 17.10 with gcc 7.2.0.

The command I'm using is:
CXX=false ./build-rr.sh xen -- -F ACLFLAGS=-no-pie

It looks like it runs until a certain point where it cannot find Xen headers, as you can see in the log:
log.txt

I have libxen-dev installed:

sudo apt-get install libxen-dev
Reading package lists... Done
Building dependency tree
Reading state information... Done
libxen-dev is already the newest version (4.9.0-0ubuntu3).
0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.

I was trying to figure out what the problem was, and I came across the fact that the variable $XEN_HEADERS was not declared before in any of the config, config.sh or config.mk files, therefore, in the function probeprereqs() in the file build-rr.sh breaks.

Any ideas how to include Xen headers or, in case these are included, how to make rumprun detect them?

[kent-mcleod]

It seems that the test program that build-rr.sh uses to check for xen headers doesn't compile.
In file included from /usr/lib/gcc/x86_64-linux-gnu/6/include/stdint.h:9:0, from xentest.c:2: /usr/include/stdint.h:286:17: error: missing binary operator before token "(" #if __GLIBC_USE (IEC_60559_BFP_EXT)
If you want to hack around this, in probeprereqs you can set XEN_HEADERS to the correct path. I don't know what causes the test program to fail, but it appears in my case that the xen headers are incompatible with the glibc version I am using.

[anttikantee]

looks more like stdint.h requiring something else to be included before it?

[kent-mcleod]

I think its library version incompatibilities. I tried on a different machine and didn't have any issues. I don't have much familiarity with using xen.

[kent-mcleod]

Broken packages:
libxen-dev:amd64 4.8.3+comet2+shim4.10.0+comet3-1+deb9u4.1 amd64 Public headers and libs for Xen
libc6-dev:amd64 2.26-2 amd64 GNU C Library: Development Libraries and Header Files
Working packages:
libxen-dev:amd64 4.8.3+comet2+shim4.10.0+comet3-1+deb9u4.1 amd64 Public
libc6-dev:amd64 2.24-11+deb9u1 amd64 GNU C Library: Development Libraries and Header Files

[kent-mcleod]

Notably, both kernels are Linux 4.9 so maybe if I was using a 4.9 libxen-dev it would work.

@roberto-torre I'd try compiling the test file:
echo -e "#include <stdint.h>\n#include <xen.h>" | gcc -I/usr/include/xen -x c - -c -o /dev/null
to see if you get the same compilation error.

[anttikantee]

does compiling a file with just #include <stdint.h> then work?

[kent-mcleod]

Only if you don't add the xen include path.

[anttikantee]

heh, well, ok, i don't know what to say

[kent-mcleod]

Tracing the included files on my working system:

gcc -I/usr/include/xen test.c -M
test.o: test.c /usr/include/stdc-predef.h \
 /usr/lib/gcc/x86_64-linux-gnu/6/include/stdint.h /usr/include/stdint.h \
 /usr/include/xen/features.h /usr/include/x86_64-linux-gnu/bits/wchar.h \
 /usr/include/x86_64-linux-gnu/bits/wordsize.h

And:

gcc  test.c -M
test.o: test.c /usr/include/stdc-predef.h \
 /usr/lib/gcc/x86_64-linux-gnu/6/include/stdint.h /usr/include/stdint.h \
 /usr/include/features.h /usr/include/x86_64-linux-gnu/sys/cdefs.h \
 /usr/include/x86_64-linux-gnu/bits/wordsize.h \
 /usr/include/x86_64-linux-gnu/gnu/stubs.h \
 /usr/include/x86_64-linux-gnu/gnu/stubs-64.h \
 /usr/include/x86_64-linux-gnu/bits/wchar.h

So xen's features.h file is replacing the system one. Removing this file on my broken machine makes it work.

[roberto-torre]

Ok, after trying some stuff I came across some interesting results:

• I edited probeprereqs() to force the XEN_HEADERS variable to /usr/include/xen. Still didn't work
• The version I'm using of libc6 is 2.26-0ubuntu2.1
• When trying to compile only stdint.h I come across this error
  echo -e "#include <stdint.h>\n#include <xen.h>" | gcc -I/usr/include/xen -x c - -c -o /dev/null
In file included from /usr/lib/gcc/x86_64-linux-gnu/7/include/stdint.h:9:0, from <stdin>:1: /usr/include/stdint.h:286:17: error: missing binary operator before token "(" #if __GLIBC_USE (IEC_60559_BFP_EXT)

So it may be that the error occurs because of libc. I will try to check other versions of it like 2.24-11, however I couldn't find it with apt

Edit: I almost crashed my computer when trying to downgrade my libc6 from 2.26 to 2.24...too many library incompatibilities...

[speas038]

Has anyone found a fix for this yet?

So this error has something to do with the order of includes in glibc, I think. I changed the line

#if __GLIBC_USE (IEC_60559_BFP_EXT)

to

#if __GLIBC_USE_IEC_60559_BFP_EXT

and got past the xen headers not found problem, but this obviously does nothing to solve the acutal problem, I just hope it may provide some clues