New eyes - Makefile.am with brew install on Ubuntu 20 LTS GitHub Actions - LD_LIBRARY_PATH needed

[ghenry]

Hi all,

For the past few days I've been trying to solve my issue. I use libmicrohttpd within SentryPeer, but the version that comes ubuntu-latest is too old for me, so I brew install libmicrohttpd. All works fine until I do:

./sentrypeer -h
./sentrypeer: error while loading shared libraries: libmicrohttpd.so.12: cannot open shared object file: No such file or directory

I'm using pkg-config from Homebrew to find it and set MHD_CFLAGS and MHD_LIBS:

https://github.com/SentryPeer/SentryPeer/blob/main/configure.ac#L53

and

https://github.com/SentryPeer/SentryPeer/blob/main/Makefile.am#L39

all is well on macOS, just Ubuntu. I've even installed a Ubuntu 20 LTS on Virtualbox and can reproduce it. If I set this, all is well, but I don't know why this isn't automatically working:

LD_LIBRARY_PATH=/home/linuxbrew/.linuxbrew/lib/ ./sentrypeer -h
Usage: sentrypeer [-h] [-V] [-s] [-v] [-d]

Options:
  -h,      Print this help
  -V,      Print version
  -s,      Enable syslog logging
  -v,      Enable verbose logging
  -d,      Enable debug mode

Report bugs to https://github.com/SentryPeer/SentryPeer/issues

See https://sentrypeer.org for more information

You can see the Action results here - https://github.com/SentryPeer/SentryPeer/runs/4413826525?check_suite_focus=true#step:4:238
and the config for the Action here - https://github.com/SentryPeer/SentryPeer/blob/main/.github/workflows/main.yml

I think this is the best place to ask? What have I missed?

Thanks for reading,
Gavin.

[danielnachun]

It is probably because the pkg-config flags don't include -Wl,-rpath,/home/linuxbrew/.linuxbrew/lib to the LDFLAGS. You should be able to use something like export LDFLAGS='-Wl,-rpath,/home/linuxbrew/.linuxbrew/lib' to add the missing flag that when you compile SentryPeer with brewed libmicrohttpd.

@Homebrew/linux what's our policy on adding RPATH flags to the Libs stanza of .pc files for libraries we ship? We don't need to this when we use those libraries as dependencies within Homebrew itself because we add the RPATH flags in the compiler shims (and the GCC spec file also adds it as well on Linux). However when using those libraries outside of Homebrew, they need the extra RPATH flag for the library to be found.

[danielnachun]

Yeah keg relocation would be a logical place to do this. All it would have to do is add an RPATH to the optdir/lib for the affected package. The only thing I'm not sure about is whether it should be Linux only or also added on macOS.

[carlocab]

Linux only. This is not needed on macOS.

[ghenry]

Yeah, no issues on macOS.

[ghenry]

It is probably because the pkg-config flags don't include -Wl,-rpath,/home/linuxbrew/.linuxbrew/lib to the LDFLAGS. You should be able to use something like export LDFLAGS='-Wl,-rpath,/home/linuxbrew/.linuxbrew/lib' to add the missing flag that when you compile SentryPeer with brewed libmicrohttpd.

Thanks Daniel. I've switched to a Ubuntu PPA but will go back now as I want v0.9.73

[ghenry]

0.9.74 was just released. Can I make this work without the export via a PR?

Thanks.

[ghenry]

Anything I can do for this? I'm hitting the same issue now with zyre, czmq and zeromq

[carlocab]

PRs welcome. You should probably reference $(brew --prefix) instead of /home/linuxbrew/.linuxbrew there for portability. You might want to query that brew is in PATH first, though.

[ghenry]

The only .pc files seem to be in brew/Library/Homebrew/os/mac/pkgconfig/* and macOS doesn't have this problem.

[danielnachun]

Sorry to be clear, this code will have to scan for .pc files in kegs, not in the code for brew. Your PR will probably need to make changes to Library/Homebrew/keg_relocate.rb, where we do other keg relocation steps. It does take a while to work through the inner logic of how all this works (I recently had to do this for a PR I'm working on). I may try to take a crack at this soon because it is kind of difficult to understand all this logic if you're not familiar with how brew works internally.

[ghenry]

Understood. Anything changed recently? I'm interested in how I've hit this rpath issue and no one else ever has? Probably my project set up :)

Thanks.

[danielnachun]

It's probably because Homebrew doesn't officially supporting using our libraries as dependencies for other projects. Despite being unsupported, on macOS this usually works okay because library paths are mostly resolved by the loader using "install names" that encode absolute paths to shared libraries in compiled binaries.

On Linux it's a different story - RPATH is almost always used by the loader to find the shared libraries. When we compile binaries with GCC in Homebrew, we use a special compiler "shim" which automatically adds -Wl,-rpath,#{HOMEBREW_PREFIX}/lib to our linker flags. While this works very nicely for us when building binaries within Homebrew, if you try to build binaries with GCC that link to Homebrew shared libraries outside of Homebrew, that RPATH obviously isn't going to be added automatically.

One other detail I neglected to mention is that if you use a Homebrew GCC installation as your compiler when building outside Homebrew, we add that RPATH to something called the GCC "spec" so that it will get added automatically when you compile. Note that this is not the case for Homebrew LLVM, which does not contain an equivalent to a spec file. And obviously when you use your host GCC, its spec file won't contain the additional RPATH either, hence why you have to add it yourself.

[ghenry]

Thanks. I'll have a play over the next few days or so.

[ghenry]

Thanks for the clear explanation Daniel. Much appreciated. I might experiment with the Homebrew gcc.