need ability to not use --enable-new-dtags option in linker scripts.

[hppritcha]

It turns out the --enable-new-dtags linker option in the Open MPI compiler scripts is causing issues for some users:

Here's what a sysadmin reported:

Many of our users want to encode an RPATH to the MPI library in their executable at link time. In fact, they like to encode paths to all of their other libraries this way. On Linux, RPATH overrides anything a user sets in LD_LIBRARY_PATH, which provides one way for our app developers to ensure that thier users are getting the correct set of libraries (they can't mess things up with a wrong LD_LIBRARY_PATH setting). The --enable-new-dtags flag changes things so that LD_LIBRARY_PATH settings now override any RPATH setting, which breaks our apps.

MVAPICH recently added the --enable-new-dtags flag to their wrappers starting with MV2-2.1, which broke many of our applications. At that point, I also looked into Open MPI and found that this option was added somewhere by the time 1.8.3 was released. Apparently, this particular set of users wasn't using Open MPI, and so we didn't notice the change in Open MPI until now.

I asked MVAPICH to add a configure flag to avoid the --enable-new-dtags option. I now configure MV2-2.1 to drop this flag in the wrappers. I was looking to see if there's a similar fix already in Open MPI that we could use, and if not, if something could be added.
-Adam

@adammoody

[jsquyres]

@adammoody Does --disable-wrapper-rpath not work for you?

[dshrader]

@jsquyres I worked with @adammoody and @hppritcha on this and am following up on it now.

Using --disable-wrapper-rpath does solve the issue in that --enable-new-dtags is no longer present in the /share/openmpi/share/openmpi/*-wrapper-data.txt files. However, this solution is undesirable as it also removes the ability for the compiler wrappers to rpath. What we're hoping to have is the ability to isolate the --enable-new-dtags link option, which is about how rpathing is implemented, from the ability of the compiler wrappers to rpath in the first place.

Removing all references of --enable-new-dtags in the /share/openmpi/share/openmpi/*-wrapper-data.txt files solves the issue in that rpathing is still possible and --enable-new-dtags is no longer used when linking, but that has to be done after make install. It would be nice to be able to specify this at Open MPI's build time.

I also wonder why --enable-new-dtags is on by default as it takes away choice from users. Having it on by default leaves Open MPI users with no method of turning it off (depending on write access); having it off by default leaves it up to the user to have it on or off.

[payerle]

Please allow an option to disable adding the --enable-new-dtags flag.
We are running into issues with it for openmpi/1.10.2 built with intel/2016.3.210 compiler suite; the
compiler environmental setup scripts sets LD_LIBRARY_PATH to include the path to Intel MPI libraries,
which are being preferred to the OpenMPI libraries set in rpath (because if DT_RUNPATH is set, DT_RPATH is ignored, so LD_LIBRARY_PATH takes precedence over both).

I can hack the configure scripts, etc. to mimic an loader which does not support --enable-new-dtags,
but it would be nice to have a configure option. Please add an option to disable this.

[adammoody]

@hppritcha we recently stumbled into this issue again. Any update here?

For now, I'm planning to hack configure to remove the --enable-new-dtags, but that's my ugly solution.

I'm hoping we could get a --disable-new-dtags configure option added to config/opal_setup_wrappers.m4 to do the same. Basically, the idea would be to skip running OPAL_SETUP_RUNPATH if --disable-new-dtags is thrown. I could post a PR for this. Just let me know. Thanks!

[hppritcha]

@adammoody let me know if the patch works

[hppritcha]

reopening issue because some sites are wanting this back ported to 3.x.y releases.