Update Open MPI to require C11 features

[hjelmn]

We are discussing updating the compiler requirement for Open MPI to require C11. The features we want to require are:

• C11 atomics. These will completely replace the hand-coded assembly in Open MPI.
• Thread local support. This feature will completely replace the thread local storage code provided by opal.
• aligned_alloc. Right now we use memalign, posix_memalign, or malloc + hand aligning. This will bump the minimum glibc up to 2.16. If 2.16 is too new we can wrap this function in opal/util.

Like to haves:

• _Static_assert
• Generics

Compiler status pages:

• Intel: https://software.intel.com/en-us/articles/c11-support-in-intel-c-compiler . Intel is missing _Atomic even in the latest versions. We will have to use atomic_* (ex atomic_long instead of _Atomic long) to support this compiler.

• GCC: https://gcc.gnu.org/wiki/C11Status

• Clang: v3.1+ started adding support in v3.1. C11 support was enabled by default in 2014.

• PGI: Supported since at least late 2014. Will have to verify older releases.

• XLC: Supported since 2012

[hjelmn]

One comment. LANL tends to support some old compiler versions to support fortran. This can be handled by using a supported compiler and using the older fortran compiler for the bindings. Example: ./configure CC=gcc-6 FC=ifort.

[hjelmn]

One question is whether we should continue to offer opal_atomic_* functions or just go straight to using atomic_* everywhere. There are benefits to both but the later is a little more work and is probably cleaner in the long run.

[hjelmn]

Basic configury is complete. Will push the branch to my repository shortly.

configure: checking if gcc-7 supports C11 without a flag
checking if gcc-7 supports C11 thread local storage... yes
checking if gcc-7 supports C11 atomic variables... yes
checking if gcc-7 supports C11 _Atomic keyword... yes
checking if gcc-7 supports C11 _Generic keyword... yes
configure: no flag required for C11 support

[hjelmn]

We may want to keep some of the inline assembly going forward. It looks like gcc v7 removed support for lock-free compare-and-set for 128-bit integers on x86-64. This was done because the load/store of these integers is not atomic unless CAS is used. We do not care about this in the lifo/fifo implementations. A similar problem affects the LL/SC fifo/lifo implementations. There is no C11 atomic support for these atomics.

For now my C11 ompi prototype I will leave the x86-64 CAS128 and PPC/ARM64 LL/SC atomics and remove the rest.

[hjelmn]

Another note about C11 atomics. The language guarantees the atomicity of arithmetic and bit-wise operators (+, -, *, ^, etc). So these are equivalent:

atomic_int foo;

atomic_fetch_add (&foo, 1);

atomic_int foo;

foo += 1;

The benefit of the atomic_* calls is the ability to add _explicit and change the memory order. The default order when not using _explicit is memory_order_seq_cst.

This behavior has an impact on the OPAL_THREAD_* macros. Those will have the cast away the _Atomic to when opal_using_threads() == false to ensure that an atomic is not emitted by the compiler. Example:

typedef _Atomic int32_t opal_atomic_int32_t;

int32_t opal_thread_add_32 (opal_atomic_int32_t *ptr, int32_t value) {
    if (opal_using_threads ()) {
        return (*p += value);
    }

    return (*((int32_t *) p) += value);
}

[bosilca]

This will prevent us from declaring the members of the structures as atomic, as not all operations on these members need to be atomic.

[hjelmn]

To some degree yes. We have to be careful and always cast away the atomic where needed. I am working through out current atomic usage in opal now. Will have a branch up next week.