rcache/vma: make rcache lock recursive

There is currently a path through the grdma mpool and vma rcache that
leads to deadlock. It happens during the rcache insert. Before the
insert the rcache mutex is locked. During the call a new vma item is
allocated and then inserted into the rcache tree. The allocation
currently goes through the malloc hooks which may (and does) call back
into the mpool if the ptmalloc heap needs to be reallocated. This
callback tries to lock the rcache mutex which leads to the
deadlock. This has been observed with multi-threaded tests and the
openib btl.

This change may lead to some minor slowdown in the rcache vma when
threading is enabled. This will only affect larger message paths in
some of the btls.

master commit open-mpi/ompi@1d56007

Signed-off-by: Nathan Hjelm <[email protected]>

Author: hjelmn

opal/mca/rcache/vma/rcache_vma.c

@@ -42,7 +42,7 @@ void mca_rcache_vma_module_init( mca_rcache_vma_module_t* rcache ) {
     rcache->base.rcache_clean = mca_rcache_vma_clean;
     rcache->base.rcache_finalize = mca_rcache_vma_finalize;
     rcache->base.rcache_dump_range = mca_rcache_vma_dump_range;
-    OBJ_CONSTRUCT(&rcache->base.lock, opal_mutex_t);
+    OBJ_CONSTRUCT(&rcache->base.lock, opal_recursive_mutex_t);
     mca_rcache_vma_tree_init(rcache);
 }