concurrently iterating over a parallel_flat_hash_map

[vondele]

I would like to concurrently iterate over a parallel_flat_hash_map (read-only access). I could imagine that this is possible if for each thread I could assign a begin(), end() range that doesn't overlap, and I could imagine this would be efficient if I could get a begin() and end() for each of the submaps.

Is there a public API for that, or is there an easy way to do it nevertheless?

[greg7mdp]

Yes, this is definitely possible.
You can use with_submap or with_submap_m (see here ) - use the second version with _m if you mutate the submaps.

You can also use get_inner. I don't have it in a current example, but here is a code fragment:

template <class K, class V, size_t N, class Mutex>
using Map = phmap::parallel_flat_hash_map<K, V, phmap::priv::hash_default_hash<K>, phmap::priv::hash_default_eq<K>,
                                          phmap::priv::Allocator<phmap::priv::Pair<const K, V>>, N, Mutex>;

using OutputTable = Map<Bitset, double, 8, std::mutex>;

OutputTable test(const Table& table1, const Table& table2) {
   OutputTable table;

   #pragma omp parallel for // schedule(static, 1)
   for (size_t i = 0; i < table1._table.subcnt(); ++i) {
      const auto& set1 = table1._table.get_inner_m(i).set_;
      const auto& set2 = table2._table.get_inner_m(i).set_;
      {
         for (const auto& [index, vec1] : set1) {
            auto it = set2.find(index);
            if (it != set2.end()) {
               const auto& vec2 = it->second;

               // create a new hashmap to contain all results sharing the common pattern
               for (const auto& e1 : vec1) {
                  for (const auto& e2 : vec2) {
                     Bitset b(e1.first);
                     b |= e2.first;
                     table.lazy_emplace_l(
                        b, [&](OutputTable::value_type& entry) { entry.second += e1.second * e2.second; },
                        [&](const OutputTable::constructor& c) { c(std::move(b), e1.second * e2.second); });
                  }
               }
            }
         }
      }
   }

   return table;
}

[vondele]

I went for the with_submap variant, works very nicely, minimal change to the code
https://github.com/vondele/cdbsubtree/blob/5ec757bf7ab1a7a5b1ed1270ae1b1872c60cc145/main.cpp#L337-L354

[greg7mdp]

Looks good!