osd: Optimized EC calculate_maxles_and_minlua needs to use ...

exclude_nonprimary_shards

When an optimized EC pool is searching for the best shard that
isn't a non-primary shard then the calculation for maxles and
minlua needs to exclude nonprimary-shards

This bug was seen in a test run where activating a PG was
interrupted by a new epoch and only a couple of non-primary
shards became active and updated les. In the next epoch
a new primary (without log) failed to find a shard that
wasn't non-primary with the latest les. The les of
non-primary shards should be ignored when looking for
an appropriate shard to get the full log from.

This is safe because an epoch cannot start I/O without
at least K shards that have updated les, and there
are always K-1 non-primary shards. If I/O has started
then we will find the latest les even if we skip
non-primary shards. If I/O has not started then the
latest les ignoring non-primary shards is the
last epoch in which I/O was started and has a good
enough log+missing list.

Signed-off-by: Bill Scales <[email protected]>

Author: bill-scales

src/osd/PeeringState.cc

@@ -1615,13 +1615,17 @@ void PeeringState::reject_reservation()
 void PeeringState::calculate_maxles_and_minlua( const map<pg_shard_t, pg_info_t> &infos,
 						epoch_t& max_last_epoch_started,
 						eversion_t& min_last_update_acceptable,
+						bool exclude_nonprimary_shards,
 						bool *history_les_bound) const
 {
   /* See doc/dev/osd_internals/last_epoch_started.rst before attempting
    * to make changes to this process.  Also, make sure to update it
    * when you find bugs! */
   max_last_epoch_started = 0;
   for (auto i = infos.begin(); i != infos.end(); ++i) {
+    if (exclude_nonprimary_shards &&
+	pool.info.is_nonprimary_shard(shard_id_t(i->first.shard)))
+      continue;
     if (!cct->_conf->osd_find_best_info_ignore_history_les &&
 	max_last_epoch_started < i->second.history.last_epoch_started) {
       if (history_les_bound) {
@@ -1665,6 +1669,7 @@ map<pg_shard_t, pg_info_t>::const_iterator PeeringState::find_best_info(
   calculate_maxles_and_minlua( infos,
 			       max_last_epoch_started,
 			       min_last_update_acceptable,
+			       exclude_nonprimary_shards,
 			       history_les_bound);
 
   if (min_last_update_acceptable == eversion_t::max())

src/osd/PeeringState.h

@@ -1691,6 +1691,7 @@ class PeeringState : public MissingLoc::MappingInfo {
   void calculate_maxles_and_minlua( const std::map<pg_shard_t, pg_info_t> &infos,
 				    epoch_t& max_last_epoch_started,
 				    eversion_t& min_last_update_acceptable,
+				    bool exclude_nonprimary_shards = false,
 				    bool *history_les_bound = nullptr) const;
 
   // acting std::set