MB-40531: [BP] Make replica eviction ignore age threshold

Backport of http://review.couchbase.org/c/kv_engine/+/134805

Normally items above a configured MFU value
item_eviction_freq_counter_age_threshold
and below a configured age item_eviction_age_percentage are not
evicted.

This avoids evicting "young" items; their MFU counter might not
reflect how hot they are yet as they were only recently stored.

However, when evicting from replica vbuckets, this may not be as
beneficial. As active resident ratio has a direct impact on cache miss
rate and average latency, it may be better to evict even young replica
items to reclaim memory where possible, before considering active vbs.

Change-Id: I3aaaa13e6a682000eb520e8971ad1451dbec7f48
Reviewed-on: http://review.couchbase.org/c/kv_engine/+/134663
Tested-by: Build Bot <[email protected]>
Well-Formed: Build Bot <[email protected]>
Reviewed-by: Jim Walker <[email protected]>
Reviewed-by: Paolo Cocchi <[email protected]>

Author: jameseh96

engines/ep/src/paging_visitor.cc

@@ -122,9 +122,23 @@ bool PagingVisitor::visit(const HashTable::HashBucketLock& lh, StoredValue& v) {
     uint64_t age = (maxCas > v.getCas()) ? (maxCas - v.getCas()) : 0;
     age = age >> ItemEviction::casBitsNotTime;
 
-    if ((storedValueFreqCounter <= freqCounterThreshold) &&
-        ((storedValueFreqCounter < freqCounterAgeThreshold) ||
-         (age >= ageThreshold))) {
+    const bool belowMFUThreshold =
+            storedValueFreqCounter <= freqCounterThreshold;
+    // age exceeds threshold (from age histogram, set by config param
+    // item_eviction_age_percentage
+    // OR
+    // MFU is below threshold set by config param
+    // item_eviction_freq_counter_age_threshold
+    // Below this threshold the item is considered "cold" enough
+    // to be evicted even if it is "young".
+    const bool meetsAgeRequirements =
+            age >= ageThreshold ||
+            storedValueFreqCounter < freqCounterAgeThreshold;
+
+    // For replica vbuckets, young items are not protected from eviction.
+    const bool isReplica = currentBucket->getState() == vbucket_state_replica;
+
+    if (belowMFUThreshold && (meetsAgeRequirements || isReplica)) {
         /*
          * If the storedValue is eligible for eviction then add its
          * frequency counter value to the histogram, otherwise add the