MB-56094: Add 'cursor_distance' to CM cursors stats

Another useful information at diagnostic, distance from checkpoint
begin.

Example, in a state like:

  [e:1 cs:1 vbs:1 vbs:1)
                  ^

It would be:

  vb_0:persistence:cursor_distance:       3
  vb_0:persistence:cursor_op:             set_vbucket_state
  vb_0:persistence:cursor_seqno:          1

Note: We can have multiple consecutive set_vbucket_state items that
share the same seqno. A cursor placed at the first vbs would report
distance:2.

Change-Id: If969c62132ccd1731d0d4abaacf005d3dbb93792
Reviewed-on: https://review.couchbase.org/c/kv_engine/+/189985
Reviewed-by: Dave Rigby <[email protected]>
Tested-by: Build Bot <[email protected]>

Author: paolococchi

engines/ep/docs/stats.org

@@ -991,6 +991,8 @@ each stat name.
 
 | cursor_name:cursor_checkpoint_id    | Checkpoint ID at which the cursor is      |
 |                                     | name 'cursor_name' is pointing now        |
+| cursor_name:cursor_distance         | The distance of cursor from checkpoint    |
+|                                     | begin                                     |
 | cursor_name:cursor_seqno            | The seqno at which the cursor             |
 |                                     | 'cursor_name' is pointing now             |
 | cursor_name:cursor_op               | The type of operation of the item pointed |

engines/ep/src/checkpoint_manager.cc

@@ -1585,6 +1585,14 @@ void CheckpointManager::addStats(const AddStatFn& add_stat,
                             cookie);
 
             const auto pos = cursor.second->getPos();
+            checked_snprintf(buf.data(),
+                             buf.size(),
+                             "vb_%d:%s:cursor_distance",
+                             vbucketId.get(),
+                             name.c_str());
+            add_casted_stat(
+                    buf.data(), cursor.second->getDistance(), add_stat, cookie);
+
             checked_snprintf(buf.data(),
                              buf.size(),
                              "vb_%d:%s:cursor_op",

engines/ep/tests/ep_testsuite.cc

@@ -7246,6 +7246,7 @@ static enum test_result test_mb19687_fixed(EngineIface* h) {
         // Add stats which are only available for persistent buckets:
         std::initializer_list<std::string_view> persistence_stats = {
                 "vb_0:persistence:cursor_checkpoint_id",
+                "vb_0:persistence:cursor_distance",
                 "vb_0:persistence:cursor_op",
                 "vb_0:persistence:cursor_seqno",
                 "vb_0:persistence:num_visits",