mon, osd, *: expose upmap-primary in OSDMap::get_features()

This is a minimal fix to ensure only peers understanding
`pg-upmap-primary` are able to connect, and thus to exclude
the possibility of running into the `pg_upmap_primaries.empty()`
assertion in encoders.

Fixes for other problems will follow up.

The intention is to ship this patch in the very next minor
release of reef.

Manual testing
--------------

\### start using upmap-primar is presence of `quincy` client
NOTE: incompatible clients aren't disconnected but this is
known and expected as we lack the machinery.

```
[rzarzynski@o06 build]$ bin/ceph osd get-require-min-compat-client
reef
[rzarzynski@o06 build]$ bin/ceph daemon mon.a sessions | jq  -jr '.[] | .name, "\t", .con_features, "\t", .con_features_hex, "\n"' | grep client
client.?        4540701547738038271     3f03cffffffdffff
client.?        4540138320759226367     3f01cfbf7ffdffff
[rzarzynski@o06 build]$ bin/ceph osd pool create test_pool 1 1
pool 'test_pool' created
[rzarzynski@o06 build]$ bin/ceph osd pg-upmap-primary 1.0 2
change primary for pg 1.0 to osd.2
[rzarzynski@o06 build]$ bin/ceph daemon mon.a sessions | jq  -jr '.[] | .name, "\t", .con_features, "\t", .con_features_hex, "\n"' | grep client
client.?        4540701547738038271     3f03cffffffdffff
client.?        4540138320759226367     3f01cfbf7ffdffff
```

\### `main` client is still able to connect
```
[rzarzynski@o06 build]$ bin/ceph -w
  cluster:
    id:     d570a7c-84ca-4fd0-aafb-80138762c6af
    health: HEALTH_WARN
            11 mgr modules have failed dependencies
            1 pool(s) do not have an application enabled

  services:
    mon: 1 daemons, quorum a (age 64m)
    mgr: x(active, since 64m)
    osd: 3 osds: 3 up (since 64m), 3 in (since 64m)

  data:
    pools:   1 pools, 1 pgs
    objects: 0 objects, 0 B
    usage:   3.0 GiB used, 300 GiB / 303 GiB avail
    pgs:     1 active+clean
```

\### `quincy` client is refused
```
[rzarzynski@o06 build-quincy]$ bin/ceph -s -c /home/rzarzynski/ceph2/build/ceph.conf
2024-05-30T08:59:42.411+0000 7f0911a9b700 -1 --2- 127.0.0.1:0/2812481872 >> [v2:127.0.0.1:40536/0,v1:127.0.0.1:40537/0] conn(0x7f090c111500 0x7f090c1118f0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0x7f08fc0048c0 tx=0x7f08fc009e30 comp rx=0 tx=0).handle_ident_missing_features client does not support all server features: 80000000
2024-05-30T08:59:42.612+0000 7f0911a9b700  0 --2- 127.0.0.1:0/2812481872 >> [v2:127.0.0.1:40536/0,v1:127.0.0.1:40537/0] conn(0x7f090c111500 0x7f090c1118f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request get_initial_auth_request returned -2
```

\### stop using upmap-primary
```
[rzarzynski@o06 build]$ bin/ceph osd rm-pg-upmap-primary 1.0
clear 1.0 pg_upmap_primary mapping
```

\### `quincy` client may connect again
```
[rzarzynski@o06 build-quincy]$ bin/ceph -s -c /home/rzarzynski/ceph2/build/ceph.conf
  cluster:
    id:     d570a7c-84ca-4fd0-aafb-80138762c6af
    health: HEALTH_WARN
            11 mgr modules have failed dependencies
            1 pool(s) do not have an application enabled

  services:
    mon: 1 daemons, quorum a (age 77m)
    mgr: x(active, since 77m)
    osd: 3 osds: 3 up (since 76m), 3 in (since 76m)

  data:
    pools:   1 pools, 1 pgs
    objects: 0 objects, 0 B
    usage:   3.0 GiB used, 300 GiB / 303 GiB avail
    pgs:     1 active+clean

```

Fixes: https://tracker.ceph.com/issues/61948
Signed-off-by: Radoslaw Zarzynski <[email protected]>

Author: rzarzynski

src/osd/OSDMap.cc

@@ -1770,9 +1770,12 @@ uint64_t OSDMap::get_features(int entity_type, uint64_t *pmask) const
     features |= CEPH_FEATURE_CRUSH_MSR;
   mask |= CEPH_FEATURES_CRUSH;
 
-  if (!pg_upmap.empty() || !pg_upmap_items.empty() || !pg_upmap_primaries.empty())
+  if (!pg_upmap.empty() || !pg_upmap_items.empty())
     features |= CEPH_FEATUREMASK_OSDMAP_PG_UPMAP;
   mask |= CEPH_FEATUREMASK_OSDMAP_PG_UPMAP;
+  if (!pg_upmap_primaries.empty())
+    features |= CEPH_FEATUREMASK_SERVER_REEF;
+  mask |= CEPH_FEATUREMASK_SERVER_REEF;
 
   for (auto &pool: pools) {
     if (pool.second.has_flag(pg_pool_t::FLAG_HASHPSPOOL)) {
@@ -1849,6 +1852,9 @@ ceph_release_t OSDMap::get_min_compat_client() const
   if (HAVE_FEATURE(f, CRUSH_MSR)) {
     return ceph_release_t::squid;        // v19.2.0
   }
+  if (HAVE_FEATURE(f, SERVER_REEF)) {	 // v18.2.3 (upmap-primary; see #61948)
+    return ceph_release_t::reef;
+  }
   if (HAVE_FEATURE(f, OSDMAP_PG_UPMAP) ||      // v12.0.0-1733-g27d6f43
       HAVE_FEATURE(f, CRUSH_CHOOSE_ARGS)) {    // v12.0.1-2172-gef1ef28
     return ceph_release_t::luminous;  // v12.2.0