4033: add cri_losing_support metric and clarify support for containerd 1.7 will be dropped in 1.36

Signed-off-by: Peter Hunt <[email protected]>

Author: haircommander

keps/sig-node/4033-group-driver-detection-over-cri/README.md

@@ -186,10 +186,15 @@ cgroupDriver option has been deprecated and will be dropped in a future release.
 The `--cgroup-driver` flag and the cgroupDriver configuration option will be
 deprecated when support for the feature is graduated to GA.
 The configurations flags (and the related fallback behavior) will be removed in
-a later release as per the [Kubernetes deprecation policy][deprecation-policy].
+Kubernetes 1.36. This aligns well with containerd v1.7 going out of support, which is the last
+remaining supported CRI that doesn't have support for this field.
 At the point the kubelet refuses to start if the CRI runtime does not support
 the feature.
 
+Between version 1.34 and 1.36, the kubelet will emit a counter metric (`cri_losing_support`) when a CRI implementation is
+used that doesn't have support for the RuntimeConfig CRI call. This metric will have a label describing the version support will be dropped by.
+If one node in a cluster has containerd running with 1.7, the metric will look like `cri_losing_support{,version="1.36"} 1`.
+
 Kubelet startup is modified so that connection to the CRI server (container
 runtime) is established and RuntimeConfig is queried before initializing the
 kubelet internal container-manager which is responsible for kubelet-side cgroup
@@ -198,8 +203,6 @@ succeed, an error (error response or timeout) is regarded as a failed
 initialization of the runtime service and kubelet will exit with an error
 message and an error code.
 
-[deprecation-policy]: https://kubernetes.io/docs/reference/using-api/deprecation-policy/#deprecating-a-flag-or-cli
-
 ### Test Plan
 
 [x] I/we understand the owners of the involved components may require updates to
@@ -329,8 +332,8 @@ CgroupDriver as they must be today.
 
 ###### What specific metrics should inform a rollback?
 
-Nodes being in NotReady state with kubelet logs indicating an error in the
-RuntimeConfig CRI request, making kubelet fail to start.
+`cri_losing_support` metric will be populated on nodes where the CRI implementation will one day lose support. After 1.36, kubelet will fatally error,
+so admins should upgrade their out of support CRI implementations (if `version==1.36`).
 
 ###### Were upgrade and rollback tested? Was the upgrade->downgrade->upgrade path tested?
 
@@ -357,8 +360,8 @@ info`).
 
 ###### How can someone using this feature know that it is working for their instance?
 
-No metrics will expose this. Examining kubelet logs whould inform
-that the cgroup driver setting instructed by the runtime is being used.
+The metric `cri_losing_support` when `version == 1.36` will indicate those nodes will be out of support in 1.36.
+If that metric is unpopulated, the feature is on (as it's GA) and the flag fallback is not being used.
 
 After GA, the CgroupDriver configuration option and the `--cgroup-driver` flag
 will be removed in a future release, in accordance with the
@@ -378,7 +381,8 @@ N/A.
 
 ###### Are there any missing metrics that would be useful to have to improve observability of this feature?
 
-N/A.
+The metric `cri_losing_support` when `version == 1.36` will indicate those nodes will be out of support in 1.36.
+If that metric is unpopulated, the feature is on (as it's GA) and the flag fallback is not being used.
 
 ### Dependencies
 

keps/sig-node/4033-group-driver-detection-over-cri/kep.yaml

@@ -38,6 +38,5 @@ feature-gates:
       - kubelet
 disable-supported: false
 
-# The following PRR answers are required at beta release
-# No metrics will be added for this feature, as it's an internal
-# implementation detail
+metrics:
+  - cri_losing_support