docs: updated VPA CustomResourceStateMetrics docs

Signed-off-by: Sherif Ayad <[email protected]>

Author: sherifkayad

docs/customresourcestate-metrics.md

@@ -250,14 +250,9 @@ kube_customresource_ref_info{customresource_group="myteam.io", customresource_ki
 
 #### VerticalPodAutoscaler
 
-In v2.9.0 the `vericalpodautoscalers` resource was removed from the list of default resources. In order to generate metrics for `verticalpodautoscalers`, you can use the following Custom Resource State config:
+In v2.9.0 the `verticalpodautoscalers` resource was removed from the list of default resources. In order to generate Memory / CPU recommendation metrics for `verticalpodautoscalers`, you can use the following Custom Resource State config:
 
 ```yaml
-# Using --resource=verticalpodautoscalers, we get the following output:
-# HELP kube_verticalpodautoscaler_annotations Kubernetes annotations converted to Prometheus labels.
-# TYPE kube_verticalpodautoscaler_annotations gauge
-# kube_verticalpodautoscaler_annotations{namespace="default",verticalpodautoscaler="hamster-vpa",target_api_version="apps/v1",target_kind="Deployment",target_name="hamster"} 1
-# A similar result can be achieved by specifying the following in --custom-resource-state-config:
 kind: CustomResourceStateMetrics
 spec:
   resources:
@@ -272,19 +267,53 @@ spec:
         target_kind: [spec, targetRef, kind]
         target_name: [spec, targetRef, name]
       metrics:
-        - name: "annotations"
-          help: "Kubernetes annotations converted to Prometheus labels."
+        - name: "vpa_containerrecommendations_target"
+          help: "VPA container recommendations for memory."
           each:
             type: Gauge
             gauge:
-              path: [metadata, annotations]
-# This will output the following metric:
-# HELP kube_customresource_autoscaling_annotations Kubernetes annotations converted to Prometheus labels.
-# TYPE kube_customresource_autoscaling_annotations gauge
-# kube_customresource_autoscaling_annotations{customresource_group="autoscaling.k8s.io", customresource_kind="VerticalPodAutoscaler", customresource_version="v1", namespace="default",target_api_version="autoscaling.k8s.io/v1",target_kind="Deployment",target_name="hamster",verticalpodautoscaler="hamster-vpa"} 123
+              path: [status, recommendation, containerRecommendations]
+              valueFrom: [target, memory]
+              labelsFromPath:
+                container: [containerName]
+          commonLabels:
+            resource: "memory"
+            unit: "byte"
+        - name: "vpa_containerrecommendations_target"
+          help: "VPA container recommendations for cpu."
+          each:
+            type: Gauge
+            gauge:
+              path: [status, recommendation, containerRecommendations]
+              valueFrom: [target, cpu]
+              labelsFromPath:
+                container: [containerName]
+          commonLabels:
+            resource: "cpu"
+            unit: "core"
+```
+
+The above configuration was tested on [this](https://github.com/kubernetes/autoscaler/blob/master/vertical-pod-autoscaler/examples/hamster.yaml) VPA configuration, outputting the following metrics:
+
+```
+# HELP kube_customresource_vpa_containerrecommendations_target VPA container recommendations for memory.
+kube_customresource_vpa_containerrecommendations_target{container="hamster",customresource_group="autoscaling.k8s.io",customresource_kind="VerticalPodAutoscaler",customresource_version="v1",namespace="default",resource="memory",target_api_version="autoscaling.k8s.io/v1",target_kind="Deployment",target_name="hamster",unit="byte",verticalpodautoscaler="hamster-vpa"} 1.048576e+08
+
+# HELP kube_customresource_vpa_containerrecommendations_target VPA container recommendations for cpu.
+kube_customresource_vpa_containerrecommendations_target{container="hamster",customresource_group="autoscaling.k8s.io",customresource_kind="VerticalPodAutoscaler",customresource_version="v1",namespace="default",resource="cpu",target_api_version="autoscaling.k8s.io/v1",target_kind="Deployment",target_name="hamster",unit="core",verticalpodautoscaler="hamster-vpa"} 0.1
 ```
 
-The above configuration was tested on [this](https://github.com/kubernetes/autoscaler/blob/master/vertical-pod-autoscaler/examples/hamster.yaml) VPA configuration, with an added annotation (`foo: 123`).
+To ensure that the VPA Metrics will be captured seamlessly, make sure that the Cluster Role / Role assigned to the Kube State Metrics has permissions to `list` & `watch` VPA resources e.g.:
+
+```yaml
+- verbs:
+    - list
+    - watch
+  apiGroups:
+    - autoscaling.k8s.io
+  resources:
+    - verticalpodautoscalers
+```
 
 ### Metric types