Merge pull request #2354 from splunk/urbiz-OD5595-k8s-cpm

[5595]: Control Plane Metrics histograms

Author: aurbiztondo-splunk

gdi/opentelemetry/collector-kubernetes/kubernetes-config-advanced.rst

@@ -57,16 +57,16 @@ This custom configuration is merged into the default agent configuration.
 
 .. _otel-kubernetes-config-advanced-control-plane:
 
-Override a control plane configuration
+Configure control plane metrics
 ==============================================================
 
-If any of the control plane metric receivers are activated under the ``agent.controlPlaneMetrics`` configuration section, then the Helm chart will configure the Collector to use the activated receivers to collect metrics from the control plane.
+Control plane metrics are available for the following components: ``coredns``, ``etcd``, ``kube-controller-manager``, ``kubernetes-apiserver``, ``kubernetes-proxy``, and ``kubernetes-scheduler``. You can use the :ref:`Collector Helm agent <helm-chart-components-agent>` to obtain control plane metrics from a specific component by setting ``agent.controlPlaneMetrics.{otel_component}`` to ``true``. 
 
-To collect control plane metrics, the Helm chart uses the Collector on each node to use the receiver creator to represent control plane receivers at runtime. The receiver creator has a set of discovery rules that know which control plane receivers to create. The default discovery rules can vary depending on the Kubernetes distribution and version. See :ref:`receiver-creator-receiver` for more information.
+The Helm chart uses the Collector on each node to use the receiver creator to represent control plane receivers at runtime. The receiver creator has a set of discovery rules that know which control plane receivers to create. The default discovery rules can vary depending on the Kubernetes distribution and version. See :ref:`receiver-creator-receiver` for more information.
 
 If your control plane is using non-standard specifications, then you can provide a custom configuration to allow the Collector to successfully connect to it.
 
-Availability and configuration instructions
+Supported versions
 -----------------------------------------------------------------------------
 
 The Collector relies on pod-level network access to collect metrics from the control plane pods. Since most cloud Kubernetes as a service distributions don't expose the control plane pods to the end user, collecting metrics from these distributions is not supported.
@@ -90,7 +90,10 @@ The following table shows which Kubernetes distributions support control plane m
 
 See the :new-page:`agent template <https://github.com/signalfx/splunk-otel-collector-chart/blob/main/helm-charts/splunk-otel-collector/templates/config/_otel-agent.tpl>` for the default configurations for the control plane receivers.
 
-See the following documentation for information on the configuration options and supported metrics for each control plane receiver:
+Availability 
+-----------------------------------------------------------------------------
+
+The following components provide control plane metrics:
 
 * :ref:`CoreDNS <coredns>`.
 * :ref:`etcd`. To retrieve etcd metrics, see :new-page:`Setting up etcd metrics <https://github.com/signalfx/splunk-otel-collector-chart/blob/main/docs/advanced-configuration.md#setting-up-etcd-metrics>`.
@@ -99,14 +102,6 @@ See the following documentation for information on the configuration options and
 * :ref:`Kubernetes proxy <kubernetes-proxy>`.
 * :ref:`Kubernetes scheduler <kubernetes-scheduler>`.
 
-Known issue
------------------------------------------------------------------------------
-
-There is a known limitation for the Kubernetes proxy control plane receiver. When using a Kubernetes cluster created using kops, a network connectivity issue prevents proxy metrics from being collected. The limitation can be addressed by updating the kubeProxy metric bind address in the kops cluster specification:
-
-#. Set ``kubeProxy.metricsBindAddress: 0.0.0.0`` in the kops cluster specification.
-#. Run ``kops update cluster {cluster_name}`` and ``kops rolling-update cluster {cluster_name}`` to deploy the change.
-
 Use custom configurations for non-standard control plane components
 -----------------------------------------------------------------------------
 
@@ -135,6 +130,30 @@ The following example shows how to connect to a nonstandard API server that uses
                 useHTTPS: true
                 useServiceAccount: false
 
+.. _kubernetes-control-plane-prometheus:
+
+Activate Kubernetes control plane metrics with the Prometheus receiver
+-------------------------------------------------------------------------
+
+To activate control plane metrics with the OpenTelemetry Prometheus receiver instead, use the feature flag ``useControlPlaneMetricsHistogramData``:
+
+.. code-block:: yaml
+
+  featureGates:
+    useControlPlaneMetricsHistogramData: true
+
+.. Note:: Out-of-the-box dashboards and navigators for control plane metrics with the Prometheus receiver aren't supported yet, but are planned for a future release.
+
+To learn more see :ref:`prometheus-receiver`.
+
+Known issues
+-----------------------------------------------------------------------------
+
+There is a known limitation for the Kubernetes proxy control plane receiver. When using a Kubernetes cluster created using kops, a network connectivity issue prevents proxy metrics from being collected. The limitation can be addressed by updating the kubeProxy metric bind address in the kops cluster specification:
+
+#. Set ``kubeProxy.metricsBindAddress: 0.0.0.0`` in the kops cluster specification.
+#. Run ``kops update cluster {cluster_name}`` and ``kops rolling-update cluster {cluster_name}`` to deploy the change.
+
 .. _kubernetes-config-advanced-non-root:
 
 Run the container in non-root user mode

gdi/opentelemetry/collector-kubernetes/kubernetes-helm-architecture.rst

@@ -13,6 +13,8 @@ The Helm chart for the Collector has three components: agent, cluster receiver,
 
 .. note:: For use cases about the different components, see the GitHub documentation :new-page:`Splunk OpenTelemetry Collector Helm Chart Components: Use Cases <https://github.com/jvoravong/splunk-otel-collector-chart/blob/Feature-components-doc/docs/components.md#use-cases>`.
 
+.. _helm-chart-components-agent:
+
 Agent component
 ==============================================