Skip to content

[Bug]: "Current status" panel on the "Kubernetes / Networking / Namespace (Workload)" dashboard fails when any container has multiple interfaces #1033

@solidDoWant

Description

@solidDoWant

What happened?

The "Current status" panel on the "Kubernetes / Networking / Namespace (Workload)" performs a join between various cAdvisor metrics (such as container_network_receive_bytes_total) and the kube_pod_info metric. This join is performed on pod name and namespace, and only includes pods that do not use host networking. This normally works great, except when their are multiple interfaces for non-host network pods.

When there are multiple interfaces assigned to a given pod, the query for cAdvisor metrics performed by the join (e.g. container_network_receive_bytes_total{namespace="some-namespace", pod="some-pod"} returns multiple time series. If the pod does not use host networking, then the kube_pod_info{namespace="some-namespace", pod="some-pod"} query performed by the join returns a single time series. This results in the join failing, as the left side contains two time series.

One of the queries for the panel is here. Depending on what the panel is intended to show, this could be fixed in one of two ways:

  1. A sum could be performed on the cAdvisor metrics to aggregate the traffic from all interfaces. Rows in the resulting panel would show the cumulative value of the metric for the entire pod.
  2. group_left could be added to the join to join each cAdvisor time series with the kube_pod_info series. There would be a separate row in the table for each interface in each pod.

Approach (2) is taken in this dashboard in at least one other place, but it would make the panel pretty "noisy" for namespaces with many pods with multiple interfaces.

What parts of the codebase are affected?

Dashboards

I agree to the following terms:

  • I agree to follow this project's Code of Conduct.
  • I have filled out all the required information above to the best of my ability.
  • I have searched the issues of this repository and believe that this is not a duplicate.
  • I have confirmed this bug exists in the default branch of the repository, as of the latest commit at the time of submission.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingkeepaliveUse to prevent automatic closing

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions