Skip to content

log.json of a container may grow to burst the tmpfs of /run, if a k8s user configure an exec liveness probe of a non exist executable file name. #8972

@abel-von

Description

@abel-von

Description

The runc exec command prints the error log to /run/containerd/io.containerd.runtime.v2.task/k8s.io/<cid>/log.json.
When a k8s user configure a exec liveness probe with a executable file that does not exist in container, this exec command will execute periodically according to the liveness probe config, and it will fail again and again. kubelet will consider this failure as a wrong config of liveness probe and will just send an event to k8s master, rather than restart the container. so the log.json file will grow unlimitedly if the k8s user ignored the liveness probe failure as all functions seems working properly.
As a result, the k8s node will become unhealthy as the tmpfs of /run may be bursted.

Steps to reproduce the issue

  1. kubectl create a pod with liveness probe of a non exist executable path
  2. check size growth of the log.json file in /run/containerd/io.containerd.runtime.v2.task/k8s.io/<cid>/log.json
  3. size of the file grow along the time.

Describe the results you received and expected

expected: The misconfiguration of a container will not break the whole node.

received: the tmpfs of /run will burst and the node will be broken.

What version of containerd are you using?

v1.6.14

Any other relevant information

runc version: 1.1.3

Show configuration if it is related to CRI plugin.

config.toml is not related to this issue.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    Projects

    Status

    Todo

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions