CNTRLPLANE-1544: staticpod: guard: Use user namespace

[tchap]

Test for Downstream Components

• kube-controller-manager: WIP: Test guard user namespace cluster-kube-controller-manager-operator#887
• kube-apiserver: WIP: Test user namespace guard cluster-kube-apiserver-operator#1936
• etcd: WIP: Test guard user namespace cluster-etcd-operator#1498
• kube-scheduler: WIP: Test user namespace guard cluster-kube-scheduler-operator#582

[openshift-ci-robot]

@tchap: This pull request references CNTRLPLANE-1544 which is a valid jira issue.

In response to this:

This is not a big deal, but we should use use namespaces for as many pods as possible.

Tests run via openshift/cluster-kube-controller-manager-operator#887

Checked manually that the guard pods are using user namespace.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

[p0lyn0mial]

since the guard pod is used by more than one static pod, maybe we should create proof PRs for all of them?

[openshift-ci-robot]

@tchap: This pull request references CNTRLPLANE-1544 which is a valid jira issue.

In response to this:

The current configuration is equivalent to restricted-v3 scc.
The scc annotation can only be set on a workload, though, not on a
static pod, but we can still apply relevant settings manually.

This is not a big deal, but we should use use namespaces for as many pods as possible.

Tests run via openshift/cluster-kube-controller-manager-operator#887

Checked manually that the guard pods are using user namespace.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

[tchap]

/hold

until we create and test proof PRs for all affected components.

[p0lyn0mial]

The current configuration is equivalent to restricted-v3 scc.
The scc annotation can only be set on a workload, though, not on a
static pod, but we can still apply relevant settings manually

I think that the guard pods are normal pods not static pods.

[p0lyn0mial]

The current configuration is equivalent to restricted-v3 scc.

do we need to set scc to use hostUsers: false?

[haircommander]

do we need to set scc to use hostUsers: false?

nope, everyone has access to it

[haircommander]

I think all of the security knobs you've set here are good, but it may be tricky to debug problems if you do them all at once. The kernel isn't very informative why things fail when they do. If the tests pass in all of the components that inherit this, that's fine, but I would recommend starting with hostUsers: false and adding seccomp+dropped caps pieces after.

[p0lyn0mial]

@haircommander thank you for your input.

@haircommander what is your recommendation for setting the scc? Would it be better to set the openshift.io/required-scc: restricted-v3 annotation and rely on the admission to apply the details?

[haircommander]

if this is creating static pods are they subject to SCC? I would expect not, as static pods don't go through apiserver to be created

[p0lyn0mial]

if this is creating static pods are they subject to SCC?

I don't think this creates a static pod. I think the guard-pod is just a regular pod.

Since setting hostUsers: false doesn’t require SCC and is enough to enable to use "user namespaces", why not setting only this field first?

[p0lyn0mial]

@haircommander one more question, are we right that it’s safe to set hostUsers in 4.21 (current master) , because user namespaces went GA in 4.20, and for odd versions the kubelet can only be one version behind?

[haircommander]

Yes that seems safe to me!

[haircommander]

Since setting hostUsers: false doesn’t require SCC and is enough to enable to use "user namespaces", why not setting only this field first?

yeah just hostUsers: false would work with me

[tchap]

Yeah, honestly I don't think we have to care about openshift.io/required-scc: restricted-v3 that much. I am working on user namespaces for the whole control plane and some of the namespaces have the run level annotation, which disables SCC admission altogether. It's very possible some of these pods are run in namespaces with SCC disabled as well. So we can't really rely on that.

And yeah, I've just checked, these are not static pods, sorry for the confusion.

[openshift-ci-robot]

@tchap: This pull request references CNTRLPLANE-1544 which is a valid jira issue.

In response to this:

Enable user namespaces and use a non-root user.

Tests run via openshift/cluster-kube-controller-manager-operator#887

Checked manually that the guard pods are using user namespace.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

[tchap]

I ended up setting only hostUsers: false and setting a non-root user/group IDs, for now. Actually the issue doesn't require ensuring everything runs as restricted-v3, I am just enabling that wherever possible, because why not. But we are fine with only user namespaces here.

[p0lyn0mial]

I ended up setting only hostUsers: false and setting a non-root user/group IDs, for now. Actually the issue doesn't require ensuring everything runs as restricted-v3, I am just enabling that wherever possible, because why not. But we are fine with only user namespaces here.

@tchap can we just set hostUsers: false ?

[tchap]

can we just set hostUsers: false ?

Ok, amended.

[openshift-ci-robot]

@tchap: This pull request references CNTRLPLANE-1544 which is a valid jira issue.

In response to this:

Tests run via openshift/cluster-kube-controller-manager-operator#887

Checked manually that the guard pods are using user namespace.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

[tchap]

I still need to create testing PRs for all components using guard pods, will get to that later today, I hope.

[p0lyn0mial]

/lgtm

/hold
for #2031 (comment)

[openshift-ci]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: p0lyn0mial, tchap

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• pkg/operator/staticpod/OWNERS [p0lyn0mial]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[openshift-ci-robot]

@tchap: This pull request references CNTRLPLANE-1544 which is a valid jira issue.

In response to this:

Checked manually that the guard pods are using user namespace.

Test for Downstream Components

• kube-controller-manager: WIP: Test guard user namespace cluster-kube-controller-manager-operator#887
• kube-apiserver: WIP: Test user namespace guard cluster-kube-apiserver-operator#1936
• etcd
• kube-scheduler

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

[openshift-ci-robot]

@tchap: This pull request references CNTRLPLANE-1544 which is a valid jira issue.

In response to this:

Checked manually that the guard pods are using user namespace.

Test for Downstream Components

• kube-controller-manager: WIP: Test guard user namespace cluster-kube-controller-manager-operator#887
• kube-apiserver: WIP: Test user namespace guard cluster-kube-apiserver-operator#1936
• etcd
• kube-scheduler: WIP: Test user namespace guard cluster-kube-scheduler-operator#582

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

[openshift-ci-robot]

@tchap: This pull request references CNTRLPLANE-1544 which is a valid jira issue.

In response to this:

Test for Downstream Components

• kube-controller-manager: WIP: Test guard user namespace cluster-kube-controller-manager-operator#887
• kube-apiserver: WIP: Test user namespace guard cluster-kube-apiserver-operator#1936
• etcd: WIP: Test guard user namespace cluster-etcd-operator#1498
• kube-scheduler: WIP: Test user namespace guard cluster-kube-scheduler-operator#582

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

[openshift-ci-robot]

@tchap: This pull request references CNTRLPLANE-1544 which is a valid jira issue.

In response to this:

Test for Downstream Components

• kube-controller-manager: WIP: Test guard user namespace cluster-kube-controller-manager-operator#887
• kube-apiserver: WIP: Test user namespace guard cluster-kube-apiserver-operator#1936
• etcd: WIP: Test guard user namespace cluster-etcd-operator#1498
• kube-scheduler: WIP: Test user namespace guard cluster-kube-scheduler-operator#582

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

[openshift-ci-robot]

@tchap: This pull request references CNTRLPLANE-1544 which is a valid jira issue.

In response to this:

Test for Downstream Components

• kube-controller-manager: WIP: Test guard user namespace cluster-kube-controller-manager-operator#887
• kube-apiserver: WIP: Test user namespace guard cluster-kube-apiserver-operator#1936
• etcd: WIP: Test guard user namespace cluster-etcd-operator#1498
• kube-scheduler: WIP: Test user namespace guard cluster-kube-scheduler-operator#582

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

[openshift-ci-robot]

@tchap: This pull request references CNTRLPLANE-1544 which is a valid jira issue.

In response to this:

Test for Downstream Components

• kube-controller-manager: WIP: Test guard user namespace cluster-kube-controller-manager-operator#887
• kube-apiserver: WIP: Test user namespace guard cluster-kube-apiserver-operator#1936
• etcd: WIP: Test guard user namespace cluster-etcd-operator#1498
• kube-scheduler: WIP: Test user namespace guard cluster-kube-scheduler-operator#582

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

[openshift-ci-robot]

@tchap: This pull request references CNTRLPLANE-1544 which is a valid jira issue.

In response to this:

Test for Downstream Components

• kube-controller-manager: WIP: Test guard user namespace cluster-kube-controller-manager-operator#887
• kube-apiserver: WIP: Test user namespace guard cluster-kube-apiserver-operator#1936
• etcd: WIP: Test guard user namespace cluster-etcd-operator#1498
• kube-scheduler: WIP: Test user namespace guard cluster-kube-scheduler-operator#582

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

[openshift-ci-robot]

@tchap: This pull request references CNTRLPLANE-1544 which is a valid jira issue.

In response to this:

Test for Downstream Components

• kube-controller-manager: WIP: Test guard user namespace cluster-kube-controller-manager-operator#887
• kube-apiserver: WIP: Test user namespace guard cluster-kube-apiserver-operator#1936
• etcd: WIP: Test guard user namespace cluster-etcd-operator#1498
• kube-scheduler: WIP: Test user namespace guard cluster-kube-scheduler-operator#582

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

[openshift-ci-robot]

@tchap: This pull request references CNTRLPLANE-1544 which is a valid jira issue.

In response to this:

Test for Downstream Components

• kube-controller-manager: WIP: Test guard user namespace cluster-kube-controller-manager-operator#887
• kube-apiserver: WIP: Test user namespace guard cluster-kube-apiserver-operator#1936
• etcd: WIP: Test guard user namespace cluster-etcd-operator#1498
• kube-scheduler: WIP: Test user namespace guard cluster-kube-scheduler-operator#582

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

[p0lyn0mial]

tests are green in the components repo

/hold cancel

[openshift-ci]

@tchap: all tests passed!

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.