Use ClusterRoles instead of Roles by default

[TGPSKI]

Greetings,

I want to get some thoughts on an upstream PR I put together.

As part of initial technical exploration using chaostoolkit/kubernetes-crd, I wanted to quickly test the chaostoolkit-kubernetes extensions with the CRD. After building my own runner docker image, I still was unable to modify resources outside of the chaostoolkit-run namespace. I see this path as being critical for first line evaluation of the crd: users will incorporate the CRD and manifests into their clusters, and then want to be able to operate on any namespace from the experiment pod.

I updated the CRD schema to support a new spec property, clusterRoleBindNamespaces. Each namespace specified in the list will generate an additional RoleBinding, allowing the service account associated with an experiment to interact with the k8s api for the given namespace.

TGPSKI#2

[dmartin35]

I think the CRD was initially built with deny-all rather than allow-all permissions on namespaces by default
I'll have to double check with @Lawouach

The initial take was to have no rights, but to be explicitly allowed, rather than implicitly being able to allowed to do any pod manipulation in any namespace. I think some users might not accept such broad permissions in their cluster, but I may be wrong.

However, there is a simple way of bypassing that. You can create your own service account, with CRUD permissions on PODs on all namespaces (with a cluster role), then you can explicitly use that service account within the Experiment CRO

Here is the manifest, that I used to create my own SA with PODs permissions in any namespace:

---
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: ctk-can-crud-pods-in-other-namespaces
rules:
- apiGroups:
  - ""
  resources:
  - pods
  verbs:
  - create
  - get
  - delete
  - list
  - patch

---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: ctk-can-crud-pods-in-other-namespaces
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: ctk-can-crud-pods-in-other-namespaces
subjects:
- kind: ServiceAccount
  name: chaostoolkit-pod
  namespace: chaostoolkit-run

---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: chaostoolkit-pod
  namespace: chaostoolkit-run

And how I used the SA (override in the experiment CRO)

---
apiVersion: chaostoolkit.org/v1
kind: ChaosToolkitExperiment
spec:
  namespace: chaostoolkit-run
  serviceaccount:
    name: chaostoolkit-pod