config v2 proposal: allow adding user files to controller join API #6626
Description
A proposal for #4822
When a new controller joins the cluster, it uses the /ca endpoint of the controller API to get the kubernetes-ca material. It would be very convenient if k0s config allowed specifying additional files that are automatically sent to the new controller nodes via an endpoint like /user-files.
We use k0s with many additional configuration files that we have to manually send to the new nodes before joining. This includes:
- encryption configuration
- OIDC provider configuration
- audit configuration
In our case, these files have identical content. We use them to configure the Kube API server with the extraArgs flags. We manually send these files to the new controllers before joining, create kube-apiserver user and set the ownership so that the kube API server can read these files. And we repeat this for every controller that joins the cluster.