docs/tilt: Collection of fixes and improvements for provider implementer's guide and Tilt workflow

Author: AndiDog

Tiltfile

@@ -192,20 +192,27 @@ tilt_helper_dockerfile_header = """
 FROM golang:1.18.3 as tilt-helper
 # Support live reloading with Tilt
 RUN go install github.com/go-delve/delve/cmd/dlv@latest
-RUN wget --output-document /restart.sh --quiet https://raw.githubusercontent.com/windmilleng/rerun-process-wrapper/master/restart.sh  && \
-    wget --output-document /start.sh --quiet https://raw.githubusercontent.com/windmilleng/rerun-process-wrapper/master/start.sh && \
-    chmod +x /start.sh && chmod +x /restart.sh && chmod +x /go/bin/dlv
+RUN wget --output-document /restart.sh --quiet https://raw.githubusercontent.com/tilt-dev/rerun-process-wrapper/master/restart.sh  && \
+    wget --output-document /start.sh --quiet https://raw.githubusercontent.com/tilt-dev/rerun-process-wrapper/master/start.sh && \
+    chmod +x /start.sh && chmod +x /restart.sh && chmod +x /go/bin/dlv && \
+    touch /process.txt && chmod 0777 /process.txt `# pre-create PID file to allow even non-root users to run the image`
 """
 
 tilt_dockerfile_header = """
 FROM gcr.io/distroless/base:debug as tilt
 WORKDIR /
+COPY --from=tilt-helper /process.txt .
 COPY --from=tilt-helper /start.sh .
 COPY --from=tilt-helper /restart.sh .
 COPY --from=tilt-helper /go/bin/dlv .
 COPY $binary_name .
 """
 
+tilt_dockerfile_footer = """
+# Use same USER instruction as in kubebuilder-generated Dockerfile
+USER 65532:65532
+"""
+
 def build_go_binary(context, reload_deps, debug, go_main, binary_name, label):
     # Set up a local_resource build of a go binary. The target repo is expected to have a main.go in
     # the context path or the main.go must be provided via go_main option. The binary is written to .tiltbuild/bin/{$binary_name}.
@@ -268,6 +275,7 @@ def build_docker_image(image, context, binary_name, additional_docker_build_comm
         additional_docker_helper_commands,
         tilt_dockerfile_header,
         additional_docker_build_commands,
+        tilt_dockerfile_footer,
     ])
 
     # Set up an image build for the provider. The live update configuration syncs the output from the local_resource

docs/book/src/developer/providers/contracts.md

@@ -20,17 +20,17 @@ Each value MUST point to an available version in your CRD Spec.
 The label allows Cluster API controllers to perform automatic conversions for object references, the controllers will pick the last available version in the list if multiple versions are found.
 To apply the label to CRDs it’s possible to use commonLabels in your kustomize.yaml file, usually in config/crd.
 
-In this example we show how to map a particular Cluster API contract version to your own CRD using Kustomize’s commonLabels feature:
+In this example we show how to map a particular Cluster API contract version to your own CRD using Kustomize’s `commonLabels` feature, in your `config/crd/kustomization.yaml`:
 
 ```yaml
 commonLabels:
-cluster.x-k8s.io/v1alpha2: v1alpha1
-cluster.x-k8s.io/v1alpha3: v1alpha2
-cluster.x-k8s.io/v1beta1:  v1beta1
+  cluster.x-k8s.io/v1alpha2: v1alpha1
+  cluster.x-k8s.io/v1alpha3: v1alpha2
+  cluster.x-k8s.io/v1beta1: v1beta1
 ```
 
 An example of this is in the [Kubeadm Bootstrap provider](https://github.com/kubernetes-sigs/cluster-api/blob/release-1.1/controlplane/kubeadm/config/crd/kustomization.yaml).
 
 ## Improving and contributing to the contract
 
-The definition of the contract between Cluster API and providers may be changed in future versions of Cluster API. The Cluster API maintainers welcome feedback and contributions to the contract in order to improve how it's defined, its clarity and visibility to provider implementers and its suitability across the different kinds of Cluster API providers. To provide feedback or open a discussion about the provider contract please [open an issue on the Cluster API](https://github.com/kubernetes-sigs/cluster-api/issues/new?assignees=&labels=&template=feature_request.md) repo or add an item to the agenda in the [Cluster API community meeting](http://git.k8s.io/community/sig-cluster-lifecycle/README.md#cluster-api).
+The definition of the contract between Cluster API and providers may be changed in future versions of Cluster API. The Cluster API maintainers welcome feedback and contributions to the contract in order to improve how it's defined, its clarity and visibility to provider implementers and its suitability across the different kinds of Cluster API providers. To provide feedback or open a discussion about the provider contract please [open an issue on the Cluster API](https://github.com/kubernetes-sigs/cluster-api/issues/new?assignees=&labels=&template=feature_request.md) repo or add an item to the agenda in the [Cluster API community meeting](http://git.k8s.io/community/sig-cluster-lifecycle/README.md#cluster-api).

docs/book/src/developer/providers/implementers-guide/building_running_and_testing.md

@@ -28,16 +28,17 @@ kubectl apply -f https://github.com/cert-manager/cert-manager/releases/download/
 
 ### Cluster API
 
-Before you can deploy the infrastructure controller, you'll need to deploy Cluster API itself.
+Before you can deploy the infrastructure controller, you'll need to deploy Cluster API itself to the management cluster.
 
-You can use a precompiled manifest from the release page, or clone [`cluster-api`][capi] and apply its manifests using `kustomize`.
+You can use a precompiled manifest from the [release page][releases], run `clusterctl init`, or clone [`cluster-api`][capi] and apply its manifests using `kustomize`:
 
 ```bash
 cd cluster-api
-kustomize build config/ | kubectl apply -f-
+make envsubst
+kustomize build config/default | ./hack/tools/bin/envsubst | kubectl apply -f -
 ```
 
-Check the status of the manager to make sure it's running properly
+Check the status of the manager to make sure it's running properly:
 
 ```bash
 kubectl describe -n capi-system pod | grep -A 5 Conditions
@@ -52,17 +53,29 @@ Conditions:
 ```
 
 [capi]: https://github.com/kubernetes-sigs/cluster-api
+[releases]: https://github.com/kubernetes-sigs/cluster-api/releases
 
 ### Your provider
 
+In this guide, we are building an _infrastructure provider_. We must tell cluster-api and its developer tooling which type of provider it is. Edit `config/default/kustomization.yaml` and add the following common label. The prefix `infrastructure-` [is used][label_prefix] to detect the provider type.
+
+```sh
+commonLabels:
+  cluster.x-k8s.io/provider: infrastructure-mailgun
+```
+
 Now you can apply your provider as well:
 
 ```bash
 cd cluster-api-provider-mailgun
-kustomize build config/ | envsubst | kubectl apply -f-
+
+# Install CRD and controller to current kubectl context
+make install deploy
+
 kubectl describe -n cluster-api-provider-mailgun-system pod | grep -A 5 Conditions
 ```
-```bash
+
+```text
 Conditions:
   Type              Status
   Initialized       True
@@ -71,32 +84,61 @@ Conditions:
   PodScheduled      True
 ```
 
+[label_prefix]: https://github.com/kubernetes-sigs/cluster-api/search?q=%22infrastructure-%22
+
 ### Tiltfile
 Cluster API development requires a lot of iteration, and the "build, tag, push, update deployment" workflow can be very tedious.
 [Tilt](https://tilt.dev) makes this process much simpler by watching for updates, then automatically building and deploying them.
 
-You can visit [some example repositories][capidev], but you can get started by writing out a yaml manifest and using the following [`Tiltfile`][tiltfile]
-`kustomize build config/ | envsubst > capm.yaml`
+See [Developing Cluster API with Tilt](../../tilt.md) on all details how to develop both Cluster API and your provider at the same time. In short, you need to perform these steps for a basic Tilt-based development environment:
 
-[capidev]: https://github.com/chuckha/capi-dev
-[tiltfile]: https://docs.tilt.dev/tiltfile_concepts.html
+- Create file `tilt-provider.yaml` in your provider directory:
 
-```starlark
-allow_k8s_contexts('kubernetes-admin@kind)
+```yaml
+name: mailgun
+config:
+  image: controller:latest # change to remote image name if desired
+  label: CAPM
+  live_reload_deps: ["main.go", "go.mod", "go.sum", "api", "controllers", "pkg"]
+```
 
-k8s_yaml('capm.yaml')
+- Create file `tilt-settings.yaml` in the cluster-api directory:
 
-docker_build('<your docker username or repository url>/cluster-api-controller-mailgun-amd64', '.')
+```yaml
+default_registry: "" # change if you use a remote image registry
+provider_repos:
+  # This refers to your provider directory and loads settings
+  # from `tilt-provider.yaml`
+  - ../cluster-api-provider-mailgun
+enable_providers:
+  - mailgun
 ```
 
-You can then use Tilt to watch the logs coming off your container.
+- Create a kind cluster. By default, Tiltfile assumes the kind cluster is named `capi-test`.
+
+```bash
+kind create cluster --name capi-test
+
+# If you want a more sophisticated setup of kind cluster + image registry, try:
+# ---
+# cd cluster-api
+# hack/kind-install-for-capd.sh
+```
+
+- Run `tilt up` in the cluster-api folder
+
+You can then use Tilt to watch the container logs.
+
+On any changed file in the listed places (`live_reload_deps` and those watched inside cluster-api repo), Tilt will build and deploy again. In the regular case of a changed file, only your controller's binary gets rebuilt, copied into the running container, and the process restarted. This is much faster than a full re-build and re-deployment of a Docker image and restart of the Kubernetes pod.
+
+You best watch the Kubernetes pods with something like `k9s -A` or `watch kubectl get pod -A`. Particularly in case your provider implementation crashes, Tilt has no chance to deploy any code changes into the container since it might be crash-looping indefinitely. In such a case – which you will notice in the log output – terminate Tilt (hit Ctrl+C) and start it again to deploy the Docker image from scratch.
 
 ## Your first Cluster
 
 Let's try our cluster out. We'll make some simple YAML:
 
 ```yaml
-apiVersion: cluster.x-k8s.io/v1alpha2
+apiVersion: cluster.x-k8s.io/v1beta1
 kind: Cluster
 metadata:
   name: hello-mailgun
@@ -105,11 +147,11 @@ spec:
     pods:
       cidrBlocks: ["192.168.0.0/16"]
   infrastructureRef:
-    apiVersion: infrastructure.cluster.x-k8s.io/v1alpha3
+    apiVersion: infrastructure.cluster.x-k8s.io/v1alpha1
     kind: MailgunCluster
     name: hello-mailgun
 ---
-apiVersion: infrastructure.cluster.x-k8s.io/v1alpha3
+apiVersion: infrastructure.cluster.x-k8s.io/v1alpha1
 kind: MailgunCluster
 metadata:
   name: hello-mailgun

docs/book/src/developer/providers/implementers-guide/configure.md

@@ -88,63 +88,35 @@ resources:
 You can now (hopefully) generate your yaml!
 
 ```bash
-kustomize build config/
-```
-
-## RBAC Role
-
-The default [RBAC role][role] contains permissions for accessing your cluster infrastructure CRDs, but not for accessing Cluster API resources.
-You'll need to add these to `config/rbac/role.yaml`
-
-[role]: https://kubernetes.io/docs/reference/access-authn-authz/rbac/
-
-```diff
-diff --git a/config/rbac/role.yaml b/config/rbac/role.yaml
-index e9352ce..29008db 100644
---- a/config/rbac/role.yaml
-+++ b/config/rbac/role.yaml
-@@ -6,6 +6,24 @@ metadata:
-   creationTimestamp: null
-   name: manager-role
- rules:
-+- apiGroups:
-+  - cluster.x-k8s.io
-+  resources:
-+  - clusters
-+  - clusters/status
-+  verbs:
-+  - get
-+  - list
-+  - watch
-+- apiGroups:
-+  - cluster.x-k8s.io
-+  resources:
-+  - machines
-+  - machines/status
-+  verbs:
-+  - get
-+  - list
-+  - watch
- - apiGroups:
-   - infrastructure.cluster.x-k8s.io
-   resources:
+kustomize build config/default
 ```
 
 ## EnvSubst
 
 _A tool like [direnv](https://direnv.net/) can be used to help manage environment variables._
 
-
 `kustomize` does not handle replacing those `${VARIABLES}` with actual values.
 For that, we use [`envsubst`][envsubst].
 
 You'll need to have those environment variables (`MAILGUN_API_KEY`, `MAILGUN_DOMAIN`, `MAILGUN_RECIPIENT`) in your environment when you generate the final yaml file.
 
-Then we call envsubst in line, like so:
+Change `Makefile` to include the call to `envsubst`:
+
+```diff
+-	$(KUSTOMIZE) build config/default | kubectl apply -f -
++	$(KUSTOMIZE) build config/default | envsubst | kubectl apply -f -
+```
+
+To generate the manifests, call envsubst in line, like so:
 
 ```bash
-kustomize build config/ | envsubst
+kustomize build config/default | envsubst
 ```
 
-[envsubst]: https://linux.die.net/man/1/envsubst
+Or to build and deploy the CRDs and manifests directly:
+
+```bash
+make install deploy
+```
 
+[envsubst]: https://github.com/drone/envsubst

docs/book/src/developer/providers/implementers-guide/controllers_and_reconciliation.md

@@ -39,25 +39,34 @@ func (r *MailgunClusterReconciler) Reconcile(ctx context.Context, req ctrl.Reque
 
 ## RBAC Roles
 
-Those `// +kubebuilder...` lines tell kubebuilder to generate [RBAC] roles so the manager we're writing can access its own managed resources.
-We also need to add roles that will let it retrieve (but not modify) Cluster API objects.
-So we'll add another annotation for that:
+The `// +kubebuilder...` lines tell kubebuilder to generate [RBAC] roles so the manager we're writing can access its own managed resources. These should already exist in `controllers/mailguncluster_controller.go`:
 
-```
+```go
 // +kubebuilder:rbac:groups=infrastructure.cluster.x-k8s.io,resources=mailgunclusters,verbs=get;list;watch;create;update;patch;delete
 // +kubebuilder:rbac:groups=infrastructure.cluster.x-k8s.io,resources=mailgunclusters/status,verbs=get;update;patch
+```
+
+We also need to add rules that will let it retrieve (but not modify) Cluster API objects.
+So we'll add another annotation for that, right below the other lines:
+
+```go
 // +kubebuilder:rbac:groups=cluster.x-k8s.io,resources=clusters;clusters/status,verbs=get;list;watch
 ```
 
-Make sure to add the annotation to both `MailgunClusterReconciler` and `MailgunMachineReconciler`.
+Make sure to add this annotation to `MailgunClusterReconciler`.
 
-Regenerate the RBAC roles after you are done:
+For `MailgunMachineReconciler`, access to Cluster API `Machine` object is needed, so you must add this annotation in `controllers/mailgunmachine_controller.go`:
 
+```go
+// +kubebuilder:rbac:groups=cluster.x-k8s.io,resources=machines;machines/status,verbs=get;list;watch
 ```
+
+Regenerate the RBAC roles after you are done:
+
+```bash
 make manifests
 ```
 
-
 [RBAC]: https://kubernetes.io/docs/reference/access-authn-authz/rbac/#role-and-clusterrole
 
 ## State
@@ -90,7 +99,7 @@ func (r *MailgunClusterReconciler) Reconcile(ctx context.Context, req ctrl.Reque
 	ctx := context.Background()
 	_ = r.Log.WithValues("mailguncluster", req.NamespacedName)
 
-	var cluster infrastructurev1alpha3.MailgunCluster
+	var cluster infrav1.MailgunCluster
 	if err := r.Get(ctx, req.NamespacedName, &cluster); err != nil {
 		return ctrl.Result{}, err
 	}
@@ -103,7 +112,7 @@ By returning an error, we request that our controller will get `Reconcile()` cal
 That may not always be what we want - what if the object's been deleted? So let's check that:
 
 ```
-var cluster infrastructurev1alpha3.MailgunCluster
+var cluster infrav1.MailgunCluster
 if err := r.Get(ctx, req.NamespacedName, &cluster); err != nil {
     // 	import apierrors "k8s.io/apimachinery/pkg/api/errors"
     if apierrors.IsNotFound(err) {
@@ -198,7 +207,7 @@ if err := helper.Patch(ctx, &mgCluster); err != nil {
 return ctrl.Result{}, nil
 ```
 
-[cluster]: https://godoc.org/sigs.k8s.io/cluster-api/api/v1alpha3#Cluster
+[cluster]: https://godoc.org/sigs.k8s.io/cluster-api/api/v1beta1#Cluster
 [getowner]: https://godoc.org/sigs.k8s.io/cluster-api/util#GetOwnerMachine
 [idempotent]: https://stackoverflow.com/questions/1077412/what-is-an-idempotent-operation
 

docs/book/src/developer/providers/implementers-guide/generate_crds.md

@@ -37,8 +37,8 @@ Here you will be asked if you want to generate resources and controllers.
 You'll want both of them:
 
 ```bash
-kubebuilder create api --group infrastructure --version v1alpha3 --kind MailgunCluster
-kubebuilder create api --group infrastructure --version v1alpha3 --kind MailgunMachine
+kubebuilder create api --group infrastructure --version v1alpha1 --kind MailgunCluster
+kubebuilder create api --group infrastructure --version v1alpha1 --kind MailgunMachine
 ```
 
 ```bash
@@ -48,6 +48,8 @@ Create Controller under pkg/controller [y/n]?
 y
 ```
 
+The latest API version of Cluster API and the version of your provider do not need to be in sync. Instead, prefer choosing a version that matches the stability of the provider API and its backward compatibility guarantees.
+
 ### Add Status subresource
 
 The [status subresource][status] lets Spec and Status requests for custom resources be addressed separately so requests don't conflict with each other.
@@ -85,7 +87,7 @@ make manifests
 
 The cluster API CRDs should be further customized:
 
-- [Apply the contract version label to support conversions](../contracts.md#api-version-labels)
+- [Apply the contract version label to support conversions](../contracts.md#api-version-labels) (required to deploy _any_ custom resource of your provider)
 - [Ensure you are compliant with the clusterctl provider contract](../../../clusterctl/provider-contract.md#components-yaml)
 
 ### Commit your changes

docs/book/src/developer/providers/implementers-guide/naming.md

@@ -44,7 +44,7 @@ identifies it.
 
 For example, our cluster object will be:
 ```yaml
-apiVersion: infrastructure.cluster.x-k8s.io/v1alpha3
+apiVersion: infrastructure.cluster.x-k8s.io/v1alpha1
 kind: MailgunCluster
 ```