[release-1.7] 🌱 Allow control plane provider to set endpoint (#10800)

* Allow control plane provider to set endpoint

Signed-off-by: Vince Prignano <[email protected]>

* Update cluster_controller_phases_test.go

Co-authored-by: Stefan Büringer <[email protected]>

---------

Signed-off-by: Vince Prignano <[email protected]>
Co-authored-by: Stefan Büringer <[email protected]>

Author: vincepri

docs/book/src/developer/architecture/controllers/cluster.md

@@ -22,7 +22,10 @@ provisions EC2 instances that will become a Kubernetes cluster through some boot
 
 The cluster controller will set an OwnerReference on the infrastructureCluster. This controller should normally take no action during reconciliation until it sees the OwnerReference.
 
-An infrastructureCluster controller is expected to eventually have its `spec.controlPlaneEndpoint` set by the user/controller.
+An infrastructureCluster controller is expected to either supply a controlPlaneEndpoint (via its own `spec.controlPlaneEndpoint` field),
+or rely on `spec.controlPlaneEndpoint` in its parent [Cluster](./cluster.md) object.
+
+If an endpoint is not provided, the implementer should exit reconciliation until it sees `cluster.spec.controlPlaneEndpoint` populated.
 
 The Cluster controller bubbles up `spec.controlPlaneEndpoint` and `status.ready` into `status.infrastructureReady` from the infrastructureCluster.
 
@@ -50,6 +53,9 @@ is a map, defined as `map[string]FailureDomainSpec`. A unique key must be used f
     - `controlPlane` (bool): indicates if failure domain is appropriate for running control plane instances.
     - `attributes` (`map[string]string`): arbitrary attributes for users to apply to a failure domain.
 
+Note: once any of `failureReason` or `failureMessage` surface on the cluster who is referencing the infrastructureCluster object,
+they cannot be restored anymore (it is considered a terminal error; the only way to recover is to delete and recreate the cluster).
+
 Example:
 ```yaml
 kind: MyProviderCluster

docs/book/src/developer/architecture/controllers/control-plane.md

@@ -41,12 +41,14 @@ Kubernetes control plane consisting of the following services:
 
 The Cluster controller will set an OwnerReference on the Control Plane. The Control Plane controller should normally take no action during reconciliation until it sees the ownerReference.
 
-A Control Plane controller implementation should exit reconciliation until it sees `cluster.spec.controlPlaneEndpoint` populated.
+A Control Plane controller implementation must either supply a controlPlaneEndpoint (via its own `spec.controlPlaneEndpoint` field),
+or rely on `spec.controlPlaneEndpoint` in its parent [Cluster](./cluster.md) object.
 
-The Cluster controller bubbles up `status.ready` into `status.controlPlaneReady`  and `status.initialized` into a `controlPlaneInitialized` condition from the Control Plane CR.
+If an endpoint is not provided, the implementer should exit reconciliation until it sees `cluster.spec.controlPlaneEndpoint` populated.
+
+A Control Plane controller can optionally provide a `controlPlaneEndpoint`
 
-The `ImplementationControlPlane` *must* rely on the existence of
-`status.controlplaneEndpoint` in its parent [Cluster](./cluster.md) object.
+The Cluster controller bubbles up `status.ready` into `status.controlPlaneReady`  and `status.initialized` into a `controlPlaneInitialized` condition from the Control Plane CR.
 
 ### CRD contracts
 
@@ -110,6 +112,35 @@ documentation][scale].
   deletion. A duration of 0 will retry deletion indefinitely. It defaults to 10 seconds on the
   Machine.
 
+#### Optional `spec` fields for implementations providing endpoints
+
+The `ImplementationControlPlane` object may provide a `spec.controlPlaneEndpoint` field to inform the Cluster
+controller where the endpoint is located.
+
+Implementers might opt to choose the `APIEndpoint` struct exposed by Cluster API types, or the following:
+
+<table>
+  <tr>
+    <th> Field </th>
+    <th> Type </th>
+    <th> Description </th>
+  </tr>
+  <tr>
+    <td><code>host</code></td>
+    <td>String</td>
+    <td>
+      The hostname on which the API server is serving.
+    </td>
+  </tr>
+  <tr>
+    <td><code>port</code></td>
+    <td>Integer</td>
+    <td>
+      The port on which the API server is serving.
+    </td>
+  </tr>
+</table>
+
 #### Required `status` fields
 
 The `ImplementationControlPlane` object **must** have a `status` object.

docs/proposals/20230407-flexible-managed-k8s-endpoints.md

@@ -76,6 +76,7 @@ More specifically we would like to introduce first class support for two scenari
 
 - Permit omitting the `<Infra>Cluster` entirely, thus making it simpler to use with Cluster API all the Managed Kubernetes implementations which do not require any additional Kubernetes Cluster Infrastructure (network settings, security groups, etc) on top of what is provided out of the box by the managed Kubernetes primitive offered by a Cloud provider.
 - Allow the `ControlPlane Provider` component to take ownership of the responsibility of creating the control plane endpoint, thus making it simpler to use with Cluster API all the Managed Kubernetes implementations which are taking care out of the box of this piece of Cluster Infrastructure.
+  - Note: In May 2024 [this pull request](https://github.com/kubernetes-sigs/cluster-api/pull/10667) added the ability for the control plane provider to provide the endpoint the same way the infrastructure cluster would.
 
 The above capabilities can be used alone or in combination depending on the requirements of a specific Managed Kubernetes or on the specific architecture/set of Cloud components being implemented.
 

internal/controllers/cluster/cluster_controller_phases.go

@@ -199,7 +199,7 @@ func (r *Reconciler) reconcileInfrastructure(ctx context.Context, cluster *clust
 
 	// Get and parse Spec.ControlPlaneEndpoint field from the infrastructure provider.
 	if !cluster.Spec.ControlPlaneEndpoint.IsValid() {
-		if err := util.UnstructuredUnmarshalField(infraConfig, &cluster.Spec.ControlPlaneEndpoint, "spec", "controlPlaneEndpoint"); err != nil {
+		if err := util.UnstructuredUnmarshalField(infraConfig, &cluster.Spec.ControlPlaneEndpoint, "spec", "controlPlaneEndpoint"); err != nil && err != util.ErrUnstructuredFieldNotFound {
 			return ctrl.Result{}, errors.Wrapf(err, "failed to retrieve Spec.ControlPlaneEndpoint from infrastructure provider for Cluster %q in namespace %q",
 				cluster.Name, cluster.Namespace)
 		}
@@ -218,6 +218,8 @@ func (r *Reconciler) reconcileInfrastructure(ctx context.Context, cluster *clust
 
 // reconcileControlPlane reconciles the Spec.ControlPlaneRef object on a Cluster.
 func (r *Reconciler) reconcileControlPlane(ctx context.Context, cluster *clusterv1.Cluster) (ctrl.Result, error) {
+	log := ctrl.LoggerFrom(ctx)
+
 	if cluster.Spec.ControlPlaneRef == nil {
 		return ctrl.Result{}, nil
 	}
@@ -274,6 +276,19 @@ func (r *Reconciler) reconcileControlPlane(ctx context.Context, cluster *cluster
 		}
 	}
 
+	if !ready {
+		log.V(3).Info("Control Plane provider is not ready yet")
+		return ctrl.Result{}, nil
+	}
+
+	// Get and parse Spec.ControlPlaneEndpoint field from the control plane provider.
+	if !cluster.Spec.ControlPlaneEndpoint.IsValid() {
+		if err := util.UnstructuredUnmarshalField(controlPlaneConfig, &cluster.Spec.ControlPlaneEndpoint, "spec", "controlPlaneEndpoint"); err != nil && err != util.ErrUnstructuredFieldNotFound {
+			return ctrl.Result{}, errors.Wrapf(err, "failed to retrieve Spec.ControlPlaneEndpoint from control plane provider for Cluster %q in namespace %q",
+				cluster.Name, cluster.Namespace)
+		}
+	}
+
 	return ctrl.Result{}, nil
 }