Chained and efficient upgrades for Clusters with managed topologies

Author: fabriziopandini

docs/book/src/images/runtime-sdk-lifecycle-hooks.png

@@ -14,7 +14,8 @@ The lifecycle hooks allow hooking into the Cluster lifecycle. The following diag
 
 ![Lifecycle Hooks overview](../../../images/runtime-sdk-lifecycle-hooks.png)
 
-Please see the corresponding [CAEP](https://github.com/kubernetes-sigs/cluster-api/blob/main/docs/proposals/20220414-runtime-hooks.md) 
+Please see the corresponding [proposal: Runtime hooks for Add-on Management (lifecycle hooks)](https://github.com/kubernetes-sigs/cluster-api/blob/main/docs/proposals/20220414-lifecycle-hooks.md) and
+also [proposal: Chained and efficient upgrades for Clusters with managed topologies](https://github.com/kubernetes-sigs/cluster-api/blob/main/docs/proposals/20250513-chained-and-efficient-upgrades-for-clusters-with-managed-topologies.md)]
 for additional background information. 
 
 ## Guidelines
@@ -139,8 +140,17 @@ cluster:
    ...
   status:
    ...
-fromKubernetesVersion: "v1.21.2"
-toKubernetesVersion: "v1.22.0"
+fromKubernetesVersion: "v1.30.0"
+toKubernetesVersion: "v1.33.0"
+upgradePlan:
+  controlPlane:
+  - v1.30.0
+  - v1.31.0
+  - v1.32.3
+  - v1.33.0
+  workers:
+  - v1.32.3
+  - v1.33.0
 ```
 
 #### Example Response:
@@ -159,12 +169,68 @@ For additional details, you can see the full schema in <button onclick="openSwag
   if previous upgrades or worker machine rollouts are still in progress, the system waits for those operations 
   to complete before starting the new upgrade.
 
+###  BeforeControlPlaneUpgrade
+
+This hook is called before a new version is propagated to the control plane object. Runtime Extension implementers
+can use this hook to execute pre-upgrade add-on tasks and block upgrades of the ControlPlane.
+
+Note: 
+- When an upgrade is starting, BeforeControlPlaneUpgrade will be called after BeforeClusterUpgrade is completed.
+- When an upgrade is in progress BeforeControlPlaneUpgrade will be called for each intermediate version that will
+  be applied to the control plane (instead BeforeClusterUpgrade will be called only once at the beginning of the upgrade).
+
+#### Example Request:
+
+```yaml
+apiVersion: hooks.runtime.cluster.x-k8s.io/v1alpha1
+kind: BeforeControlPlaneUpgradeRequest
+settings: <Runtime Extension settings>
+cluster:
+  apiVersion: cluster.x-k8s.io/v1beta1
+  kind: Cluster
+  metadata:
+   name: test-cluster
+   namespace: test-ns
+  spec:
+   ...
+  status:
+   ...
+fromKubernetesVersion: "v1.30.0"
+toKubernetesVersion: "v1.33.0"
+upgradePlan:
+  controlPlane:
+  - v1.30.0
+  - v1.31.0
+  - v1.32.3
+  - v1.33.0
+  workers:
+  - v1.32.3
+  - v1.33.0
+```
+
+Note: The upgrade plan in the request contains only missing steps to reach the target version.
+
+#### Example Response:
+
+```yaml
+apiVersion: hooks.runtime.cluster.x-k8s.io/v1alpha1
+kind: BeforeControlPlaneUpgradeResponse
+status: Success # or Failure
+message: "error message if status == Failure"
+retryAfterSeconds: 10
+```
+
+For additional details, you can see the full schema in <button onclick="openSwaggerUI()">Swagger UI</button>.
+
 ###  AfterControlPlaneUpgrade
 
-This hook is called after the entire control plane has been upgraded to the version specified in `spec.topology.version`,
-and immediately before the new version is going to be propagated to the MachineDeployments of the Cluster. 
-Runtime Extension implementers can use this hook to execute post-upgrade add-on tasks and block upgrades to workers
-until everything is ready.
+This hook is called after the entire control plane has been upgraded to the version specified in `spec.topology.version` 
+or to an intermediate version in the upgrade plan and:
+- if workers upgrade can be skipped for this version, immediately before the next intermediate version is applied to the control plane
+- if workers upgrade must be performed for this version, immediately before the new version is going to be propagated to the MachineDeployments of the Cluster.
+
+Runtime Extension implementers can use this hook to execute post-upgrade add-on tasks and block upgrades to the next
+version of the control plane or to workers until everything is ready.
 
 Note: While the MachineDeployments upgrade is blocked changes made to existing MachineDeployments and creating new MachineDeployments
 will be delayed while the object is waiting for upgrade. Example: modifying MachineDeployments (think scale up),
@@ -188,9 +254,122 @@ cluster:
    ...
   status:
    ...
-kubernetesVersion: "v1.22.0"
+kubernetesVersion: "v1.30.0"
+upgradePlan:
+  controlPlane:
+    - v1.31.0
+    - v1.32.3
+    - v1.33.0
+  workers:
+    - v1.32.3
+    - v1.33.0
+```
+
+Note: The upgrade plan in the request contains only missing steps to reach the target version, if any.
+
+#### Example Response:
+
+```yaml
+apiVersion: hooks.runtime.cluster.x-k8s.io/v1alpha1
+kind: AfterControlPlaneUpgradeResponse
+status: Success # or Failure
+message: "error message if status == Failure"
+retryAfterSeconds: 10
+```
+
+For additional details, you can see the full schema in <button onclick="openSwaggerUI()">Swagger UI</button>.
+
+###  BeforeWorkersUpgrade
+
+This hook is called before a new version is propagated to workers. Runtime Extension implementers
+can use this hook to execute pre-upgrade add-on tasks and block upgrades of Workers.
+
+Note:
+- This hook will be called only if workers upgrade must be performed for an intermediate version of of a chained upgrade
+  or when upgrading to the target `spec.topology.version`.
+
+#### Example Request:
+
+```yaml
+apiVersion: hooks.runtime.cluster.x-k8s.io/v1alpha1
+kind: BeforeWorkersUpgradeRequest
+settings: <Runtime Extension settings>
+cluster:
+  apiVersion: cluster.x-k8s.io/v1beta1
+  kind: Cluster
+  metadata:
+   name: test-cluster
+   namespace: test-ns
+  spec:
+   ...
+  status:
+   ...
+fromKubernetesVersion: "v1.30.0"
+toKubernetesVersion: "v1.33.0"
+upgradePlan:
+  controlPlane:
+  - v1.30.0
+  - v1.31.0
+  - v1.32.3
+  - v1.33.0
+  workers:
+  - v1.32.3
+  - v1.33.0
 ```
 
+Note: The upgrade plan in the request contains only missing steps to reach the target version.
+
+#### Example Response:
+
+```yaml
+apiVersion: hooks.runtime.cluster.x-k8s.io/v1alpha1
+kind: BeforeControlPlaneUpgradeResponse
+status: Success # or Failure
+message: "error message if status == Failure"
+retryAfterSeconds: 10
+```
+
+For additional details, you can see the full schema in <button onclick="openSwaggerUI()">Swagger UI</button>.
+
+###  AfterWorkersUpgrade
+
+This hook is called after all the workers have been upgraded to the version specified in `spec.topology.version`
+or to an intermediate version in the upgrade plan, and:
+- if the upgrade plan is completed and the entire cluster is at `spec.topology.version`, immediately before calling the AfterClusterUpgrade hook
+- if the upgrade plan is not complete and the entrire cluster is now at one of the intermediate versions, immediately before calling BeforeControlPlaneUpgrade hook for the next intermediate step
+
+Runtime Extension implementers can use this hook to execute post-upgrade add-on tasks; if the upgrade plan is not completed,
+this hook allows to block upgrades to the next version of the control plane until everything is ready.
+
+#### Example Request:
+
+```yaml
+apiVersion: hooks.runtime.cluster.x-k8s.io/v1alpha1
+kind: AfterWorkersRequest
+settings: <Runtime Extension settings>
+cluster:
+  apiVersion: cluster.x-k8s.io/v1beta1
+  kind: Cluster
+  metadata:
+   name: test-cluster
+   namespace: test-ns
+  spec:
+   ...
+  status:
+   ...
+kubernetesVersion: "v1.30.0"
+upgradePlan:
+  controlPlane:
+    - v1.31.0
+    - v1.32.3
+    - v1.33.0
+  workers:
+    - v1.32.3
+    - v1.33.0
+```
+
+Note: The upgrade plan in the request contains only missing steps to reach the target version, if any.
+
 #### Example Response:
 
 ```yaml
@@ -201,6 +380,8 @@ message: "error message if status == Failure"
 retryAfterSeconds: 10
 ```
 
+Note: retryAfterSeconds is ignored when workers version is equal to `spec.topology.version`.
+
 For additional details, you can see the full schema in <button onclick="openSwaggerUI()">Swagger UI</button>.
 
 ###  AfterClusterUpgrade
@@ -237,8 +418,6 @@ status: Success # or Failure
 message: "error message if status == Failure"
 ```
 
-For additional details, refer to the [Draft OpenAPI spec](https://editor.swagger.io/?url=https://raw.githubusercontent.com/kubernetes-sigs/cluster-api/main/docs/proposals/images/runtime-hooks/runtime-hooks-openapi.yaml).
-
 ###  BeforeClusterDelete
 
 This hook is called after the Cluster deletion has been triggered by the user and immediately before the topology

docs/book/src/tasks/experimental-features/runtime-sdk/implement-lifecycle-hooks.md

@@ -26,7 +26,7 @@ Additional documentation:
 * Background information:
     * [Runtime SDK CAEP](https://github.com/kubernetes-sigs/cluster-api/blob/main/docs/proposals/20220221-runtime-SDK.md)
     * [Topology Mutation Hook CAEP](https://github.com/kubernetes-sigs/cluster-api/blob/main/docs/proposals/20220330-topology-mutation-hook.md)
-    * [Runtime Hooks for Add-on Management CAEP](https://github.com/kubernetes-sigs/cluster-api/blob/main/docs/proposals/20220414-runtime-hooks.md)
+    * [Runtime Hooks for Add-on Management CAEP](https://github.com/kubernetes-sigs/cluster-api/blob/main/docs/proposals/20220414-lifecycle-hooks.md)
 * For Runtime Extension developers:
     * [Implementing Runtime Extensions](./implement-extensions.md)
     * [Implementing Lifecycle Hook Extensions](./implement-lifecycle-hooks.md)

docs/book/src/tasks/experimental-features/runtime-sdk/index.md

@@ -209,6 +209,8 @@ at high level the new CRD contains:
 - A list of patches, allowing to change above templates for each specific Cluster.
 - A list of variable definitions, defining a set of additional values the users can provide on each specific cluster;
   those values can be used in patches. 
+- A list of Kubernetes versions to be used when performing chained upgrades for clusters using this Cluster class, see
+  [proposal: Chained and efficient upgrades for Clusters with managed topologies](20250513-chained-and-efficient-upgrades-for-clusters-with-managed-topologies.md).
 
 The following paragraph provides some additional context on some of the above values; more info can
 be found in [writing a ClusterClass](https://cluster-api.sigs.k8s.io/tasks/experimental-features/cluster-class/write-clusterclass.html).
@@ -378,8 +380,19 @@ as well as in
           apiVersion: infrastructure.cluster.x-k8s.io/v1beta1
           kind: VSphereClusterTemplate
           name: vsphere-prod-cluster-template
+      upgrade:
+        versions:
+        - v1.28.0
+        - v1.29.0
+        - v1.30.0
+        - v1.30.1
+        - v1.31.2
+        - ... 
    ```
 
+   see [proposal: Chained and efficient upgrades for Clusters with managed topologies](20250513-chained-and-efficient-upgrades-for-clusters-with-managed-topologies.md) for more options
+   for configuring Kubernetes version upgrade of clusters using managed topologies.
+
 2. User creates a cluster using the class name and defining the topology.
    ```yaml
     apiVersion: cluster.x-k8s.io/v1beta1
@@ -433,6 +446,9 @@ This section talks about updating a Cluster which was created using a `ClusterCl
 
 ![Update cluster with ClusterClass](./images/cluster-class/update.png)
 
+see [proposal: Chained and efficient upgrades for Clusters with managed topologies](20250513-chained-and-efficient-upgrades-for-clusters-with-managed-topologies.md) 
+for more considerations about Kubernetes version upgrade of clusters using managed topologies. 
+
 #### Behavior with patches
 
 This section highlights how the basic behavior discussed above changes when patches are used. This is an important use case because without 
@@ -485,6 +501,9 @@ like e.g. a different HTTP proxy configuration, a different image to be used for
            valueFrom:
              variable: machineType
    ```
+   
+See [proposal: topology mutation hook](20220330-topology-mutation-hook.md) for a powerful alternative to
+inline patches.
 
 ##### Create a new Cluster with patches
 
@@ -613,6 +632,7 @@ The initial plan is to rollout Cluster Class and support for managed topologies
 - 10/04/2021: Added support for patches and variables
 - 01/10/2022: Added support for MachineHealthChecks
 - 12/20/2022: Cleaned up outdated implementation details by linking the book's pages instead. This will make it easier to keep the proposal up to date.
+- 05/13/2025: Added support for Upgrade Plans; see [proposal: Chained and efficient upgrades for Clusters with managed topologies](20250513-chained-and-efficient-upgrades-for-clusters-with-managed-topologies.md)
 
 <!-- Links -->
 [community meeting]: https://docs.google.com/document/d/1Ys-DOR5UsgbMEeciuG0HOgDQc8kZsaWIWJeKJ1-UfbY

docs/proposals/20210526-cluster-class-and-managed-topologies.md

@@ -115,15 +115,18 @@ As a developer of an add-ons orchestration solution:
 * **Before a Cluster is Created** I want to automatically check if enough disk space is available for allocation to the cluster for persistent storage of collected metrics values.
 * **After the Control Plane** **is Initialized** I want to automatically install a metrics database and associated add-ons in the workload cluster.
 * **Before the Cluster is Upgraded** I want to install a new version of the metrics database with a new version of the custom metrics apiservice to interact directly with the Kubernetes apiserver.
-* **After the ControlPlane is Upgraded** I want to automatically check that the new version of the custom metrics apiservice is working and correctly fulfilled by my metrics database.
+* **Before the ControlPlane is Upgraded** I want to install a new version of the metrics database with a new version of the custom metrics apiservice to interact directly with the Kubernetes apiserver.
+* **After the ControlPlane is Upgraded** I want to install new versions of metrics collectors to each upgraded node in the cluster.
+* **Before workers are Upgraded** I want to install a new version of the metrics database with a new version of the custom metrics apiservice to interact directly with the Kubernetes apiserver.
+* **After workers are Upgraded** I want to install new versions of metrics collectors to each upgraded node in the cluster
 * **After the Cluster is Upgraded** I want to install new versions of metrics collectors to each upgraded node in the cluster.
 * **Before the Cluster is Deleted** I want to automatically back up persistent volumes used by the metrics database.
 
 ### Runtime hook definitions
 
 Below is a description for the Runtime Hooks introduced by this proposal.
 
-![runtime-hooks](images/runtime-hooks/runtime-hooks.png)
+![runtime-hooks](images/lifecycle-hooks/lifecycle-hooks.png)
 
 The remainder of this section has been moved to the Cluster API [book](../../docs/book/src/tasks/experimental-features/runtime-sdk/implement-lifecycle-hooks.md#definitions)
 to avoid duplication.
@@ -210,6 +213,7 @@ See [upgrade strategy](#upgrade-strategy).
 * [x] 2022-04-04: Opened corresponding [issue](https://github.com/kubernetes-sigs/cluster-api/issues/6374)
 * [x] 2022-04-06: Presented proposal at a [community meeting]
 * [x] 2022-04-14: Opened proposal PR
+* [x] 2025-05-13: Added runtime hooks for chained upgrades; see [proposal: Chained and efficient upgrades for Clusters with managed topologies](20250513-chained-and-efficient-upgrades-for-clusters-with-managed-topologies.md)
 
 <!-- Links -->
 [community meeting]: https://docs.google.com/document/d/1ushaVqAKYnZ2VN_aa3GyKlS4kEd6bSug13xaXOakAQI/edit#heading=h.pxsq37pzkbdq