Tracking issue for v1Beta2

[fabriziopandini]

Introduce the new API type

P0: mandatory for first alpha
P1: nice 2 have for first alpha

• v1beta2 API (not strictly related to this issue)
  • Prepare main branch for v1.11: 🌱 Prepare main for v1.11 development #12000
  • Create v1beta2 types for core, bootstrap, controlplane, make controllers/webhooks to use it
    • Drop hardcoded v1beta1 references 🌱 Drop hardcoded v1beta1 references #12027
    • Create API types ⚠️ Add v1beta2 types #12037
    • Promote v1Beta2Conditions
      • Introduce deprecated struct, move v1beta2 top level ⚠️ Promote v1beta2 conditions #12066
      • Move conditions fields as the first field in the struct
        • rif. 📖 Proposal: Improving status in CAPI resources #10897 (comment)
    • Add minReadySeconds to machine, remove it from KCP. MD ⚠️ Add MinReadySeconds to Machine and remove it from MachineDeployment, MachineSet, MachinePool. #12153
    • Implement v1beta2 contract fields:
      • Cluster: InfrastructureReady, ControlPlaneReady ⚠️ Add initialization to Cluster status #12098
      • Machine: BootstrapReady, InfrastructureReady ⚠️ Add initialization to Machine Status #12101
      • KubeadmConfig: Ready, DataSecretName ⚠️ Implement v1beta2 contract in cluster controller, KCP, CABPK #12094
      • KCP: Initialized, Ready ⚠️ Implement v1beta2 contract in cluster controller, KCP, CABPK #12094
      • MachinePool: BootstrapReady, InfrastructureReady ⚠️ Add initialization to MachinePool Status #12102
      • Ensure both CABPK & KCP are declaring the v1beta2 contract (via labels on their CRDs)
  • Create v1beta2 types for CAPD(ev) (ensure v1beta1 uses v1beta1 conditions, ...)
    • Create v1beta2 CAPD API as a mirror copy of CAPD v1beta1 API: ⚠️ Add CAPD v1beta2 types #12226
    • Move v1beta2 conditions top level, v1beta1 conditions to deprecated: ⚠️ Promote v1beta2 condition in CAPD #12362
    • Rename conditions const: ⚠️ conditions: add V1Beta1 suffix and remove V1Beta2 suffix from condition types and reasons in CAPD v1beta2 packages #12393
    • Implement v1beta2 contract ⚠️ Implement v1beta2 contract in CAPD #12409
      • Contract changes
        • InfraCluster: failure domains https://main.cluster-api.sigs.k8s.io/developer/providers/contracts/infra-cluster#infracluster-failure-domains
        • InfraCluster: Add initialization struct, move ready into it: https://main.cluster-api.sigs.k8s.io/developer/providers/contracts/infra-cluster#infracluster-initialization-completed
        • InfraMachine: Add initialization struct, ... https://main.cluster-api.sigs.k8s.io/developer/providers/contracts/infra-machine#inframachine-initialization-completed
      • Fix conversions
      • Bump v1.11 in test/e2e/data/shared/main/docker/metadata.yaml to v1beta2
      • Change contract labels in test/infrastructure/docker/config/crd/kustomization.yaml
    • Stop using v1beta1 status in CAPD controllers ⚠️ Stop using v1beta1 status in CAPD controllers #12438
    • Migrate E2E tests to v1beta2 (including test extension): ⚠️ Migrate E2E tests to v1beta2 #12451
    • Ensure CAPD v1beta1 API types are not using clusterv1beta2 types: 🌱 Ensure CAPD v1beta1 API package only imports core v1beta1 #12405
    • Update test/infrastructure/docker/examples: ⚠️ Update example manifests #12480
    • Align CAPD conversion to conversion of other providers (e.g. compare with initialization conversion in CABPK): ⚠️ Align CAPD conversion to conversion of other providers #12481
  • Add v1beta2 ExtensionConfig (ensure v1alpha1 uses v1beta1 conditions, ...): ⚠️ Add v1beta2 API for ExtensionConfig #12197
• Documentation
  • Migration guide for v1beta1 => v1beta2 📖 Update contract documentation #12093
    • Think about limitations of v1beta1 <=> v1beta2 conversion: round-trip only works if clients have v1.9 v1beta1 structs
      • => Recommendation (e.g. v1.10=>v1.11 guide, maybe link in versions.md): use at least CAPI v1.9 structs, if you can't only patch specific fields and avoid e.g. patching the entire status field
    • GitOps: look into fields that we "lose", e.g. removed fields, remove pointers from struct fields (empty objects are going away through roundtrip), remove pointers from string ("" goes away through roundtrip + special cases)
    • Consider writing up benefits of API improvements in v1beta2
  • Update book (e.g. all the examples v1beta1 --> v1beta2)
    • Update examples: Update examples & code snippets in Cluster API book to v1beta2 #12448
  • Revisit our own API conventions in our docs: 📖 Update API conventions #12581
• Change metrics & dashboards (similar to 🌱 metrics: use v1beta2 for condition metrics and add metrics for dockercluster devcluster dockermachine devmachine extensionconfig ipaddressclaim and crs #12006)
• Others
  • Move APIs from api/v1beta2 => api/core/v1beta2: ⚠️ Move APIs to ./api #12262
  • Deprecate v1beta1 packages: ✨ Deprecate v1alpha1 & v1beta1 API packages #12254

Introduce support for compatible contracts

• Add support for compatible contracts in clusterctl 🌱 Add support for compatible contracts to clusterctl #12018
• Implement v1beta2 contract changes to the contract package + handle fallback to v1beta2 contract
  • Machine controller ✨ Implement v1beta2 contract in Machine controller #12038
    • Improve "path .status.ready: field not found" error logs (xref: https://storage.googleapis.com/kubernetes-ci-logs/logs/periodic-cluster-api-e2e-main/1909623599107411968/artifacts/clusters/bootstrap/logs/capi-system/capi-controller-manager/capi-controller-manager-655d6cb6cc-c5wsz/manager.log) => doesn't occur anymore. Probably fixed when implementing the contract if/else for v1beta1 & v1beta2
  • Cluster controller ⚠️ Implement v1beta2 contract in cluster controller, KCP, CABPK #12094
  • Stop calling external.FailuresFrom => we still need this for deprecated.v1beta1 fields
    ⚠️ Implement v1beta2 contract in cluster controller, KCP, CABPK #12094
• Update contract docs 📖 Update contract documentation #12093

Improving status Implementation Phase2 (1.11)

• Promote v1beta conditions
  • Rename packages for condition utils (conditions-->deprecated conditions, v1beta2conditions --> conditions) ⚠️ Swap condition packages #12086
  • Rename condition interfaces and method implementations (current--> deprecated, v1beta2--> current) ⚠️ Swap condition packages #12086
  • Rename condition consts (current--> deprecated, v1beta2--> current) ; mark old conditions type and util as deprecated ⚠️ conditions: add V1Beta1 suffix and remove V1Beta2 suffix from condition types and reasons in v1beta2 packages #12091
• Use new conditions / Audit usage of old conditions: 🌱 Drop usage of v1beta1 conditions #12109
• Stop using failure reason and message ⚠️ Stop using FailureReason and FailureMessage in controllers #12148
• Stop using deprecated replica counters / Audit usage of old fields ⚠️ Stop using deprecated replica counters in controllers #12149
• Make all replica counter status fields pointers + omitempty: ⚠️ Change .status.replicas fields to pointer + omitempty #12250
• Other notes (implemented as part of 🌱 Drop usage of v1beta1 conditions #12109)
  • MachineDeployment controller
    • Also fix: Machinedeployment replicas not in sync with Machineset #12035
  • MachineHealthCheck controller
    • Change test for ControlPlaneInitializedCondition in reconcile
    • Get MachineHealthCheckSucceededCondition in patchUnhealthyTargets
    • Drop failureReson/failureMessage test from needsRemediation
    • Change test for ControlPlaneInitializedCondition, InfrastructureReadyCondition in needsRemediation (few places)
  • KCP controller
    • Change test for MachineEtcdMemberHealthyCondition in canSafelyRemoveEtcdMember + add a test for ClusterHealthy if not already there
    • We should use the Initialized condition instead of the Available condition to calculate remoteConditionsGracePeriod
• Clusterctl
  • clusterctl describe v1beta2 flag: ⚠️ Clusterctl describe defaults to v1beta2 #12369

Others

• Adapt ClusterClass Metadata for eventual removal: ⚠️ Rename deprecated ClusterClass Metadata fields to DeprecatedV1Beta1Metadata #12273
• Drop DefaulterRemoveUnknownOrOmitableFields option on CAPI mutating webhooks: ⚠️ Remove DefaulterRemoveUnknownOrOmitableFields mutating webhook option #12231

Open issues we should look into before the release

• Machinedeployment replicas not in sync with Machineset #12035
• MD.Status.ReadyReplicas changes from 3 to 0 when machineset_controller updateStatus() hits "Unable to retrieve Node status" error #10195
• ClusterClass continuously reconciling resource due to incorrect apiVersion #9557 (@sbueringer)
• KubeadmControlPlane controller hangs during cert expiry check in unstable network conditions #12460
  • 🐛 Fix timeout handling in GetAPIServerCertificateExpiry and DialContext #12554
• Removing fields from template doesn't remove it in the resulting resource #11773
• [P2] MachineSetPreflightChecks + maxSurge zero results in MDs downscaled for a time longer than desirable #12187
• [P2] A 5 control-plane nodes cluster does not recover if loosing 2 nodes at the same time #10125

Testing

• Check conditions in e2e tests: 🌱 add checker to validate conditions for v1beta2 #12111
• Consider extending test/extension to produce a patch for extraArgs (v1beta1+v1beta2) for additional coverage
  • 🌱 Set extraArgs in test extension #12557
• [P0] Fix unit / e2e test flakes, e.g.
  • Failing e2e tests due to networking conformance test #12449
  • Objects with Changed Resource Versions #12334
• Audit e2e test artifact YAMLs: we only expect fields that we set + fields that are defaulted
• Consider to step up test coverage where API version is relevant (e.g. CC, topology controller, Runtime Extension)
  • Add quickstart with v1beta1 YAML with ClusterClass + RTX: 🌱 Add quickstart e2e test with v1beta1 with ClusterClass and RuntimeSDK #12577
• [P1] Look at our logs and see if we can reduce the amount of logged errors
  • Combine with going through a few basic scenarios with watch on clusterctl describe (clusterctl upgrade / rollout / ... e.g. by running e2e test cases)
  • Filter out "well-known" API warnings in controllers (deprecated API is still used)
    • Consider hiding: " is deprecated; use controlplane.cluster.x-k8s.io/v1beta2 KubeadmControlPlaneTemplate"
  • The following errors don't need the entire stack trace:
    • "error creating watch machine-watchNodes for *v1.Node: connection down" (+ connection down error in general)
    • check for other cases