Skip to content

Commit 8cb10a4

Browse files
marek-veberclaude
andcommitted
Address review feedback for ARO HCP proposal
Updates based on willie-yao's review comments: - Update metadata: add reviewers, fix dates, change status to implementable - Fix API versions: v1beta2 → v1beta1 throughout examples - Add "Alternatives Considered" section explaining why separate CRDs are needed - Add "Maintenance and Ownership" section clarifying ARO team ownership - Document ASO migration plan with dependency chain - Add references to ARO-HCP repository and API specifications - Note API versions: 2024-06-10-preview (private) vs 2025-12-23-preview (public) - Add ASO issue reference for key management gap - Update graduation criteria with ownership and migration commitments Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
1 parent 3949aeb commit 8cb10a4

File tree

1 file changed

+61
-11
lines changed

1 file changed

+61
-11
lines changed

docs/proposals/20250425-aro-hcp.md

Lines changed: 61 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -4,10 +4,11 @@ authors:
44
- "@serngawy"
55
- "@marek-veber"
66
reviewers:
7-
- ""
8-
creation-date: 2025-04-23
9-
last-updated: 2026-02-20
10-
status: implemented
7+
- "@willie-yao"
8+
- "@jackfrancis"
9+
creation-date: 2025-04-25
10+
last-updated: 2026-03-11
11+
status: implementable
1112
---
1213

1314

@@ -23,6 +24,8 @@ status: implemented
2324
- [Proposal](#proposal)
2425
- [User Stories](#user-stories)
2526
- [Implementation Details](#implementation-details)
27+
- [Alternatives Considered](#alternatives-considered)
28+
- [Maintenance and Ownership](#maintenance-and-ownership)
2629
- [Risks and Mitigations](#risks-and-mitigations)
2730
- [Graduation Criteria](#graduation-criteria)
2831
- [Implementation History](#implementation-history)
@@ -33,6 +36,8 @@ status: implemented
3336

3437
The Cluster API Provider for Azure (CAPZ) extends Kubernetes Cluster API functionality to Microsoft Azure environments, facilitating the management and lifecycle of Kubernetes clusters on Azure. This proposal outlines the integration of Azure Red Hat OpenShift (ARO) Hosted Control Plane (HCP) clusters within CAPZ using Azure Service Operator (ASO) for resource provisioning.
3538

39+
ARO HCP is an evolution of Azure Red Hat OpenShift that uses a hosted control plane architecture. For more information about ARO HCP, see the [ARO-HCP repository](https://github.com/Azure/ARO-HCP).
40+
3641
The implementation leverages ASO resources embedded directly in Custom Resource specifications, providing a declarative, Kubernetes-native approach to managing ARO HCP infrastructure.
3742

3843

@@ -115,9 +120,13 @@ type AROControlPlaneSpec struct {
115120
// - HcpOpenShiftCluster resource
116121
// - HcpOpenShiftClustersExternalAuth resource (if external auth is needed)
117122
//
118-
// Example resources:
123+
// Example resources (using private preview API v1api20240610preview):
119124
// - redhatopenshift.azure.com/v1api20240610preview/HcpOpenShiftCluster
120125
// - redhatopenshift.azure.com/v1api20240610preview/HcpOpenShiftClustersExternalAuth
126+
//
127+
// Note: v1api20240610preview is the private preview API. Migration to public preview
128+
// API (v1api20251223preview) is planned. See API specs at:
129+
// https://github.com/Azure/ARO-HCP/tree/main/api/redhatopenshift/resource-manager/Microsoft.RedHatOpenShift/hcpclusters/preview
121130
Resources []runtime.RawExtension `json:"resources,omitempty"`
122131

123132
// IdentityRef is a reference to an identity to be used when reconciling the ARO control plane.
@@ -203,7 +212,7 @@ type ResourceStatus struct {
203212
An example of AROControlPlane CR using resources mode:
204213

205214
```yaml
206-
apiVersion: controlplane.cluster.x-k8s.io/v1beta2
215+
apiVersion: controlplane.cluster.x-k8s.io/v1beta1
207216
kind: AROControlPlane
208217
metadata:
209218
name: my-cluster
@@ -343,7 +352,7 @@ status:
343352
This example shows AROControlPlane using ASO credentials instead of identityRef:
344353
345354
```yaml
346-
apiVersion: controlplane.cluster.x-k8s.io/v1beta2
355+
apiVersion: controlplane.cluster.x-k8s.io/v1beta1
347356
kind: AROControlPlane
348357
metadata:
349358
name: hcp-cluster
@@ -471,7 +480,7 @@ type AROMachinePoolSpec struct {
471480
// Resources are embedded ASO resources to be managed by this AROMachinePool.
472481
// Must include HcpOpenShiftClustersNodePool resource.
473482
//
474-
// Example:
483+
// Example (using private preview API v1api20240610preview):
475484
// - redhatopenshift.azure.com/v1api20240610preview/HcpOpenShiftClustersNodePool
476485
Resources []runtime.RawExtension `json:"resources"`
477486
}
@@ -498,7 +507,7 @@ type AROMachinePoolStatus struct {
498507
An example of AROMachinePool CR:
499508

500509
```yaml
501-
apiVersion: infrastructure.cluster.x-k8s.io/v1beta2
510+
apiVersion: infrastructure.cluster.x-k8s.io/v1beta1
502511
kind: AROMachinePool
503512
metadata:
504513
name: my-cluster-mp1
@@ -620,7 +629,7 @@ type AROClusterInitializationStatus struct {
620629
An example of AROCluster CR:
621630

622631
```yaml
623-
apiVersion: infrastructure.cluster.x-k8s.io/v1beta2
632+
apiVersion: infrastructure.cluster.x-k8s.io/v1beta1
624633
kind: AROCluster
625634
metadata:
626635
name: my-cluster
@@ -1066,7 +1075,7 @@ spec.properties.etcd.dataEncryption.customerManaged.kms.activeKey.version
10661075
**Solution**: Add the Vault resource to **AROCluster.spec.resources[]** (not AROControlPlane):
10671076
10681077
```yaml
1069-
apiVersion: infrastructure.cluster.x-k8s.io/v1beta2
1078+
apiVersion: infrastructure.cluster.x-k8s.io/v1beta1
10701079
kind: AROCluster
10711080
metadata:
10721081
name: my-cluster
@@ -1148,6 +1157,10 @@ spec:
11481157
11491158
- **Azure SDK**: Used only for encryption key management (when identityRef is set)
11501159
- KeyVault keys (create/retrieve key versions within existing vault)
1160+
- **Note**: This is a temporary gap in ASO coverage. ASO supports Vault CRDs but not key management. See upstream tracking: [Azure/azure-service-operator#4481](https://github.com/Azure/azure-service-operator/issues/4481)
1161+
- This SDK dependency will be removed once ASO adds key version retrieval support
1162+
- **Note**: This is a temporary gap in ASO's coverage. ASO currently supports Vault CRDs but not Key management within vaults. There is an upstream ASO issue tracking this: [Azure/azure-service-operator#4481](https://github.com/Azure/azure-service-operator/issues/4481)
1163+
- **Future**: Once ASO adds support for key version retrieval, the keyvaults SDK service can be removed entirely
11511164
11521165
### Validation and Testing
11531166
@@ -1193,6 +1206,40 @@ Resource Deployment
11931206
- Testing of error conditions and recovery
11941207
- Testing of both authentication modes (CAPZ managed vs ASO credential-based)
11951208
1209+
## Alternatives Considered
1210+
1211+
### Using AzureASOManagedControlPlane
1212+
1213+
AzureASOManagedControlPlane is designed for AKS managed clusters using `containerservice.azure.com` resources. ARO HCP uses fundamentally different Azure resource types (`redhatopenshift.azure.com/HcpOpenShiftCluster`) with ARO-specific requirements including ETCD encryption with Key Vault integration, external authentication resources, and different node pool APIs. The architectural differences are significant enough that separate CRDs provide clearer API boundaries and simpler controller logic than trying to unify both platforms under a single type.
1214+
1215+
### Pure ASO Without CAPZ Controllers
1216+
1217+
While ASO provides Azure resource management, CAPZ controllers add Cluster API integration (lifecycle management, dependency orchestration, status aggregation) and automation features like encryption key version injection. Users who don't need CAPI integration can use ASO directly with `identityRef: nil` mode.
1218+
1219+
## Maintenance and Ownership
1220+
1221+
The ARO HCP integration is **owned and maintained by the Azure Red Hat OpenShift (ARO) team at Red Hat**. This includes the AROControlPlane, AROMachinePool, and AROCluster CRDs, controllers, webhooks, and ARO-specific documentation.
1222+
1223+
**CAPZ core team responsibilities** are limited to maintaining shared infrastructure (ASO integration patterns, common libraries) and reviewing ARO pull requests for CAPZ architectural alignment. The CAPZ core team is **not expected to own or maintain ARO-specific logic**.
1224+
1225+
### Azure SDK Usage and ASO Migration Plan
1226+
1227+
The keyvaults service currently uses Azure SDK for key version retrieval because ASO doesn't yet support Key Vault key management CRDs (only Vault resources). This SDK usage is temporary and will be removed once ASO coverage is available.
1228+
1229+
**ARO HCP API Versions**:
1230+
- **2024-06-10-preview**: Private preview API version (currently used in this implementation)
1231+
- **2025-12-23-preview**: Public preview API version (planned migration target)
1232+
1233+
API specifications are maintained in the [ARO-HCP repository](https://github.com/Azure/ARO-HCP/tree/main/api/redhatopenshift/resource-manager/Microsoft.RedHatOpenShift/hcpclusters/preview).
1234+
1235+
**Dependency chain for full ASO migration**:
1236+
1. ARO HCP public preview completion (pending Microsoft release formalities)
1237+
2. ARO HCP API specification merged into Azure REST API specs repository
1238+
3. ASO generation of ARO HCP CRDs from updated specs
1239+
4. Backporting ASO changes to the release version used by CAPZ
1240+
1241+
Once these dependencies are resolved and ASO supports key management, the keyvaults SDK service will be removed entirely. The ARO team commits to migrating to pure ASO-native implementation at that time.
1242+
11961243
## Risks and Mitigations
11971244
11981245
- **ASO API Changes**: Monitor ASO releases and update resource definitions accordingly
@@ -1209,6 +1256,9 @@ Resource Deployment
12091256
- ✅ Encryption key management working correctly
12101257
- ✅ ExternalAuth configuration functioning properly
12111258
- ✅ Clean code with field-based mode removed (~5,400 lines removed)
1259+
- ✅ Maintenance ownership documented and agreed upon
1260+
- ✅ Plan documented to remove keyvaults SDK service when ASO supports key version retrieval
1261+
- ✅ Acknowledgment that CAPZ's long-term direction is toward pure ASO-native controllers
12121262
12131263
## Implementation History
12141264

0 commit comments

Comments
 (0)