Add Aro-hcp proposal by serngawy · Pull Request #5605 · kubernetes-sigs/cluster-api-provider-azure

serngawy · 2025-05-01T15:54:33Z

What type of PR is this?

/kind documentation

What this PR does / why we need it:

Proposal for ARO-HCP feature

Which issue(s) this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close the issue(s) when PR gets merged):
Fixes #

Special notes for your reviewer:

TODOs:

squashed commits
[ X] includes documentation
adds unit tests
cherry-pick candidate

Release note:

None

codecov · 2025-05-01T16:05:01Z

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 54.68%. Comparing base (c3f0631) to head (2643a59).
⚠️ Report is 596 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #5605      +/-   ##
==========================================
+ Coverage   53.28%   54.68%   +1.40%     
==========================================
  Files         272      279       +7     
  Lines       29537    31174    +1637     
==========================================
+ Hits        15739    17049    +1310     
- Misses      12983    13205     +222     
- Partials      815      920     +105

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

serngawy · 2025-05-05T14:38:48Z

@marek-veber

marek-veber · 2025-05-07T12:51:27Z

docs/proposals/20250425-aro-hcp.md

+
+ // Resource group name where the Aro-hcp will be attached to it.
+ ResourceGroup string `json:"resourceGroup,omitempty"`
+


IMHO we will also need the:

tenant_id,

subscription_id,

managed_resource_group_name (not only resource_group_name)
to create a cluster, isn't it?

yes, those info are provided at the azureclusteridentity

marek-veber · 2025-05-07T13:15:35Z

docs/proposals/20250425-aro-hcp.md

+// PlatformProfile represents the Azure platform configuration.
+type PlatformProfile struct {
+ // Azure subnet id
+ Subnet string `json:"subnet,omitempty"`


Why Subnet & SubnetRef ?
subnet id or cluster id or node pool id ? I'm not sure base on: https://github.com/Azure/ARO-HCP/blob/main/cluster-service/cluster-creation.md#creating-node-pools

Replicas & autorepare will be used from capi MachinePool?

subnetRef will be referring to VirtualNetworksSubnet CR by name.
yes, replicas defined at CAPI machinePool CR

marek-veber · 2025-05-07T17:59:32Z

docs/proposals/20250425-aro-hcp.md

+ ControlPlaneOperators  map[string]string `json:"controlPlaneOperators,omitempty"`
+
+
+ // DataPlaneOperators ref to Microsoft.ManagedIdentity/userAssignedIdentities
+ DataPlaneOperators     map[string]string `json:"dataPlaneOperators,omitempty"`
+
+
+ // ServiceManagedIdentity ref to Microsoft.ManagedIdentity/userAssignedIdentities
+ ServiceManagedIdentity string            `json:"serviceManagedIdentity,omitempty"`


What are the keys in those maps?
What about to replace maps with structures for it?

I updated the managed Identities to include all the required identities names

bryan-cox · 2025-05-09T19:24:07Z

docs/proposals/20250425-aro-hcp.md

+ ServiceManagedIdentity string            `json:"serviceManagedIdentity,omitempty"`
+}
+
+type ControlPlaneOperators struct {


Are we not planning to reuse the HyperShift API for this, rather than duplicating API efforts?

no, we will re-use the Hypershift API. The naming for CRD fields is a bit different but internally with API call will set the naming as here

bryan-cox · 2025-05-09T19:24:27Z

docs/proposals/20250425-aro-hcp.md

+	KmsManagedIdentities string `json:"kmsManagedIdentities,omitempty"`
+}
+
+type DataPlaneOperators struct {


Similar question as above, re: reusing HyperShift API

bryan-cox · 2025-05-09T19:27:37Z

docs/proposals/20250425-aro-hcp.md

+ ServiceManagedIdentity string            `json:"serviceManagedIdentity,omitempty"`
+}
+
+type ControlPlaneOperators struct {


Should we also list the exact role definitions each one of these will get?

not sure if I understand, The field desc mentioning the role definition "control-plane" and similar for every UAMI

Every control plane and data plane are assigned a specific Azure role definition(s) - CNTRLPLANE-171.

For example, the ingress managed identity is only assigned the "Azure Red Hat OpenShift Cluster Ingress Operator" role over the resource groups containing the VNET and the DNS Zone.

okay, that is fine we will use the same role naming and keys. Would you mention the ref for it from the Hypershift repo . I don't think the Jira url above is accessible for everyone.

The mapping of control plane to role definition to resource group does not reside from the HyperShift repo. I am not sure if it is available in some public fashion on the ARO side 🤔

I checked the hypershift repo, I don't think we need to set the Azure role definition, In CAPZ we will set the role key name similar to the example here ex; control-plane, cluster-api-azure, ingress, ..etc then hypershift will set the role definition. Hypershift has the role ref here

it is here in the ocm-cluster-service

FWIW, we do not set the role definitions in HyperShift. That is assumed to be done prior to creating the HostedCluster.

The HyperShift link you mentioned is related to the HyperShift CLI, which does set the role definitions but that is only for development and testing use.

hmm, so we need to have this defined some where yes/no ?
Also this is more like implementation details for the UserAssignedIdentity CR that will be created we will need to set the right spec info

I think it would be nice to be defined somewhere.

Maybe it goes with the UserAssignedIdentity CR; is that defined somewhere?

willie-yao · 2025-05-29T16:05:58Z

/assign

marek-veber · 2025-06-02T11:28:28Z

docs/proposals/20250425-aro-hcp.md

+	ControlPlaneManagedIdentities string `json:"controlPlaneOperatorsManagedIdentities,omitempty"`
+
+    // ClusterApiAzureManagedIdentities "cluster-api-azure" Microsoft.ManagedIdentity/userAssignedIdentities
+	ClusterApiAzureManagedIdentities string `json:clusterApiAzureManagedIdentities",omitempty"`


sometimes is here badly formatted json option:
json:clusterApiAzureManagedIdentities",omitempty" -> json:"clusterApiAzureManagedIdentities,omitempty"

The " should be moved after json: . There is such a typo multiple times.

Thanks fixed

marek-veber

identityRef - need to be *corev1.ObjectReference in go
and in yaml:

identityRef:
   kind: AzureClusterIdentity
   name: aro-identity
   namespace: default

marek-veber · 2025-06-02T23:01:55Z

docs/proposals/20250425-aro-hcp.md

+
+ // IdentityRef is a reference to an identity to be used when reconciling the aro control plane.
+ // If no identity is specified, the default identity for this controller will be used.
+ IdentityRef *infrav1.AzureClusterIdentity `json:"identityRef,omitempty"`


Suggested change

IdentityRef *infrav1.AzureClusterIdentity `json:"identityRef,omitempty"`

IdentityRef *corev1.ObjectReference `json:"identityRef,omitempty"`

marek-veber · 2025-06-02T23:04:21Z

docs/proposals/20250425-aro-hcp.md

+ identityRef:
+   name: aro-identity


Suggested change

identityRef:

name: aro-identity

identityRef:

kind: AzureClusterIdentity

name: aro-identity

namespace: default

docs/proposals/20250425-aro-hcp.md

Signed-off-by: serngawy <serngawy@gmail.com>

willie-yao · 2025-06-18T23:27:07Z

Is this ready for final review? @serngawy

serngawy · 2025-06-19T15:46:01Z

Is this ready for final review? @serngawy

yes

marek-veber · 2025-06-23T18:58:26Z

docs/proposals/20250425-aro-hcp.md

+	CloudNetworkConfigManagedIdentities string `json:"cloudNetworkConfigManagedIdentities,omitempty"`
+
+    // KmsManagedIdentities "kms" Microsoft.ManagedIdentity/userAssignedIdentities
+	KmsManagedIdentities string `json:"kmsManagedIdentities,omitempty"`


This is not possible to use with openshift 4.18.1

Correct. ARO HCP is only supported for OpenShift 4.19+

marek-veber · 2025-07-09T11:45:04Z

/lgtm
/approved

k8s-ci-robot · 2025-07-09T11:45:09Z

@marek-veber: changing LGTM is restricted to collaborators

Details

In response to this:

/lgtm
/approved

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

willie-yao · 2025-08-01T22:56:28Z

/assign @jackfrancis

k8s-triage-robot · 2025-11-10T21:33:06Z

The Kubernetes project currently lacks enough contributors to adequately respond to all PRs.

This bot triages PRs according to the following rules:

After 90d of inactivity, lifecycle/stale is applied
After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
After 30d of inactivity since lifecycle/rotten was applied, the PR is closed

You can:

Mark this PR as fresh with /remove-lifecycle stale
Close this PR with /close
Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

k8s-triage-robot · 2025-12-10T21:37:34Z

The Kubernetes project currently lacks enough active contributors to adequately respond to all PRs.

This bot triages PRs according to the following rules:

After 90d of inactivity, lifecycle/stale is applied
After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
After 30d of inactivity since lifecycle/rotten was applied, the PR is closed

You can:

Mark this PR as fresh with /remove-lifecycle rotten
Close this PR with /close
Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle rotten

k8s-triage-robot · 2026-01-09T22:20:03Z

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages PRs according to the following rules:

After 90d of inactivity, lifecycle/stale is applied
After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
After 30d of inactivity since lifecycle/rotten was applied, the PR is closed

You can:

Reopen this PR with /reopen
Mark this PR as fresh with /remove-lifecycle rotten
Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/close

k8s-ci-robot · 2026-01-09T22:20:09Z

@k8s-triage-robot: Closed this PR.

Details

In response to this:

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages PRs according to the following rules:

After 90d of inactivity, lifecycle/stale is applied

After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied

After 30d of inactivity since lifecycle/rotten was applied, the PR is closed

You can:

Reopen this PR with /reopen

Mark this PR as fresh with /remove-lifecycle rotten

Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

k8s-triage-robot · 2026-02-23T21:30:24Z

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages PRs according to the following rules:

After 90d of inactivity, lifecycle/stale is applied
After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
After 30d of inactivity since lifecycle/rotten was applied, the PR is closed

You can:

Reopen this PR with /reopen
Mark this PR as fresh with /remove-lifecycle rotten
Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/close

willie-yao · 2026-03-03T18:31:34Z

/reopen

k8s-ci-robot · 2026-03-03T18:31:41Z

@willie-yao: Reopened this PR.

Details

In response to this:

/reopen

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

k8s-ci-robot · 2026-03-03T18:31:56Z

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please ask for approval from jackfrancis. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details

Needs approval from an approver in each of these files:

OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

willie-yao · 2026-03-03T18:34:14Z

/remove-lifecycle rotten

github-project-automation bot moved this to Todo in CAPZ Planning May 1, 2025

github-project-automation bot added this to CAPZ Planning May 1, 2025

k8s-ci-robot added release-note-none Denotes a PR that doesn't merit a release note. kind/documentation Categorizes issue or PR as related to documentation. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. labels May 1, 2025

k8s-ci-robot requested review from mboersma and nojnhuh May 1, 2025 15:54

k8s-ci-robot added the size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. label May 1, 2025

serngawy force-pushed the aro-proposal branch 2 times, most recently from 22ad098 to 35b0abc Compare May 2, 2025 21:01

marek-veber reviewed May 7, 2025

View reviewed changes

serngawy force-pushed the aro-proposal branch from 35b0abc to 169d04e Compare May 7, 2025 20:35

bryan-cox reviewed May 9, 2025

View reviewed changes

k8s-ci-robot assigned willie-yao May 29, 2025

marek-veber reviewed Jun 2, 2025

View reviewed changes

serngawy force-pushed the aro-proposal branch from 169d04e to d9bc85b Compare June 2, 2025 20:21

marek-veber reviewed Jun 2, 2025

View reviewed changes

marek-veber mentioned this pull request Jun 3, 2025

ARO ... define structures #5671

Closed

4 tasks

marek-veber reviewed Jun 3, 2025

View reviewed changes

docs/proposals/20250425-aro-hcp.md Outdated Show resolved Hide resolved

docs/proposals/20250425-aro-hcp.md Outdated Show resolved Hide resolved

docs/proposals/20250425-aro-hcp.md Outdated Show resolved Hide resolved

docs/proposals/20250425-aro-hcp.md Outdated Show resolved Hide resolved

serngawy force-pushed the aro-proposal branch from d9bc85b to 44837d6 Compare June 4, 2025 18:07

marek-veber suggested changes Jun 9, 2025

View reviewed changes

docs/proposals/20250425-aro-hcp.md Outdated Show resolved Hide resolved

docs/proposals/20250425-aro-hcp.md Outdated Show resolved Hide resolved

github-project-automation bot moved this from Todo to Wait-On-Author in CAPZ Planning Jun 9, 2025

marek-veber suggested changes Jun 11, 2025

View reviewed changes

docs/proposals/20250425-aro-hcp.md Outdated Show resolved Hide resolved

docs/proposals/20250425-aro-hcp.md Outdated Show resolved Hide resolved

docs/proposals/20250425-aro-hcp.md Outdated Show resolved Hide resolved

Add Aro-hcp proposal

2643a59

Signed-off-by: serngawy <serngawy@gmail.com>

serngawy force-pushed the aro-proposal branch from 44837d6 to 2643a59 Compare June 11, 2025 21:22

mboersma added this to the v1.21 milestone Jun 12, 2025

marek-veber reviewed Jun 23, 2025

View reviewed changes

willie-yao moved this from Wait-On-Author to Needs Review in CAPZ Planning Jun 26, 2025

k8s-ci-robot assigned jackfrancis Aug 1, 2025

jackfrancis modified the milestones: v1.21, next Aug 12, 2025

k8s-ci-robot added the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Nov 10, 2025

k8s-ci-robot added lifecycle/rotten Denotes an issue or PR that has aged beyond stale and will be auto-closed. and removed lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. labels Dec 10, 2025

k8s-ci-robot closed this Jan 9, 2026

github-project-automation bot moved this from Needs Review to Done in CAPZ Planning Jan 9, 2026

k8s-ci-robot reopened this Mar 3, 2026

github-project-automation bot moved this from Done to In Progress in CAPZ Planning Mar 3, 2026

k8s-ci-robot removed the lifecycle/rotten Denotes an issue or PR that has aged beyond stale and will be auto-closed. label Mar 3, 2026


		// Resource group name where the Aro-hcp will be attached to it.
		ResourceGroup string `json:"resourceGroup,omitempty"`

	IdentityRef *infrav1.AzureClusterIdentity `json:"identityRef,omitempty"`
	IdentityRef *corev1.ObjectReference `json:"identityRef,omitempty"`

Conversation

serngawy commented May 1, 2025

Uh oh!

codecov bot commented May 1, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

serngawy commented May 5, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

willie-yao commented May 29, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

marek-veber left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

willie-yao commented Jun 18, 2025

Uh oh!

serngawy commented Jun 19, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

marek-veber commented Jul 9, 2025

Uh oh!

k8s-ci-robot commented Jul 9, 2025

codecov bot commented May 1, 2025 •

edited

Loading