Azure Stack Hub HybridEnvironment Support

[patrickdillon]

/kind feature

Describe the solution you'd like
Support using CAPZ to provision clusters on Azure Stack Hub.

Anything else you would like to add:
Achieving this goal would entail:

• Adding Azure Stack Cloud (or HybridEnvironment) as a valid AzureEnvironment
• Configuring cloud endpoints for the azure stack environment
• Adding support for Basic Load Balancers (only supported LB on Azure Stack Hub)
• Updating API version to a compatible profile as necessary (I believe this is only needed for resources that are not provisioned using ASO, so we can either migrate them to ASO or update the profiles. I'm currently working through NSGs, not sure if more are needed).

Environment:

• cluster-api-provider-azure version:
• Kubernetes version: (use kubectl version):
• OS (e.g. from /etc/os-release):

[nojnhuh]

FYI @mboersma I think this will probably intersect with #4780.

I see ASO exposes only the AZURE_AUTHORITY_HOST, AZURE_RESOURCE_MANAGER_ENDPOINT, and AZURE_RESOURCE_MANAGER_AUDIENCE variables instead what CAPZ does now with the pre-baked sets of those for public cloud, China, Gov cloud, etc. @patrickdillon Would a way to set the equivalents of those variables in CAPZ be sufficient?

And since CAPZ would ultimately have to pass this config along to ASO, we should iron out Azure/azure-service-operator#3447 first before we commit to any API changes in CAPZ.

[patrickdillon]

Sorry @nojnhuh I missed your comment the first time around, and only noticed it now that I have opened a PR for the implementation.

This commit from my PR provides context for both of your question. It shows how the variables you mention are set. Also you are 100% correct: AFAIK there is no equivalent for go-autorest's EnvironmentFromURL(armEndpoint) function in the V2 SDK, so there is a dependency on go-autorest. FWIW that dependency is limited to a single file.

Unfortunately, as you can see from my PR (and the length of time between my comments), setting these variables per se is not sufficient, and much further work has been required.

[k8s-triage-robot]

The Kubernetes project currently lacks enough contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue as fresh with /remove-lifecycle stale
• Close this issue with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

[k8s-triage-robot]

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue as fresh with /remove-lifecycle rotten
• Close this issue with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle rotten

[patrickdillon]

/remove-lifecycle rotten

Sorry sorry I will update this asap