Skip to content

Commit dcab7be

Browse files
authored
Merge pull request #3447 from nojnhuh/aso-proposal-links
fix broken links in ASO proposal
2 parents 2326522 + f445ab3 commit dcab7be

File tree

1 file changed

+7
-7
lines changed

1 file changed

+7
-7
lines changed

docs/proposals/20230123-azure-service-operator.md

Lines changed: 7 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -121,23 +121,23 @@ Integration with ASO will not require any updates to CAPZ resources and CAPZ's C
121121

122122
ASO is a self-contained component published independently of CAPZ. When CAPZ is relying on ASO for critical functionality, CAPZ's installation process should also automatically install ASO.
123123

124-
When installing CAPZ with `clusterctl init`, the Kubernetes resources defined by CAPZ's `infrastructure-components.yaml` published on the corresponding release are installed. ASO can also be [installed with a YAML manifest](https://azure.github.io/azure-service-operator/introduction/installing-from-yaml/) published for each of its releases describing its components that can be incorporated with CAPZ's `infrastructure-components.yaml`. This method installs ASO in the `azureserviceoperator-system` namespace and expects a Secret named `aso-controller-settings` to exist with Service Principal credentials. The easiest and most flexible way to customize that manifest further would likely be to generate a manifest from ASO's Helm chart with any necessary values overridden.
124+
When installing CAPZ with `clusterctl init`, the Kubernetes resources defined by CAPZ's `infrastructure-components.yaml` published on the corresponding release are installed. ASO can also be [installed with a YAML manifest](https://azure.github.io/azure-service-operator/guide/installing-from-yaml/) published for each of its releases describing its components that can be incorporated with CAPZ's `infrastructure-components.yaml`. This method installs ASO in the `azureserviceoperator-system` namespace and expects a Secret named `aso-controller-settings` to exist with Service Principal credentials. The easiest and most flexible way to customize that manifest further would likely be to generate a manifest from ASO's Helm chart with any necessary values overridden.
125125

126126
Other than integrating with CAPZ's `infrastructure-components.yaml`, the `capz-controller-manager` container itself could assume responsibility for installing ASO by using the Helm Go library to install the ASO chart when it starts. Installing ASO this way allows CAPZ to hook in additional logic, such as to check if ASO is already installed in the cluster or to verify ASO is healthy after it's installed, which would not be possible if a static manifest was embedded in `infrastructure-components.yaml`. However, installing ASO components from CAPZ itself would also require CAPZ to have additional RBAC permissions to create, update, and delete ASO's Deployment, Service, and other components which would not be required if the components were managed from `infrastructure-components.yaml`.
127127

128128
For either method of installing the rest of ASO, ASO's CRDs will be installed from `infrastructure-components.yaml` to more easily facilitate `clusterctl move` and to keep the `capz-controller-manager` Pod from requiring RBAC permissions to create, update, and delete CRDs.
129129

130-
In terms of specific options, ASO offers both [single- and multi-operator configurations](https://azure.github.io/azure-service-operator/introduction/multitenant-deployment/). The single-operator option is the default, and assumes a single ASO installation managing resources for the whole cluster. The multi-operator method allows several ASO installations to coexist, each monitoring resources in a unique set of namespaces to enable multitenancy. ASO's single-operator configuration also allows multitenancy with more tightly-scoped credentials. Overall, a single ASO installation will likely be the easier path forward since it's consistent with CAPZ's multitenancy model and doesn't involve managing ASO's CRDs and webhooks across multiple installations.
130+
In terms of specific options, ASO offers both [single- and multi-operator configurations](https://azure.github.io/azure-service-operator/guide/multitenant-deployment/). The single-operator option is the default, and assumes a single ASO installation managing resources for the whole cluster. The multi-operator method allows several ASO installations to coexist, each monitoring resources in a unique set of namespaces to enable multitenancy. ASO's single-operator configuration also allows multitenancy with more tightly-scoped credentials. Overall, a single ASO installation will likely be the easier path forward since it's consistent with CAPZ's multitenancy model and doesn't involve managing ASO's CRDs and webhooks across multiple installations.
131131

132132
The recommendation in this proposal is to install a single-operator, ["reduced permissions"](https://azure.github.io/azure-service-operator/design/adr-2023-02-helm-chart-size-limitations/#decision) configuration of ASO entirely from `infrastructure-components.yaml` to keep CAPZ from needing any new RBAC permissions to install ASO and to align with how CAPZ handles its existing dependency on AAD Pod Identity.
133133

134134
#### `clusterctl move`
135135

136-
`clusterctl move` is expected to work automatically following ASO's [documented best practices](https://azure.github.io/azure-service-operator/introduction/frequently-asked-questions/#what-is-the-best-practice-for-transferring-aso-resources-from-one-cluster-to-another) to move its resources pending a solution to this issue in Cluster API: https://github.com/kubernetes-sigs/cluster-api/issues/8473
136+
`clusterctl move` is expected to work automatically following ASO's [documented best practices](https://azure.github.io/azure-service-operator/guide/frequently-asked-questions/#what-is-the-best-practice-for-transferring-aso-resources-from-one-cluster-to-another) to move its resources pending a solution to this issue in Cluster API: https://github.com/kubernetes-sigs/cluster-api/issues/8473
137137

138138
### Security Model
139139

140-
ASO requires credentials to interact with Azure which are documented here: https://azure.github.io/azure-service-operator/introduction/authentication/. Specifically, ASO needs the values of `$AZURE_CLIENT_ID`, `$AZURE_CLIENT_SECRET` or `$AZURE_CLIENT_CERTIFICATE`, `$AZURE_SUBSCRIPTION_ID`, and `$AZURE_TENANT_ID` for Service Principal authentication. These credentials must be defined in a Kubernetes Secret that either applies to all ASO resources in all namespaces, all ASO resources in a particular namespace, or individual ASO resources referring to the Secret in its `serviceoperator.azure.com/credential-from` annotation. ASO allows different credentials to be used for different resources to enable multitenancy, similar to CAPZ and its AzureClusterIdentity. To mediate between the two mechanisms, CAPZ will ensure a cluster's AzureClusterIdentity or default parameters from environment variables are reflected in a Secret to be used by ASO and ensure each ASO resource it manages is created with the annotation pointing to the Secret.
140+
ASO requires credentials to interact with Azure which are documented here: https://azure.github.io/azure-service-operator/guide/authentication/. Specifically, ASO needs the values of `$AZURE_CLIENT_ID`, `$AZURE_CLIENT_SECRET` or `$AZURE_CLIENT_CERTIFICATE`, `$AZURE_SUBSCRIPTION_ID`, and `$AZURE_TENANT_ID` for Service Principal authentication. These credentials must be defined in a Kubernetes Secret that either applies to all ASO resources in all namespaces, all ASO resources in a particular namespace, or individual ASO resources referring to the Secret in its `serviceoperator.azure.com/credential-from` annotation. ASO allows different credentials to be used for different resources to enable multitenancy, similar to CAPZ and its AzureClusterIdentity. To mediate between the two mechanisms, CAPZ will ensure a cluster's AzureClusterIdentity or default parameters from environment variables are reflected in a Secret to be used by ASO and ensure each ASO resource it manages is created with the annotation pointing to the Secret.
141141

142142
Consumers of ASO's APIs (such as CAPZ) also need Kubernetes RBAC permissions to interact with ASO resources. For example, this ClusterRole snippet is required by CAPZ to manage resource groups with ASO:
143143
```yaml
@@ -162,15 +162,15 @@ The most immediate risks with adopting ASO are regressions of various forms. Fir
162162
163163
Along those same lines, there is a risk that if a service is implemented both with the SDK and with ASO in different supported versions of CAPZ, then the fix for an issue affecting both versions will need to be implemented separately for each version. Such issues are not expected to be common given the implementation for each ASO service is expected to be mostly disjoint from its SDK-based implementation. When they do arise, however, the cost of the extra work required for a second implementation of a fix will be accepted since the risk will only be present for a few months at most as newer CAPZ releases push SDK-backed service implementations out of support.
164164
165-
Another shorter-term risk is that although ASO v2 is [officially supported by Microsoft](https://azure.github.io/azure-service-operator/introduction/frequently-asked-questions/#what-is-the-support-model), it has not yet published a GA release. The remaining work to be completed before ASO v2's first GA release is being tracked here: https://github.com/Azure/azure-service-operator/milestone/20.
165+
Another shorter-term risk is that although ASO v2 is [officially supported by Microsoft](https://azure.github.io/azure-service-operator/guide/frequently-asked-questions/#what-is-the-support-model), it has not yet published a GA release. The remaining work to be completed before ASO v2's first GA release is being tracked here: https://github.com/Azure/azure-service-operator/milestone/20.
166166
167167
Similarly, ASO does not currently support some Azure services used by CAPZ currently which are being tracked here: https://github.com/Azure/azure-service-operator/issues/2237. If a service is not yet implemented in ASO by the time CAPZ is ready for it, a track 2 SDK-backed implementation can be added instead. Additionally, some Azure services like Resource Health do not represent infrastructure and therefore don't closely align with ASO's model for creating, updating, and deleting resources. If those kinds of services are determined not to ever be a good fit for ASO, then SDK-backed implementations for those interfaces could be added as well.
168168
169169
Further down the road, there is an apparent increased risk that CAPZ's lifespan may extend beyond that of its Azure interface when ASO is used instead of the API or SDK directly, as ASO has not yet been proven as a mission-critical interface to Azure to the same extent as Azure's REST API and Go SDK.
170170
171171
Since ASO can be freely installed by users, there is also a risk that a CAPZ-installed ASO may conflict with a user-installed instance. Before CAPZ installs ASO, it should verify that ASO is already either not installed or installed and configured as expected by CAPZ.
172172
173-
The difference between CAPZ's and ASO's metrics stories may also be considered a risk. CAPZ's metrics currently include traces that measure how long each step in a reconciliation takes, including all Azure API calls. Once those equivalent API calls are being done from ASO instead of CAPZ, CAPZ's metrics will lose that data. However, that lost data may no longer be meaningful to CAPZ since Azure API calls will be happening out-of-band with CAPZ reconciliations. ASO does measure how many Azure API calls are made and how long each takes and exposes those in the Prometheus format, along with other metrics as documented at https://azure.github.io/azure-service-operator/introduction/metrics/. However, users relying on the current structure of metrics will likely experience breaking changes.
173+
The difference between CAPZ's and ASO's metrics stories may also be considered a risk. CAPZ's metrics currently include traces that measure how long each step in a reconciliation takes, including all Azure API calls. Once those equivalent API calls are being done from ASO instead of CAPZ, CAPZ's metrics will lose that data. However, that lost data may no longer be meaningful to CAPZ since Azure API calls will be happening out-of-band with CAPZ reconciliations. ASO does measure how many Azure API calls are made and how long each takes and exposes those in the Prometheus format, along with other metrics as documented at https://azure.github.io/azure-service-operator/guide/metrics/. However, users relying on the current structure of metrics will likely experience breaking changes.
174174
175175
Dependency on ASO may also negatively impact the time between new features being available in Azure and CAPZ being able to take advantage of them. Although ASO can publish updated CRDs with new Azure API versions independently from the SDK, a few factors contribute to the likelihood of ASO lagging a bit behind the SDK:
176176
- The Azure SDK's code generation pipeline is currently more mature than ASO's.
@@ -217,7 +217,7 @@ All of the resource types above are always assumed to be owned by CAPZ except fo
217217

218218
After that, a CAPZ upgrade may include a bump in the API version used for some ASO resource types. ASO will handle those changes transparently from CAPZ's perspective with its own conversion webhooks. The limit on the size of Kubernetes resources will likely force ASO to remove certain API versions of resources in CRDs in the future. When a version is deprecated in ASO, CAPZ will have to upgrade its use of the resource to a newer API version.
219219

220-
Separate from API version upgrades of resources, upgrades to ASO itself may be included in new versions of CAPZ. [ASO's documentation](https://azure.github.io/azure-service-operator/introduction/upgrading/) recommends upgrading using the same mechanism used to initially install. At the same time when CAPZ installs ASO, it will also upgrade an existing ASO it installed if necessary.
220+
Separate from API version upgrades of resources, upgrades to ASO itself may be included in new versions of CAPZ. [ASO's documentation](https://azure.github.io/azure-service-operator/guide/upgrading/) recommends upgrading using the same mechanism used to initially install. At the same time when CAPZ installs ASO, it will also upgrade an existing ASO it installed if necessary.
221221

222222
## Additional Details
223223

0 commit comments

Comments
 (0)