-
Notifications
You must be signed in to change notification settings - Fork 229
Add comprehensive documentation for resource extension points #5004
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from all commits
27da1cb
eee496c
5a324d2
2e6ab43
4a619aa
c04a5c3
df58f62
3583cf2
e06e9e9
c8af1ad
acb7acd
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,75 @@ | ||
| --- | ||
| title: Extension Points | ||
| linktitle: Extension Points | ||
| weight: 100 | ||
| menu: | ||
| main: | ||
| parent: Contributing | ||
| description: "How to extend Azure Service Operator v2 resources with custom behavior" | ||
| --- | ||
|
|
||
| Azure Service Operator v2 provides several extension points that allow customization of resource behavior. These extension points enable contributors to compensate for variation in the behavior of resources by adding custom logic at various stages of the resource lifecycle. | ||
|
|
||
| ## Overview | ||
|
|
||
| Extension points are Go interfaces defined in `v2/pkg/genruntime/extensions/` that resources can implement to customize their behavior. When a resource implements an extension interface, the controller will invoke the custom logic at the appropriate time during reconciliation. | ||
|
|
||
| ### Extension Implementation Pattern | ||
|
|
||
| Extensions are typically implemented in resource-specific files under `v2/api/<service>/customizations/<resource>_extensions.go`. The general pattern is: | ||
|
|
||
| 1. Declare that your extension type implements the interface: | ||
| ```go | ||
| var _ extensions.ARMResourceModifier = &MyResourceExtension{} | ||
| ``` | ||
|
|
||
| 2. Implement the required method(s) of the interface | ||
|
|
||
| 3. The controller automatically detects and uses the extension through a type-check at the appropriate time | ||
|
|
||
| ### Available Extension Points | ||
|
|
||
| The following extension points are available for customizing resource behavior: | ||
|
|
||
| | Extension Point | Purpose | When Invoked | | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. minor, style: maybe format this table to be more human readable in the raw markdown? |
||
| |-----------------|---------|--------------| | ||
| | [ARMResourceModifier]({{< relref "arm-resource-modifier" >}}) | Modify the ARM payload before sending to Azure | Just before PUT/PATCH to ARM | | ||
| | [Deleter]({{< relref "deleter" >}}) | Customize resource deletion behavior | When resource is being deleted | | ||
| | [ErrorClassifier]({{< relref "error-classifier" >}}) | Classify ARM errors as retryable or fatal | When ARM returns an error | | ||
| | [Importer]({{< relref "importer" >}}) | Customize resource import behavior | During `asoctl import` operations | | ||
| | [KubernetesSecretExporter]({{< relref "kubernetes-secret-exporter" >}}) | Export secrets to Kubernetes | After successful reconciliation | | ||
| | [PostReconciliationChecker]({{< relref "post-reconciliation-checker" >}}) | Perform post-reconciliation validation | After ARM reconciliation succeeds | | ||
| | [PreReconciliationChecker]({{< relref "pre-reconciliation-checker" >}}) | Validate before reconciling | Before sending requests to ARM | | ||
| | [PreReconciliationOwnerChecker]({{< relref "pre-reconciliation-owner-checker" >}}) | Validate owner state before reconciling | Before any ARM operations (including GET) | | ||
| | [SuccessfulCreationHandler]({{< relref "successful-creation-handler" >}}) | Handle successful resource creation | After initial resource creation | | ||
|
|
||
| ## When to Use Extensions | ||
|
|
||
| Extensions should be used when: | ||
|
|
||
| - The generated code doesn't handle a specific Azure resource quirk | ||
| - Additional validation or logic is needed before/after ARM operations | ||
| - Custom error handling is required for specific scenarios | ||
| - Resources need special handling during creation, deletion, or import | ||
| - Secrets or configuration need custom export logic | ||
|
|
||
| Extensions should **not** be used for: | ||
|
|
||
| - Changes that could be made to the generator itself | ||
| - Logic that applies to all resources (consider modifying the controller instead) | ||
| - Working around bugs in the generator (fix the generator instead) | ||
|
|
||
| ## Development Guidelines | ||
|
|
||
| 1. **Keep extensions minimal**: Only add logic that cannot be handled by the generator | ||
| 2. **Document thoroughly**: Explain why the extension is needed | ||
| 3. **Type assert hub versions**: Include hub type assertions to catch breaking changes | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. May want an example? |
||
| 4. **Handle errors gracefully**: Return appropriate error types and messages | ||
| 5. **Test thoroughly**: Add unit tests for extension logic | ||
| 6. **Call next**: Most extensions use a chain pattern - remember to call the `next` function | ||
|
|
||
| ## Related Resources | ||
|
|
||
| - [Adding a new code-generated resource]({{< relref "../add-a-new-code-generated-resource" >}}) | ||
| - [Generator overview]({{< relref "../generator-overview" >}}) | ||
| - [Testing]({{< relref "../testing" >}}) | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,243 @@ | ||
| --- | ||
| title: ARMResourceModifier | ||
| linktitle: ARMResourceModifier | ||
| weight: 10 | ||
| --- | ||
|
|
||
| ## Description | ||
|
|
||
| `ARMResourceModifier` allows resources to modify the payload that will be sent to Azure Resource Manager (ARM) immediately before it is transmitted. This extension point is invoked after the standard resource conversion and validation, but before the HTTP request is made to ARM. | ||
|
|
||
| The interface is invoked during PUT and PATCH operations to ARM, giving the resource an opportunity to make last-minute adjustments to the ARM payload based on runtime conditions, Azure state, or complex business logic that cannot be expressed in the generated code. | ||
|
|
||
| ## Interface Definition | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Wondering if we should just link to the code here? Or directly embed the code via link (does Hugo support that? probably not). Calling the exact structure + what the parameters mean out here just seems like writing an interface/function comment and I'd sorta rather have a single place where we document it well (ideally in the code) and then reference that place from the docs?
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. We'd still keep the "when to use", motivation, examples, etc here. |
||
|
|
||
| ```go | ||
| type ARMResourceModifier interface { | ||
| ModifyARMResource( | ||
| ctx context.Context, | ||
| armClient *genericarmclient.GenericClient, | ||
| armObj genruntime.ARMResource, | ||
| obj genruntime.ARMMetaObject, | ||
| kubeClient kubeclient.Client, | ||
| resolver *resolver.Resolver, | ||
| log logr.Logger, | ||
| ) (genruntime.ARMResource, error) | ||
| } | ||
| ``` | ||
|
|
||
| **Parameters:** | ||
| - `ctx`: The current operation context | ||
| - `armClient`: Client for making additional ARM API calls if needed | ||
| - `armObj`: The ARM resource representation about to be sent to Azure | ||
| - `obj`: The Kubernetes resource being reconciled | ||
| - `kubeClient`: Client for accessing the Kubernetes cluster | ||
| - `resolver`: Helper for resolving resource references | ||
| - `log`: Logger for the current operation | ||
|
|
||
| **Returns:** | ||
| - Modified `genruntime.ARMResource` that will be sent to ARM | ||
| - Error if modification fails (will block the ARM request) | ||
|
|
||
| ## Motivation | ||
|
|
||
| The `ARMResourceModifier` extension exists to handle cases where: | ||
|
|
||
| 1. **Azure resource state affects the payload**: Some Azure resources require different payloads based on their current state in Azure (e.g., creation vs. update) | ||
|
|
||
| 2. **Complex conditional logic**: Business logic that depends on multiple factors and cannot be expressed declaratively in the resource schema | ||
|
|
||
| 3. **Azure-specific quirks**: Handling special cases or undocumented Azure behavior that varies by resource type | ||
|
|
||
| 4. **Dynamic payload construction**: Building parts of the payload at runtime based on information retrieved from Azure or Kubernetes | ||
|
|
||
| 5. **Soft-delete scenarios**: Resources with soft-delete capabilities (like Key Vault) may need special handling to recover or purge existing resources | ||
|
|
||
| ## When to Use | ||
|
|
||
| Implement `ARMResourceModifier` when: | ||
|
|
||
| - ✅ The resource has different creation modes or requires conditional field population | ||
| - ✅ You need to query Azure or Kubernetes state to determine the correct payload | ||
| - ✅ The resource has soft-delete and requires recovery or purge logic | ||
| - ✅ Child resources need to be included in parent payloads (e.g., VNET subnets) | ||
| - ✅ Field values must be computed at reconciliation time based on external state | ||
|
|
||
| Do **not** use `ARMResourceModifier` when: | ||
|
|
||
| - ❌ The logic could be handled by field defaults or validation | ||
| - ❌ The change should apply to all resources (modify the generator instead) | ||
| - ❌ Simple field transformations (use conversion functions) | ||
| - ❌ The issue is a generator bug (fix the generator) | ||
|
|
||
| ## Example: Key Vault CreateMode Handling | ||
|
|
||
| The Key Vault resource uses `ARMResourceModifier` to handle different creation modes based on whether a soft-deleted vault exists: | ||
|
|
||
| ```go | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Another place where we should maybe link to the example code rather than duplicate it here? |
||
| var _ extensions.ARMResourceModifier = &VaultExtension{} | ||
|
|
||
| func (ex *VaultExtension) ModifyARMResource( | ||
| ctx context.Context, | ||
| armClient *genericarmclient.GenericClient, | ||
| armObj genruntime.ARMResource, | ||
| obj genruntime.ARMMetaObject, | ||
| kubeClient kubeclient.Client, | ||
| resolver *resolver.Resolver, | ||
| log logr.Logger, | ||
| ) (genruntime.ARMResource, error) { | ||
| // Type assert to the specific resource type | ||
| kv, ok := obj.(*keyvault.Vault) | ||
| if !ok { | ||
| return nil, eris.Errorf( | ||
| "Cannot run VaultExtension.ModifyARMResource() with unexpected resource type %T", | ||
| obj) | ||
| } | ||
|
|
||
| // Type assert hub version to catch breaking changes | ||
| var _ conversion.Hub = kv | ||
|
|
||
| // Exit early if no special handling needed | ||
| if kv.Spec.Properties == nil || kv.Spec.Properties.CreateMode == nil { | ||
| return armObj, nil | ||
| } | ||
|
|
||
| // Get resource context | ||
| id, err := ex.getOwner(ctx, kv, resolver) | ||
| if err != nil { | ||
| return nil, eris.Wrap(err, "failed to get and parse resource ID from KeyVault owner") | ||
| } | ||
|
|
||
| // Create Azure SDK client to check for soft-deleted vaults | ||
| vc, err := armkeyvault.NewVaultsClient(id.SubscriptionID, armClient.Creds(), armClient.ClientOptions()) | ||
| if err != nil { | ||
| return nil, eris.Wrap(err, "failed to create new VaultsClient") | ||
| } | ||
|
|
||
| // Determine the correct create mode based on Azure state | ||
| createMode := *kv.Spec.Properties.CreateMode | ||
| if createMode == CreateMode_CreateOrRecover { | ||
| // Check if soft-deleted vault exists and adjust createMode accordingly | ||
| createMode, err = ex.handleCreateOrRecover(ctx, kv, vc, id, log) | ||
| if err != nil { | ||
| return nil, eris.Wrapf(err, "error checking for existence of soft-deleted KeyVault") | ||
| } | ||
| } | ||
|
|
||
| if createMode == CreateMode_PurgeThenCreate { | ||
| // Purge the soft-deleted vault before creating | ||
| err = ex.handlePurgeThenCreate(ctx, kv, vc, log) | ||
| if err != nil { | ||
| return nil, eris.Wrapf(err, "error purging soft-deleted KeyVault") | ||
| } | ||
| createMode = CreateMode_Default | ||
| } | ||
|
|
||
| // Modify the ARM payload with the determined createMode | ||
| spec := armObj.Spec() | ||
| err = reflecthelpers.SetProperty(spec, "Properties.CreateMode", &createMode) | ||
| if err != nil { | ||
| return nil, eris.Wrapf(err, "error setting CreateMode to %s", createMode) | ||
| } | ||
|
|
||
| return armObj, nil | ||
| } | ||
| ``` | ||
|
|
||
| **Key aspects of this example:** | ||
|
|
||
| 1. **Type assertions**: Both for the resource type and hub version | ||
| 2. **Early exit**: Returns original payload if no modification needed | ||
| 3. **External queries**: Checks Azure for soft-deleted vault state | ||
| 4. **Conditional logic**: Different behavior based on createMode and vault state | ||
| 5. **Payload modification**: Uses reflection to safely modify the ARM payload | ||
| 6. **Error handling**: Returns detailed errors that prevent ARM submission | ||
|
|
||
| ## Common Patterns | ||
|
|
||
| ### Pattern 1: Querying Current Azure State | ||
|
|
||
| ```go | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Here the code snippets make more sense - wondering if we want links to code where they are used too, though? Applies to all of these "patterns" |
||
| func (ex *ResourceExtension) ModifyARMResource(...) (genruntime.ARMResource, error) { | ||
| // Get the resource ID | ||
| resourceID, hasResourceID := genruntime.GetResourceID(obj) | ||
| if !hasResourceID { | ||
| // Not yet claimed, return unmodified | ||
| return armObj, nil | ||
| } | ||
|
|
||
| // Query current state from Azure | ||
| raw := make(map[string]any) | ||
| _, err = armClient.GetByID(ctx, resourceID, apiVersion, &raw) | ||
| if err != nil { | ||
| // Handle NotFound appropriately | ||
| var responseError *azcore.ResponseError | ||
| if eris.As(err, &responseError) && responseError.StatusCode == http.StatusNotFound { | ||
| return armObj, nil | ||
| } | ||
| return nil, err | ||
| } | ||
|
|
||
| // Use Azure state to modify payload | ||
| // ... | ||
|
|
||
| return armObj, nil | ||
| } | ||
| ``` | ||
|
|
||
| ### Pattern 2: Including Child Resources | ||
|
|
||
| ```go | ||
| func (ex *ResourceExtension) ModifyARMResource(...) (genruntime.ARMResource, error) { | ||
| // Get existing child resources from Azure | ||
| children, err := getRawChildCollection(raw, "subnets") | ||
| if err != nil { | ||
| return nil, eris.Wrapf(err, "failed to get child resources") | ||
| } | ||
|
|
||
| // Merge with desired state in payload | ||
| err = setChildCollection(armObj.Spec(), children, "Subnets") | ||
| if err != nil { | ||
| return nil, eris.Wrapf(err, "failed to set child resources") | ||
| } | ||
|
|
||
| return armObj, nil | ||
| } | ||
| ``` | ||
|
|
||
| ### Pattern 3: Conditional Field Population | ||
|
|
||
| ```go | ||
| func (ex *ResourceExtension) ModifyARMResource(...) (genruntime.ARMResource, error) { | ||
| // Determine value based on runtime conditions | ||
| value, err := ex.computeValue(ctx, obj, armClient) | ||
| if err != nil { | ||
| return nil, err | ||
| } | ||
|
|
||
| // Set the computed value on the ARM resource | ||
| spec := armObj.Spec() | ||
| err = reflecthelpers.SetProperty(spec, "Properties.FieldName", value) | ||
| if err != nil { | ||
| return nil, err | ||
| } | ||
|
|
||
| return armObj, nil | ||
| } | ||
| ``` | ||
|
|
||
| ## Testing | ||
|
|
||
| When testing `ARMResourceModifier` extensions: | ||
|
|
||
| 1. **Test all code paths**: Cover each conditional branch | ||
| 2. **Mock ARM responses**: Use envtest with recorded responses | ||
| 3. **Verify payload changes**: Assert that the returned armObj has expected modifications | ||
| 4. **Test error handling**: Ensure errors are properly classified and reported | ||
| 5. **Test edge cases**: No ResourceID, resource not found, nil fields, etc. | ||
|
|
||
| ## Related Extension Points | ||
|
|
||
| - [PreReconciliationChecker]({{< relref "pre-reconciliation-checker" >}}): For validation before ARM operations | ||
| - [PostReconciliationChecker]({{< relref "post-reconciliation-checker" >}}): For validation after ARM operations | ||
| - [ErrorClassifier]({{< relref "error-classifier" >}}): For handling errors from modified requests | ||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I thought lower weight is "higher" on the page. I guess this is the index so this weight is about where this lands compared to othher contributing docs?
So this is at the end?
Just confirming this is correct as it seemed a little surprising to me