diff --git a/content/master/composition/compositions.md b/content/master/composition/compositions.md index e933ad2ec..ec67327b3 100644 --- a/content/master/composition/compositions.md +++ b/content/master/composition/compositions.md @@ -58,7 +58,7 @@ A [composite resource]({{}}) or XR is a custom API. You use two Crossplane types to create a new custom API: * A [Composite Resource Definition]({{}}) - (XRD) - Defines the XR's schema. + (XRD) - Defines the XR's schema. * A Composition - This page. Configures how the XR creates other resources. {{}} @@ -659,11 +659,11 @@ A function can change: * The `status` of the composite resource. * The `metadata` and `spec` of any composed resource. -A function can also change the connection details and readiness of the composite -resource. A function indicates that the composite resource is ready by telling -Crossplane whether its composed resources are ready. When the function pipeline -tells Crossplane that all composed resources are ready, Crossplane marks the -composite resource as ready. +A function can also change the readiness of the composite resource. A function +indicates that the composite resource is ready by telling Crossplane whether its +composed resources are ready. When the function pipeline tells Crossplane that +all composed resources are ready, Crossplane marks the composite resource as +ready. A function can't change: @@ -812,20 +812,20 @@ spec: kind: Script script: | from crossplane.function import request - + def compose(req, rsp): observed_xr = req.observed.composite.resource - + # Access the required ConfigMap using the helper function config_map = request.get_required_resource(req, "app-config") - + if not config_map: # Fallback image if ConfigMap not found image = "nginx:latest" else: # Read image from ConfigMap data image = config_map.get("data", {}).get("image", "nginx:latest") - + # Create deployment with the configured image rsp.desired.resources["deployment"].resource.update({ "apiVersion": "apps/v1", @@ -877,33 +877,33 @@ spec: kind: Script script: | from crossplane.function import request, response - + def compose(req, rsp): observed_xr = req.observed.composite.resource - + # Always request the ConfigMap to ensure stable requirements config_name = observed_xr["spec"].get("configName", "default-config") namespace = observed_xr["metadata"].get("namespace", "default") - + response.require_resources( - rsp, + rsp, name="dynamic-config", api_version="v1", kind="ConfigMap", match_name=config_name, namespace=namespace ) - + # Check if we have the required ConfigMap config_map = request.get_required_resource(req, "dynamic-config") - + if not config_map: # ConfigMap not found yet - Crossplane will call us again return - + # ConfigMap found - use the image data to create deployment image = config_map.get("data", {}).get("image", "nginx:latest") - + rsp.desired.resources["deployment"].resource.update({ "apiVersion": "apps/v1", "kind": "Deployment", @@ -955,12 +955,12 @@ context. ### Function response cache {{}} -Function response caching is an alpha feature. Enable it by setting the +Function response caching is an alpha feature. Enable it by setting the `--enable-function-response-cache` feature flag. {{< /hint >}} -Crossplane can cache function responses to improve performance by reducing -repeated function calls. When enabled, Crossplane caches responses from +Crossplane can cache function responses to improve performance by reducing +repeated function calls. When enabled, Crossplane caches responses from composition functions that include a time to live (TTL) value. The cache works by: @@ -981,5 +981,5 @@ Control the cache behavior with these Crossplane pod arguments: - `--xfn-cache-max-ttl` - Maximum cache duration (default: 24 hours) The cache stores files in the `/cache/xfn/` directory in the Crossplane pod. -For better performance, consider using an in-memory cache by mounting an +For better performance, consider using an in-memory cache by mounting an emptyDir volume with `medium: Memory`. diff --git a/content/master/guides/connection-details-composition.md b/content/master/guides/connection-details-composition.md index 12ac3f973..c01fd2ff9 100644 --- a/content/master/guides/connection-details-composition.md +++ b/content/master/guides/connection-details-composition.md @@ -171,6 +171,34 @@ from the tabs below. {{< tabs >}} +{{< tab "YAML" >}} + +Create this composition function to install YAML support: + +```yaml +apiVersion: pkg.crossplane.io/v1 +kind: Function +metadata: + name: function-patch-and-transform +spec: + package: xpkg.crossplane.io/crossplane-contrib/function-patch-and-transform:v0.10.0 +``` + +Save the function as `fn.yaml` and apply it: + +```shell +kubectl apply -f fn.yaml +``` + +Check that Crossplane installed the function: + +```shell {copy-lines="1"} +kubectl get -f fn.yaml +NAME INSTALLED HEALTHY PACKAGE AGE +function-patch-and-transform True True xpkg.crossplane.io/crossplane-contrib/function-patch-and-transform:v0.10.0 8s +``` +{{< /tab >}} + {{< tab "Templated YAML" >}} Templated YAML is a good choice if you're used to writing [Helm charts](https://helm.sh). @@ -306,6 +334,124 @@ exposes their credentials as the composite resource's connection details `Secret {{< tabs >}} +{{< tab "YAML" >}} + +```yaml {label="comp-pt"} +apiVersion: apiextensions.crossplane.io/v1 +kind: Composition +metadata: + name: useraccesskeys-patch-and-transform +spec: + compositeTypeRef: + apiVersion: example.org/v1alpha1 + kind: UserAccessKey + mode: Pipeline + pipeline: + - step: patch-and-transform + functionRef: + name: function-patch-and-transform + input: + apiVersion: pt.fn.crossplane.io/v1beta1 + kind: Resources + writeConnectionSecretToRef: + patches: + - type: FromCompositeFieldPath + fromFieldPath: spec.writeConnectionSecretToRef.name + toFieldPath: name + resources: + - name: user + base: + apiVersion: iam.aws.m.upbound.io/v1beta1 + kind: User + spec: + forProvider: {} + - name: accesskey-0 + base: + apiVersion: iam.aws.m.upbound.io/v1beta1 + kind: AccessKey + spec: + forProvider: + userSelector: + matchControllerRef: true + writeConnectionSecretToRef: + name: accesskey-secret-0 + connectionDetails: + - name: user-0 + type: FromConnectionSecretKey + fromConnectionSecretKey: username + - name: password-0 + type: FromConnectionSecretKey + fromConnectionSecretKey: password + patches: + - type: FromCompositeFieldPath + fromFieldPath: metadata.name + toFieldPath: spec.writeConnectionSecretToRef.name + transforms: + - type: string + string: + type: Format + fmt: "%s-accesskey-secret-0" + - name: accesskey-1 + base: + apiVersion: iam.aws.m.upbound.io/v1beta1 + kind: AccessKey + spec: + forProvider: + userSelector: + matchControllerRef: true + writeConnectionSecretToRef: + name: accesskey-secret-1 + connectionDetails: + - name: user-1 + type: FromConnectionSecretKey + fromConnectionSecretKey: username + - name: password-1 + type: FromConnectionSecretKey + fromConnectionSecretKey: password + patches: + - type: FromCompositeFieldPath + fromFieldPath: metadata.name + toFieldPath: spec.writeConnectionSecretToRef.name + transforms: + - type: string + string: + type: Format + fmt: "%s-accesskey-secret-1" + - step: ready + functionRef: + name: function-auto-ready +``` + + + +**How this Composition exposes connection details:** + +* Each composed {{}}AccessKey{{}} has + {{}}writeConnectionSecretToRef{{}} set. This + tells each `AccessKey` to write its credentials to an individual `Secret`. +* Each {{}}AccessKey{{}} defines + {{}}connectionDetails{{}} that specify which + keys from its connection secret should be included in the XR's + aggregated connection details secret. +* The {{}}name{{}} field in each connection + detail entry sets the key name in the aggregated secret. +* The {{}}fromConnectionSecretKey{{}} field + specifies which key to read from the composed resource's individual connection secret. +* The function's input includes a top-level + {{}}writeConnectionSecretToRef{{}} section + that allows you to specify where to create the connection secret. +* The {{}}patches{{}} in the + `writeConnectionSecretToRef` section read the secret name from the XR's + `.spec.writeConnectionSecretToRef.name` field. +* The function automatically includes a `Secret` object in the XR's composed + resources that represents the XR's aggregated connection details. +* You don't need to create or compose this `Secret` yourself, it's done + automatically for you. + + + +{{< /tab >}} + {{< tab "Templated YAML" >}} ```yaml {label="comp-gotmpl"} @@ -754,22 +900,27 @@ kubectl get secret -n default -l crossplane.io/composite=my-keys -o jsonpath='{. ## Understanding how composing connection details works -The basic steps to expose connection details for a composite resource are: +You can expose connection details for a composite resource using two approaches: + +### Manual composition (most functions) + +With functions like `function-go-templating`, `function-python`, `function-kcl`, and others, +manually compose a `Secret` resource: 1. **Compose resources**: Create composed resources as usual in your composition, such as IAM `User` and `AccessKeys`. These resources expose their connection details in a `Secret`. 2. **Set `writeConnectionSecretToRef`**: Each composed resource that should have - connection details stored in their own individual `Secret` should have their - `writeConnectionSecretToRef` set in the composition. + connection details stored should have their `writeConnectionSecretToRef` set + in the composition. 3. **Observed connection details**: Crossplane observes the actual state of each composed resource, including its connection details, and makes this data - available when it runs the function. + available when the function runs. -4. **Compose the combined `Secret`**: With the observed connection details of - your composed resources in hand, compose a `Secret` resource that combines +4. **Compose the combined `Secret`**: Compose a `Secret` resource that reads + from the observed connection details of your composed resources and combines the important connection details you want to expose for the XR. Consider allowing the consumer of the XR to specify the name they want this secret to have. @@ -779,6 +930,30 @@ The basic steps to expose connection details for a composite resource are: Composition should handle these cases by checking if resources and their connection details exist before accessing them. +### Automatic aggregation (`function-patch-and-transform`) + +`function-patch-and-transform` automatically observes connection details from +composed resources and creates the aggregated connection secret to +maintain backward compatibility with v1 behavior. + +You don't need to manually compose a `Secret` resource yourself. + +1. **Compose resources**: Create composed resources as usual in your + composition, such as IAM `User` and `AccessKeys`. These resources expose + their connection details in a `Secret`. + +2. **Set `writeConnectionSecretToRef`**: Each composed resource that should have + connection details stored should have their `writeConnectionSecretToRef` set + in the composition. + +3. **Define `connectionDetails`**: On each composed resource, define which + connection secret keys to include in the aggregated secret using the + `connectionDetails` field. + +4. **Configure the `Secret`**: Add a `writeConnectionSecretToRef` section in the + function's `input` to set the aggregated secret's name and namespace as + needed. Use patches to configure these values using data from the XR if + needed. ## Troubleshooting @@ -789,7 +964,8 @@ The basic steps to expose connection details for a composite resource are: * Composed resources don't have `writeConnectionSecretToRef` set * Composed resources aren't ready/healthy yet -* Not handling initial nil state correctly in the Composition +* (`function-patch-and-transform`) Missing `connectionDetails` field on composed resources +* (Manual composition) Not handling initial nil state correctly in the Composition @@ -800,7 +976,10 @@ The basic steps to expose connection details for a composite resource are: * Wait for composed resources to become ready (`kubectl get` and check the `READY` column) * Verify the composed resource is actually producing connection details: `kubectl get secret -o yaml` -* Add nil/empty checks in your Composition logic to safeguard access to data that may not exist yet +* (`function-patch-and-transform`) Ensure each composed resource has a + `connectionDetails` section that maps the desired secret keys +* (Manual composition) Add nil/empty checks in your Composition logic to + safeguard access to data that may not exist yet @@ -810,11 +989,14 @@ The basic steps to expose connection details for a composite resource are: **Cause:** Not encoding the combined secret data correctly in your Composition logic -**Solution:** Ensure that your connection details data is correctly encoded for -the function you're using. For example, `function-python` requires you to -convert connection details to base64-encoded strings, while connection details -in `function-go-templating` and `function-kcl` are already encoded this way and -require no conversion logic. +**Solution:** This only applies to manual composition approaches. Ensure that +your connection details data is correctly encoded for the function you're using. +For example, `function-python` requires you to convert connection details to +base64-encoded strings, while connection details in `function-go-templating` and +`function-kcl` are already encoded this way and require no conversion logic. + +`function-patch-and-transform` handles encoding when automatically creating the +composed connection secret. diff --git a/content/master/guides/function-patch-and-transform.md b/content/master/guides/function-patch-and-transform.md index 4ca1b43b8..712a22141 100644 --- a/content/master/guides/function-patch-and-transform.md +++ b/content/master/guides/function-patch-and-transform.md @@ -670,7 +670,7 @@ For example, after Crossplane creates a new managed resource, take the value `hostedZoneID` and store it in the composite resource's status. {{< hint "important" >}} -To patch to composite resource status fields, you must first define the custom +To patch to composite resource status fields, you must first define the custom status fields in the CompositeResourceDefinition. {{< /hint >}} @@ -1738,7 +1738,129 @@ patches: ## Composite resource connection details -Function patch and Transform must define the specific secret keys a resource +Function Patch and Transform automatically aggregates connection details from +composed resources. Unlike other composition functions, Function Patch and +Transform can't access observed connection details to enable manually composing a +`Secret` resource. Instead, it provides built-in automatic aggregation. + +To expose connection details for a composite resource: + +1. Set `writeConnectionSecretToRef` on each composed resource that produces + connection details +2. Define `connectionDetails` on each resource to specify which secret keys to + include in the aggregated secret +3. Configure where to write the aggregated connection details secret (see + [options below](#setting-the-connection-secret-name-and-namespace)) + +The function automatically creates a `Secret` composed resource containing all +the aggregated connection details. + +{{}} +For a complete example of connection details aggregation with Function Patch and +Transform, see the [Connection Details Composition guide]({{}}). + +If you need to transform connection details or manually compose a `Secret` +resource with more complex logic, use other functions like +`function-go-templating`, `function-python`, or `function-kcl` instead. Those +functions have access to observed connection details, which Function Patch and +Transform doesn't provide. +{{}} + + +### v1 vs v2 behavior + + +This function handles composite resource connection details differently +depending on if the XR is Crossplane `v1` or `v2` style. + +* `v1`: The function pipeline returns connection details and + Crossplane creates a separate connection secret for the XR/claim. +* `v2`: This function automatically composes a `Secret` containing the connection + details and includes it along with the XR's other composed resources. + +### Setting the connection secret name and namespace + +The function determines where to write the aggregated connection details secret +in this priority order: + +#### Composite resource reference + +If you have included a `spec.writeConnectionSecretToRef` field in your XR's +schema and the XR has this field set, the function uses its values to +configure the name and namespace of the connection details secret. + +```yaml +apiVersion: example.org/v1alpha1 +kind: UserAccessKey +metadata: + namespace: default + name: my-keys +spec: + writeConnectionSecretToRef: + name: my-keys-connection-details +``` + +You don't need to configure anything else in the Composition. + +{{}} +The XRD must include `spec.writeConnectionSecretToRef` in its schema for users +to set this field. +{{}} + +#### Function input + +You can also configure the secret's name and namespace directly in the +Composition's function input using the `writeConnectionSecretToRef` field. This +field supports both static values and patches. + +Use patches to read values from the XR: + +```yaml {label="input-patches"} +apiVersion: pt.fn.crossplane.io/v1beta1 +kind: Resources +writeConnectionSecretToRef: + patches: + - type: FromCompositeFieldPath + fromFieldPath: spec.connection.name + toFieldPath: name + - type: FromCompositeFieldPath + fromFieldPath: spec.team + toFieldPath: namespace +resources: + # ... composed resources with connectionDetails +``` + +Or use static values: + +```yaml {label="input-static"} +apiVersion: pt.fn.crossplane.io/v1beta1 +kind: Resources +writeConnectionSecretToRef: + name: fixed-secret-name + namespace: fixed-namespace +resources: + # ... composed resources with connectionDetails +``` + +This approach gives you full control and is useful for Cluster-scoped XRs where +you need to explicitly set the namespace, or when you want to transform the +secret name. + +#### Automatically generated + +If the function can't determine the secret's name from the XR reference or +function input, it automatically generates a name based on the XR's name using +the format `-connection`. + +For namespaced XRs, the function creates the secret in the same namespace as the XR. + +For Cluster-scoped XRs, you must use either the XR reference or function input +approach to specify a namespace. Automatic generation doesn't work for +Cluster-scoped XRs because the function can't determine which namespace to use. + +### Connection detail types + +Function Patch and Transform must define the specific secret keys a composed resource creates with the `connectionDetails` object. {{}} @@ -1830,13 +1952,12 @@ myStaticSecret: 18 bytes ``` {{}} -The CompositeResourceDefinition can also limit which keys Crossplane stores from -the composite resources. +The `CompositeResourceDefinition` can also limit which keys Crossplane stores +for `v1` composite resources. By default an XRD writes all secret keys listed in the composed resources `connectionDetails` to the combined secret object. - For more information on connection secrets read about [managed resources]({{}}). {{}} diff --git a/content/master/whats-new/_index.md b/content/master/whats-new/_index.md index 2f2d803d2..224c0234a 100644 --- a/content/master/whats-new/_index.md +++ b/content/master/whats-new/_index.md @@ -171,9 +171,9 @@ deprecate and remove cluster scoped MRs at a future date. Read more about Crossplane v2's [backward compatibility](#backward-compatibility). {{}} -Crossplane v2 also introduces -[managed resource definitions]({{}}) -for selective activation of provider resources, reducing cluster overhead by +Crossplane v2 also introduces +[managed resource definitions]({{}}) +for selective activation of provider resources, reducing cluster overhead by installing only the managed resources you actually need. ## Compose any resource @@ -250,7 +250,7 @@ spec: Operations support three modes: * **Operation** - Run once to completion -* **CronOperation** - Run on a scheduled basis +* **CronOperation** - Run on a scheduled basis * **WatchOperation** - Run when resources change Operations can read existing resources and optionally change them. This enables @@ -268,6 +268,7 @@ Crossplane v2 makes the following breaking changes: * It removes native patch and transform composition. * It removes the `ControllerConfig` type. * It removes support for external secret stores. +* It removes composite resource connection details support. * It removes the default registry for Crossplane Packages. Crossplane deprecated native patch and transform composition in Crossplane @@ -279,6 +280,10 @@ Crossplane deprecated the `ControllerConfig` type in v1.11. It's replaced by the Crossplane added external secret stores in v1.7. External secret stores have remained in alpha for over two years and are now unmaintained. +Composite resources no longer have native connection details support. You +can recreate this feature by composing your own connection details `Secret` +as described in the [connection details composition guide]({{}}). + Crossplane v2 drops the `--registry` flag that allowed users to specify a default registry value and now requires users to always specify a fully qualified URL when installing packages, both directly via `spec.package` and indirectly as dependencies. @@ -299,9 +304,9 @@ Run `kubectl get pkg` to look for any packages that aren't fully qualified, then update or rebuild any Packages to use fully qualified images as needed. {{}} -Crossplane v2 supports legacy v1-style XRs and MRs. Most users can upgrade from +Crossplane v2 supports legacy v1-style XRs and MRs. Most users can upgrade from v1.x to Crossplane v2 without breaking changes. Existing Compositions require minor updates to work with Crossplane v2 -style XRs and MRs. Follow the [Crossplane v2 upgrade guide]({{}}) +style XRs and MRs. Follow the [Crossplane v2 upgrade guide]({{}}) for step-by-step migration instructions.