|
| 1 | +# API Lifecycle |
| 2 | + |
| 3 | +In only the rarest of cases will the first version of a CRD be also its final version. Instead usually |
| 4 | +CRDs evolve over time and Kubernetes has strong, though sometimes hard to use, support for managing |
| 5 | +different versions of CRDs and their resources. |
| 6 | + |
| 7 | +To understand how CRDs work in the context of the Sync Agent, it's important to first get familiar |
| 8 | +with the [regular Kubernetes behaviour](https://kubernetes.io/docs/tasks/extend-kubernetes/custom-resources/custom-resource-definition-versioning/) |
| 9 | +regarding CRD versioning. |
| 10 | + |
| 11 | +## Basics |
| 12 | + |
| 13 | +The Sync Agent will, whenever a published CRD changes (this can also happen when the projection rules |
| 14 | +inside a `PublishedResource` are updated), create a new `APIResourceSchema` (ARS) in kcp. The name and |
| 15 | +version of this ARS are based on a hash of the projected CRD. Undoing a change would make the agent |
| 16 | +re-use the previously created ARS (ARS are immutable). |
| 17 | + |
| 18 | +After every reconciliation, the list of latest resource schemas in the configured `APIExport` is |
| 19 | +updated. For this the agent will find all ARS that belong to it (based on an ownership label) and |
| 20 | +then merge them into the `APIExport`. Resource schemas for unknown group/resource combinations are |
| 21 | +left untouched, so admins are free to add additional resource schemas to an `APIExport`. |
| 22 | + |
| 23 | +This means that every change to a CRD on the service cluster is applied practically immediately in |
| 24 | +each workspace that consumes the `APIExport`. Administrators are wise to act carefully when working |
| 25 | +with their CRDs on their service cluster. Sometimes it can make sense to turn-off the agent before |
| 26 | +testing new CRDs, even though this will temporarily suspend the synchronization. |
| 27 | + |
| 28 | +## Single-Version CRDs |
| 29 | + |
| 30 | +A very common scenario is to only ever have a single version inside each CRD and keeping this version |
| 31 | +perpetually backwards-compatible. As long as all consumers are aware that certain fields might not |
| 32 | +be set yet in older objects, this scheme works out generally fine. |
| 33 | + |
| 34 | +The agent will handle this scenario just fine by itself. Whenever a CRD is updated, it will reflect |
| 35 | +those changes back into a new `APIResourceSchema` and update the `APIExport`, making the changes |
| 36 | +immediately available to all consumers. Since the agent itself doesn't much care for the contents of |
| 37 | +objects, it itself is not affected by any structural changes in CRDs, as long as it is able to apply |
| 38 | +them on the underlying Kubernetes cluster. |
| 39 | + |
| 40 | +## Multi-Version CRDs |
| 41 | + |
| 42 | +Having multiple versions in a single CRD is immediately much more work, since in Kubernetes all |
| 43 | +versions of a CRD must be _losslessly_ convertible to every other version. Without CEL expressions |
| 44 | +or a dedicated conversion webhook this is practically impossible to achieve. |
| 45 | + |
| 46 | +At the moment kcp does not support CEL-based conversions, and there is no support for configuring a |
| 47 | +conversion webhook inside the Sync Agent either. This is because such a webhook would need to run |
| 48 | +very close to the kcp shards and it's simply out of scope for such a component to be described and |
| 49 | +deployed by the Sync Agent, let alone a trust nightmare for the kcp operators who would have to run |
| 50 | +foreign webhooks on their cluster. |
| 51 | + |
| 52 | +Since both conversion mechanisms are not usable in the current state of kcp and the Sync Agent, |
| 53 | +having multiple versions in a CRD can be difficult to manage. |
| 54 | + |
| 55 | +Generally the Sync Agent itself does not care much about the schemas of each CRD version or the |
| 56 | +convertibility between them. The synchronization works by using unstructured clients to the storage |
| 57 | +versison of the CRD on both sides (in kcp and on the service cluster). Which version is the storage |
| 58 | +version is up to the CRD author. |
| 59 | + |
| 60 | +When publishing multiple versions of a CRD |
| 61 | + |
| 62 | +* only those versions marked as `served` can be picked and |
| 63 | +* if no `storage` version is picked, the latest (highest) version will be chosen automatically as |
| 64 | + the storage version in kcp. |
0 commit comments