Skip to content

Commit e279938

Browse files
c-piuslindnerbyLeelaChacha
authored
docs: Introduce Module Migration Guide (#2452)
* docs: Introduce Module Migration Guide * add module version deletion * refer to New submission pipeline for deletion * stash * text * adapt text * shorten guide 5 * remove iteration review url * Apply suggestions from code review Co-authored-by: Christoph Schwägerl <[email protected]> * add suggestions from review --------- Co-authored-by: Benjamin Lindner <[email protected]> Co-authored-by: Benjamin Lindner <[email protected]> Co-authored-by: Raj <[email protected]>
1 parent 70855e9 commit e279938

File tree

3 files changed

+253
-0
lines changed

3 files changed

+253
-0
lines changed

‎docs/contributor/06-module-migration-concept.md‎

Lines changed: 79 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,79 @@
1+
# New Module Submission and Promotion Concept
2+
3+
The existing module metadata is in channel-based ModuleTemplate custom resources (CRs). The new module metadata sources are version-based ModuleTemplates accompanied by ModuleReleaseMeta CRs. This document describes the target module submission process for module development teams, migrating a Kyma module from the existing module metadata to the new module metadata.
4+
5+
To go directly to the migration procedure, see [New Module Submission and Promotion Process: Migration Guide](./07-module-migration-guide.md).
6+
7+
> [!Tip]
8+
> Before you start the migration, see the custom resource definitions (CRDs) related to the new module metadata and additional information around the changes.
9+
> - [ModuleTemplate](./resources/03-moduletemplate.md)
10+
> - [ModuleReleaseMeta](./resources/05-modulereleasemeta.md)
11+
> - The Architecture Decision Record backing the migration: [#984](https://github.com/kyma-project/community/issues/984)
12+
> - The PowerPoint presentation with an update given in the 2024-11-26 Kyma Iteration Review meeting: [2024-11-26 Update On Module Metadata](https://sap-my.sharepoint.com/:p:/p/c_schwaegerl/EbvSNmRnr3JEjaLoZ__cI9UB9lu5tt0qaly-f7yQO2Gwbw?e=slfiDf) <!-- markdown-link-check-disable-line -->
13+
14+
## The Target Module Submission and Promotion Processes
15+
16+
The following diagram shows the target module submission and promotion process using the Telemetry module. The process consists of the following stages:
17+
18+
1. Submitting a new module version
19+
2. Submitting the channel mapping
20+
3. Promoting ModuleTemplate and ModuleReleaseMeta CRs
21+
4. Deleting a module version
22+
23+
![Target Process](./assets/module-migration.svg)
24+
25+
### 1. Submitting a New Module Version
26+
27+
You submit a new module version by creating a pull request (PR) to the `/kyma/module-manifests` repository. In the PR, you must provide a `module-config.yaml` file under the `/modules/<module-name>/<module-version>` path where `<module-version>` matches the version of the module configured in `module-config.yaml`.
28+
29+
Once the PR is opened, the submission pipeline verifies all the information. For example, the pipeline verifies if the provided `module-config.yaml` is valid (that the FQDN of the module doesn't change), it builds the module using the `modulectl` in `--dry-run` mode, and checks if the version does not exist yet.
30+
31+
Once the PR is merged, the submission pipeline builds and publishes the module using `modulectl` and pushes the generated ModuleTemplate to the `/kyma/kyma-modules` repository. The path of the generated ModuleTemplate is `/<module-name>/moduletemplate-<module-name>-<module-version>.yaml`.
32+
33+
For more information, see the [new submission pipeline](https://github.tools.sap/kyma/test-infra/blob/feature/new-submission-pipeline/ado/new-submission-pipeline-activity.md).
34+
35+
> [!Note]
36+
> The new module version submission process only builds the necessary artifacts and puts them into their respective repositories, namely, the OCI Registry and the `/kyma/kyma-modules` repository. The ModuleTemplate is NOT directly deployed into the KCP landscape.
37+
38+
### 2. Submitting a Channel Mapping
39+
40+
You submit a channel mapping by creating a PR to the `/kyma/module-manifests` repository. In the PR, you must update the `module-releases.yaml` file under the `/modules/<module-name>` path. The `module-releases.yaml` is a simple mapping file defining what version each channel should map to.
41+
42+
Once the PR is opened, the submission pipeline verifies if the mapping is correct. For example, the pipeline verifies that no version downgrade is performed for a channel and that the referenced module version exists.
43+
44+
Once the PR is merged, the submission pipeline generates a ModuleReleaseMeta CR and kustomization, and pushes them to the `/kyma/kyma-modules` repository. The kustomization includes the required ModuleTemplate and the ModuleReleaseMeta CRs.
45+
46+
For more details, see the [new submission pipeline](https://github.tools.sap/kyma/test-infra/blob/feature/new-submission-pipeline/ado/new-submission-pipeline-activity.md).
47+
48+
> [!Note]
49+
> Because, for example, the `dev` channel is only allowed in the `dev` landscape, and the `experimental` channel is allowed in the `dev` and `stage` landscapes, the ModuleReleaseMeta CR and kustomization are generated landscape-specific. It means there is a separate ModuleReleaseMeta CR and kustomization per landscape.
50+
51+
> [!Note]
52+
> The kustomization is extended only with ModuleTemplates for the versions referenced in the `module-releases.yaml`. ModuleTemplates for the not-referenced versions are NOT added. Also, versions are not automatically removed from the kustomization, even if not referenced anymore. This needs to be done manually. For more information, see step 4.
53+
54+
> [!Note]
55+
> The new module version and the channel mapping submission processes cannot be combined in one PR. First, you must submit the new module version. Only then can you update the channel mapping.
56+
57+
### 3. Promoting ModuleTemplates and ModuleReleaseMeta
58+
59+
ArgoCD detects and applies the changes from Step 2. It deploys only the ModuleTemplates and ModuleReleaseMeta CRs relevant to each landscape.
60+
61+
### 4. Deleting a Module Version
62+
63+
To delete an unused module version, create a PR to the `/kyma/module-manifests` repository. In the PR, delete the module version's `module-config.yaml` file under `/modules/<module-name>/<module-version>`.
64+
65+
Once the PR is opened, the submission pipeline checks if the version is still in use by one or more channels. If so, the PR can't be merged.
66+
67+
Once the PR is merged, the submission pipeline deletes the related ModuleTemplate under `/<module-name>/moduletemplate-<module-name>-<module-version>.yaml` in the `/kyma/kyma-modules` repository. In addition, it removes the reference to the module from the kustomization (*).
68+
69+
ArgoCD picks up these changes to `/kyma/kyma-modules` and undeploys the ModuleTemplate from the landscapes.
70+
71+
For more details, see the [new submission pipeline](https://github.tools.sap/kyma/test-infra/blob/feature/new-submission-pipeline/ado/new-submission-pipeline-activity.md).
72+
73+
> [!Note]
74+
> In the deleting a module version process, only the ModuleTemplate in `/kyma/kyma-modules` gets deleted and undeployed from KCP. The artifacts pushed to the OCI registry remain and cannot be overwritten.
75+
76+
## Related Information
77+
78+
- [New Module Submission and Promotion Process: Migration Guide](./07-module-migration-guide.md)
79+
- [Migrating from Kyma CLI to modulectl](https://github.com/kyma-project/modulectl/blob/main/docs/contributor/migration-guide.md)

‎docs/contributor/07-module-migration-guide.md‎

Lines changed: 170 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,170 @@
1+
# New Module Submission and Promotion Process: Migration Guide
2+
3+
To ensure a smooth transition, the submission pipeline, and Lifecycle Manager support **both** the old and new metadata formats. If both are present, Lifecycle Manager prefers the new format. If no new metadata format is provided, it falls back to the old channel-based metadata.
4+
5+
The migration strategy involves replicating the current state with the NEW metadata while keeping the old metadata as a fallback.
6+
7+
## Migration Procedure
8+
9+
1. Submit the existing versions using the NEW approach.
10+
11+
The following example assumes the current module versions are:
12+
13+
- `telemetry-regular` pointing to `1.32.0`
14+
- `telemetry-fast` pointing to `1.34.0`
15+
- `telemetry-experimental` pointing to `1.34.0-experimental`
16+
- `telemetry-dev` pointing to `1.35.0-rc1`
17+
18+
You must re-submit all the above versions using the NEW approach. It means you must submit the following:
19+
20+
- `/modules/telemetry/1.32.0/module-config.yaml`
21+
- `/modules/telemetry/1.34.0/module-config.yaml`
22+
- `/modules/telemetry/1.34.0-experimental/module-config.yaml`
23+
- `/modules/telemetry/1.35.0-rc1/module-config.yaml`
24+
25+
For more information on the necessary changes in the `module-config.yaml` file, see [Migrating from Kyma CLI to `modulectl`](https://github.com/kyma-project/modulectl/blob/main/docs/contributor/migration-guide.md#2-module-configuration-module-configyaml-differences).
26+
27+
Once you submit all the versions, the following ModuleTemplate custom resources (CRs) appear in the `/kyma/kyma-modules` repository:
28+
29+
- `/telemetry/moduletemplate-telemetry-1.32.0.yaml`
30+
- `/telemetry/moduletemplate-telemetry-1.34.0.yaml`
31+
- `/telemetry/moduletemplate-telemetry-1.34.0-experimental.yaml`
32+
- `/telemetry/moduletemplate-telemetry-1.35.0-rc1.yaml`
33+
34+
2. Submit the existing channel mapping using the NEW approach.
35+
36+
Create a `/module-manifests/modules/<module-name>/module-releases.yaml` that replicates the existing channel mapping. For example:
37+
38+
```yaml
39+
channels:
40+
- channel: regular
41+
version: 1.32.0
42+
- channel: fast
43+
version: 1.34.0
44+
- channel: experimental
45+
version: 1.34.0-experimental
46+
- channel: dev
47+
version: 1.35.0-rc1
48+
```
49+
50+
Once you submit the channel mapping, landscape-specific ModuleReleaseMeta CRs are created and kustomizations are updated accordingly in `/kyma/kyma-modules`.
51+
52+
- `/telemetry`
53+
- `/moduletemplate-telemetry-1.32.0.yaml`
54+
- `/moduletemplate-telemetry-1.34.0.yaml`
55+
- `/moduletemplate-telemetry-1.34.0-experimental.yaml`
56+
- `/moduletemplate-telemetry-1.35.0-rc1.yaml`
57+
- `/dev`
58+
- `module-release-meta.yaml` with
59+
- channel `regular` => `1.32.0`
60+
- channel `fast` => `1.34.0`
61+
- channel `experimental` => `1.34.0-experimental`
62+
- channel `dev` => `1.35.0-rc1`
63+
- `/stage`
64+
- `module-release-meta.yaml` with
65+
- channel `regular` => `1.32.0`
66+
- channel `fast` => `1.34.0`
67+
- channel `experimental` => `1.34.0-experimental`
68+
- `/prod`
69+
- `module-release-meta.yaml` with
70+
- channel `regular` => `1.32.0`
71+
- channel `fast` => `1.34.0`
72+
- `/kustomizations`
73+
- `/dev`
74+
- `/kustomization.yaml` with
75+
- ModuleReleaseMeta `../../telemetry/dev/module-release-meta.yaml`
76+
- ModuleTemplate `../../telemetry/moduletemplate-telemetry-1.32.0.yaml`
77+
- ModuleTemplate `../../telemetry/moduletemplate-telemetry-1.34.0.yaml`
78+
- ModuleTemplate `../../telemetry/moduletemplate-telemetry-1.34.0-experimental.yaml`
79+
- ModuleTemplate `../../telemetry/moduletemplate-telemetry-1.35.0-rc1.yaml`
80+
- ... (resources from other modules)
81+
- `/stage`
82+
- `/kustomization.yaml` with
83+
- ModuleReleaseMeta `../../telemetry/stage/module-release-meta.yaml`
84+
- ModuleTemplate `../../telemetry/moduletemplate-telemetry-1.32.0.yaml`
85+
- ModuleTemplate `../../telemetry/moduletemplate-telemetry-1.34.0.yaml`
86+
- ModuleTemplate `../../telemetry/moduletemplate-telemetry-1.34.0-experimental.yaml`
87+
- ... (resources from other modules)
88+
- `/prod`
89+
- `/kustomization.yaml` with
90+
- ModuleReleaseMeta `../../telemetry/stage/module-release-meta.yaml`
91+
- ModuleTemplate `../../telemetry/moduletemplate-telemetry-1.32.0.yaml`
92+
- ModuleTemplate `../../telemetry/moduletemplate-telemetry-1.34.0.yaml`
93+
- ... (resources from other modules)
94+
95+
3. Verify in KCP.
96+
97+
ArgoCD picks up and deploys the changes from step 2. All landscapes have the same channel-version mapping of the module described in OLD and NEW metadata.
98+
99+
Following the Telemetry module example, the following resources exist in KCP:
100+
101+
**New ModuleTemplates**
102+
- ModuleTemplate `kyma-system/telemetry-regular` pointing to `1.32.0`
103+
- ModuleTemplate `kyma-system/telemetry-fast` pointing to `1.34.0`
104+
- ModuleTemplate `kyma-system/telemetry-experimental` pointing to `1.34.0-experimental` (only in DEV and STAGE)
105+
- ModuleTemplate `kyma-system/telemetry-dev` pointing to `1.35.0-rc1` (only in DEV)
106+
- ModuleReleaseMeta `kyma-system/telemetry`
107+
108+
**Old ModuleTemplates**
109+
- ModuleTemplate `kyma-system/telemetry-1.32.0`
110+
- ModuleTemplate `kyma-system/telemetry-1.34.0`
111+
- ModuleTemplate `kyma-system/telemetry-1.34.0-experimental` (only in DEV and STAGE)
112+
- ModuleTemplate `kyma-system/telemetry-1.35.0-rc1` (only in DEV)
113+
114+
As the new module metadata takes precedence, the reconciliation of the module already happens based on the new metadata. Since all versions and channel mappings are the same, no update is performed and all modules stay in the same state as before.
115+
116+
The functionality can further be verified by enabling the module in a test SKR which will install it from scratch using the new metadata.
117+
118+
4. [OPTIONAL] Roll back the new module metadata.
119+
120+
In case of failure, the setup can be reverted to the old approach.
121+
122+
To do so, a PR can be opened to `/kyma/kyma-modules` reverting the submission from 2. ArgoCD then undeploys the new module metadata and KLM falls back to the old module metadata.
123+
124+
> [!Note]
125+
> After rollback, you can still use the old submission pipeline to submit new versions of the module while you're working on a fix.
126+
127+
5. Submit a version upgrade using the old format.
128+
129+
To avoid version downgrades, prepare for failure recovery and submit a version update using the old metadata.
130+
131+
- `/modules/telemetry/regular/module-config.yaml` pointing to `1.34.0` (before `1.32.0`)
132+
133+
Since the new metadata exists, Lifecycle Manager continues to use it. The old metadata is ignored but remains available for rollback.
134+
135+
6. Submit the updated channel mapping with the NEW approach.
136+
137+
After preparing the old metadata to rollback in case of failure, you can continue with the actual version update using the new metadata.
138+
139+
Following the example, submit the `modules/telemetry/module-releases.yaml` file:
140+
141+
```yaml
142+
channels:
143+
- channel: regular
144+
version: 1.34.0 # <= this version is bumped
145+
- channel: fast
146+
version: 1.34.0
147+
- channel: experimental
148+
version: 1.34.0-experimental
149+
- channel: dev
150+
version: 1.35.0-rc1
151+
```
152+
153+
Once you submit the mapping, the resources in `/kyma/kyma-modules` are the same as the resources from step 2, except the `regular` channel pointing to version `1.34.0` in all ModuleReleaseMeta CRs for different landscapes.
154+
155+
7. Verify if the module is updated in KCP.
156+
157+
ArgoCD picks up this change and deploys the new ModuleReleaseMeta to the different landscapes. Lifecycle Manager is now picking up the version change and updating all modules using the `regular` channel to version `1.34.0`.
158+
159+
8. [OPTIONAL] Roll back the new metadata.
160+
161+
In case of failure, you can revert the setup to the old approach.
162+
163+
To do so, a PR can be opened to `/kyma/kyma-modules` reverting the submissions from steps 2 and 6. It is important to revert completely removing the entire new metadata from KCP so that KLM falls back to the old module metadata.
164+
165+
> [!Note]
166+
> After rollback, you can still use the old submission pipeline to submit new versions of the module while you're working on a fix.
167+
168+
9. Delete all old metadata files related to the module.
169+
170+
10. Continue to use the NEW approach to provide new module versions and update the mapping of channels.

‎docs/contributor/assets/module-migration.svg‎

Lines changed: 4 additions & 0 deletions
Loading

0 commit comments

Comments
 (0)