Skip to content

Commit 8f39ca2

Browse files
feat(883): helm transfer cli (open-component-model#1846)
<!-- markdownlint-disable MD041 --> #### What this PR does / why we need it This PR implements the transformers from open-component-model#1832 It creates a 3-step-transformation for the helm chart transformation-chain: ```mermaid flowchart TD A[Transfer Helm Chart] -->B(Get Helm Chart) B --> C{Download from Repo} C -->|Has Prov| D[Download from Repo] C --> E{Pack Helm LocalBlob} D --> E E -->|GetHelmOutput| F{Convert to OCI} F -->|ConverToHelm| G{Pack OCI from Helm} F -->|ConverToHelm with prov| G{Pack OCI from Helm} G -->|upload-as localBlob/default | I{Upload as localBlob} G -->|upload-as ociArtifact | H{Upload as OCI Image} ``` 1. get helm chart with `GetHelmChartV1alpha1` 2. convert helm to OCI with `ConvertHelmToOCIV1alpha1` 3. upload a. localBlobl: the oci artifact with `OCIAddLocalResourceV1alpha1` b. ociArtifact: the oci artifact with `AddOCIArtifactType` The [spec PR](open-component-model#1832) will be kept in sync with changed from here in the `bindings/go/helm` package. #### Which issue(s) this PR fixes Contributes: open-component-model/ocm-project#883 #### Testing ##### How to test the changes ```bash #!/bin/zsh alias OCM='go run ../../main.go' REGISTRY="ghcr.io/matthiasbruns/ocm-tutorials" REGISTRY2="ghcr.io/matthiasbruns/ocm-tutorials-2" pause() { echo "\n>>> Next: $1" echo "--- Press Enter to continue ---" read } # OCM --help # create constructor.yaml # https://stefanprodan.github.io/podinfo/podinfo-6.9.1.tgz cat <<EOF > constructor.yaml components: - name: ocm.software/podinfo version: 6.9.1 provider: name: ocm.software resources: - name: podinfo version: 6.9.1 type: helmChart access: type: helm/v1 helmRepository: https://stefanprodan.github.io/podinfo helmChart: podinfo-6.9.1.tgz EOF CTF_DIR=$(mktemp -d) echo "Using temporary directory: $CTF_DIR" pause "Add component version to CTF from constructor.yaml" # add cv command OCM add cv --repository ctf::$CTF_DIR --constructor constructor.yaml --skip-reference-digest-processing HELM_REF="ctf::$CTF_DIR//ocm.software/podinfo:6.9.1" pause "Create component version ($REGISTRY)" # transfer OCM transfer component-version $HELM_REF $REGISTRY --copy-resources pause "Transfer with --upload-as localBlob (OCI registry)" # transfer --upload-as localBlob OCM transfer component-version $HELM_REF $REGISTRY --copy-resources --upload-as localBlob pause "Transfer with --upload-as ociArtifact (OCI registry)" # transfer --upload-as ociArtifact OCM transfer component-version $HELM_REF $REGISTRY --copy-resources --upload-as ociArtifact pause "Download component descriptor with oras" # download with oras oras pull $REGISTRY/component-descriptors/ocm.software/podinfo:6.9.1 --output . pause "Download resource using OCM CLI" # rm downloaded if exists rm -rf downloaded # downloadCMD resource OCM download resource https://$REGISTRY//ocm.software/podinfo:6.9.1 --identity name=podinfo,version=6.9.1 --output ./downloaded ``` You can also unpack the blob and it should contain the `podinfo` chart contents # transfer oci helm to another oci `ocm transfer component-version http://$REGISTRY//ocm.software/podinfo:6.9.1 $REGISTRY2 --copy-resources --upload-as ociArtifact` ##### Verification - [x] I have tested the changes locally by running `ocm` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Added support for transferring Helm charts as part of component version transfers * Helm charts can now be converted to OCI artifacts during transfer * Support for both local Helm chart paths and remote Helm repositories as transfer sources * **Documentation** * Updated transfer command documentation with Helm chart transfer capabilities and examples <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Signed-off-by: Matthias Bruns <git@matthiasbruns.com>
1 parent 4a07684 commit 8f39ca2

File tree

16 files changed

+1176
-50
lines changed

16 files changed

+1176
-50
lines changed

.github/config/wordlist.txt

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -612,6 +612,9 @@ integrators
612612
taskfile
613613
ociimage
614614
statefulset
615+
ConvertHelmToOCI
616+
GetHelmChart
617+
AddOCIArtifact
615618
csl
616619
async
617620
netlify

CONTRIBUTING.md

Lines changed: 15 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -61,6 +61,7 @@ task cli:build
6161
## Working with Modules
6262

6363
This is a multi-module Go workspace. Each module in `bindings/go/` has its own:
64+
6465
- `go.mod`
6566
- `Taskfile.yml` with `test`, `test/integration` (if applicable)
6667

@@ -74,8 +75,8 @@ task test
7475
## Code Style
7576

7677
- Run `golangci-lint` before committing (CI enforces this)
77-
- Convenience task to run over all modules: `task tools:lint`
78-
- If you want to apply auto-fixing: `task tools:lint -- --fix`
78+
- Convenience task to run over all modules: `task tools:lint`
79+
- If you want to apply auto-fixing: `task tools:lint -- --fix`
7980
- Generated code lives alongside source — run `task generate` if you change schemas
8081

8182
## Pull Requests
@@ -90,10 +91,20 @@ CI will run linting, tests, and CodeQL analysis automatically.
9091

9192
## Architecture Decisions
9293

93-
Design decisions are documented in [`docs/adr/`](docs/adr). If you're proposing a significant change, consider writing an ADR first.
94+
Design decisions are documented in [`docs/adr/`](docs/adr). If you're proposing a significant change, consider writing
95+
an ADR first.
9496

9597
## Questions?
9698

9799
- Check existing [issues](https://github.com/open-component-model/open-component-model/issues)
98-
- See the [community docs](docs/community/) for SIGs and meetings or check out how to engage with us on our [website](https://ocm.software/community/engagement/)!
100+
- See the [community docs](docs/community/) for SIGs and meetings or check out how to engage with us on
101+
our [website](https://ocm.software/community/engagement/)!
99102
- Review the [Code of Conduct](https://github.com/cncf/foundation/blob/main/code-of-conduct.md)
103+
104+
| Variable | Default | Description |
105+
|--------------------|----------------------|-----------------------------------------|
106+
| `IMAGE_REGISTRY` | `localhost:5001` | Registry URL for pushing/pulling images |
107+
| `IMAGE_PREFIX` | `acme.org/sovereign` | Image name prefix/organization |
108+
| `PUSH_IMAGE` | `true` | Whether to push images to registry |
109+
| `VERSION` | `1.0.0` | Component version |
110+
| `POSTGRES_VERSION` | `15` | PostgreSQL version to use |

cli/cmd/add/component-version/cmd.go

Lines changed: 1 addition & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -207,8 +207,7 @@ add component-version --%[1]s ./archive --%[2]s %[3]s.yaml
207207
enum.Var(cmd.Flags(), FlagExternalComponentVersionCopyPolicy, ExternalComponentVersionCopyPolicies(), "policy to apply when a component reference to a component version outside of the constructor or target repository is encountered")
208208
cmd.Flags().Bool(FlagSkipReferenceDigestProcessing, false, "skip digest processing for resources and sources. Any resource referenced via access type will not have their digest updated.")
209209
enum.VarP(cmd.Flags(), FlagOutput, "o", []string{render.OutputFormatTable.String(), render.OutputFormatYAML.String(), render.OutputFormatJSON.String(), render.OutputFormatNDJSON.String(), render.OutputFormatTree.String()}, "output format of the component descriptors")
210-
enum.VarP(cmd.Flags(), FlagDisplayMode, "", []string{render.StaticRenderMode, render.LiveRenderMode}, `display mode can be used in combination with --recursive
211-
static: print the output once the complete component graph is discovered
210+
enum.VarP(cmd.Flags(), FlagDisplayMode, "", []string{render.StaticRenderMode, render.LiveRenderMode}, `static: print the output once the complete component graph is discovered
212211
live (experimental): continuously updates the output to represent the current construction state of the component graph`)
213212

214213
return cmd

cli/cmd/transfer/component-version/cmd.go

Lines changed: 35 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -13,11 +13,14 @@ import (
1313
"sigs.k8s.io/yaml"
1414

1515
"ocm.software/open-component-model/bindings/go/credentials"
16+
helmaccess "ocm.software/open-component-model/bindings/go/helm/access"
17+
helmtransformer "ocm.software/open-component-model/bindings/go/helm/transformation"
18+
helmv1alpha1 "ocm.software/open-component-model/bindings/go/helm/transformation/spec/v1alpha1"
1619
"ocm.software/open-component-model/bindings/go/oci/compref"
1720
ociaccess "ocm.software/open-component-model/bindings/go/oci/spec/access"
1821
ctfv1 "ocm.software/open-component-model/bindings/go/oci/spec/repository/v1/ctf"
1922
ociv1alpha1 "ocm.software/open-component-model/bindings/go/oci/spec/transformation/v1alpha1"
20-
"ocm.software/open-component-model/bindings/go/oci/transformer"
23+
ocitransformer "ocm.software/open-component-model/bindings/go/oci/transformer"
2124
"ocm.software/open-component-model/bindings/go/plugin/manager"
2225
"ocm.software/open-component-model/bindings/go/runtime"
2326
"ocm.software/open-component-model/bindings/go/transform/graph/builder"
@@ -52,11 +55,14 @@ func New() *cobra.Command {
5255
a target repository using an internally generated transformation graph.
5356
5457
This command constructs a TransformationGraphDefinition consisting of:
55-
1. CTFGetComponentVersion / OCIGetComponentVersion
56-
2. CTFAddComponentVersion / OCIAddComponentVersion
57-
3. GetOCIArtifact / OCIAddLocalResource
58-
59-
We support OCI and CTF repositories as source and target, and the graph is built accordingly based on the provided references.
58+
1. CTFGetComponentVersion -> OCIGetComponentVersion
59+
2. CTFAddComponentVersion -> OCIAddComponentVersion
60+
3. GetOCIArtifact -> OCIAddLocalResource / AddOCIArtifact
61+
4. GetHelmChart -> ConvertHelmToOCI -> OCIAddLocalResource / AddOCIArtifact
62+
63+
We support OCI and CTF as well as Helm repositories as transfer sources.
64+
OCI and CTF repositories are supported as transfer targets, while Helm repositories are not supported.
65+
The graph is built accordingly based on the provided references.
6066
By default, only the component version itself is transferred, but with --copy-resources, all resources are also copied and transformed if necessary.
6167
6268
The graph is validated, and then executed unless --dry-run is set.`,
@@ -73,6 +79,9 @@ transfer component-version ghcr.io/source-org/ocm//ocm.software/mycomponent:1.0.
7379
# Transfer from one OCI to another using OCI artifacts (default)
7480
transfer component-version ghcr.io/source-org/ocm//ocm.software/mycomponent:1.0.0 ghcr.io/target-org/ocm --copy-resources --upload-as ociArtifact
7581
82+
# Transfer a component version containing Helm charts (access-type: helm/v1) as an OCI artifact
83+
transfer component-version ghcr.io/source-org/ocm//ocm.software/mycomponent:1.0.0 ghcr.io/target-org/ocm --copy-resources --upload-as ociArtifact
84+
7685
# Transfer including all resources (e.g. OCI artifacts)
7786
transfer component-version ctf::./my-archive//ocm.software/mycomponent:1.0.0 ghcr.io/my-org/ocm --copy-resources
7887
@@ -241,43 +250,54 @@ func graphBuilder(pm *manager.PluginManager, credentialProvider credentials.Reso
241250
transformerScheme := runtime.NewScheme()
242251
transformerScheme.MustRegisterScheme(ociv1alpha1.Scheme)
243252
transformerScheme.MustRegisterScheme(ociaccess.Scheme)
253+
transformerScheme.MustRegisterScheme(helmv1alpha1.Scheme)
244254

245-
ociGet := &transformer.GetComponentVersion{
255+
ociGet := &ocitransformer.GetComponentVersion{
246256
Scheme: transformerScheme,
247257
RepoProvider: pm.ComponentVersionRepositoryRegistry,
248258
CredentialProvider: credentialProvider,
249259
}
250-
ociAdd := &transformer.AddComponentVersion{
260+
ociAdd := &ocitransformer.AddComponentVersion{
251261
Scheme: transformerScheme,
252262
RepoProvider: pm.ComponentVersionRepositoryRegistry,
253263
CredentialProvider: credentialProvider,
254264
}
255265

256266
// Resource transformers
257-
ociGetResource := &transformer.GetLocalResource{
267+
ociGetResource := &ocitransformer.GetLocalResource{
258268
Scheme: transformerScheme,
259269
RepoProvider: pm.ComponentVersionRepositoryRegistry,
260270
CredentialProvider: credentialProvider,
261271
}
262-
ociAddResource := &transformer.AddLocalResource{
272+
ociAddResource := &ocitransformer.AddLocalResource{
263273
Scheme: transformerScheme,
264274
RepoProvider: pm.ComponentVersionRepositoryRegistry,
265275
CredentialProvider: credentialProvider,
266276
}
267277

268278
// OCI Artifact transformers
269-
ociGetOCIArtifact := &transformer.GetOCIArtifact{
279+
ociGetOCIArtifact := &ocitransformer.GetOCIArtifact{
270280
Scheme: transformerScheme,
271281
Repository: pm.ResourcePluginRegistry,
272282
CredentialProvider: credentialProvider,
273283
}
274284

275-
ociAddOCIArtifact := &transformer.AddOCIArtifact{
285+
ociAddOCIArtifact := &ocitransformer.AddOCIArtifact{
276286
Scheme: transformerScheme,
277287
Repository: pm.ResourcePluginRegistry,
278288
CredentialProvider: credentialProvider,
279289
}
280290

291+
// Helm transformers
292+
getHelmChart := &helmtransformer.GetHelmChart{
293+
Scheme: transformerScheme,
294+
ResourceConsumerIdentityProvider: &helmaccess.HelmAccess{},
295+
CredentialProvider: credentialProvider,
296+
}
297+
convertHelmToOCI := &helmtransformer.ConvertHelmChartToOCI{
298+
Scheme: transformerScheme,
299+
}
300+
281301
return builder.NewBuilder(transformerScheme).
282302
WithTransformer(&ociv1alpha1.OCIGetComponentVersion{}, ociGet).
283303
WithTransformer(&ociv1alpha1.OCIAddComponentVersion{}, ociAdd).
@@ -288,7 +308,9 @@ func graphBuilder(pm *manager.PluginManager, credentialProvider credentials.Reso
288308
WithTransformer(&ociv1alpha1.CTFGetLocalResource{}, ociGetResource).
289309
WithTransformer(&ociv1alpha1.CTFAddLocalResource{}, ociAddResource).
290310
WithTransformer(&ociv1alpha1.GetOCIArtifact{}, ociGetOCIArtifact).
291-
WithTransformer(&ociv1alpha1.AddOCIArtifact{}, ociAddOCIArtifact)
311+
WithTransformer(&ociv1alpha1.AddOCIArtifact{}, ociAddOCIArtifact).
312+
WithTransformer(&helmv1alpha1.GetHelmChart{}, getHelmChart).
313+
WithTransformer(&helmv1alpha1.ConvertHelmToOCI{}, convertHelmToOCI)
292314
}
293315

294316
func renderTGD(tgd *transformv1alpha1.TransformationGraphDefinition, format string) (io.ReadCloser, error) {

cli/cmd/transfer/component-version/internal/graph.go

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -10,6 +10,7 @@ import (
1010
dagsync "ocm.software/open-component-model/bindings/go/dag/sync"
1111
descruntime "ocm.software/open-component-model/bindings/go/descriptor/runtime"
1212
descriptorv2 "ocm.software/open-component-model/bindings/go/descriptor/v2"
13+
helmv1 "ocm.software/open-component-model/bindings/go/helm/access/spec/v1"
1314
"ocm.software/open-component-model/bindings/go/oci/compref"
1415
ociv1 "ocm.software/open-component-model/bindings/go/oci/spec/access/v1"
1516
"ocm.software/open-component-model/bindings/go/oci/spec/repository/v1/oci"
@@ -157,6 +158,17 @@ func fillGraphDefinitionWithPrefetchedComponents(ctx context.Context, d *dag.Dir
157158
if err != nil {
158159
return fmt.Errorf("cannot process OCI artifact resource: %w", err)
159160
}
161+
case *helmv1.Helm:
162+
uploadAsOCIArtifact := false
163+
if _, isOCITarget := toSpec.(*oci.Repository); isOCITarget {
164+
if uploadType == UploadAsOciArtifact {
165+
uploadAsOCIArtifact = true
166+
}
167+
}
168+
err := processHelm(resource, id, val, tgd, toSpec, resourceTransformIDs, i, uploadAsOCIArtifact)
169+
if err != nil {
170+
return fmt.Errorf("cannot process Helm Chart resource: %w", err)
171+
}
160172
default:
161173
// No transformation configured for resource with access types not listed above
162174
slog.Info("Unsupported resource access type, skipping resource. Only local blob and OCI artifact resources are supported for transformation.",
Lines changed: 69 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,69 @@
1+
package internal
2+
3+
import (
4+
"fmt"
5+
6+
v2 "ocm.software/open-component-model/bindings/go/descriptor/v2"
7+
helmv1alpha1 "ocm.software/open-component-model/bindings/go/helm/transformation/spec/v1alpha1"
8+
"ocm.software/open-component-model/bindings/go/runtime"
9+
transformv1alpha1 "ocm.software/open-component-model/bindings/go/transform/spec/v1alpha1"
10+
"ocm.software/open-component-model/bindings/go/transform/spec/v1alpha1/meta"
11+
)
12+
13+
func processHelm(resource v2.Resource, id string, val *discoveryValue, tgd *transformv1alpha1.TransformationGraphDefinition, toSpec runtime.Typed, resourceTransformIDs map[int]string, i int, uploadAsOCIArtifact bool) error {
14+
resourceIdentity := resource.ToIdentity()
15+
resourceID := identityToTransformationID(resourceIdentity)
16+
getResourceID := fmt.Sprintf("%sGet%s", id, resourceID)
17+
convertResourceID := fmt.Sprintf("%sConvert%s", id, resourceID)
18+
addResourceID := fmt.Sprintf("%sAdd%s", id, resourceID)
19+
20+
unstructured, err := runtime.UnstructuredFromMixedData(map[string]any{
21+
"resource": resource,
22+
})
23+
if err != nil {
24+
return fmt.Errorf("cannot create unstructured spec for GetHelmChartV1alpha1 transformation: %w", err)
25+
}
26+
27+
// Create GetHelmChart transformation
28+
getChartTransform := transformv1alpha1.GenericTransformation{
29+
TransformationMeta: meta.TransformationMeta{
30+
Type: helmv1alpha1.GetHelmChartV1alpha1,
31+
ID: getResourceID,
32+
},
33+
Spec: unstructured,
34+
}
35+
tgd.Transformations = append(tgd.Transformations, getChartTransform)
36+
37+
// convert chart to oci artifact transformation
38+
convertToOCITransform := transformv1alpha1.GenericTransformation{
39+
TransformationMeta: meta.TransformationMeta{
40+
Type: helmv1alpha1.ConvertHelmToOCIV1alpha1,
41+
ID: convertResourceID,
42+
},
43+
Spec: &runtime.Unstructured{Data: map[string]any{
44+
"resource": fmt.Sprintf("${%s.output.resource}", getResourceID),
45+
"chartFile": fmt.Sprintf("${%s.output.chartFile}", getResourceID),
46+
"provFile": fmt.Sprintf("${%s.output.?provFile}", getResourceID),
47+
}},
48+
}
49+
tgd.Transformations = append(tgd.Transformations, convertToOCITransform)
50+
51+
// Create upload transformations
52+
var addResourceTransform transformv1alpha1.GenericTransformation
53+
if uploadAsOCIArtifact {
54+
if addResourceTransform, err = ociUploadAsArtifact(toSpec, addResourceID, convertResourceID, imageReferenceFromAccess(convertResourceID)); err != nil {
55+
return fmt.Errorf("failed to create oci upload transformation: %w", err)
56+
}
57+
} else {
58+
if addResourceTransform, err = ociUploadAsLocalResource(toSpec, val.Descriptor.Component.Name, val.Descriptor.Component.Version, addResourceID, convertResourceID, imageReferenceFromAccess(convertResourceID)); err != nil {
59+
return fmt.Errorf("failed to create oci upload as local resource transformation: %w", err)
60+
}
61+
}
62+
63+
tgd.Transformations = append(tgd.Transformations, addResourceTransform)
64+
65+
// Track this resource's transformation
66+
resourceTransformIDs[i] = addResourceID
67+
68+
return nil
69+
}

cli/cmd/transfer/component-version/internal/helpers.go

Lines changed: 5 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -6,7 +6,6 @@ import (
66
ocispecv1 "github.com/opencontainers/image-spec/specs-go/v1"
77

88
"ocm.software/open-component-model/bindings/go/oci/looseref"
9-
ociv1 "ocm.software/open-component-model/bindings/go/oci/spec/access/v1"
109
ctfv1 "ocm.software/open-component-model/bindings/go/oci/spec/repository/v1/ctf"
1110
"ocm.software/open-component-model/bindings/go/oci/spec/repository/v1/oci"
1211
ociv1alpha1 "ocm.software/open-component-model/bindings/go/oci/spec/transformation/v1alpha1"
@@ -89,16 +88,16 @@ func ChooseAddLocalResourceType(repo runtime.Typed) (runtime.Type, error) {
8988
}
9089
}
9190

92-
func GetReferenceName(ociAccess ociv1.OCIImage) (string, error) {
93-
if ociAccess.ImageReference == "" {
91+
func GetReferenceName(imageReference string) (string, error) {
92+
if imageReference == "" {
9493
return "", fmt.Errorf("cannot get reference name from empty image reference")
9594
}
96-
imageRef, err := looseref.ParseReference(ociAccess.ImageReference)
95+
imageRef, err := looseref.ParseReference(imageReference)
9796
if err != nil {
98-
return "", fmt.Errorf("invalid OCI image reference %q: %w", ociAccess.ImageReference, err)
97+
return "", fmt.Errorf("invalid OCI image reference %q: %w", imageReference, err)
9998
}
10099
if imageRef.Repository == "" {
101-
return "", fmt.Errorf("invalid image reference %q: repository is required", ociAccess.ImageReference)
100+
return "", fmt.Errorf("invalid image reference %q: repository is required", imageRef)
102101
}
103102
referenceName := imageRef.Repository
104103
if imageRef.Tag != "" {

cli/cmd/transfer/component-version/internal/helpers_test.go

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -4,6 +4,7 @@ import (
44
"testing"
55

66
"github.com/stretchr/testify/require"
7+
78
ociv1 "ocm.software/open-component-model/bindings/go/oci/spec/access/v1"
89
)
910

@@ -156,7 +157,7 @@ func TestGetReferenceName(t *testing.T) {
156157
t.Run(tt.name, func(t *testing.T) {
157158
r := require.New(t)
158159

159-
gotReference, gotErr := GetReferenceName(tt.ociImage)
160+
gotReference, gotErr := GetReferenceName(tt.ociImage.ImageReference)
160161

161162
if tt.wantErr {
162163
r.Error(gotErr, "GetReferenceName() should return an error")

0 commit comments

Comments
 (0)