|
| 1 | +# OCI Registry Integration for k0smotron |
| 2 | + |
| 3 | +This example demonstrates how to configure k0smotron to use a k0s binary from an OCI registry instead of relying on the default installation script. |
| 4 | + |
| 5 | + |
| 6 | +## Prerquisites |
| 7 | + |
| 8 | +For this setup, you need to use the control plane and bootstrap providers for k0smotron, together with your desired infrastructure provider. In this example, we’ll use the AWS infrastructure provider. |
| 9 | + |
| 10 | +(See the [tutorial](https://cluster-api-aws.sigs.k8s.io/quick-start) on how to use AWS in CAPI for more details). Once you have a valid cluster to deploy the providers, run: |
| 11 | + |
| 12 | +```cmd |
| 13 | +clusterctl init --control-plane k0sproject-k0smotron --bootstrap k0sproject-k0smotron --infrastructure aws |
| 14 | +``` |
| 15 | + |
| 16 | +## Configure `K0sControlPlane` for using and OCI registry |
| 17 | + |
| 18 | +Configuring the `K0sControlPlane` to pull k0s from an OCI registry is straightforward. **The only requirement is that the machine being bootstrapped needs Oras CLI installed**. You can achieve this in two ways: |
| 19 | + |
| 20 | +- By using `.preStartCommands` to install the Oras CLI on the machine before pulling the binary. |
| 21 | +- By using a machine image with the Oras CLI pre-installed. |
| 22 | + |
| 23 | +```yaml |
| 24 | +apiVersion: cluster.x-k8s.io/v1beta1 |
| 25 | +kind: Cluster |
| 26 | +metadata: |
| 27 | + name: aws-test |
| 28 | + namespace: default |
| 29 | +spec: |
| 30 | + clusterNetwork: |
| 31 | + pods: |
| 32 | + cidrBlocks: |
| 33 | + - 192.168.0.0/16 |
| 34 | + serviceDomain: cluster.local |
| 35 | + services: |
| 36 | + cidrBlocks: |
| 37 | + - 10.128.0.0/12 |
| 38 | + controlPlaneRef: |
| 39 | + apiVersion: controlplane.cluster.x-k8s.io/v1beta1 |
| 40 | + kind: K0sControlPlane |
| 41 | + name: aws-test |
| 42 | + infrastructureRef: |
| 43 | + apiVersion: infrastructure.cluster.x-k8s.io/v1beta2 |
| 44 | + kind: AWSCluster |
| 45 | + name: aws-test |
| 46 | +--- |
| 47 | +apiVersion: infrastructure.cluster.x-k8s.io/v1beta2 |
| 48 | +kind: AWSMachineTemplate |
| 49 | +metadata: |
| 50 | + name: aws-test |
| 51 | + namespace: default |
| 52 | +spec: |
| 53 | + template: |
| 54 | + spec: |
| 55 | + ami: |
| 56 | + # Replace with your AMI ID |
| 57 | + id: ami-0008aa5cb0cde3400 # Ubuntu 20.04 in eu-west-1 |
| 58 | + instanceType: t3.large |
| 59 | + publicIP: true |
| 60 | + iamInstanceProfile: nodes.cluster-api-provider-aws.sigs.k8s.io # Instance Profile created by `clusterawsadm bootstrap iam create-cloudformation-stack` |
| 61 | + cloudInit: |
| 62 | + # Makes CAPA use k0s bootstrap cloud-init directly and not via SSM |
| 63 | + # Simplifies the VPC setup as we do not need custom SSM endpoints etc. |
| 64 | + insecureSkipSecretsManager: true |
| 65 | + uncompressedUserData: false |
| 66 | + sshKeyName: <your-ssh-key-name> |
| 67 | +--- |
| 68 | +apiVersion: controlplane.cluster.x-k8s.io/v1beta1 |
| 69 | +kind: K0sControlPlane |
| 70 | +metadata: |
| 71 | + name: aws-test |
| 72 | +spec: |
| 73 | + replicas: 3 |
| 74 | + version: v1.33.4+k0s.0 |
| 75 | + updateStrategy: Recreate |
| 76 | + k0sConfigSpec: |
| 77 | + # OCI URL (digest reference) for the k0s binary blob |
| 78 | + downloadURL: oci://example.com/my-repo/k0s@sha256:abcdefg123456789 |
| 79 | + # Install Oras CLI |
| 80 | + preStartCommands: |
| 81 | + - VERSION="1.3.0" |
| 82 | + - curl -LO "https://github.com/oras-project/oras/releases/download/v${VERSION}/oras_${VERSION}_linux_amd64.tar.gz" |
| 83 | + - mkdir -p oras-install/ |
| 84 | + - tar -zxf oras_${VERSION}_*.tar.gz -C oras-install/ |
| 85 | + - sudo mv oras-install/oras /usr/local/bin/ |
| 86 | + - rm -rf oras_${VERSION}_*.tar.gz oras-install/ |
| 87 | + args: |
| 88 | + - --enable-worker |
| 89 | + k0s: |
| 90 | + apiVersion: k0s.k0sproject.io/v1beta1 |
| 91 | + kind: ClusterConfig |
| 92 | + metadata: |
| 93 | + name: k0s |
| 94 | + spec: |
| 95 | + api: |
| 96 | + extraArgs: |
| 97 | + anonymous-auth: "true" |
| 98 | + telemetry: |
| 99 | + enabled: false |
| 100 | + machineTemplate: |
| 101 | + infrastructureRef: |
| 102 | + apiVersion: infrastructure.cluster.x-k8s.io/v1beta2 |
| 103 | + kind: AWSMachineTemplate |
| 104 | + name: aws-test |
| 105 | + namespace: default |
| 106 | +--- |
| 107 | +apiVersion: infrastructure.cluster.x-k8s.io/v1beta2 |
| 108 | +kind: AWSCluster |
| 109 | +metadata: |
| 110 | + name: aws-test |
| 111 | + namespace: default |
| 112 | +spec: |
| 113 | + region: eu-west-1 |
| 114 | + sshKeyName: <your-ssh-key-name> |
| 115 | + controlPlaneLoadBalancer: |
| 116 | + healthCheckProtocol: TCP |
| 117 | + network: |
| 118 | + additionalControlPlaneIngressRules: |
| 119 | + - description: "k0s controller join API" |
| 120 | + protocol: tcp |
| 121 | + fromPort: 9443 |
| 122 | + toPort: 9443 |
| 123 | +``` |
| 124 | +
|
| 125 | +As shown above, we use the `downloadURL` field to reference a k0s binary blob via its digest. The URL must use the `oci://` schema. |
| 126 | + |
| 127 | +## Authentication |
| 128 | + |
| 129 | +If your OCI registry requires authentication, you need to provide credentials in a `config.json` file, following the [Oras CLI authentication mechanism](https://oras.land/docs/how_to_guides/authentication/). You can make this file available to the node by adding it as a *file* entry containing the authentication credentials under the `files` field in the `K0sControlPlane` spec. For example: |
| 130 | + |
| 131 | +```yaml |
| 132 | +apiVersion: controlplane.cluster.x-k8s.io/v1beta1 |
| 133 | +kind: K0sControlPlane |
| 134 | +metadata: |
| 135 | + name: aws-test |
| 136 | +spec: |
| 137 | + replicas: 3 |
| 138 | + version: v1.33.4+k0s.0 |
| 139 | + updateStrategy: Recreate |
| 140 | + k0sConfigSpec: |
| 141 | + # OCI URL (digest reference) for the k0s binary blob |
| 142 | + downloadURL: oci://example.com/my-private-repo/k0s@sha256:abcdefg123456789 |
| 143 | + # We add a new file with a secret reference for the needed credentials used by Oras |
| 144 | + files: |
| 145 | + - contentFrom: |
| 146 | + secretRef: |
| 147 | + name: my-oras-config |
| 148 | + key: .dockerconfigjson |
| 149 | + path: /root/.docker/config.json |
| 150 | + preStartCommands: |
| 151 | + - VERSION="1.3.0" |
| 152 | + - curl -LO "https://github.com/oras-project/oras/releases/download/v${VERSION}/oras_${VERSION}_linux_amd64.tar.gz" |
| 153 | + - mkdir -p oras-install/ |
| 154 | + - tar -zxf oras_${VERSION}_*.tar.gz -C oras-install/ |
| 155 | + - sudo mv oras-install/oras /usr/local/bin/ |
| 156 | + - rm -rf oras_${VERSION}_*.tar.gz oras-install/ |
| 157 | + - export DOCKER_CONFIG=/root/.docker # In addition to downloading hours, we need to make oras use the proper `.docker/config.json` by setting the directoty of the desired config |
| 158 | + args: |
| 159 | + - --enable-worker |
| 160 | + k0s: |
| 161 | + apiVersion: k0s.k0sproject.io/v1beta1 |
| 162 | + kind: ClusterConfig |
| 163 | + metadata: |
| 164 | + name: k0s |
| 165 | + spec: |
| 166 | + api: |
| 167 | + extraArgs: |
| 168 | + anonymous-auth: "true" |
| 169 | + telemetry: |
| 170 | + enabled: false |
| 171 | + machineTemplate: |
| 172 | + infrastructureRef: |
| 173 | + apiVersion: infrastructure.cluster.x-k8s.io/v1beta2 |
| 174 | + kind: AWSMachineTemplate |
| 175 | + name: aws-test |
| 176 | + namespace: default |
| 177 | +``` |
| 178 | +
|
| 179 | +In this example, a new file entry is configured that references a secret containing the authentication credentials. |
| 180 | +
|
| 181 | +!!! note "Do not forget to set `DOCKER_CONFIG`" |
| 182 | + To let the Oras CLI use the authentication credentials, export the `DOCKER_CONFIG` environment variable in your `.preStartCommands`, so that it points to the directory containing `config.json` when the machine boots. |
0 commit comments