Skip to content

Commit 1fa06ef

Browse files
authored
Merge pull request #690 from Danil-Grigorev/document-oci
📖 Document OCI source usage, publish and preload subcommands
2 parents 36c2c84 + a7f01b3 commit 1fa06ef

File tree

6 files changed

+349
-59
lines changed

6 files changed

+349
-59
lines changed
Lines changed: 1 addition & 47 deletions
Original file line numberDiff line numberDiff line change
@@ -1,49 +1,3 @@
11
# Plugin installation
22

3-
The `cluster-api-operator` plugin can be installed using krew, the kubectl plugin manager.
4-
5-
## Prerequisites
6-
7-
[krew][] installed on your system. See the krew installation guide for instructions.
8-
9-
[krew]: [https://krew.sigs.k8s.io/docs/user-guide/setup/install/]
10-
11-
## Steps
12-
13-
1. Add the cluster-api-operator plugin index to krew:
14-
```bash
15-
kubectl krew index add operator https://github.com/kubernetes-sigs/cluster-api-operator.git
16-
```
17-
18-
2. Install the cluster-api-operator plugin:
19-
```bash
20-
kubectl krew install operator/clusterctl-operator
21-
```
22-
23-
3. Verify the installation:
24-
```bash
25-
kubectl operator
26-
```
27-
28-
This should print help information for the kubectl operator plugin.
29-
30-
The `cluster-api-operator` plugin is now installed and ready to use with `kubectl`.
31-
32-
### Optionally: installing as a `clusterctl` plugin
33-
Typically the plugin is installed under `~/.krew/bin/kubectl-operator`, which would be present under your `$PATH` after correct `krew` installation. If you want to use plugin with `clusterctl`, you need to rename this file to be prefixed with `clusterctl-` instead, like so:
34-
```bash
35-
cp ~/.krew/bin/kubectl-operator ~/.krew/bin/clusterctl-operator
36-
```
37-
38-
After that plugin is available to use as a `clusterctl` plugin:
39-
```bash
40-
clusterctl operator --help
41-
```
42-
43-
## Upgrade
44-
45-
To upgrade your plugin with the new release of `cluster-api-operator` you will need to run:
46-
47-
```bash
48-
kubectl krew upgrade
49-
```
3+
Please refer to [plugin installation](../03_topics/03_plugin/01_installation.md) section.

docs/book/src/03_topics/02_configuration/01_air-gapped-environtment.md

Lines changed: 120 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -4,18 +4,20 @@ To install Cluster API providers in an air-gapped environment using the operator
44

55
1. Configure the operator for an air-gapped environment:
66
- Manually fetch and store a helm chart for the operator.
7-
- Provide image overrides for the operator in from an accessible image repository.
7+
- Provide image overrides for the operator from an accessible image repository.
88
2. Configure providers for an air-gapped environment:
9-
- Provide fetch configuration for each provider from an accessible location (e.g., an internal GitHub repository) or from pre-created ConfigMaps within the cluster.
9+
- Provide fetch configuration for each provider from an accessible location: e.g., an OCI artifact, internal Github/Gitlab repository URL or from pre-created ConfigMaps within the cluster.
1010
- Provide image overrides for each provider to pull images from an accessible image repository.
1111

1212
**Example Usage:**
1313

1414
As an admin, I need to fetch the Azure provider components from within the cluster because I am working in an air-gapped environment.
1515

16+
### Using ConfigMap
17+
1618
In this example, there is a ConfigMap in the `capz-system` namespace that defines the components and metadata of the provider.
1719

18-
The Azure InfrastructureProvider is configured with a `fetchConfig` specifying the label selector, allowing the operator to determine the available versions of the Azure provider. Since the provider's version is marked as `v1.9.3`, the operator uses the components information from the ConfigMap with matching label to install the Azure provider.
20+
The Azure InfrastructureProvider is configured with a `fetchConfig` specifying the label selector, allowing the operator to determine the available versions of the Azure provider. Since the provider's version is marked as `v1.9.3`, the operator uses the components information from the ConfigMap with a matching label to install the Azure provider.
1921

2022
```yaml
2123
---
@@ -47,39 +49,145 @@ spec:
4749
provider-components: azure
4850
```
4951
50-
### Situation when manifests do not fit into configmap
52+
### Using OCI Artifact
53+
54+
OCI artifact files can follow these naming patterns:
55+
56+
- `<registry>/<repository>:<tag>` (e.g., `my-registry.example.com/my-provider:v1.9.3`)
57+
- `<registry>/<repository>` (e.g., my-registry.example.com/my-provider), in which case the tag is substituted by provider version.
58+
59+
When working with metadata and component files within OCI artifacts, the files stored in the artifact should follow these naming conventions:
60+
61+
- **Metadata Files**:
62+
- Default: `metadata.yaml`
63+
- Versioned: `fmt.Sprintf("%s-%s-%s-metadata.yaml", p.GetType(), p.GetName(), p.GetSpec().Version)`, Example: `infrastructure-azure-v1.9.3-metadata.yaml`
64+
65+
- **Component Files**:
66+
- Default: `components.yaml`
67+
- Typed: `fmt.Sprintf("%s-components.yaml", p.GetType())`, Example: `infrastructure-components.yaml`
68+
- Versioned: `fmt.Sprintf("%s-%s-%s-components.yaml", p.GetType(), p.GetName(), p.GetSpec().Version)`, Example: `infrastructure-azure-v1.9.3-components.yaml`
69+
70+
Versioned files allow to use single image for hosting multiple provider manifests and versions simultaneously, without overlapping each other.
71+
72+
Typed allow to store multiple provider types inside single image, which is needed for example for `bootstrap` and `control-plane` providers.
73+
74+
Example layout for a `kubeadm` provider may look like:
75+
- `metadata.yaml`
76+
- `control-plane-components.yaml`
77+
- `bootstrap-components.yaml`
78+
79+
To fetch provider components which are stored as an OCI artifact, you can configure `fetchConfig.oci` field to pull them directly from an OCI registry:
80+
81+
```yaml
82+
apiVersion: operator.cluster.x-k8s.io/v1alpha2
83+
kind: InfrastructureProvider
84+
metadata:
85+
name: azure
86+
namespace: capz-system
87+
spec:
88+
version: v1.9.3
89+
configSecret:
90+
name: azure-variables
91+
fetchConfig:
92+
oci: "my-oci-registry.example.com/my-provider:v1.9.3"
93+
```
94+
95+
## OCI Authentication
96+
97+
To securely authenticate with an OCI registry, environment variables are used for user credentials. The following environment variables are involved:
98+
99+
- **`OCI_USERNAME`**: The username for the OCI registry.
100+
- **`OCI_PASSWORD`**: The password associated with the username.
101+
- **`OCI_ACCESS_TOKEN`**: A token used for authentication.
102+
- **`OCI_REFRESH_TOKEN`**: A refresh token to obtain new access tokens.
103+
104+
### Fetching Provider Components from a secure OCI Registry
105+
106+
To fetch provider components stored as an OCI artifact, you can configure the `fetchConfig.oci` field to pull them directly from an OCI registry. The `configSecret` field references a Kubernetes `Secret` that should contain the necessary OCI credentials (such as username and password, or token), ensuring that sensitive information is securely stored.
107+
108+
Here’s an example of how to configure the `InfrastructureProvider` resource to fetch a specific version of a provider component from an OCI registry:
109+
110+
```yaml
111+
apiVersion: operator.cluster.x-k8s.io/v1alpha2
112+
kind: InfrastructureProvider
113+
metadata:
114+
name: azure
115+
namespace: capz-system
116+
spec:
117+
version: v1.9.3
118+
configSecret:
119+
name: azure-variables # Secret containing the OCI registry credentials
120+
fetchConfig:
121+
oci: "my-oci-registry.example.com/my-provider:v1.9.3" # Reference to the OCI artifact (provider)
122+
```
123+
124+
The reference secret can could contain OCI authentication data:
125+
126+
```yaml
127+
apiVersion: v1
128+
kind: Secret
129+
metadata:
130+
name: azure-variables # Name of the secret referenced in the InfrastructureProvider
131+
namespace: capz-system # Namespace where the secret resides
132+
type: Opaque
133+
data:
134+
OCI_USERNAME: <secret>
135+
OCI_PASSWORD: <secret>
136+
OCI_ACCESS_TOKEN: <secret>
137+
OCI_REFRESH_TOKEN: <secret>
138+
```
139+
140+
### Using Github/Gitlab URL
141+
142+
If the provider components are hosted at a specific repository URL, you can use `fetchConfig.url` to retrieve them directly.
143+
144+
```yaml
145+
apiVersion: operator.cluster.x-k8s.io/v1alpha2
146+
kind: InfrastructureProvider
147+
metadata:
148+
name: azure
149+
namespace: capz-system
150+
spec:
151+
version: v1.9.3
152+
configSecret:
153+
name: azure-variables
154+
fetchConfig:
155+
url: "https://my-internal-repo.example.com/providers/azure/v1.9.3.yaml"
156+
```
157+
158+
## Situation when manifests do not fit into ConfigMap
51159

52-
There is a limit on the [maximum size](https://kubernetes.io/docs/concepts/configuration/configmap/#motivation) of a configmap - 1MiB. If the manifests do not fit into this size, Kubernetes will generate an error and provider installation fail. To avoid this, you can archive the manifests and put them in the configmap that way.
160+
There is a limit on the [maximum size](https://kubernetes.io/docs/concepts/configuration/configmap/#motivation) of a ConfigMap - 1MiB. If the manifests do not fit into this size, Kubernetes will generate an error and provider installation will fail. To avoid this, you can archive the manifests and put them in the ConfigMap that way.
53161

54-
For example, you have two files: `components.yaml` and `metadata.yaml`. To create a working config map you need:
162+
For example, you have two files: `components.yaml` and `metadata.yaml`. To create a working ConfigMap, you need:
55163

56-
1. Archive components.yaml using `gzip` cli tool
164+
1. Archive components.yaml using `gzip` CLI tool:
57165

58166
```sh
59167
gzip -c components.yaml > components.gz
60168
```
61169

62-
2. Create a configmap manifest from the archived data
170+
2. Create a ConfigMap manifest from the archived data:
63171

64172
```sh
65173
kubectl create configmap v1.9.3 --namespace=capz-system --from-file=components=components.gz --from-file=metadata=metadata.yaml --dry-run=client -o yaml > configmap.yaml
66174
```
67175

68-
3. Edit the file by adding "provider.cluster.x-k8s.io/compressed: true" annotation
176+
3. Edit the file by adding "provider.cluster.x-k8s.io/compressed: true" annotation:
69177

70178
```sh
71179
yq eval -i '.metadata.annotations += {"provider.cluster.x-k8s.io/compressed": "true"}' configmap.yaml
72180
```
73181

74-
**Note**: without this annotation operator won't be able to determine if the data is compressed or not.
182+
**Note**: Without this annotation, the operator won't be able to determine if the data is compressed or not.
75183

76-
4. Add labels that will be used to match the configmap in `fetchConfig` section of the provider
184+
4. Add labels that will be used to match the ConfigMap in the `fetchConfig` section of the provider:
77185

78186
```sh
79187
yq eval -i '.metadata.labels += {"my-label": "label-value"}' configmap.yaml
80188
```
81189

82-
5. Create a configmap in your kubernetes cluster using kubectl
190+
5. Create the ConfigMap in your Kubernetes cluster using kubectl:
83191

84192
```sh
85193
kubectl create -f configmap.yaml
Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
# Plugin
2+
3+
This section descibes plugin commands with usage and examples
Lines changed: 49 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,49 @@
1+
# Plugin installation
2+
3+
The `cluster-api-operator` plugin can be installed using krew, the kubectl plugin manager.
4+
5+
## Prerequisites
6+
7+
[krew][] installed on your system. See the krew installation guide for instructions.
8+
9+
[krew]: [https://krew.sigs.k8s.io/docs/user-guide/setup/install/]
10+
11+
## Steps
12+
13+
1. Add the cluster-api-operator plugin index to krew:
14+
```bash
15+
kubectl krew index add operator https://github.com/kubernetes-sigs/cluster-api-operator.git
16+
```
17+
18+
2. Install the cluster-api-operator plugin:
19+
```bash
20+
kubectl krew install operator/clusterctl-operator
21+
```
22+
23+
3. Verify the installation:
24+
```bash
25+
kubectl operator
26+
```
27+
28+
This should print help information for the kubectl operator plugin.
29+
30+
The `cluster-api-operator` plugin is now installed and ready to use with `kubectl`.
31+
32+
### Optionally: installing as a `clusterctl` plugin
33+
Typically the plugin is installed under `~/.krew/bin/kubectl-operator`, which would be present under your `$PATH` after correct `krew` installation. If you want to use plugin with `clusterctl`, you need to rename this file to be prefixed with `clusterctl-` instead, like so:
34+
```bash
35+
cp ~/.krew/bin/kubectl-operator ~/.krew/bin/clusterctl-operator
36+
```
37+
38+
After that plugin is available to use as a `clusterctl` plugin:
39+
```bash
40+
clusterctl operator --help
41+
```
42+
43+
## Upgrade
44+
45+
To upgrade your plugin with the new release of `cluster-api-operator` you will need to run:
46+
47+
```bash
48+
kubectl krew upgrade
49+
```
Lines changed: 77 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,77 @@
1+
# Using the `preload` Plugin for Kubernetes Operator
2+
3+
## Overview
4+
5+
The `preload` subcommand allows users to preload provider `ConfigMaps` into a management cluster from an OCI (Open Container Initiative) artifact, known provider source, or URL override. Users can supply any number of provider stings or discover and use existing provider manifests from the cluster.
6+
7+
## Command Syntax
8+
The basic syntax for using the `preload` command is:
9+
10+
```sh
11+
kubectl operator preload [flags]
12+
```
13+
14+
## Flags and Options
15+
| Flag | Short | Description |
16+
|------|-------|-------------|
17+
| `--kubeconfig` | | Path to the kubeconfig file for the source management cluster. Uses default discovery rules if unspecified. |
18+
| `--existing` | `-e` | Discover all providers in the cluster and prepare `ConfigMap` for each of them. |
19+
| `--core` | | Specifies the core provider and version (e.g., `cluster-api:v1.1.5`). Defaults to the latest release. |
20+
| `--infrastructure` | `-i` | Specifies infrastructure providers and versions (e.g., `aws:v0.5.0`). |
21+
| `--bootstrap` | `-b` | Specifies bootstrap providers and versions (e.g., `kubeadm:v1.1.5`). |
22+
| `--control-plane` | `-c` | Specifies control plane providers and versions (e.g., `kubeadm:v1.1.5`). |
23+
| `--ipam` | | Specifies IPAM providers and versions (e.g., `infoblox:v0.0.1`). |
24+
| `--runtime-extension` | | Specifies runtime extension providers and versions (e.g., `my-extension:v0.0.1`). |
25+
| `--addon` | | Specifies add-on providers and versions (e.g., `helm:v0.1.0`). |
26+
| `--target-namespace` | `-n` | Specifies the target namespace where the operator should be deployed. Defaults to `capi-operator-system`. |
27+
| `--artifact-url` | `-u` | Specifies the URL of the OCI artifact containing component manifests. |
28+
29+
## Examples
30+
31+
### Load CAPI Operator Manifests from an OCI Source
32+
```sh
33+
kubectl operator preload --core cluster-api
34+
```
35+
This command loads the `cluster-api` core provider manifests into the management cluster. If no version is specified, the latest release is used.
36+
37+
### Load CAPI Operator Manifests from Existing Providers in the Cluster
38+
```sh
39+
kubectl operator preload -e
40+
```
41+
This command discovers all existing providers in the cluster and prepares ConfigMaps containing their manifests.
42+
43+
### Prepare Provider ConfigMap from OCI for a Specific Infrastructure Provider
44+
```sh
45+
kubectl operator preload --infrastructure=aws -u my-registry.example.com/infrastructure-provider
46+
```
47+
This command fetches the latest available version of the `aws` infrastructure provider from the specified OCI registry and creates a ConfigMap.
48+
49+
### Prepare Provider ConfigMap with a Specific Version
50+
```sh
51+
kubectl operator preload --infrastructure=aws::v2.3.0 -u my-registry.example.com/infrastructure-provider
52+
```
53+
This command loads the AWS infrastructure provider version `v2.3.0` from the OCI registry into the default namespace.
54+
55+
### Prepare Provider ConfigMap with a Custom Namespace
56+
```sh
57+
kubectl operator preload --infrastructure=aws:custom-namespace -u my-registry.example.com/infrastructure-provider
58+
```
59+
This command loads the latest version of the AWS infrastructure provider into the `custom-namespace`.
60+
61+
### Prepare Provider ConfigMap with a Specific Version and Namespace
62+
```sh
63+
kubectl operator preload --infrastructure=aws:custom-namespace:v2.3.0 -u my-registry.example.com/infrastructure-provider
64+
```
65+
This command loads AWS provider version `v2.3.0` into `custom-namespace`.
66+
67+
### Prepare Provider ConfigMap for Multiple Infrastructure Providers
68+
```sh
69+
kubectl operator preload --infrastructure=aws --infrastructure=vsphere -u my-registry.example.com/infrastructure-provider
70+
```
71+
This command fetches and loads manifests for both AWS and vSphere infrastructure providers from the OCI registry.
72+
73+
### Prepare Provider ConfigMap with a Custom Target Namespace
74+
```sh
75+
kubectl operator preload --infrastructure aws --target-namespace foo -u my-registry.example.com/infrastructure-provider
76+
```
77+
This command loads the AWS infrastructure provider into the `foo` namespace, ensuring that the operator uses a customized deployment location.

0 commit comments

Comments
 (0)