Skip to content

Commit bb87eb6

Browse files
FrankYang0529jillian-maroket
FrankYang0529
and
jillian-maroket
authored
feat: add steps to deploying to the RKE2 custom cluster (#525)
* feat: add steps to deploying to the RKE2 custom cluster Signed-off-by: PoAn Yang <[email protected]> Co-authored-by: Jillian <[email protected]>
1 parent da8a921 commit bb87eb6

File tree

7 files changed

+240
-51
lines changed

7 files changed

+240
-51
lines changed

‎docs/rancher/cloud-provider.md‎

Lines changed: 80 additions & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,7 @@ title: "Harvester Cloud Provider"
55
keywords:
66
- Harvester
77
- harvester
8-
- RKE
8+
- RKE
99
- rke
1010
- RKE2
1111
- rke2
@@ -47,12 +47,12 @@ When spinning up an RKE cluster using the Harvester node driver, you can perform
4747
1. Select `Harvester(Out-of-tree)` option.
4848

4949
![](/img/v1.2/rancher/rke-cloud-provider.png)
50-
50+
5151
2. Install `Harvester Cloud Provider` from the Rancher marketplace.
5252

5353
![](/img/v1.2/rancher/install-harvester-cloud-provider.png)
5454

55-
55+
5656
### Deploying to the RKE2 Cluster with Harvester Node Driver
5757

5858
When spinning up an RKE2 cluster using the Harvester node driver, select the `Harvester` cloud provider. The node driver will then help deploy both the CSI driver and CCM automatically.
@@ -62,25 +62,88 @@ When spinning up an RKE2 cluster using the Harvester node driver, select the `Ha
6262
### Deploying to the RKE2 custom cluster (experimental)
6363

6464
![](/img/v1.2/rancher/custom.png)
65-
1. Use `generate_addon.sh` to generate a cloud-config and place it into the directory `/etc/kubernetes/cloud-config` on every custom node.
65+
1. Generate cloud config data using the script `generate_addon.sh`, and then place the data on every custom node (directory: `/etc/kubernetes/cloud-config`).
6666

67-
```
68-
curl -sfL https://raw.githubusercontent.com/harvester/cloud-provider-harvester/master/deploy/generate_addon.sh | bash -s <serviceaccount name> <namespace>
69-
```
67+
```bash
68+
curl -sfL https://raw.githubusercontent.com/harvester/cloud-provider-harvester/master/deploy/generate_addon.sh | bash -s <serviceaccount name> <namespace>
69+
```
7070

7171
:::note
7272

73-
The `generate_addon.sh` script depends on `kubectl` and `jq` to operate the Harvester cluster.
73+
The script depends on `kubectl` and `jq` when operating the Harvester cluster, and functions only when given access to the `Harvester Cluster` kubeconfig file.
7474

75-
The script needs access to the `Harvester Cluster` kubeconfig to work. You can find the `kubeconfig` file from one of the Harvester management nodes in the `/etc/rancher/rke2/rke2.yaml` path.
75+
You can find the `kubeconfig` file in one of the Harvester management nodes in the `/etc/rancher/rke2/rke2.yaml` path. The server IP must be replaced with the VIP address.
76+
77+
Example of content:
78+
79+
```yaml
80+
apiVersion: v1
81+
clusters:
82+
- cluster:
83+
certificate-authority-data: <redacted>
84+
server: https://127.0.0.1:6443
85+
name: default
86+
# ...
87+
```
7688

77-
The namespace needs to be the namespace in which the guest cluster will be created.
89+
You must specify the namespace in which the guest cluster will be created.
7890

7991
:::
8092

93+
Example of output:
8194

82-
2. Configure the **Cloud Provider** to `Harvester` and select **Create** to spin up the cluster.
83-
![](/img/v1.2/rancher/create-custom-rke2.png)
95+
```yaml
96+
########## cloud config ############
97+
apiVersion: v1
98+
clusters:
99+
- cluster:
100+
certificate-authority-data: <CACERT>
101+
server: https://HARVESTER-ENDPOINT/k8s/clusters/local
102+
name: local
103+
contexts:
104+
- context:
105+
cluster: local
106+
namespace: default
107+
user: harvester-cloud-provider-default-local
108+
name: harvester-cloud-provider-default-local
109+
current-context: harvester-cloud-provider-default-local
110+
kind: Config
111+
preferences: {}
112+
users:
113+
- name: harvester-cloud-provider-default-local
114+
user:
115+
token: <TOKEN>
116+
117+
########## cloud-init user data ############
118+
write_files:
119+
- encoding: b64
120+
content: <CONTENT>
121+
owner: root:root
122+
path: /etc/kubernetes/cloud-config
123+
permissions: '0644'
124+
```
125+
126+
1. Create a VM in the Harvester cluster with the following settings:
127+
128+
- **Basics** tab: The minimum requirements are 2 CPUs and 4 GiB of RAM. The required disk space depends on the VM image.
129+
130+
![](/img/v1.3/rancher/custom-cluster-vm-cpu-and-ram.png)
131+
132+
- **Networks** tab: Specify a network name with the format `nic-<number>`.
133+
134+
![](/img/v1.3/rancher/custom-cluster-vm-network.png)
135+
136+
- **Advanced Options** tab: Copy and paste the content of the **Cloud Config User Data** screen.
137+
138+
![](/img/v1.3/rancher/custom-cluster-vm-user-data.png)
139+
140+
1. On the **Basics** tab of the **Cluster Configuration** screen, select **Harvester** as the **Cloud Provider** and then select **Create** to spin up the cluster.
141+
142+
![](/img/v1.2/rancher/create-custom-rke2.png)
143+
144+
1. On the **Registration** tab, perform the steps required to run the RKE2 registration command on the VM.
145+
146+
![](/img/v1.3/rancher/custom-cluster-registration.png)
84147

85148
### Deploying to the K3s cluster with Harvester node driver (experimental)
86149

@@ -115,8 +178,8 @@ When spinning up a K3s cluster using the Harvester node driver, you can perform
115178
- name: harvester-cloud-provider-default-local
116179
user:
117180
token: <TOKEN>
118-
119-
181+
182+
120183
########## cloud-init user data ############
121184
write_files:
122185
- encoding: b64
@@ -188,12 +251,12 @@ The cloud provider can be upgraded by upgrading the RKE2 version. You can upgrad
188251
RKE/K3s upgrade cloud provider via the Rancher UI, as follows:
189252
1. Click **☰ > RKE/K3s Cluster > Apps > Installed Apps**.
190253
2. Find the cloud provider chart and select ⋮ **> Edit/Upgrade**.
191-
3. Select **Version**.
254+
3. Select **Version**.
192255
4. Click **Next > Update**.
193256

194257
:::info
195258

196-
The upgrade process for a [single-node guest cluster](../advanced/singlenodeclusters) may stall when the new `harvester-cloud-provider` pod is stuck in the *Pending* state. This issue is caused by a section in the `harvester-cloud-provider` deployment that describes the rolling update strategy. Specifically, the default value conflicts with the `podAntiAffinity` configuration in single-node clusters.
259+
The upgrade process for a [single-node guest cluster](../advanced/singlenodeclusters) may stall when the new `harvester-cloud-provider` pod is stuck in the *Pending* state. This issue is caused by a section in the `harvester-cloud-provider` deployment that describes the rolling update strategy. Specifically, the default value conflicts with the `podAntiAffinity` configuration in single-node clusters.
197260

198261
For more information, see [this GitHub issue comment](https://github.com/harvester/harvester/issues/5348#issuecomment-2055453709). To address the issue, manually delete the old `harvester-cloud-provider` pod. You might need to do this multiple times until the new pod can be successfully scheduled.
199262

@@ -203,7 +266,7 @@ For more information, see [this GitHub issue comment](https://github.com/harvest
203266
Once you've deployed the Harvester cloud provider, you can leverage the Kubernetes `LoadBalancer` service to expose a microservice within the guest cluster to the external world. Creating a Kubernetes `LoadBalancer` service assigns a dedicated Harvester load balancer to the service, and you can make adjustments through the `Add-on Config` within the Rancher UI.
204267
205268
![](/img/v1.2/rancher/lb-svc.png)
206-
269+
207270
208271
### IPAM
209272
Harvester's built-in load balancer offers both **DHCP** and **Pool** modes, and you can configure it by adding the annotation `cloudprovider.harvesterhci.io/ipam: $mode` to its corresponding service. Starting from Harvester cloud provider >= v0.2.0, it also introduces a unique **Share IP** mode. A service shares its load balancer IP with other services in this mode.

‎static/img/v1.3/rancher/custom-cluster-registration.png‎

217 KB
Loading

‎static/img/v1.3/rancher/custom-cluster-vm-cpu-and-ram.png‎

154 KB
Loading

‎static/img/v1.3/rancher/custom-cluster-vm-network.png‎

178 KB
Loading

‎static/img/v1.3/rancher/custom-cluster-vm-user-data.png‎

504 KB
Loading

‎versioned_docs/version-v1.2/rancher/cloud-provider.md‎

Lines changed: 80 additions & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,7 @@ title: "Harvester Cloud Provider"
55
keywords:
66
- Harvester
77
- harvester
8-
- RKE
8+
- RKE
99
- rke
1010
- RKE2
1111
- rke2
@@ -47,12 +47,12 @@ When spinning up an RKE cluster using the Harvester node driver, you can perform
4747
1. Select `Harvester(Out-of-tree)` option.
4848

4949
![](/img/v1.2/rancher/rke-cloud-provider.png)
50-
50+
5151
2. Install `Harvester Cloud Provider` from the Rancher marketplace.
5252

5353
![](/img/v1.2/rancher/install-harvester-cloud-provider.png)
5454

55-
55+
5656
### Deploying to the RKE2 Cluster with Harvester Node Driver
5757

5858
When spinning up an RKE2 cluster using the Harvester node driver, select the `Harvester` cloud provider. The node driver will then help deploy both the CSI driver and CCM automatically.
@@ -62,25 +62,88 @@ When spinning up an RKE2 cluster using the Harvester node driver, select the `Ha
6262
### Deploying to the RKE2 custom cluster (experimental)
6363

6464
![](/img/v1.2/rancher/custom.png)
65-
1. Use `generate_addon.sh` to generate a cloud-config and place it into the directory `/etc/kubernetes/cloud-config` on every custom node.
65+
1. Generate cloud config data using the script `generate_addon.sh`, and then place the data on every custom node (directory: `/etc/kubernetes/cloud-config`).
6666

67-
```
68-
curl -sfL https://raw.githubusercontent.com/harvester/cloud-provider-harvester/master/deploy/generate_addon.sh | bash -s <serviceaccount name> <namespace>
69-
```
67+
```bash
68+
curl -sfL https://raw.githubusercontent.com/harvester/cloud-provider-harvester/master/deploy/generate_addon.sh | bash -s <serviceaccount name> <namespace>
69+
```
7070

7171
:::note
7272

73-
The `generate_addon.sh` script depends on `kubectl` and `jq` to operate the Harvester cluster.
73+
The script depends on `kubectl` and `jq` when operating the Harvester cluster, and functions only when given access to the `Harvester Cluster` kubeconfig file.
7474

75-
The script needs access to the `Harvester Cluster` kubeconfig to work. You can find the `kubeconfig` file from one of the Harvester management nodes in the `/etc/rancher/rke2/rke2.yaml` path.
75+
You can find the `kubeconfig` file in one of the Harvester management nodes in the `/etc/rancher/rke2/rke2.yaml` path. The server IP must be replaced with the VIP address.
76+
77+
Example of content:
78+
79+
```yaml
80+
apiVersion: v1
81+
clusters:
82+
- cluster:
83+
certificate-authority-data: <redacted>
84+
server: https://127.0.0.1:6443
85+
name: default
86+
# ...
87+
```
7688

77-
The namespace needs to be the namespace in which the guest cluster will be created.
89+
You must specify the namespace in which the guest cluster will be created.
7890

7991
:::
8092

93+
Example of output:
8194

82-
2. Configure the **Cloud Provider** to `Harvester` and select **Create** to spin up the cluster.
83-
![](/img/v1.2/rancher/create-custom-rke2.png)
95+
```yaml
96+
########## cloud config ############
97+
apiVersion: v1
98+
clusters:
99+
- cluster:
100+
certificate-authority-data: <CACERT>
101+
server: https://HARVESTER-ENDPOINT/k8s/clusters/local
102+
name: local
103+
contexts:
104+
- context:
105+
cluster: local
106+
namespace: default
107+
user: harvester-cloud-provider-default-local
108+
name: harvester-cloud-provider-default-local
109+
current-context: harvester-cloud-provider-default-local
110+
kind: Config
111+
preferences: {}
112+
users:
113+
- name: harvester-cloud-provider-default-local
114+
user:
115+
token: <TOKEN>
116+
117+
########## cloud-init user data ############
118+
write_files:
119+
- encoding: b64
120+
content: <CONTENT>
121+
owner: root:root
122+
path: /etc/kubernetes/cloud-config
123+
permissions: '0644'
124+
```
125+
126+
1. Create a VM in the Harvester cluster with the following settings:
127+
128+
- **Basics** tab: The minimum requirements are 2 CPUs and 4 GiB of RAM. The required disk space depends on the VM image.
129+
130+
![](/img/v1.3/rancher/custom-cluster-vm-cpu-and-ram.png)
131+
132+
- **Networks** tab: Specify a network name with the format `nic-<number>`.
133+
134+
![](/img/v1.3/rancher/custom-cluster-vm-network.png)
135+
136+
- **Advanced Options** tab: Copy and paste the content of the **Cloud Config User Data** screen.
137+
138+
![](/img/v1.3/rancher/custom-cluster-vm-user-data.png)
139+
140+
1. On the **Basics** tab of the **Cluster Configuration** screen, select **Harvester** as the **Cloud Provider** and then select **Create** to spin up the cluster.
141+
142+
![](/img/v1.2/rancher/create-custom-rke2.png)
143+
144+
1. On the **Registration** tab, perform the steps required to run the RKE2 registration command on the VM.
145+
146+
![](/img/v1.3/rancher/custom-cluster-registration.png)
84147

85148
### Deploying to the K3s cluster with Harvester node driver (experimental)
86149

@@ -115,8 +178,8 @@ When spinning up a K3s cluster using the Harvester node driver, you can perform
115178
- name: harvester-cloud-provider-default-local
116179
user:
117180
token: <TOKEN>
118-
119-
181+
182+
120183
########## cloud-init user data ############
121184
write_files:
122185
- encoding: b64
@@ -188,12 +251,12 @@ The cloud provider can be upgraded by upgrading the RKE2 version. You can upgrad
188251
RKE/K3s upgrade cloud provider via the Rancher UI, as follows:
189252
1. Click **☰ > RKE/K3s Cluster > Apps > Installed Apps**.
190253
2. Find the cloud provider chart and select ⋮ **> Edit/Upgrade**.
191-
3. Select **Version**.
254+
3. Select **Version**.
192255
4. Click **Next > Update**.
193256

194257
:::note
195258

196-
The upgrade process for a [single-node guest cluster](../advanced/singlenodeclusters) may stall when the new `harvester-cloud-provider` pod is stuck in the *Pending* state. This issue is caused by a section in the `harvester-cloud-provider` deployment that describes the rolling update strategy. Specifically, the default value conflicts with the `podAntiAffinity` configuration in single-node clusters.
259+
The upgrade process for a [single-node guest cluster](../advanced/singlenodeclusters) may stall when the new `harvester-cloud-provider` pod is stuck in the *Pending* state. This issue is caused by a section in the `harvester-cloud-provider` deployment that describes the rolling update strategy. Specifically, the default value conflicts with the `podAntiAffinity` configuration in single-node clusters.
197260

198261
For more information, see [this GitHub issue comment](https://github.com/harvester/harvester/issues/5348#issuecomment-2055453709). To address the issue, manually delete the old `harvester-cloud-provider` pod. You might need to do this multiple times until the new pod can be successfully scheduled.
199262

@@ -203,7 +266,7 @@ For more information, see [this GitHub issue comment](https://github.com/harvest
203266
Once you've deployed the Harvester cloud provider, you can leverage the Kubernetes `LoadBalancer` service to expose a microservice within the guest cluster to the external world. Creating a Kubernetes `LoadBalancer` service assigns a dedicated Harvester load balancer to the service, and you can make adjustments through the `Add-on Config` within the Rancher UI.
204267
205268
![](/img/v1.2/rancher/lb-svc.png)
206-
269+
207270
208271
### IPAM
209272
Harvester's built-in load balancer offers both **DHCP** and **Pool** modes, and you can configure it by adding the annotation `cloudprovider.harvesterhci.io/ipam: $mode` to its corresponding service. Starting from Harvester cloud provider >= v0.2.0, it also introduces a unique **Share IP** mode. A service shares its load balancer IP with other services in this mode.

0 commit comments

Comments
 (0)