Skip to content

Commit 3615943

Browse files
andfasanoandfasano
committed
add node-joiner.sh script and related documentation
1 parent 3a580b0 commit 3615943

File tree

2 files changed

+228
-0
lines changed

2 files changed

+228
-0
lines changed

‎docs/user/agent/add-nodes.md‎

Lines changed: 97 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,97 @@
1+
# Adding a node via the node-joiner tool
2+
3+
## Pre-requisites
4+
1. The `oc` tool must be available in the execution environment (the "user host").
5+
2. The user host has a valid network connection to the target OpenShift cluster to be expanded.
6+
3. The user host has a valid pull-secret.
7+
8+
## Setup
9+
1. Download the [node-joiner.sh](./node-joiner.sh) script in a working directory in
10+
the user host (the "assets folder").
11+
2. Create a `nodes-config.yaml` in the assets folder. This configuration file must contain the
12+
list of all the nodes that the user wants to add to the target cluster. For each node it must be
13+
specified at least the name and the primary interface mac address, for example:
14+
```
15+
hosts:
16+
- hostname: extra-worker-0
17+
interfaces:
18+
- name: eth0
19+
macAddress: 00:02:46:e3:9e:7c
20+
- hostname: extra-worker-1
21+
interfaces:
22+
- name: eth0
23+
macAddress: 00:02:46:e3:9e:8c
24+
- hostname: extra-worker-2
25+
interfaces:
26+
- name: eth0
27+
macAddress: 00:02:46:e3:9e:9c
28+
```
29+
3. Optionally, it's possible to specify - for each node - an `NMState` configuration block
30+
(it will be applied during the first boot), for example:
31+
```
32+
hosts:
33+
- hostname: extra-worker-0
34+
interfaces:
35+
- name: eth0
36+
macAddress: 00:02:46:e3:9e:7c
37+
networkConfig:
38+
interfaces:
39+
- name: eth0
40+
type: ethernet
41+
state: up
42+
mac-address: 00:02:46:e3:9e:7c
43+
ipv4:
44+
enabled: true
45+
address:
46+
- ip: 192.168.111.90
47+
prefix-length: 24
48+
dhcp: false
49+
dns-resolver:
50+
config:
51+
server:
52+
- 192.168.111.1
53+
routes:
54+
config:
55+
- destination: 0.0.0.0/0
56+
next-hop-address: 192.168.111.1
57+
next-hop-interface: eth0
58+
table-id: 254
59+
- hostname: extra-worker-1
60+
interfaces:
61+
- name: eth0
62+
macAddress: 00:02:46:e3:9e:8c
63+
- hostname: extra-worker-2
64+
interfaces:
65+
- name: eth0
66+
macAddress: 00:02:46:e3:9e:9c
67+
68+
## ISO generation
69+
Run the [node-joiner.sh](./node-joiner.sh) by specifying the location of the current pull secret:
70+
```bash
71+
$ ./node-joiner.sh ~/config/pull-secret
72+
```
73+
The script will generate a temporary namespace `openshift-node-joiner` in the target cluster,
74+
where a pod will be launched to execute the effective node-joiner workload.
75+
In case of success, the `agent-addnodes.x86_64.iso` ISO image will be downloaded in the assets folder.
76+
77+
## Nodes joining
78+
Use the iso image to boot all the nodes listed in the `nodes-config.yaml` file, and wait for the related
79+
certificate signing requests (CSRs) to appear. When adding a new node to the cluster, two pending CSRs will
80+
be generated, and they must be manually approved by the user.
81+
Use the following command to monitor the pending certificates:
82+
```
83+
$ oc get csr
84+
```
85+
User the `oc` `approve` command to approve them:
86+
```
87+
$ oc adm certificate approve <csr_name>
88+
```
89+
Once all the pendings certificates will be approved, then the new node will become available:
90+
```
91+
$ oc get nodes
92+
NAME STATUS ROLES AGE VERSION
93+
extra-worker-0 Ready worker 1h v1.29.3+8628c3c
94+
master-0 Ready control-plane,master 31h v1.29.3+8628c3c
95+
master-1 Ready control-plane,master 32h v1.29.3+8628c3c
96+
master-2 Ready control-plane,master 32h v1.29.3+8628c3c
97+
```

‎docs/user/agent/node-joiner.sh‎

Lines changed: 131 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,131 @@
1+
#!/bin/bash
2+
3+
if [ $# -lt 1 ]; then
4+
echo "./node-joiner.sh <pull secret path>"
5+
echo "Usage example:"
6+
echo "$ ./node-joiner.sh ~/config/my-pull-secret"
7+
8+
exit 1
9+
fi
10+
pullSecret=$1
11+
12+
# Extract the installer image pullspec and release version.
13+
releaseImage=$(oc get clusterversion version -o=jsonpath='{.status.history[?(@.state == "Completed")].image}')
14+
nodeJoinerPullspec=$(oc adm release info -a "$pullSecret" --image-for=installer "$releaseImage")
15+
16+
# Create the namespace to run the node-joiner, along with the required roles and bindings.
17+
staticResources=$(cat <<EOF
18+
apiVersion: v1
19+
kind: Namespace
20+
metadata:
21+
name: openshift-node-joiner
22+
---
23+
apiVersion: v1
24+
kind: ServiceAccount
25+
metadata:
26+
name: node-joiner
27+
namespace: openshift-node-joiner
28+
---
29+
apiVersion: rbac.authorization.k8s.io/v1
30+
kind: ClusterRole
31+
metadata:
32+
name: node-joiner
33+
rules:
34+
- apiGroups:
35+
- config.openshift.io
36+
resources:
37+
- clusterversions
38+
- proxies
39+
verbs:
40+
- get
41+
- apiGroups:
42+
- ""
43+
resources:
44+
- secrets
45+
- configmaps
46+
- nodes
47+
verbs:
48+
- get
49+
- list
50+
---
51+
apiVersion: rbac.authorization.k8s.io/v1
52+
kind: ClusterRoleBinding
53+
metadata:
54+
name: node-joiner
55+
subjects:
56+
- kind: ServiceAccount
57+
name: node-joiner
58+
namespace: openshift-node-joiner
59+
roleRef:
60+
kind: ClusterRole
61+
name: node-joiner
62+
apiGroup: rbac.authorization.k8s.io
63+
EOF
64+
)
65+
echo "$staticResources" | oc apply -f -
66+
67+
# Generate a configMap to store the user configuration
68+
oc create configmap nodes-config --from-file=nodes-config.yaml -n openshift-node-joiner -o yaml --dry-run=client | oc apply -f -
69+
70+
# Runt the node-joiner pod to generate the ISO
71+
nodeJoinerPod=$(cat <<EOF
72+
apiVersion: v1
73+
kind: Pod
74+
metadata:
75+
name: node-joiner
76+
namespace: openshift-node-joiner
77+
annotations:
78+
openshift.io/scc: anyuid
79+
labels:
80+
app: node-joiner
81+
spec:
82+
restartPolicy: Never
83+
serviceAccountName: node-joiner
84+
securityContext:
85+
seccompProfile:
86+
type: RuntimeDefault
87+
containers:
88+
- name: node-joiner
89+
imagePullPolicy: IfNotPresent
90+
image: $nodeJoinerPullspec
91+
volumeMounts:
92+
- name: nodes-config
93+
mountPath: /config
94+
- name: assets
95+
mountPath: /assets
96+
command: ["/bin/sh", "-c", "cp /config/nodes-config.yaml /assets; HOME=/assets node-joiner add-nodes --dir=/assets --log-level=debug; echo \$? > /assets/completed; sleep 600"]
97+
volumes:
98+
- name: nodes-config
99+
configMap:
100+
name: nodes-config
101+
namespace: openshift-node-joiner
102+
- name: assets
103+
emptyDir:
104+
sizeLimit: "4Gi"
105+
EOF
106+
)
107+
echo "$nodeJoinerPod" | oc apply -f -
108+
109+
# Wait until the node-joiner was completed.
110+
while true; do
111+
if oc exec node-joiner -n openshift-node-joiner -- test -e /assets/completed >/dev/null 2>&1; then
112+
break
113+
else
114+
echo "Waiting for node-joiner pod to complete..."
115+
sleep 10s
116+
fi
117+
done
118+
119+
# In case of success, let's extract the ISO, otherwise the logs are shown for troubleshooting the error.
120+
completed=$(oc exec node-joiner -n openshift-node-joiner -- cat /assets/completed)
121+
if [ "$completed" = 0 ]; then
122+
echo "node-joiner successfully completed, extracting ISO image..."
123+
oc cp -n openshift-node-joiner node-joiner:/assets/agent-addnodes.x86_64.iso agent-addnodes.x86_64.iso
124+
else
125+
oc logs node-joiner -n openshift-node-joiner
126+
echo "node-joiner failed"
127+
fi
128+
129+
# Remove all the resources previously created.
130+
echo "Cleaning up"
131+
oc delete namespace openshift-node-joiner --grace-period=0 >/dev/null 2>&1 &

0 commit comments

Comments
 (0)