Skip to content

Commit bdc7939

Browse files
ivandelicivandelic
committed
Adding contetn for memory and CPU reservations for Kubernetes system daemons
1 parent 85b2bcc commit bdc7939

File tree

2 files changed

+119
-0
lines changed

2 files changed

+119
-0
lines changed

‎app-dev/oke/oke-kube-reserved-cloud-init/README.md

Lines changed: 71 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,71 @@
1+
# Reserve Memory and CPU for Kubernetes System Daemons
2+
3+
Kubernetes resources can be reserved at the node pool level and applied to every worker node with cloud-init execution at boot time. Remember, OKE uses cloud-init to set up the worker node customizations.
4+
5+
If you wonder what the magic number for CPU and memory reservations is, the answer is hidden in the node shape and size correlation. The implementation below is based on well-known algorithms from the market described [here](https://learnk8s.io/allocatable-resources).
6+
7+
The script reserves memory with the following rules:
8+
- 255 MiB for every 1 GB of memory (up to 4 GB)
9+
- 205 MiB for every 1 GB of memory (from 4 GB up to 8 GB)
10+
- 105 MiB for every 1 GB of memory (from 8 GB up to 16 GB)
11+
- 160 MiB for every 1 GB of memory (from 16 GB up to 128 GB)
12+
- 20 MiB for every 1 GB of memory (from 128 GB)
13+
14+
The script reserves CPU with the following rules:
15+
- 60 milicores for the first physical core (OCPU)
16+
- 10 milicores for the second physical core (OCPU)
17+
- 5 milicores for the third and fourth physical core (OCPU)
18+
- 3 milicores for every further physical core (OCPU)
19+
20+
Paste the following [cloud-init script](/app-dev/oke/oke-kube-reserved-cloud-init/cloud-init.sh) to every node pool where you want to reserve kube resources. The quantity of reserved resources depends on the node’s memory and OCPUs. The process of adding cloud-init scripts to OKE is defined [here](https://docs.oracle.com/en-us/iaas/Content/ContEng/Tasks/contengusingcustomcloudinitscripts.htm#contengusingcustomcloudinitscripts_topic_Using_the_Console).
21+
22+
```bash
23+
#!/bin/bash
24+
curl --fail -H "Authorization: Bearer Oracle" -L0 http://169.254.169.254/opc/v2/instance/metadata/oke_init_script | base64 --decode >/var/run/oke-init.sh
25+
26+
node_memory_gb=$(curl -s --fail -H "Authorization: Bearer Oracle" -L0 http://169.254.169.254/opc/v2/instance/ | jq '.shapeConfig.memoryInGBs' | awk '{print int($0)}')
27+
node_cpu_count=$(curl -s --fail -H "Authorization: Bearer Oracle" -L0 http://169.254.169.254/opc/v2/instance/ | jq '.shapeConfig.ocpus' | awk '{print int($0)}')
28+
29+
kube_cpu_allocation=0
30+
kube_memory_allocation=0
31+
32+
# Calculate CPU allocations for system daemons
33+
i=1
34+
while [ $i -le $node_cpu_count ]
35+
do
36+
if (( i == 1 )); then
37+
kube_cpu_allocation=$((kube_cpu_allocation + 60))
38+
elif (( i == 2 )); then
39+
kube_cpu_allocation=$((kube_cpu_allocation + 10))
40+
elif (( i == 3 || i == 4 )); then
41+
kube_cpu_allocation=$((kube_cpu_allocation + 5))
42+
elif (( i > 4 )); then
43+
kube_cpu_allocation=$((kube_cpu_allocation + 3))
44+
fi
45+
((i++))
46+
done
47+
48+
# Calculate Memory allocations for system daemons
49+
i=1
50+
while [ $i -le $node_memory_gb ]
51+
do
52+
if (( i <= 4 )); then
53+
kube_memory_allocation=$((kube_memory_allocation + 255))
54+
elif (( i > 4 && i <= 8 )); then
55+
kube_memory_allocation=$((kube_memory_allocation + 205))
56+
elif (( i > 8 && i <= 16 )); then
57+
kube_memory_allocation=$((kube_memory_allocation + 105))
58+
elif (( i > 16 && i <= 128 )); then
59+
kube_memory_allocation=$((kube_memory_allocation + 60))
60+
elif (( i > 128 )); then
61+
kube_memory_allocation=$((kube_memory_allocation + 20))
62+
fi
63+
((i++))
64+
done
65+
66+
echo "CPU $kube_cpu_allocation in total"
67+
echo "Memory $kube_memory_allocation in total"
68+
69+
# configure kubelet with image credential provider
70+
bash /var/run/oke-init.sh --kubelet-extra-args "--kube-reserved=cpu="$kube_cpu_allocation"m,memory="$kube_memory_allocation"Mi"
71+
```

‎app-dev/oke/oke-kube-reserved-cloud-init/cloud-init.sh

Lines changed: 48 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,48 @@
1+
#!/bin/bash
2+
curl --fail -H "Authorization: Bearer Oracle" -L0 http://169.254.169.254/opc/v2/instance/metadata/oke_init_script | base64 --decode >/var/run/oke-init.sh
3+
4+
node_memory_gb=$(curl -s --fail -H "Authorization: Bearer Oracle" -L0 http://169.254.169.254/opc/v2/instance/ | jq '.shapeConfig.memoryInGBs' | awk '{print int($0)}')
5+
node_cpu_count=$(curl -s --fail -H "Authorization: Bearer Oracle" -L0 http://169.254.169.254/opc/v2/instance/ | jq '.shapeConfig.ocpus' | awk '{print int($0)}')
6+
7+
kube_cpu_allocation=0
8+
kube_memory_allocation=0
9+
10+
# Calculate CPU allocations for system daemons
11+
i=1
12+
while [ $i -le $node_cpu_count ]
13+
do
14+
if (( i == 1 )); then
15+
kube_cpu_allocation=$((kube_cpu_allocation + 60))
16+
elif (( i == 2 )); then
17+
kube_cpu_allocation=$((kube_cpu_allocation + 10))
18+
elif (( i == 3 || i == 4 )); then
19+
kube_cpu_allocation=$((kube_cpu_allocation + 5))
20+
elif (( i > 4 )); then
21+
kube_cpu_allocation=$((kube_cpu_allocation + 3))
22+
fi
23+
((i++))
24+
done
25+
26+
# Calculate Memory allocations for system daemons
27+
i=1
28+
while [ $i -le $node_memory_gb ]
29+
do
30+
if (( i <= 4 )); then
31+
kube_memory_allocation=$((kube_memory_allocation + 255))
32+
elif (( i > 4 && i <= 8 )); then
33+
kube_memory_allocation=$((kube_memory_allocation + 205))
34+
elif (( i > 8 && i <= 16 )); then
35+
kube_memory_allocation=$((kube_memory_allocation + 105))
36+
elif (( i > 16 && i <= 128 )); then
37+
kube_memory_allocation=$((kube_memory_allocation + 60))
38+
elif (( i > 128 )); then
39+
kube_memory_allocation=$((kube_memory_allocation + 20))
40+
fi
41+
((i++))
42+
done
43+
44+
echo "CPU $kube_cpu_allocation in total"
45+
echo "Memory $kube_memory_allocation in total"
46+
47+
# configure kubelet with image credential provider
48+
bash /var/run/oke-init.sh --kubelet-extra-args "--kube-reserved=cpu="$kube_cpu_allocation"m,memory="$kube_memory_allocation"Mi"

0 commit comments

Comments
 (0)