Skip to content

Commit e7aaafb

Browse files
SyNuOraSyNuOra
authored
Merge branch 'main' into synuora-data-development-CICD
2 parents c8bbe8b + 1a86de1 commit e7aaafb

File tree

343 files changed

+7621
-3652
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

343 files changed

+7621
-3652
lines changed

‎app-dev/README.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -7,6 +7,8 @@ EMEA AppDev is an expert team of Cloud Native architects and developers focusing
77

88
# Team Publications
99

10+
- [Reserve Memory and CPU for Kubernetes System Daemons](/app-dev/oke/oke-kube-reserved-cloud-init/README.md)
11+
- [Image Credential Provider for OKE](https://github.com/oracle-devrel/oke-credential-provider-for-ocir/)
1012
- [Cloud Coaching - OCI Observability for Kubernetes monitoring](https://www.youtube.com/watch?v=DLce-UReoKo)[November 2023]
1113
- [Github Workflow or OCI DevOps for deploying a Springboot App on OCI Container Instances](https://www.youtube.com/watch?v=Nc4lMeEuq5c)[October 2023]
1214
- [OKE in a Hub-Spoke Architecture](https://ivan-delic.medium.com/advanced-kubernetes-networking-oke-in-a-hub-spoke-architectures-f0ba2256e824)[September 2023]

‎app-dev/oke/oke-kube-reserved-cloud-init/README.md

Lines changed: 71 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,71 @@
1+
# Reserve Memory and CPU for Kubernetes System Daemons
2+
3+
Kubernetes resources can be reserved at the node pool level and applied to every worker node with cloud-init execution at boot time. Remember, OKE uses cloud-init to set up the worker node customizations.
4+
5+
If you wonder what the magic number for CPU and memory reservations is, the answer is hidden in the node shape and size correlation. The implementation below is based on well-known algorithms from the market described [here](https://learnk8s.io/allocatable-resources).
6+
7+
The script reserves memory with the following rules:
8+
- 255 MiB for every 1 GB of memory (up to 4 GB)
9+
- 205 MiB for every 1 GB of memory (from 4 GB up to 8 GB)
10+
- 105 MiB for every 1 GB of memory (from 8 GB up to 16 GB)
11+
- 160 MiB for every 1 GB of memory (from 16 GB up to 128 GB)
12+
- 20 MiB for every 1 GB of memory (from 128 GB)
13+
14+
The script reserves CPU with the following rules:
15+
- 60 milicores for the first physical core (OCPU)
16+
- 10 milicores for the second physical core (OCPU)
17+
- 5 milicores for the third and fourth physical core (OCPU)
18+
- 3 milicores for every further physical core (OCPU)
19+
20+
Paste the following [cloud-init script](/app-dev/oke/oke-kube-reserved-cloud-init/cloud-init.sh) to every node pool where you want to reserve kube resources. The quantity of reserved resources depends on the node’s memory and OCPUs. The process of adding cloud-init scripts to OKE is defined [here](https://docs.oracle.com/en-us/iaas/Content/ContEng/Tasks/contengusingcustomcloudinitscripts.htm#contengusingcustomcloudinitscripts_topic_Using_the_Console).
21+
22+
```bash
23+
#!/bin/bash
24+
curl --fail -H "Authorization: Bearer Oracle" -L0 http://169.254.169.254/opc/v2/instance/metadata/oke_init_script | base64 --decode >/var/run/oke-init.sh
25+
26+
node_memory_gb=$(curl -s --fail -H "Authorization: Bearer Oracle" -L0 http://169.254.169.254/opc/v2/instance/ | jq '.shapeConfig.memoryInGBs' | awk '{print int($0)}')
27+
node_cpu_count=$(curl -s --fail -H "Authorization: Bearer Oracle" -L0 http://169.254.169.254/opc/v2/instance/ | jq '.shapeConfig.ocpus' | awk '{print int($0)}')
28+
29+
kube_cpu_allocation=0
30+
kube_memory_allocation=0
31+
32+
# Calculate CPU allocations for system daemons
33+
i=1
34+
while [ $i -le $node_cpu_count ]
35+
do
36+
if (( i == 1 )); then
37+
kube_cpu_allocation=$((kube_cpu_allocation + 60))
38+
elif (( i == 2 )); then
39+
kube_cpu_allocation=$((kube_cpu_allocation + 10))
40+
elif (( i == 3 || i == 4 )); then
41+
kube_cpu_allocation=$((kube_cpu_allocation + 5))
42+
elif (( i > 4 )); then
43+
kube_cpu_allocation=$((kube_cpu_allocation + 3))
44+
fi
45+
((i++))
46+
done
47+
48+
# Calculate Memory allocations for system daemons
49+
i=1
50+
while [ $i -le $node_memory_gb ]
51+
do
52+
if (( i <= 4 )); then
53+
kube_memory_allocation=$((kube_memory_allocation + 255))
54+
elif (( i > 4 && i <= 8 )); then
55+
kube_memory_allocation=$((kube_memory_allocation + 205))
56+
elif (( i > 8 && i <= 16 )); then
57+
kube_memory_allocation=$((kube_memory_allocation + 105))
58+
elif (( i > 16 && i <= 128 )); then
59+
kube_memory_allocation=$((kube_memory_allocation + 60))
60+
elif (( i > 128 )); then
61+
kube_memory_allocation=$((kube_memory_allocation + 20))
62+
fi
63+
((i++))
64+
done
65+
66+
echo "CPU $kube_cpu_allocation in total"
67+
echo "Memory $kube_memory_allocation in total"
68+
69+
# configure kubelet with image credential provider
70+
bash /var/run/oke-init.sh --kubelet-extra-args "--kube-reserved=cpu="$kube_cpu_allocation"m,memory="$kube_memory_allocation"Mi"
71+
```

‎app-dev/oke/oke-kube-reserved-cloud-init/cloud-init.sh

Lines changed: 48 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,48 @@
1+
#!/bin/bash
2+
curl --fail -H "Authorization: Bearer Oracle" -L0 http://169.254.169.254/opc/v2/instance/metadata/oke_init_script | base64 --decode >/var/run/oke-init.sh
3+
4+
node_memory_gb=$(curl -s --fail -H "Authorization: Bearer Oracle" -L0 http://169.254.169.254/opc/v2/instance/ | jq '.shapeConfig.memoryInGBs' | awk '{print int($0)}')
5+
node_cpu_count=$(curl -s --fail -H "Authorization: Bearer Oracle" -L0 http://169.254.169.254/opc/v2/instance/ | jq '.shapeConfig.ocpus' | awk '{print int($0)}')
6+
7+
kube_cpu_allocation=0
8+
kube_memory_allocation=0
9+
10+
# Calculate CPU allocations for system daemons
11+
i=1
12+
while [ $i -le $node_cpu_count ]
13+
do
14+
if (( i == 1 )); then
15+
kube_cpu_allocation=$((kube_cpu_allocation + 60))
16+
elif (( i == 2 )); then
17+
kube_cpu_allocation=$((kube_cpu_allocation + 10))
18+
elif (( i == 3 || i == 4 )); then
19+
kube_cpu_allocation=$((kube_cpu_allocation + 5))
20+
elif (( i > 4 )); then
21+
kube_cpu_allocation=$((kube_cpu_allocation + 3))
22+
fi
23+
((i++))
24+
done
25+
26+
# Calculate Memory allocations for system daemons
27+
i=1
28+
while [ $i -le $node_memory_gb ]
29+
do
30+
if (( i <= 4 )); then
31+
kube_memory_allocation=$((kube_memory_allocation + 255))
32+
elif (( i > 4 && i <= 8 )); then
33+
kube_memory_allocation=$((kube_memory_allocation + 205))
34+
elif (( i > 8 && i <= 16 )); then
35+
kube_memory_allocation=$((kube_memory_allocation + 105))
36+
elif (( i > 16 && i <= 128 )); then
37+
kube_memory_allocation=$((kube_memory_allocation + 60))
38+
elif (( i > 128 )); then
39+
kube_memory_allocation=$((kube_memory_allocation + 20))
40+
fi
41+
((i++))
42+
done
43+
44+
echo "CPU $kube_cpu_allocation in total"
45+
echo "Memory $kube_memory_allocation in total"
46+
47+
# configure kubelet with image credential provider
48+
bash /var/run/oke-init.sh --kubelet-extra-args "--kube-reserved=cpu="$kube_cpu_allocation"m,memory="$kube_memory_allocation"Mi"

0 commit comments

Comments
 (0)