k8s

Test k8s
========
- kubectl run -it test1 --image=busybox --restart=Never -- /bin/sh
- kubectl run test2 --image =busybox --restart=Never -- sleep 100d
- kubectl exec -it test2 /bin/sh


check pod ns
============
- get pod container id
  $ kubectl get po <POD-NAME> -o jsonpath='{.status.containerStatuses[0].containerID}' | cut -c 10-21
- get container pid
  $ docker inspect --format '{{ .State.Pid }}' <ContainerID>
- check ns
  $ nsenter -t <PID> -n ip a


add node manually
=================
- /etc/sysconfig/flanneld
  https://kubernetes.io/docs/getting-started-guides/fedora/flannel_multi_node_cluster/

- /etc/kubernetes/kubelet
  https://kubernetes.io/docs/getting-started-guides/fedora/fedora_manual_config/

- /etc/environment

- /etc/kubernetes/config

- /etc/kubernetes/manifests/kube-proxy.yaml


tricks
======
- deploy/cmd/destroy
  kubectl run test1 --rm --restart=Never -it --image=busybox -- ip a
- deploy
  kubectl run test1 --image=docker.io/busybox:latest -it sh
- get pod id
  kubectl get pod | grep test1
- run command
  kubectl exec test1-xxxxxxxxx-xxxxx -t -- ip a


flannel
=======
container to internet
---------------------
container veth -> veth in host bridge -> iptable (masq) -> internet

container to container
----------------------
container veth -> veth in host bridge -> host routing -> flannel -> vxlan -> peer flannel
-> peer host routing -> veth in peer host bridge -> peer container veth


deploy
======
https://vitux.com/install-and-deploy-kubernetes-on-ubuntu/
https://stackoverflow.com/questions/58024643/kubernetes-master-node-not-ready-state

[master+node]
$ sudo apt install docker.io
# cat > /etc/docker/daemon.json <<EOF
{
  "exec-opts": ["native.cgroupdriver=systemd"],
  "log-driver": "json-file",
  "log-opts": {
    "max-size": "100m"
  },
  "storage-driver": "overlay2"
}
EOF

$ mkdir -p /etc/systemd/system/docker.service.d
$ systemctl daemon-reload
$ systemctl restart docker

$ sudo systemctl enable docker
$ curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo apt-key add
$ sudo apt-add-repository "deb http://apt.kubernetes.io/ kubernetes-xenial main"
$ sudo apt install kubeadm
$ sudo sh -c 'kubectl completion bash >/etc/bash_completion.d/kubectl'
$ sudo swapoff -a
$ sudo sysctl net.bridge.bridge-nf-call-iptables=1


[master]
$ sudo kubeadm init --pod-network-cidr=10.244.0.0/16
$ mkdir -p $HOME/.kube
$ sudo cp /etc/kubernetes/admin.conf $HOME/.kube/config
$ sudo chown $(id -u):$(id -g) $HOME/.kube/config

// k8s 1.16 bug: "cniVersion": "0.2.0"
$ sudo kubectl apply -f https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml

[node]
$ sudo kubeadm join ....


ignore prechecking error
========================
--extra-config=kubeadm.ignore-preflight-errors=SystemVerification


minikube coredns crashloop
==========================
https://github.com/coredns/coredns/blob/master/plugin/loop/README.md
$ cp /run/systemd/resolve/resolv.conf /tmp
$ sudo systemctl stop systemd-resolved
$ sudo systemctl disable systemd-resolved
$ sudo mv /tmp/resolv.conf /etc/resolv.conf


hugepage sysfs
==============
echo 2147483648 > /sys/fs/cgroup/hugetlb/kubepods/hugetlb.2MB.limit_in_bytes