diff --git a/kubernetes-eks/templates/README.md b/kubernetes-eks/templates/README.md new file mode 100644 index 0000000..742995b --- /dev/null +++ b/kubernetes-eks/templates/README.md @@ -0,0 +1,124 @@ +# Deployment of Streaming Server on EKS + +1. create EKS Cluster + +```bash +export cluster_name=test-cluster-sr +export node_group_name=node-west-1 +export service_account=test_eks_sa + +eksctl create cluster \ +--name ${cluster_name} \ +--version 1.23 \ +--region eu-west-1 \ +--nodegroup-name ${node_group_name} \ +--node-type m5.large \ +--nodes 2 + +1. Connect using the Kubeconfig file created and create alias + +```bash +export KUBECONFIG=/Users/jusi/.kube/config +alias k='kubectl' +alias kap='kubectl apply -f' +``` + +2. Check if there is storage class or any pvs + +## Get storage class + +```bash +k get sc && k get pv +NAME PROVISIONER RECLAIMPOLICY VOLUMEBINDINGMODE ALLOWVOLUMEEXPANSION AGE +gp2 (default) kubernetes.io/aws-ebs Delete WaitForFirstConsumer false 19m +``` + +There is already by default a storage class on the cluster but EKS role doesn't have permission to create dynamic EBS volume. + +As the Storage Class is already created on K8S v1.23 and already set as default, I don't need to create any Storage Class. :relaxed: + +Moreover, The Storage class is defined as "WaitForFirstConsumer" so the PVC will not be bind until the pod using this pvc is created. + +:info: If we try to create any PVC without creating any PV or without dynamic provisioning using storage class, the PVC will stay in pending mode. + +3. Give the permission to the EKS Cluster + +This documentation was followed to add the permission for the Service account to be able to create dynamic EBS volume + +[eks-persistent-storage](https://aws.amazon.com/premiumsupport/knowledge-center/eks-persistent-storage/) + +Once, this is configured, no need to create any Persistent Volume and we will be able to use only Persistent Volume Claim dynamically. + +## Deployment of the pods, pvc and services + +This command will deploy all the yml files in the manifests directory. + +```bash +k apply -f manifests +``` + +## Description and explanation + +I've add a InitContainer for the streaming server deployment to create a directory and give permission as the container was never starting and has this error: + +```bash +k logs streaming-server-6998587d9c-7m5qm +nginx: [emerg] mkdir() "/var/lib/streaming/hls/" failed (13: Permission denied) +``` + +3 services(svc) were created: +- 1 ClusterIP ( default ) as it will be only internal +- 2 Load Balancer's Type to be publicly accessible + +I was able to connect on both ALB URL ( on port 1935 with OBS and from curl for port ) that I get from the svc but I have issue on the Nginx/streaming server ( Forbidden ) + + +```bash +k get svc +NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE +kubernetes ClusterIP 10.100.0.1 443/TCP 10h +streaming-consumer LoadBalancer 10.100.177.117 a56c0f9bec4bf4b05bbe0b00d5b2f8d6-1896631646.eu-west-1.elb.amazonaws.com 9999:31077/TCP 7h22m +streaming-server-external LoadBalancer 10.100.43.5 a549243e74b1a4c4fb481f18d7361d65-106262341.eu-west-1.elb.amazonaws.com 1935:31095/TCP 3h57m +streaming-server-internal ClusterIP 10.100.10.47 8080/TCP 6h2m +``` + +Issue with Nginx: + +``` +curl http://a56c0f9bec4bf4b05bbe0b00d5b2f8d6-1896631646.eu-west-1.elb.amazonaws.com:9999/ + +403 Forbidden + +

403 Forbidden

+
nginx/1.15.8
+ + +``` + +I have the same issue using the docker-compose. + +From my test and research, it's not an error from my deployment and configuration but an error from the docker image and the permission. + + +### Cleanup + + + +```bash +k delete -f manifests +persistentvolumeclaim "streaming-consumer-claim0" deleted +deployment.apps "streaming-consumer" deleted +service "streaming-consumer" deleted +persistentvolumeclaim "streaming-server-claim0" deleted +deployment.apps "streaming-server" deleted +service "streaming-server-external" deleted +service "streaming-server-internal" deleted +``` + +### Deleting EKS Cluster + +```bash +eksctl delete cluster --name ${cluster_name} +``` + +And delete the role and policies created for the service account diff --git a/kubernetes-eks/templates/manifests/streaming-consumer-claim0-persistentvolumeclaim.yaml b/kubernetes-eks/templates/manifests/streaming-consumer-claim0-persistentvolumeclaim.yaml new file mode 100644 index 0000000..9211163 --- /dev/null +++ b/kubernetes-eks/templates/manifests/streaming-consumer-claim0-persistentvolumeclaim.yaml @@ -0,0 +1,15 @@ +apiVersion: v1 +kind: PersistentVolumeClaim +metadata: + creationTimestamp: null + labels: + io.kompose.service: streaming-consumer-claim0 + name: streaming-consumer-claim0 +spec: + accessModes: + - ReadWriteOnce + storageClassName: gp2 + resources: + requests: + storage: 100Mi +status: {} diff --git a/kubernetes-eks/templates/manifests/streaming-consumer-deployment.yaml b/kubernetes-eks/templates/manifests/streaming-consumer-deployment.yaml new file mode 100644 index 0000000..0a7c078 --- /dev/null +++ b/kubernetes-eks/templates/manifests/streaming-consumer-deployment.yaml @@ -0,0 +1,43 @@ +apiVersion: apps/v1 +kind: Deployment +metadata: + annotations: + kompose.cmd: kompose convert -c + kompose.version: 1.26.1 (HEAD) + creationTimestamp: null + labels: + io.kompose.service: streaming-consumer + name: streaming-consumer +spec: + replicas: 1 + selector: + matchLabels: + io.kompose.service: streaming-consumer + strategy: + type: Recreate + template: + metadata: + annotations: + kompose.cmd: kompose convert -c + kompose.version: 1.26.1 (HEAD) + creationTimestamp: null + labels: + io.kompose.service: streaming-consumer + spec: + containers: + - image: codeworksio/nginx + name: streaming-consumer + ports: + - containerPort: 8080 + resources: {} + volumeMounts: + - mountPath: /var/www + name: streaming-consumer-claim0 + readOnly: false + restartPolicy: Always + volumes: + - name: streaming-consumer-claim0 + persistentVolumeClaim: + claimName: streaming-consumer-claim0 + readOnly: false +status: {} diff --git a/kubernetes-eks/templates/manifests/streaming-consumer-service.yaml b/kubernetes-eks/templates/manifests/streaming-consumer-service.yaml new file mode 100644 index 0000000..210172a --- /dev/null +++ b/kubernetes-eks/templates/manifests/streaming-consumer-service.yaml @@ -0,0 +1,17 @@ +apiVersion: v1 +kind: Service +metadata: + creationTimestamp: null + labels: + io.kompose.service: streaming-consumer + name: streaming-consumer +spec: + # This will configure an ELB to handle incoming traffic to this service. + type: LoadBalancer + ports: + - protocol: TCP + name: "9999" + port: 9999 + targetPort: 8080 + selector: + io.kompose.service: streaming-consumer diff --git a/kubernetes-eks/templates/manifests/streaming-server-claim0-persistentvolumeclaim.yaml b/kubernetes-eks/templates/manifests/streaming-server-claim0-persistentvolumeclaim.yaml new file mode 100644 index 0000000..712c21c --- /dev/null +++ b/kubernetes-eks/templates/manifests/streaming-server-claim0-persistentvolumeclaim.yaml @@ -0,0 +1,15 @@ +apiVersion: v1 +kind: PersistentVolumeClaim +metadata: + creationTimestamp: null + labels: + io.kompose.service: streaming-server-claim0 + name: streaming-server-claim0 +spec: + accessModes: + - ReadWriteOnce + storageClassName: gp2 + resources: + requests: + storage: 1Gi +status: {} diff --git a/kubernetes-eks/templates/manifests/streaming-server-deployment.yaml b/kubernetes-eks/templates/manifests/streaming-server-deployment.yaml new file mode 100644 index 0000000..a73eee6 --- /dev/null +++ b/kubernetes-eks/templates/manifests/streaming-server-deployment.yaml @@ -0,0 +1,49 @@ +apiVersion: apps/v1 +kind: Deployment +metadata: + annotations: + kompose.cmd: kompose convert -c + kompose.version: 1.26.1 (HEAD) + creationTimestamp: null + labels: + io.kompose.service: streaming-server + name: streaming-server +spec: + replicas: 1 + selector: + matchLabels: + io.kompose.service: streaming-server + strategy: + type: Recreate + template: + metadata: + annotations: + kompose.cmd: kompose convert -c + kompose.version: 1.26.1 (HEAD) + creationTimestamp: null + labels: + io.kompose.service: streaming-server + spec: + initContainers: + - name: init + image: busybox:1.28 + command: ['sh', '-c', "mkdir -p /var/lib/streaming/hls/ && chmod -R 777 /var/lib/streaming/"] + volumeMounts: + - mountPath: /var/lib/streaming + name: streaming-server-claim0 + containers: + - image: codeworksio/streaming-server + name: streaming-server + ports: + - containerPort: 1935 + - containerPort: 8080 + resources: {} + volumeMounts: + - mountPath: /var/lib/streaming + name: streaming-server-claim0 + restartPolicy: Always + volumes: + - name: streaming-server-claim0 + persistentVolumeClaim: + claimName: streaming-server-claim0 +status: {} diff --git a/kubernetes-eks/templates/manifests/streaming-server-service-external.yaml b/kubernetes-eks/templates/manifests/streaming-server-service-external.yaml new file mode 100644 index 0000000..a279263 --- /dev/null +++ b/kubernetes-eks/templates/manifests/streaming-server-service-external.yaml @@ -0,0 +1,19 @@ +apiVersion: v1 +kind: Service +metadata: + annotations: + kompose.cmd: kompose convert -c + kompose.version: 1.26.1 (HEAD) + creationTimestamp: null + labels: + io.kompose.service: streaming-server + name: streaming-server-external +spec: + type: LoadBalancer + ports: + # This will configure an ELB to handle incoming traffic to this service. + - name: "1935" + port: 1935 + targetPort: 1935 + selector: + io.kompose.service: streaming-server diff --git a/kubernetes-eks/templates/manifests/streaming-server-service-internal.yaml b/kubernetes-eks/templates/manifests/streaming-server-service-internal.yaml new file mode 100644 index 0000000..54c19d8 --- /dev/null +++ b/kubernetes-eks/templates/manifests/streaming-server-service-internal.yaml @@ -0,0 +1,18 @@ +apiVersion: v1 +kind: Service +metadata: + annotations: + kompose.cmd: kompose convert -c + kompose.version: 1.26.1 (HEAD) + creationTimestamp: null + labels: + io.kompose.service: streaming-server + name: streaming-server-internal +spec: + ports: + # This Exposes the service on a cluster-internal IP. Choosing this value makes the service only reachable from within the cluster. + - name: "8080" + port: 8080 + targetPort: 8080 + selector: + io.kompose.service: streaming-server