Ampernetacle

This is a Terraform configuration to deploy a Kubernetes cluster on Oracle Cloud Infrastructure. It creates a few virtual machines and uses kubeadm to install a Kubernetes control plane on the first machine, and join the other machines as worker nodes.

By default, it deploys a 4-node cluster using ARM machines. Each machine has 1 OCPU and 6 GB of RAM, which means that the cluster fits within Oracle's (pretty generous if you ask me) free tier.

It is not meant to run production workloads, but it's great if you want to learn Kubernetes with a "real" cluster (i.e. a cluster with multiple nodes) without breaking the bank, and if you want to develop or test applications on ARM.

Getting started

1. Create an Oracle Cloud Infrastructure account (just follow this link).
2. Have installed or install kubernetes.
3. Have installed or install terraform.
4. Have installed or install OCI CLI .
5. Configure OCI credentials.
6. Download this project and enter its folder.
7. terraform init
8. terraform apply

That's it!

At the end of the terraform apply, a kubeconfig file is generated in this directory. To use your new cluster, you can do:

Linux

export KUBECONFIG=$PWD/kubeconfig
kubectl get nodes

Windows

$env:KUBECONFIG="$pwd\kubeconfig"
kubectl get nodes

The command above should show you 4 nodes, named node1 to node4.

You can also log into the VMs. At the end of the Terraform output you should see a command that you can use to SSH into the first VM (just copy-paste the command).

Windows

It works with Windows 10/Powershell 5.1.

It may be necesssary to change the execution policy to unrestricted.

PowerShell ExecutionPolicy

Availability Domain

If you get a message like the following:

Error: 500-InternalError
│ ...
│ Service: Core Instance
│ Error Message: Out of host capacity.

...then you can try to switch to a different availability domain. This can be done by changing the availability_domain input variable. (Thanks @uknbr for the contribution!)

Customization

Check variables.tf to see tweakable parameters. You can change the number of nodes, the size of the nodes, or switch to Intel/AMD instances if you'd like. Keep in mind that if you switch to Intel/AMD instances, you won't get advantage of the free tier.

Stopping the cluster

terraform destroy

Implementation details

This Terraform configuration:

• generates an OpenSSH keypair and a kubeadm token
• deploys 4 VMs using Ubuntu 20.04
• uses cloud-init to install and configure everything
• installs Docker and Kubernetes packages
• runs kubeadm init on the first VM
• runs kubeadm join on the other VMs
• installs the Weave CNI plugin
• transfers the kubeconfig file generated by kubeadm
• patches that file to use the public IP address of the machine

Caveats

There is no cloud controller manager, which means that you cannot create services with type: LoadBalancer; or rather, if you create such services, their EXTERNAL-IP will remain <pending>.

To expose services, use NodePort.

Likewise, there is no ingress controller and no storage class.

(These might be added in a later iteration of this project.)

Edit:

You can use persistent storage with longhorn, here is an example database:

---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: mysql-pv-claim
spec:
  storageClassName: longhorn
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 5Gi
---

apiVersion: v1
kind: Service
metadata:
  name: mysql
spec:
  ports:
  - port: 3306
  selector:
    app: mysql
  clusterIP: None
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: mysql
spec:
  selector:
    matchLabels:
      app: mysql
  strategy:
    type: Recreate
  template:
    metadata:
      labels:
        app: mysql
    spec:
      containers:
      - image: mariadb:10.2
        name: mysql
        env:
          # Use secret in real usage
        - name: MYSQL_ROOT_PASSWORD
          value: "8*&#i7fj2j47d"
        ports:
        - containerPort: 3306
          name: mysql
        volumeMounts:
        - name: mysql-persistent-storage
          mountPath: /var/lib/mysql
      volumes:
      - name: mysql-persistent-storage
        persistentVolumeClaim:
          claimName: mysql-pv-claim

Longhorn will maintain replica volumes in HA on 3 worker nodes and has a UI interface to view volume status.

Edit2: Added Ingress Controller NGINX on master node public ip.

Remarks

Oracle Cloud also has a managed Kubernetes service called Container Engine for Kubernetes (or OKE). That service doesn't have the caveats mentioned above; however, it's not part of the free tier.

What does "Ampernetacle" mean?

It's a porte-manteau between Ampere, Kubernetes, and Oracle. It's probably not the best name in the world but it's the one we have! If you have an idea for a better name let us know. 😊