Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
76 changes: 74 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,2 +1,74 @@
# operations
Repository for Evoc Devops Configs.
# EvOC Operations (Infrastructure & DevOps)

Welcome to the **operations** repository for **EvOC (Evolutionary Algorithms on Click)**. This repository contains all the configuration files, automation scripts, and deployment manifests required to set up and manage the EvOC infrastructure across different environments.

## Overview

This repository is designed to automate the deployment of the EvOC microservices architecture. It supports a robust, production-ready environment using Kubernetes and Ansible, while maintaining a simple Docker Compose setup for local development.

### Key Features
- **Automated Kubernetes Deployment**: Full lifecycle management (init/join) of a multi-node K8s cluster.
- **CRI-Dockerd Integration**: Seamlessly bridges Kubernetes with Docker Engine (v0.3.21) for reliable container orchestration.
- **Architecture Aware**: Automatic detection and support for both `amd64` and `arm64` (Apple Silicon) architectures.
- **Infrastructure-as-Code (IaC)**: Ansible playbooks for automated provisioning of CockroachDB, MinIO, RabbitMQ, and Kubernetes components.
- **Microservices Orchestration**: Pre-configured Kubernetes manifests for the Auth, Runner, Frontend, and Controller services.

---

## Repository Structure

| Directory | Description |
|-----------|-------------|
| [`/ansible`](./ansible) | Ansible playbooks and inventory for server-side automation and cluster setup. |
| [`/kubernetes`](./kubernetes) | K8s manifests (Deployments, Services, Secrets) for microservice orchestration. |
| [`/docker`](./docker) | Docker Compose configurations for quick local development. |

---

## Prerequisites

Before starting, ensure you have the following tools installed on your local machine:
- [Ansible CLI](https://docs.ansible.com/ansible/latest/installation_guide/intro_installation.html)
- [Docker Engine](https://docs.docker.com/get-docker/)
- [SSH Access](https://www.ssh.com/academy/ssh/public-key-authentication) to target nodes.

---

## Quick Start

### 1. Local Development (Docker Compose)
For a quick local setup, navigate to the `docker` directory and follow the instructions in its README.
```bash
cd docker
docker-compose up -d
```

### 2. Multi-Node Production Cluster (Ansible + K8s)
For setting up a full production-ready cluster, refer to the [Ansible README](./ansible/README.md).
1. Configure your nodes in `ansible/inventory.ini`.
2. Define your secrets in `ansible/vars/`.
3. Run the playbook:
```bash
cd ansible
ansible-playbook -i inventory.ini playbook.yaml -kK
```

---

## Security & Secrets

Sensitive information (API keys, database credentials) are managed via:
- **Ansible Vars**: Stored in `ansible/vars/` (excluded from Git).
- **K8s Secrets**: Defined in `kubernetes/evolve-secrets.yaml`.

*Always ensure your local secrets files are excluded from version control.*

---

## Contributing
Please follow the standard Git workflow:
1. Fork the repo.
2. Create your feature branch (`git checkout -b feature/amazing-feature`).
3. Commit your changes (`git commit -m 'Add some amazing feature'`).
4. Push to the branch (`git push origin feature/amazing-feature`).
5. Open a Pull Request.
103 changes: 57 additions & 46 deletions ansible/README.md
Original file line number Diff line number Diff line change
@@ -1,22 +1,25 @@
# Deploy EvOC on Infrastructure
# 🤖 Automated Deployment with Ansible

# Requirements
This directory contains the automation logic to provision target servers from scratch, configure a multi-node Kubernetes cluster, and deploy all EvOC infrastructure components.

- `Ansible CLI` installed
- `SSH sudo access` to the target server
## 🛠 Prerequisites

# Configure Hosts
- **Ansible 2.10+** installed locally.
- **Ubuntu 22.04 / 24.04** on target servers (Recommended).
- **SSH access** with sudo privileges on all nodes.

In the inventory file, specify the target server's IP address or hostname under the `[master]` and `[worker]` group. Also, specify the SSH user that has sudo access to the target server.
---

```ini
# Define all unique hosts and their
# connection details ONCE using logical names.
## ⚙️ Configuration

### 1. Inventory Setup
Edit `inventory.ini` to define your cluster topology.

```ini
[nodes]
master-node ansible_host=<your_master_node_ip>
worker-node1 ansible_host=<your_worker_node1_ip>
worker-node2 ansible_host=<your_worker_node2_ip>
master-node ansible_host=1.2.3.4
worker-node1 ansible_host=1.2.3.5
worker-node2 ansible_host=1.2.3.6

[master]
master-node
Expand All @@ -25,51 +28,59 @@ master-node
worker-node1
worker-node2

[cockroachdb]
worker-node1
worker-node2

# ------IMPORTANT---------
# Exactly one node only
# should be used for MinIO.
[minio]
worker-node2
[all:vars]
ansible_user=ubuntu
```

# ------IMPORTANT---------
# Exactly one node only
# should be used for RabbitMQ.
[rabbitmq]
worker-node3
### 2. Variable Configuration
Create the following secret files in `ansible/vars/` (these are ignored by Git):

[all:children]
master
workers
#### `ansible/vars/crdb_secrets.yml`
```yaml
crdb_enterprise_license_key: "YOUR_LICENSE_KEY_HERE"
```

[all:vars]
ansible_user=<your_ssh_user>
#### `ansible/vars/minio_secrets.yml`
```yaml
minio_root_user: "admin"
minio_root_password: "a-very-secure-password"
```

# CockroachDB License Key
---

## 🚀 Execution

Get a cockroachdb enterprise-free license key by declaring usage of cockroachdb for `academic research` as that's the goal of this project. You can get the license key from the [CockroachDB docs](https://www.cockroachlabs.com/docs/stable/licensing-faqs). After that, create a file in path `ansible/vars/crdb_secrets.yml` and paste the license key in the file. The file should look like this:
Run the deployment from the root or `ansible` directory:

```yml
crdb_enterprise_license_key: "<<YOUR_LICENSE_KEY>>"
```bash
ansible-playbook -i inventory.ini playbook.yaml -kK
```

# Minio Config
### What This Playbook Does:
1. **System Preparation**: Disables swap, configures kernel modules (`overlay`, `br_netfilter`), and sets sysctl parameters.
2. **Container Runtime**: Installs Docker and the `cri-dockerd` (v0.3.21) bridge to enable Kubernetes-Docker compatibility.
3. **Kubernetes Stack**: Installs `kubeadm`, `kubelet`, and `kubectl` (v1.31).
4. **Cluster Lifecycle**:
- Initializes the Master node.
- Generates and shares join tokens dynamically with Workers.
- Installs **Flannel CNI** for cross-node networking.
5. **Infrastructure**: Provisions **CockroachDB** (Multi-node), **MinIO**, and **RabbitMQ**.
6. **Application**: Automatically copies and applies all manifests from the `/kubernetes` directory.

To use Minio as the object storage for EvOC, you need to create a file in path `ansible/vars/minio_secrets.yml` and paste the access key and secret key in the file. The file should look like this:
---

```yml
minio_root_user: "<username>"
minio_root_password: "<password>"
```
## 🧩 Advanced Features

# Execute the Playbook
### 🔌 CRI-Dockerd Integration
As Kubernetes has deprecated direct Docker support, this playbook integrates **`cri-dockerd`**. This shim allows you to continue using the Docker Engine as the container runtime while maintaining full compatibility with the Kubernetes CRI.

Run the following command to execute the playbook from the `ansible` directory on your local machine. It will prompt for the SSH password and the sudo password for the target server. Do provide the correct passwords when prompted.
### 💻 Multi-Architecture Support
The playbook automatically detects the processor architecture of your nodes. Whether you are running on **x86_64 (Intel/AMD)** or **aarch64 (ARM/Apple Silicon)**, the correct binaries (Docker, CRDB, MinIO) will be downloaded and installed.

```bash
ansible-playbook -i inventory.ini playbook.yml -kK
```
---

## 🔍 Troubleshooting

- **CRI Socket**: If `kubelet` fails to start, verify the socket exists: `ls /var/run/cri-dockerd.sock`.
- **Node Status**: Check cluster health: `kubectl get nodes`.
- **Logs**: View service logs on nodes: `journalctl -u cri-docker`.
6 changes: 3 additions & 3 deletions ansible/inventory.ini
Original file line number Diff line number Diff line change
Expand Up @@ -25,9 +25,9 @@ worker-node2

# ------IMPORTANT---------
# Exactly one node only
# should be used for RabbitMQ.
[rabbitmq]
worker-node3
# should be used for Redis.
[redis]
worker-node1

[all:children]
master
Expand Down
Loading