Skip to content

Commit a7660b3

Browse files
stewstew
committed
commital init
0 parents  commit a7660b3

38 files changed

+2689
-0
lines changed

‎.github/workflows/ci.yml‎

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,19 @@
1+
on:
2+
pull_request:
3+
push:
4+
jobs:
5+
check:
6+
runs-on: ubuntu-24.04
7+
steps:
8+
- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # 4.2.2
9+
- uses: nixbuild/nix-quick-install-action@63ca48f939ee3b8d835f4126562537df0fee5b91 # v32
10+
- uses: nix-community/cache-nix-action@135667ec418502fa5a3598af6fb9eb733888ce6a # 6.1.3
11+
with:
12+
# restore and save a cache using this key
13+
primary-key: nix-${{ runner.os }}-${{ runner.arch }}-check-${{ hashFiles('**/*.nix', '**/flake.lock', '**.*.tf') }}
14+
# if there's no cache hit, restore a cache by this prefix
15+
restore-prefixes-first-match: |
16+
nix-${{ runner.os }}-${{ runner.arch }}-check-
17+
- run: |
18+
nix flake check
19+
nix run .#check

‎.gitignore‎

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,14 @@
1+
*~
2+
output
3+
*private
4+
.terraform/
5+
unseal_key*
6+
vault_token
7+
.vscode
8+
.direnv
9+
terraform.tfvars
10+
terraform.tfstate*
11+
nimbus-secrets.json
12+
secrets.env
13+
.envrc
14+
outputs/

‎README.md‎

Lines changed: 28 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,28 @@
1+
# Support for running your own unison.cloud cluster
2+
3+
## Overview
4+
5+
This repository contains instructions and configuration for running your own unison.cloud "Bring Your Own Cloud" (BYOC) cluster. There are multiple options based on how you'd like to deploy your cluster:
6+
7+
- [Docker Compose instructions](docker/README.md) for a simple deployment on a local cluster.
8+
- [EKS instructions](eks/README.md) to use Terraform (or OpenTofu) to deploy on a Kubernetes cluster via AWS Elastic Kubernetes Service (EKS).
9+
10+
11+
## Scope
12+
13+
These configurations are intended to demonstrate minimalist deployments of unison.cloud. They are **not production-ready** but serve testing, development, and demonstration purposes. They provide an easy way to test unison.cloud in your own environment, with the expectation that you might later integrate these learnings into an existing production environment.
14+
15+
## Prerequisites
16+
17+
### Cluster name
18+
19+
To register a Unison Cloud cluster, you'll need a cluster name. You can come up with your own or have one generated for you, but it must be unique across all Unison Cloud clusters. It should be a valid DNS subdomain, meaning it must consist of only lowercase letters, numbers, and hyphens, and must start and end with a letter or number.
20+
21+
### Register a BYOC cluster
22+
23+
See [the Unison Cloud BYOC website](https://www.unison.cloud/byoc/) for more info.
24+
25+
26+
### Cluster token
27+
28+
Before deploying your cluster you will need a cluster token to authenticate your nodes with the Unison Cloud control plane. You will receive one of these when your cluster is registered.

‎docker/.env‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
UNISON_CLOUD_DOCKER_IMAGE=unisoncomputing/unison-cloud:cloud_nimbus_releases_41_1_2_20399
2+
UNISON_CLOUD_ENVIRONMENTS_MOUNT=unison-cloud-environments
3+
VAULT_PORT=8200
4+
# TODO we shouldn't assume that the table name is test
5+
UNISON_CLOUD_STORAGE_TABLE_NAME=test
6+
UNISON_CLOUD_SERVICES_BUCKET_NAME=unison-cloud-services
7+
UNISON_CLOUD_USER_BLOBS_BUCKET_NAME=unison-cloud-user-blobs

‎docker/README.md‎

Lines changed: 95 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,95 @@
1+
# Unison Cloud on Docker Compose
2+
3+
**NOTE:** all paths/files in this documentation assume you are already within the `docker` directory (not the project root).
4+
5+
## Scope and disclaimers
6+
7+
** This is not a production setup.**
8+
9+
This Docker Compose configurations is intended to demonstrate minimalist deployments of unison.cloud in a local environment. It is **not production-ready** but serves testing, development, and demonstration purposes. It provides an easy way to test unison.cloud in your own environment, with the expectation that you might later integrate these learnings into an existing production environment. Specifically, please be aware of the following caveats:
10+
11+
- The secret store (used by [Environment.setValue](https://share.unison-lang.org/@unison/cloud/code/releases/20.16.1/latest/terms/Environment/setValue) and [Environment.Config.lookup](https://share.unison-lang.org/@unison/cloud/code/releases/20.16.1/latest/types/Environment/Config)) stores values insecurely and in-memory: **they will not persist when the Vault container is terminated**.
12+
- Your [storage](https://share.unison-lang.org/@unison/cloud/code/releases/20.16.1/latest/types/Storage), [blobs](https://share.unison-lang.org/@unison/cloud/code/releases/20.16.1/latest/types/Blobs), and [services](https://share.unison-lang.org/@unison/cloud/code/releases/20.16.1/latest/types/Services) are persisted only to named Docker volumes on your machine. If your hard drive fails or you perform some overzealous Docker pruning, they are gone forever. The centralized Unison Cloud control plane stores only some metadata like IDs and permissions; it cannot restore your data.
13+
14+
15+
## Prerequisites
16+
17+
### Install Docker
18+
19+
See [instructions on docker.com](https://docs.docker.com/engine/install/).
20+
21+
### Register a cluster
22+
23+
See [instructions](../README.md#register-a-byoc-cluster).
24+
25+
## Configure your cluster
26+
27+
1. Copy [nimbus-secrets-example.json](nimbus-secrets-example.json) to `nimbus-secrets.json`.
28+
2. Replace the `cloudApiToken` value with the token you received when registering your cluster.
29+
3. In `cloudApiInstances.uri`, replace `YOUR_CLUSTER_NAME` with the cluster name that you registered.
30+
31+
Some insecure development credentials (ex: `INSECURE_DEV_TOKEN`) have been provided for the services running on your local cluster. These are fine for local development, so you can leave them for now, but you will want to change them before opening up your cluster outside of your personal computer.
32+
33+
## Run your cluster
34+
35+
Run this `docker compose` command to start up your cluster:
36+
37+
```sh
38+
docker compose --env-file secrets-example.env --env-file .env up --remove-orphans
39+
```
40+
41+
Logs will print to the screen. When you see messages that start with `Starting public HTTP server` from your `unison-cloud-nimbus` instances, your cluster is ready to serve requests!
42+
43+
If you would prefer to run your containers in the background, you can add `--detach` (or `-d`) to the end of the command:
44+
45+
```sh
46+
docker compose --env-file secrets-example.env --env-file .env up --remove-orphans --detach
47+
```
48+
49+
In detached mode logs will not print to the screen and you can use your terminal window to run other commands.
50+
51+
See [Tear down your cluster](#tear-down-your-cluster) when you are ready to stop your cluster.
52+
53+
## Use your new cluster!
54+
55+
Now you can point your [Unison Cloud client](https://share.unison-lang.org/@unison/cloud) to your own cluster to submit batch jobs, deploy services and more!
56+
57+
First you'll need to know the control plane endpoint for your cluster. It has the form `{your_cluster_name}.byoc.unison.cloud` (ex: `alice-homelab.byoc.unison.cloud`).
58+
59+
There are two ways to point your cloud client to your cluster: via an environment variable _or_ with a custom [Cloud.ClientConfig](https://share.unison-lang.org/@unison/cloud/code/releases/20.16.1/latest/types/Cloud/ClientConfig).
60+
61+
### Configure via environment variable
62+
63+
Set the `UNISON_CLOUD_HOST` environment variable to your cluster control plane endpoint. You could set this in your shell init file (ex: `.bashrc`) or you could just set it in the scope of your `ucm` session:
64+
65+
```sh
66+
UNISON_CLOUD_HOST=alice-homelab.byoc.unison.cloud ucm
67+
```
68+
69+
You can now use [Cloud.run](https://share.unison-lang.org/@unison/cloud/code/releases/20.16.1/latest/terms/Cloud/run) and it will point to your cluster.
70+
71+
### Configure via Cloud.ClientConfig
72+
73+
Instead of setting the `UNISON_CLOUD_HOST` environment variable, you can programmatically configure your cloud client to point to your cluster. Note that this approach requires using [Cloud.run.withConfig](https://share.unison-lang.org/@unison/cloud/code/releases/20.16.1/latest/terms/Cloud/run/withConfig) instead of `Cloud.run`.
74+
75+
```unison
76+
main : '{IO, Exception} Nat
77+
main = do
78+
homelabConfig =
79+
Cloud.ClientConfig.default() |> ClientConfig.host.set (HostName "alice-homelab.byoc.unison.cloud")
80+
Cloud.run.withConfig homelabConfig do
81+
Cloud.submit Environment.default() do
82+
1 + 1
83+
```
84+
85+
## Tear down your cluster
86+
87+
When you are done using your local cluster, you can terminate it. If you ran `docker compose` in attached mode (without the `--detach`/`-d` flag), then you can hit `<ctrl>-c` to stop the running containers.
88+
89+
If you ran `docker compose` in detached mode, then you'll need to run the `down` variant of the `up` command that you previously ran:
90+
91+
```sh
92+
docker compose --env-file secrets-example.env --env-file .env down
93+
```
94+
95+
Even if you ran `docker compose` in attached mode you still may want to run the above `down` command to perform some cleanup that `<ctrl>-c` doesn't trigger like removing stopped containers and the cluster network.

‎docker/caddy/Caddyfile‎

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,9 @@
1+
http://:8080 {
2+
reverse_proxy {
3+
dynamic a {
4+
name unison-cloud-nimbus
5+
port 8082
6+
}
7+
}
8+
}
9+

‎docker/docker-compose.yml‎

Lines changed: 155 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,155 @@
1+
# run this with:
2+
#
3+
# docker compose -f docker-compose.yml --env-file secrets-example.env --env-file .env up
4+
5+
secrets:
6+
nimbus-config.json:
7+
file: ./nimbus-config-example.json
8+
nimbus-secrets.json:
9+
# copy nimbus-secrets-example.json, apply your changes, and save it as nimbus-secrets.json
10+
file: ./nimbus-secrets.json
11+
12+
volumes:
13+
unison-cloud-s3-data:
14+
unison-cloud-dynamodb-data:
15+
16+
networks:
17+
default:
18+
name: unison-cloud
19+
20+
services:
21+
22+
# The dynamodb-local docker image runs as a dynamodblocal user, so we need to
23+
# change the ownership of the mounted directory to avoid permissions issues.
24+
unison-cloud-init-volumes:
25+
image: alpine
26+
restart: "no"
27+
entrypoint: |
28+
/bin/sh -c "chown 1000:1000 /home/dynamodblocal/data"
29+
volumes:
30+
- type: volume
31+
source: unison-cloud-dynamodb-data
32+
target: /home/dynamodblocal/data
33+
34+
unison-cloud-vault:
35+
image: hashicorp/vault
36+
environment:
37+
VAULT_ADDR: "http://127.0.0.1:${VAULT_PORT}"
38+
VAULT_DEV_ROOT_TOKEN_ID: "$UNISON_CLOUD_VAULT_TOKEN"
39+
healthcheck:
40+
test: "vault status"
41+
start_period: 30s
42+
timeout: 1s
43+
cap_add:
44+
- IPC_LOCK
45+
46+
unison-cloud-init-vault:
47+
image: hashicorp/vault
48+
depends_on:
49+
unison-cloud-vault:
50+
condition: service_healthy
51+
environment:
52+
VAULT_ADDR: "http://unison-cloud-vault:${VAULT_PORT}"
53+
VAULT_TOKEN: "$UNISON_CLOUD_VAULT_TOKEN"
54+
command:
55+
- /bin/sh
56+
- '-c'
57+
- 'vault secrets list | grep -q ${UNISON_CLOUD_ENVIRONMENTS_MOUNT} || vault secrets enable -path=${UNISON_CLOUD_ENVIRONMENTS_MOUNT} kv-v2'
58+
59+
unison-cloud-s3:
60+
image: quay.io/minio/minio
61+
command: server /data
62+
volumes:
63+
- type: volume
64+
source: unison-cloud-s3-data
65+
target: /data
66+
healthcheck:
67+
test: "mc ready local"
68+
start_period: 30s
69+
timeout: 1s
70+
networks:
71+
default:
72+
aliases:
73+
- ${UNISON_CLOUD_SERVICES_BUCKET_NAME}.unison-cloud-s3
74+
- ${UNISON_CLOUD_USER_BLOBS_BUCKET_NAME}.unison-cloud-s3
75+
environment:
76+
MINIO_ROOT_USER: "unison-cloud"
77+
MINIO_ROOT_PASSWORD: "$UNISON_CLOUD_S3_TOKEN"
78+
MINIO_DOMAIN: "unison-cloud-s3"
79+
80+
unison-cloud-init-s3:
81+
image: minio/mc
82+
depends_on:
83+
unison-cloud-s3:
84+
condition: service_healthy
85+
entrypoint: ""
86+
command:
87+
- /bin/sh
88+
- '-c'
89+
- |
90+
mc alias set unison-cloud http://unison-cloud-s3:9000 unison-cloud INSECURE_DEV_TOKEN &&
91+
mc mb --ignore-existing unison-cloud/${UNISON_CLOUD_SERVICES_BUCKET_NAME} &&
92+
mc mb --ignore-existing unison-cloud/${UNISON_CLOUD_USER_BLOBS_BUCKET_NAME}
93+
94+
95+
unison-cloud-dynamo:
96+
image: amazon/dynamodb-local:3.0.0
97+
command: "-jar DynamoDBLocal.jar -sharedDb -dbPath /home/dynamodblocal/data"
98+
volumes:
99+
- type: volume
100+
source: unison-cloud-dynamodb-data
101+
target: /home/dynamodblocal/data
102+
depends_on:
103+
unison-cloud-init-volumes:
104+
condition: service_completed_successfully
105+
healthcheck:
106+
test: "curl localhost:8000"
107+
start_period: 10s
108+
timeout: 1s
109+
110+
unison-cloud-init-dynamo:
111+
image: amazon/aws-cli
112+
depends_on:
113+
unison-cloud-dynamo:
114+
condition: service_healthy
115+
environment:
116+
AWS_REGION: ""
117+
AWS_ACCESS_KEY_ID: "$UNISON_CLOUD_AWS_ACCESS_KEY_ID"
118+
AWS_SECRET_ACCESS_KEY: "$UNISON_CLOUD_AWS_SECRET_ACCESS_KEY"
119+
AWS_ENDPOINT_URL: "http://unison-cloud-dynamo:8000"
120+
entrypoint: ""
121+
command:
122+
- /bin/sh
123+
- '-c'
124+
- 'aws dynamodb list-tables | grep "\"${UNISON_CLOUD_STORAGE_TABLE_NAME}\"" >/dev/null || aws dynamodb create-table --table-name ${UNISON_CLOUD_STORAGE_TABLE_NAME} --attribute-definitions AttributeName=K,AttributeType=B AttributeName=NS,AttributeType=S --key-schema AttributeName=K,KeyType=HASH AttributeName=NS,KeyType=RANGE --provisioned-throughput ReadCapacityUnits=5,WriteCapacityUnits=5'
125+
126+
unison-cloud-nimbus:
127+
image: "$UNISON_CLOUD_DOCKER_IMAGE"
128+
deploy:
129+
replicas: 2
130+
depends_on:
131+
unison-cloud-init-s3:
132+
condition: service_completed_successfully
133+
unison-cloud-init-dynamo:
134+
condition: service_completed_successfully
135+
unison-cloud-init-vault:
136+
condition: service_completed_successfully
137+
unison-cloud-s3:
138+
condition: service_healthy
139+
unison-cloud-dynamo:
140+
condition: service_healthy
141+
unison-cloud-vault:
142+
condition: service_healthy
143+
environment:
144+
NIMBUS_CONFIG_DIR: '/run/secrets'
145+
secrets:
146+
- nimbus-config.json
147+
- nimbus-secrets.json
148+
stop_grace_period: "6m"
149+
150+
unison-cloud-load-balancer:
151+
image: caddy
152+
ports:
153+
- "8080:8080"
154+
volumes:
155+
- ./caddy:/etc/caddy

‎docker/nimbus-config-example.json‎

Lines changed: 36 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,36 @@
1+
{
2+
"environment" : "local",
3+
"bindPorts": {
4+
"publicHttp" : 8082,
5+
"gossipHttp" : 8081
6+
},
7+
"blobFetcher": {
8+
"bucket": "unison-cloud-services",
9+
"awsConfig": {
10+
"hostName": "unison-cloud-s3",
11+
"port": 9000,
12+
"https": false,
13+
"region": "us-west-2"
14+
}
15+
},
16+
"userBlobFetcher": {
17+
"bucket": "unison-cloud-user-blobs",
18+
"awsConfig": {
19+
"hostName": "unison-cloud-s3",
20+
"port": 9000,
21+
"https": false,
22+
"region": "us-west-2"
23+
}
24+
},
25+
"vault": {
26+
"host": "unison-cloud-vault",
27+
"port": 8200,
28+
"scheme": "http"
29+
},
30+
"environmentsMount": "unison-cloud-environments",
31+
"dynamo": {
32+
"host": "unison-cloud-dynamo",
33+
"port": 8000,
34+
"type": "local"
35+
}
36+
}

0 commit comments

Comments
 (0)