Skip to content

Commit ba24eba

Browse files
JuArceklaus993
JuArce
and
klaus993
authored
ci: deploy batcher with ansible (#1384)
Co-authored-by: Klaus Lungwitz <[email protected]>
1 parent 47e2bc4 commit ba24eba

33 files changed

+1058
-0
lines changed

‎.gitignore‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -14,3 +14,5 @@ volume
1414
config-files/*.last_processed_batch.json
1515

1616
nonce_*.bin
17+
18+
infra/ansible/playbooks/ini/**.ini

‎Makefile‎

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1069,3 +1069,21 @@ setup_local_aligned_all:
10691069

10701070
tmux new-window -t aligned_layer -n telemetry
10711071
tmux send-keys -t aligned_layer:telemetry 'docker compose -f telemetry-docker-compose.yaml down && make telemetry_create_env && make telemetry_run_db && make open_telemetry_start && make telemetry_start' C-m
1072+
1073+
__ANSIBLE__: ## ____
1074+
1075+
ansible_batcher_create_env: ## Create empty variables files for the Batcher deploy
1076+
@cp -n infra/ansible/playbooks/ini/caddy-batcher.ini.example infra/ansible/playbooks/ini/caddy-batcher.ini
1077+
@cp -n infra/ansible/playbooks/ini/config-batcher.ini.example infra/ansible/playbooks/ini/config-batcher.ini
1078+
@cp -n infra/ansible/playbooks/ini/env-batcher.ini.example infra/ansible/playbooks/ini/env-batcher.ini
1079+
@echo "Config files for the Batcher created in infra/ansible/playbooks/ini"
1080+
@echo "Please complete the values and run make ansible_batcher_deploy"
1081+
1082+
ansible_batcher_deploy: ## Deploy the Batcher. Parameters: INVENTORY, KEYSTORE
1083+
@if [ -z "$(INVENTORY)" ] || [ -z "$(KEYSTORE)" ]; then \
1084+
echo "Error: Both INVENTORY and KEYSTORE must be set."; \
1085+
exit 1; \
1086+
fi
1087+
@ansible-playbook infra/ansible/playbooks/batcher.yaml \
1088+
-i $(INVENTORY) \
1089+
-e "keystore_path=$(KEYSTORE)"

‎infra/ansible/README.md‎

Lines changed: 57 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,57 @@
1+
# Guide to Deploy
2+
3+
## Batcher
4+
5+
To deploy the Batcher you need to set some variables and then run the Batcher playbook
6+
7+
Create the variables files:
8+
9+
```shell
10+
make ansible_batcher_create_env
11+
```
12+
13+
This will create the following files in `infra/ansible/playbooks/ini`
14+
15+
- `config-batcher.ini`
16+
- `env-batcher.ini`
17+
- `caddy-batcher.ini`
18+
19+
The `config-batcher.ini` contains the following variables:
20+
21+
| Variable | Description | Stage | Testnet | Mainnet |
22+
|-------------------------------------------|------------------------------------------------------------|-----------------------------------------------------------------------------------------------------------------|------------------------------------------------------------------------------------------------------------|-------------------------------------|
23+
| aligned_layer_deployment_config_file_path | JSON with Aligned contracts addresses | /home/app/repos/batcher/aligned_layer/contracts/script/output/holesky/alignedlayer_deployment_output.stage.json | /home/app/repos/operator/aligned_layer/contracts/script/output/holesky/alignedlayer_deployment_output.json | TBD |
24+
| eigen_layer_deployment_config_file_path | JSON with EigenLayer contracts addresses | /home/app/repos/batcher/aligned_layer/contracts/script/output/holesky/eigenlayer_deployment_output.json | /home/app/repos/operator/aligned_layer/contracts/script/output/holesky/eigenlayer_deployment_output.json | TBD |
25+
| eth_rpc_url | HTTP RPC url | <your_rpc_http_provider> | <your_rpc_http_provider> | <your_rpc_http_provider> |
26+
| eth_rpc_url_fallback | HTTP RPC fallback url. Must be different than eth_rpc_url | https://ethereum-holesky-rpc.publicnode.com | https://ethereum-holesky-rpc.publicnode.com | https://ethereum-rpc.publicnode.com |
27+
| eth_ws_url | WS RPC url | <your_rpc_ws_provider> | <your_rpc_ws_provider> | <your_rpc_ws_provider> |
28+
| eth_ws_url_fallback | WS RPC fallback url. Must be different than eth_ws_rpc_url | wss://ethereum-holesky-rpc.publicnode.com | wss://ethereum-holesky-rpc.publicnode.com | wss://ethereum-rpc.publicnode.com |
29+
| ecdsa_private_key_store_password | Password of the ECDSA keystore | <your_ecdsa_keystore_password> | <your_ecdsa_keystore_password> | <your_ecdsa_keystore_password> |
30+
| telemetry_ip_port_address | IP:Port of the Telemetry service | <IP>:4001 | <IP>:4001 | TBD |
31+
| batcher_replacement_private_key | This is the private key for the non-paying users | - | - | - |
32+
33+
The `env-batcher.ini` contains the following variables:
34+
35+
| Variable | Description | Stage | Testnet | Mainnet |
36+
|-------------------|------------------------------------------------------------|-------------------------------------|---------------------------------------|-------------------------------|
37+
| secret_access_key | Secret access key for user with access to the Storage (S3) | <your_secret_access_key> | <your_secret_access_key> | <your_secret_access_key> |
38+
| region | Region of the Storage | <us-east-1> | <us-east-1> | <us-east-1> |
39+
| access_key_id | Access key for the user with access to the Storage (S3) | <your_access_key_id> | <your_access_key_id> | <your_access_key_id> |
40+
| bucket_name | Bucket name | <stage.storage.example.com> | <holesky.storage.example.com> | <storage.example.com> |
41+
| download_endpoint | Public endpoint to download batcher | <https://stage.storage.example.com> | <https://holesky.storage.example.com> | <https://storage.example.com> |
42+
| log_level | Log level | info | info | info |
43+
44+
The `caddy-batcher.ini` contains the following variables:
45+
46+
| Variable | Description | Stage | Testnet | Mainnet |
47+
|----------------|--------------------------------------|-----------------------------|-------------------------------|-----------------------|
48+
| batcher_domain | Domain of the Batcher to send proofs | <stage.batcher.example.com> | <holesky.batcher.example.com> | <batcher.example.com> |
49+
50+
> [!WARNING]
51+
> You need to previously set the `batcher_domain` in your DNS provider to point to the Batcher IP.
52+
53+
Deploy the Batcher:
54+
55+
```shell
56+
make ansible_batcher_deploy INVENTORY=</path/to/inventory> KEYSTORE=<path/to/keystore/in/your/local>
57+
```

‎infra/ansible/aggregator.yaml‎

Lines changed: 61 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,61 @@
1+
- import_playbook: setup.yaml
2+
- import_playbook: go.yaml
3+
- import_playbook: rust.yaml
4+
- import_playbook: eigenlayer-cli.yaml
5+
6+
- hosts: aggregator
7+
vars:
8+
service: "aggregator"
9+
10+
tasks:
11+
- name: Update apt and install required system packages
12+
become: true
13+
apt:
14+
pkg:
15+
- pkg-config
16+
- libssl-dev
17+
- gcc
18+
state: latest
19+
update_cache: true
20+
21+
- name: Create directories for each service if do not exist
22+
become: true
23+
ansible.builtin.file:
24+
path: /home/{{ app_user }}/repos/{{ service }}
25+
state: directory
26+
mode: '0755'
27+
owner: '{{ app_user }}'
28+
group: '{{ app_user }}'
29+
loop:
30+
- aggregator
31+
32+
- name: Clone Aligned repository
33+
become: true
34+
ansible.builtin.git:
35+
repo: https://github.com/yetanotherco/aligned_layer.git
36+
dest: /home/{{ app_user }}/repos/{{ service }}/aligned_layer
37+
version: v0.10.2
38+
loop:
39+
- aggregator
40+
41+
- name: Set permissions for cloned repository
42+
become: true
43+
ansible.builtin.file:
44+
path: /home/{{ app_user }}/repos/{{ service }}/aligned_layer
45+
mode: '0755'
46+
owner: '{{ app_user }}'
47+
group: '{{ app_user }}'
48+
recurse: yes
49+
50+
- name: Compile all FFIs
51+
become: true
52+
shell:
53+
chdir: /home/{{ app_user }}/repos/aggregator/aligned_layer
54+
cmd: sudo -u {{ app_user }} . /home/{{ app_user }}/.profile && sudo -u {{ app_user }} make build_all_ffi_linux
55+
56+
- name: Allow access to tcp port 8090
57+
become: true
58+
ufw:
59+
rule: allow
60+
port: 8090
61+
proto: tcp

‎infra/ansible/playbooks/avs-sync.yaml‎

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,16 @@
1+
- import_playbook: setup.yaml
2+
- import_playbook: go.yaml
3+
4+
- hosts: aligned-holesky-others
5+
become: true
6+
vars:
7+
user: "{{ user }}"
8+
9+
tasks:
10+
# Clone avs-sync repository
11+
- name: Clone Aligned repository
12+
ansible.builtin.git:
13+
repo: https://github.com/Layr-Labs/avs-sync.git
14+
dest: /home/{{ user }}/repos/avs-sync
15+
become_user: "{{ user }}"
16+

‎infra/ansible/playbooks/batcher.yaml‎

Lines changed: 142 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,142 @@
1+
- name: Run setup playbook
2+
ansible.builtin.import_playbook: setup.yaml
3+
vars:
4+
host: batcher
5+
6+
- name: Run webserver playbook
7+
ansible.builtin.import_playbook: webserver.yaml
8+
vars:
9+
host: batcher
10+
11+
- name: Run go playbook
12+
ansible.builtin.import_playbook: go.yaml
13+
vars:
14+
host: batcher
15+
16+
- name: Run rust playbook
17+
ansible.builtin.import_playbook: rust.yaml
18+
vars:
19+
host: batcher
20+
21+
- name: Run eigenlayer-cli playbook
22+
ansible.builtin.import_playbook: eigenlayer-cli.yaml
23+
vars:
24+
host: batcher
25+
26+
- hosts: batcher
27+
28+
tasks:
29+
# Install required packages
30+
- name: Update apt and install required system packages
31+
become: true
32+
apt:
33+
pkg:
34+
- pkg-config
35+
- libssl-dev
36+
state: latest
37+
update_cache: true
38+
vars:
39+
ansible_ssh_user: "{{ admin_user }}"
40+
41+
# Create directories for each service
42+
- name: Create directories for each service if do not exist
43+
ansible.builtin.file:
44+
path: /home/{{ ansible_user }}/repos/batcher
45+
state: directory
46+
mode: '0755'
47+
owner: '{{ ansible_user }}'
48+
group: '{{ ansible_user }}'
49+
50+
# Clone Aligned repository for the service
51+
- name: Clone Aligned repository
52+
ansible.builtin.git:
53+
repo: https://github.com/yetanotherco/aligned_layer.git
54+
dest: /home/{{ ansible_user }}/repos/batcher/aligned_layer
55+
version: v0.10.2
56+
recursive: false
57+
58+
# Build the batcher
59+
- name: Install deps
60+
make:
61+
chdir: /home/{{ ansible_user }}/repos/batcher/aligned_layer
62+
target: deps
63+
environment:
64+
PATH: "{{ ansible_env.PATH }}:/usr/local/go/bin:/home/{{ ansible_user }}/.cargo/bin"
65+
66+
- name: Install batcher
67+
make:
68+
chdir: /home/{{ ansible_user }}/repos/batcher/aligned_layer
69+
target: install_batcher
70+
environment:
71+
PATH: "{{ ansible_env.PATH }}:/usr/local/go/bin:/home/{{ ansible_user }}/.cargo/bin"
72+
73+
- name: Create systemd services directory
74+
file:
75+
path: "/home/{{ ansible_user }}/.config/systemd/user/"
76+
state: directory
77+
78+
- name: Add service to systemd
79+
template:
80+
src: services/batcher.service.j2
81+
dest: "/home/{{ ansible_user }}/.config/systemd/user/batcher.service"
82+
force: no
83+
84+
- name: Upload config file for batcher
85+
template:
86+
src: config-files/config-batcher.yaml.j2
87+
dest: "/home/{{ ansible_user }}/config/config-batcher.yaml"
88+
vars:
89+
aligned_layer_deployment_config_file_path: "{{ lookup('ini', 'aligned_layer_deployment_config_file_path', file='ini/config-batcher.ini') }}"
90+
eigen_layer_deployment_config_file_path: "{{ lookup('ini', 'eigen_layer_deployment_config_file_path', file='ini/config-batcher.ini') }}"
91+
eth_rpc_url: "{{ lookup('ini', 'eth_rpc_url', file='ini/config-batcher.ini') }}"
92+
eth_rpc_url_fallback: "{{ lookup('ini', 'eth_rpc_url_fallback', file='ini/config-batcher.ini') }}"
93+
eth_ws_url: "{{ lookup('ini', 'eth_ws_url', file='ini/config-batcher.ini') }}"
94+
eth_ws_url_fallback: "{{ lookup('ini', 'eth_ws_url_fallback', file='ini/config-batcher.ini') }}"
95+
ecdsa_private_key_store_path: "/home/{{ ansible_user }}/.keystores/batcher"
96+
ecdsa_private_key_store_password: "{{ lookup('ini', 'ecdsa_private_key_store_password', file='ini/config-batcher.ini') }}"
97+
telemetry_ip_port_address: "{{ lookup('ini', 'telemetry_ip_port_address', file='ini/config-batcher.ini') }}"
98+
batcher_replacement_private_key: "{{ lookup('ini', 'batcher_replacement_private_key', file='ini/config-batcher.ini') }}"
99+
100+
101+
- name: Upload env file for batcher
102+
template:
103+
src: config-files/env-batcher.j2
104+
dest: "/home/{{ ansible_user }}/config/.env.batcher"
105+
vars:
106+
secret_access_key: "{{ lookup('ini', 'secret_access_key', file='ini/env-batcher.ini') }}"
107+
region: "{{ lookup('ini', 'region', file='ini/env-batcher.ini') }}"
108+
access_key_id: "{{ lookup('ini', 'access_key_id', file='ini/env-batcher.ini') }}"
109+
bucket_name: "{{ lookup('ini', 'bucket_name', file='ini/env-batcher.ini') }}"
110+
download_endpoint: "{{ lookup('ini', 'download_endpoint', file='ini/env-batcher.ini') }}"
111+
log_level: "{{ lookup('ini', 'log_level', file='ini/env-batcher.ini') }}"
112+
113+
- name: Add reverse proxy to Caddy
114+
become: true
115+
template:
116+
src: caddy/Caddyfile.batcher.j2
117+
dest: "/etc/caddy/Caddyfile"
118+
vars:
119+
ansible_ssh_user: "{{ admin_user }}"
120+
batcher_domain: "{{ lookup('ini', 'batcher_domain', file='ini/caddy-batcher.ini') }}"
121+
122+
- name: Copy keystore to server
123+
ansible.builtin.copy:
124+
src: '{{ keystore_path }}'
125+
dest: /home/{{ ansible_user }}/.keystores/batcher
126+
owner: '{{ ansible_user }}'
127+
group: '{{ ansible_user }}'
128+
129+
- name: Start Batcher service
130+
ansible.builtin.systemd_service:
131+
name: batcher
132+
state: started
133+
enabled: true
134+
scope: user
135+
136+
- name: Restart Caddy service
137+
become: true
138+
ansible.builtin.systemd_service:
139+
name: caddy
140+
state: restarted
141+
vars:
142+
ansible_ssh_user: "{{ admin_user }}"

‎infra/ansible/playbooks/docker.yaml‎

Lines changed: 47 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,47 @@
1+
- name: Docker Setup
2+
hosts: all
3+
become: true
4+
vars:
5+
user: "{{ user }}"
6+
7+
tasks:
8+
- name: Install required system packages
9+
apt:
10+
pkg:
11+
- apt-transport-https
12+
- ca-certificates
13+
- curl
14+
- software-properties-common
15+
- python3-pip
16+
- virtualenv
17+
- python3-setuptools
18+
state: latest
19+
update_cache: true
20+
21+
- name: Add Docker GPG apt Key
22+
apt_key:
23+
url: https://download.docker.com/linux/ubuntu/gpg
24+
state: present
25+
26+
- name: Add Docker Repository
27+
apt_repository:
28+
repo: deb https://download.docker.com/linux/ubuntu focal stable
29+
state: present
30+
31+
- name: Update apt and install docker-ce
32+
apt:
33+
name: docker-ce
34+
state: latest
35+
update_cache: true
36+
37+
- name: Ensure group "docker" exists
38+
ansible.builtin.group:
39+
name: docker
40+
state: present
41+
42+
- name: Add user to docker group
43+
user:
44+
name: "{{ user }}"
45+
groups: docker
46+
append: yes
47+
state: present

‎infra/ansible/playbooks/eigenlayer-cli.yaml‎

Lines changed: 26 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,26 @@
1+
- name: Eigenlayer CLI Setup
2+
hosts: "{{ host }}"
3+
4+
tasks:
5+
- name: check if eigenlayer is installed
6+
stat:
7+
path: /home/{{ ansible_user }}/bin/eigenlayer
8+
register: eigenlayer_exists
9+
10+
- name: Download Eigenlayer CLI
11+
when: not eigenlayer_exists.stat.exists
12+
get_url:
13+
url: https://raw.githubusercontent.com/layr-labs/eigenlayer-cli/master/scripts/install.sh
14+
dest: /tmp/eigenlayer.sh
15+
mode: '0755'
16+
17+
- name: Install Eigenlayer CLI
18+
when: not eigenlayer_exists.stat.exists
19+
shell: /tmp/eigenlayer.sh
20+
21+
- name: Add eigenlayer path to ~/.profile
22+
when: not eigenlayer_exists.stat.exists
23+
lineinfile:
24+
path: ~/.profile
25+
line: 'export PATH=$PATH:~/bin'
26+
state: present

0 commit comments

Comments
 (0)