You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
This guide will help you deploy an Open Terms Archive collection to a server.
8
+
This guide will help you deploy an Open Terms Archive collection to a server. The deployment is automated using [Ansible](https://docs.ansible.com/ansible/latest/index.html), a configuration management tool that will set up the Open Terms Archive engine and configure it to track your collection's terms.
9
+
10
+
## System Overview
11
+
12
+
Before diving into the deployment steps, here's a high-level overview of how Open Terms Archive works:
13
+
14
+
1.**Repository structure**: Each collection uses three GitHub repositories:
15
+
-`declarations`: Contains the terms to track, engine configuration and deployment configuration
16
+
-`versions`: Automatically managed repository storing versions history
17
+
-`snapshots`: Automatically managed repository storing snapshots history
18
+
19
+
2.**Deployment process**: The deployment is automated through GitHub Actions. Ansible configures the server and sets up the Open Terms Archive engine. On your server, [PM2](https://pm2.keymetrics.io) starts and controls the engine.
20
+
21
+
3.**Operation**: The engine runs continuously on your server, periodically checking for changes in the tracked terms. When changes are detected, it automatically commits them to the versions and snapshots repositories. Email notifications are sent when issues occur.
22
+
23
+
## Prerequisites
24
+
25
+
Before starting, ensure you have:
26
+
27
+
- A server with admin access
28
+
- All collections repositories created, if not, see the [guide to create repositories]({{< relref "collections/how-to/create-repositories" >}})
29
+
- At least one declaration added to your collection
30
+
- A GitHub user account dedicated to bot-related actions (commit entries in versions and snapshots repositories, report issues when tracking fails, publish releases, …)
31
+
-[Ansible](https://docs.ansible.com/ansible/latest/installation_guide/intro_installation.html) installed on your local machine
32
+
33
+
> **Note**: This guide is intended for both Open Terms Archive organization members and external contributors. Some steps marked with "_Specific to Open Terms Archive organization members_" are only relevant for organization members as they involve access to the organization's shared password database. External contributors should adapt these steps to their own security practices while following the same deployment principles.
9
34
10
35
## 1. Configure the server
11
36
12
37
First, ensure your server provides unsupervised access:
13
38
14
-
1. Check the SSH host key:
39
+
1. Check the SSH host key and get the SSH fingerprint by running the following command on your local machine:
40
+
15
41
```shell
16
-
ssh-keyscan --type=ed25519 <server_address>
42
+
ssh-keyscan -t ed25519 <server_address>
17
43
```
18
-
If no Ed25519 key appears, generate one on the server:
44
+
45
+
If no Ed25519 key appears, generate one by running the following commands on the server:
> **Note**: A server fingerprint is a unique identifier for your server's SSH key. It helps verify that you're connecting to the correct server and not a malicious one. The fingerprint is a hash of the server's public key and is used to prevent man-in-the-middle attacks. You'll need this fingerprint in the next steps for secure deployment.
53
+
54
+
2. Create a dedicated user account specifically for deployment purposes, by running the following commands on the server:
55
+
25
56
```shell
26
-
adduser <user>
27
-
usermod --append --groups=sudo <user>
57
+
adduser <deployment_user>
58
+
usermod --append --groups=sudo <deployment_user>
28
59
```
29
60
30
-
3. Grant passwordless sudo access:
61
+
> **Note**: The `adduser` command might not be installed by default on your system. It can be installed with `sudo apt-get install adduser`.
62
+
63
+
3. Configure sudo access for this user, by running the adding the following line to the `/etc/sudoers` file on the server:
64
+
31
65
```shell
32
-
# Add to /etc/sudoers:
33
-
<user> ALL=(ALL) NOPASSWD:ALL
66
+
<deployment_user> ALL=(ALL) NOPASSWD:ALL
34
67
```
35
68
69
+
> **Note**: While passwordless sudo access does reduce security compared to requiring a password, it is essential for full automation in deployment workflows with Ansible. The deployment process requires system-level operations (like installing packages and configuring services) that must be executed without manual intervention. To mitigate security risks, this configuration is limited to a dedicated deployment user that should only be used for deployment purposes, and the server must be properly secured with SSH key authentication.
70
+
36
71
## 2. Set up the deployment configuration
37
72
38
-
1. Clone the collection declarations repository:
73
+
1. Clone the collection declarations repository that you want to deploy:
2. Configure the inventory file `deployment/inventory.yml`:
79
+
2. Configure the inventory file `deployment/inventory.yml` with your server's IP address, deployment user (from step 1), server fingerprint (from step 1) and the repository URL:
- Go to `https://github.com/OpenTermsArchive/<collection_name>-declarations/settings/secrets/actions`
88
+
3. Add the server fingerprint to GitHub, to allow the deployment workflow to uniquely identify the server:
89
+
- Go to `https://github.com/<organization>/<collection_id>-declarations/settings/secrets/actions`
52
90
- Create a new secret named `SERVER_FINGERPRINT` with your Ed25519 fingerprint
53
91
54
92
## 3. Configure SSH deployment keys
55
93
56
-
1. On the server, generate a deployment key:
94
+
1. On the server, generate a deployment key, which will be used by the continuous deployment workflow to connect to the server to deploy the collection:
- Create `SERVER_SSH_KEY` with the private key content
101
+
2. Add the private key to GitHub, to allow the deployment workflow to connect to the server:
102
+
- Go to `https://github.com/<organization>/<collection_id>-declarations/settings/secrets/actions`
103
+
- Create a new secret named `SERVER_SSH_KEY` with the private key content
65
104
66
-
3. Back up the keys:
67
-
- Store both public and private keys in the shared password database
68
-
- Create an entry titled "Deployment SSH key" in the collection folder
105
+
3. > _Specific to Open Terms Archive organization members_
106
+
>
107
+
> Back up the keys in the shared password database by creating an entry titled "Deployment SSH Key" in the collection folder and storing both public and private keys in this entry
69
108
70
109
## 4. Set up GitHub permissions
71
110
72
-
1. Create a fine-grained GitHub token:
73
-
- Log in as OTA-Bot
111
+
1. Log in as the user account dedicated to bot-related actions in GitHub
112
+
113
+
2. Create a fine-grained GitHub token:
74
114
- Create a new token at github.com/settings/personal-access-tokens/new
75
-
- Set repository access for both declarations and versions repos
115
+
- Set repository access for both declarations and versions repositories
76
116
- Grant "Contents" and "Issues" write permissions
77
117
78
-
2. Back up the token:
79
-
- Store it in the shared password database under "GitHub Token"
80
-
81
118
3. Get the token approved:
82
119
- Have an organization admin approve the token request
83
120
84
-
## 5. Configure secrets
121
+
4. > _Specific to Open Terms Archive organization members_
122
+
>
123
+
> Back up the token in the shared password database by creating an entry titled "GitHub Token" in the collection folder and storing the token in this entry
124
+
125
+
## 5. Configure and encrypt secrets
126
+
127
+
This section uses [Ansible Vault](https://docs.ansible.com/ansible/latest/vault_guide/index.html), a feature of Ansible that allows you to encrypt sensitive data like passwords and keys. The encrypted files can be safely committed to version control while keeping the actual secrets secure. The vault key you'll create will be used to encrypt and decrypt these secrets.
85
128
86
129
1. Generate and store a vault key:
87
130
- Generate a secure password without quotes/backticks
88
-
- Store it in the password database
89
-
- Create `deployment/vault.key` with the password
90
-
- Add it as `ANSIBLE_VAULT_KEY` in GitHub secrets
131
+
- Inside the collection folder, create `deployment/vault.key` with the password.
132
+
- Add the same password as `ANSIBLE_VAULT_KEY` in GitHub secrets.
91
133
92
-
2. Store GitHub token:
93
-
```
94
-
# In deployment/.env:
134
+
> **Note**: The same vault key is used in two places:
135
+
> - Locally as `vault.key` to encrypt/decrypt files during development
136
+
> - In GitHub Actions as `ANSIBLE_VAULT_KEY` to decrypt files during automated deployment
137
+
138
+
2. Store the GitHub token (from step 4), in `deployment/.env`:
139
+
140
+
```shell
95
141
OTA_ENGINE_GITHUB_TOKEN=your_token
96
142
```
97
143
98
-
3. Encrypt the `.env` file:
144
+
3. Encrypt the `.env` file by running the following command inside the `deployment` folder of the collection:
145
+
99
146
```shell
100
147
ansible-vault encrypt .env
101
148
```
102
149
150
+
> **Note**: Running the command above from the `deployment` folder will ensure that the vault.key file is used as the vault key, as this folder contains an ansible.cfg file that explicitly configures this behavior.
151
+
>
152
+
> To decrypt an encrypted file, use:
153
+
> ```shell
154
+
> ansible-vault decrypt deployment/.env
155
+
> ```
156
+
> After making changes, re-encrypt it:
157
+
> ```shell
158
+
> ansible-vault encrypt deployment/.env
159
+
> ```
160
+
161
+
4. Commit the changes to the repository
162
+
163
+
5. > _Specific to Open Terms Archive organization members_
164
+
>
165
+
> Back up the vault key in the shared password database by creating an entry titled "Vault Key" in the collection folder and storing the vault key in this entry
166
+
103
167
## 6. Set up collection-specific SSH key
104
168
105
-
1. Generate a new key:
169
+
1. Generate a new key, which will be used by the Open Terms Archive engine to perform actions on GitHub as the bot user:
2. Store the private key in `deployment/github-bot-private-key`
176
+
177
+
3. Encrypt the private key file by running the following command inside the `deployment` folder of the collection:
178
+
111
179
```shell
112
-
# Copy private key to deployment/github-bot-private-key
113
180
ansible-vault encrypt github-bot-private-key
114
181
```
115
182
116
-
3. Add the public key to OTA-Bot's GitHub account:
183
+
4. Commit the changes to the repository
184
+
185
+
5. Add the public key to bot user's GitHub account:
117
186
- Go to github.com/settings/ssh/new
118
187
- Add the public key with title "<collection_name> collection"
119
188
189
+
6. > _Specific to Open Terms Archive organization members_
190
+
>
191
+
> Back up the key in the shared password database by creating an entry titled "OTA-Bot GitHub SSH key" in the collection folder and storing both public and private keys in this entry
192
+
120
193
## 7. Configure email notifications
121
194
122
-
1. Generate SMTP credentials:
123
-
- Create a new SMTP key in Brevo
124
-
- Name it "<collection_name> collection"
195
+
This section describes how to configure the engine to use a specific SMTP server to send email notifications when it encounters errors during the tracking process. This helps you stay informed about issues that need attention and allows you to restart the tracking process if needed.
196
+
197
+
1. Get the SMTP credentials (host, username, password) from your email provider
198
+
199
+
2. Update collection SMTP configuration within the `logger` key of `@opentermsarchive/engine` in the `config/production.json` file:
200
+
201
+
```json
202
+
"logger": {
203
+
"smtp": {
204
+
"host": "<smtp_host>",
205
+
"username": "<smtp_username>"
206
+
},
207
+
},
208
+
```
209
+
210
+
3. Store the password in `deployment/.env`:
125
211
126
-
2. Store the credentials:
127
212
```shell
128
-
# In deployment/.env:
129
213
OTA_ENGINE_SMTP_PASSWORD=your_smtp_key
130
214
```
131
215
132
-
3. Encrypt the `.env` file:
216
+
> **Note**: To decrypt the file encrypted in a previous step in order to add the password, run `ansible-vault decrypt .env`
217
+
218
+
4. Encrypt the `.env` file:
219
+
133
220
```shell
134
221
ansible-vault encrypt .env
135
222
```
136
223
224
+
5. > _Specific to Open Terms Archive organization members_
225
+
> Create a new SMTP key in Brevo and name it "<collection_name> collection"
226
+
> Back up the key in the shared password database by creating an entry titled "SMTP Key" in the collection folder and storing the credentials in this entry
227
+
137
228
## 8. Test the deployment
138
229
139
230
1. Via GitHub Actions:
140
231
- Check that the `deploy` action completes successfully
0 commit comments