Skip to content

Commit e646b16

Browse files
committed
Add a VM upgrade runbook
1 parent 75dfb67 commit e646b16

File tree

2 files changed

+78
-12
lines changed

2 files changed

+78
-12
lines changed

docs/runbook/README.md

Lines changed: 11 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -6,15 +6,14 @@ This page gathers the available guides to operate a Mithril network.
66

77
# Guides
88

9-
| Operation | Location | Description
10-
|------------|------------|------------
11-
| **Genesis manually** | [manual-genesis](./genesis-manually/README.md) | Proceed to manual (re)genesis of the aggregator certificate chain.
12-
| **Era markers** | [era-markers](./era-markers/README.md) | Create and update era markers on the Cardano chain.
13-
| **Downloads statistics** | [downloads statistics](./statistics/README.md) | Display the number of downloads per day.
14-
| **Signer registrations monitoring** | [registrations-monitoring](./registrations-monitoring/README.md) | Gather aggregated data about signer registrations (versions, stake, ...).
15-
| **Update protocol parameters** | [protocol-parameters](./protocol-parameters/README.md) | Update the protocol parameters of a Mithril network.
16-
| **Recompute certificates hash** | [recompute-certificates-hash](./recompute-certificates-hash/README.md) | Recompute the certificates has of an aggregator.
17-
| **Fix terraform lock** | [terraform-lock](./terraform-lock/README.md) | Fix a terraform lock in CD workflows.
18-
| **Manage SSH access to infrastructure** | [ssh-access](./ssh-access/README.md) | Manage SSH access on the VM of the infrastructure for a user.
19-
20-
9+
| Operation | Location | Description |
10+
| --------------------------------------- | ---------------------------------------------------------------------- | ------------------------------------------------------------------------- |
11+
| **Genesis manually** | [manual-genesis](./genesis-manually/README.md) | Proceed to manual (re)genesis of the aggregator certificate chain. |
12+
| **Era markers** | [era-markers](./era-markers/README.md) | Create and update era markers on the Cardano chain. |
13+
| **Downloads statistics** | [downloads statistics](./statistics/README.md) | Display the number of downloads per day. |
14+
| **Signer registrations monitoring** | [registrations-monitoring](./registrations-monitoring/README.md) | Gather aggregated data about signer registrations (versions, stake, ...). |
15+
| **Update protocol parameters** | [protocol-parameters](./protocol-parameters/README.md) | Update the protocol parameters of a Mithril network. |
16+
| **Recompute certificates hash** | [recompute-certificates-hash](./recompute-certificates-hash/README.md) | Recompute the certificates hash of an aggregator. |
17+
| **Fix terraform lock** | [terraform-lock](./terraform-lock/README.md) | Fix a terraform lock in CD workflows. |
18+
| **Manage SSH access to infrastructure** | [ssh-access](./ssh-access/README.md) | Manage SSH access on the VM of the infrastructure for a user. |
19+
| **Upgrade VM of infrastructure** | [upgrade-vm](./upgrade-vm/README.md) | Upgrade the VM of the infrastructure of a Mithril network. |

docs/runbook/upgrade-vm/README.md

Lines changed: 67 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,67 @@
1+
# Upgrade the VM of the infrastructure of a Mithril network.
2+
3+
## Introduction
4+
5+
From time to time, we may need to upgrade or downgrade the VM that is powering the infrastructure of a Mithril network (Mithril aggregator, Mithril signers, Cardano nodes, ...).
6+
7+
## Find the workflow used to deploy a Mithril network
8+
9+
Currently, the following [Mithril networks](https://mithril.network/doc/manual/developer-docs/references#mithril-networks) are generally available, and deployed with `terraform`:
10+
11+
- `testing-preview`: with the workflow [`.github/workflows/ci.yml`](../../github/workflows/ci.yml)
12+
- `pre-release-preview`: with the workflow [`.github/workflows/pre-release.yml`](../../github/workflows/pre-release.yml)
13+
- `release-preprod`: with the workflow [`.github/workflows/release.yml`](../../github/workflows/release.yml)
14+
- `release-mainnet`: with the workflow [`.github/workflows/release.yml`](../../github/workflows/release.yml)
15+
16+
## Update the VM instance type
17+
18+
Update the following value of the targeted VM (`google_machine_type`) in the deployment matrix of the network with the new value that need to be used:
19+
20+
```yaml
21+
google_machine_type: e2-highmem-4
22+
```
23+
24+
Which will be replaced eg with:
25+
26+
```yaml
27+
google_machine_type: e2-highmem-8
28+
```
29+
30+
The list of available machines on GCP is available [here](https://cloud.google.com/compute/all-pricing)
31+
32+
The modifications should be created in a dedicated PR, and the result of the **Plan** job of the terraform deployment should be analyzed precisely to make sure that the change has been taken into consideration.
33+
34+
:warning: The modification of the VM is **usually** an operation that can be performed **in place** by terraform. This means that the modification is transparent and does not imply any **loss of data**. If the **Plan** does not explicitely state that the modification will be done in place, it means that a more complex upgrade operation needs to be implemented.
35+
36+
Example of a terraform plan with an in place modification of the VM:
37+
38+
```bash
39+
Terraform will perform the following actions:
40+
41+
# google_compute_instance.vm_instance will be updated in-place
42+
~ resource "google_compute_instance" "vm_instance" {
43+
id = "projects/iog-hydra/zones/europe-west1-b/instances/mithril-testing-preview-vm"
44+
~ machine_type = "e2-highmem-4" -> "e2-highmem-8"
45+
name = "mithril-testing-preview-vm"
46+
tags = [
47+
"mithril",
48+
"mithril-testing-preview",
49+
"preview",
50+
"testing",
51+
]
52+
# (20 unchanged attributes hidden)
53+
54+
# (5 unchanged blocks hidden)
55+
}
56+
```
57+
58+
## Deployment of the new protocol parameters
59+
60+
The update of the new protocol parameters will take place as detailed in the following table:
61+
| Workflow | Deployed at | Effective at
62+
|------------|------------|------------
63+
| [`.github/workflows/ci.yml`](../../github/workflows/ci.yml) | Merge on `main` branch | Immediately
64+
| [`.github/workflows/pre-release.yml`](../../github/workflows/pre-release.yml) | Pre-release of a distribution | Immediately
65+
| [`.github/workflows/release.yml`](../../github/workflows/release.yml) | Release of a distribution | Immediately
66+
67+
For more information about the CD, please refer to [Release process and versioning](https://mithril.network/doc/adr/3).

0 commit comments

Comments
 (0)