Vaultvisor: Vault Auto-Updater

[daniel-savu]

Is your feature request related to a problem? Please describe.
The vault client is released with breaking changes on a regular basis, forcing operators to check the docs website and manually update. This is both error-prone and inconvenient. Providing operators with an opt-in updating solution can ensure higher system availability.

This problem multiplies when an operator runs vaults on multiple networks, having to keep track of multiple releases.

Describe alternatives you've considered

• Cosmos releases can be run within a process manager that handles auto-updates, called Cosmovisor. It listens for on-chain version upgrade events to download and run new releases. Positional arguments are passed through to the node binary. Cosmovisor is implemented in Go, so it's not possible to just swap the cosmos sdk event listener with subxt.
• Google open-sourced Omaha, the updater used by Chrome and Google Earth. Similarly, Omaha checks an update server and then downloads both the new binary and updater logic (e.g. migrations). It seems to only target Windows, and the update server implies trusting the host.

Describe the solution you'd like
Implement Vaultvisor, a wrapper software similar to the Cosmovisor that listens for on-chain version release events and updates the vault client. Given a JSON file with input arguments, the Vaultvisor can run and auto-update binaries across the Kintsugi, Interlay, and their respective testnet networks.

• Vaultvisor spawns a separate process to run the vault binary. Positional arguments are specified in a JSON file that can have migrations applied to it.
• Add a new extrinsic in the Vault Registry: set_vault_release(version, binary_checksum), which emits a VaultClientUpdate(version, binary_checksum) event. The extrinsic writes to a storage item which is read when the Vaultvisor is first started up.
• Vaultvisor uses subxt to listen for the VaultClientUpdate event. Based on the connected parachain network and the emitted version, it can derive the GitHub URI of the new release. Then, it:
  • Downloads the binary and ensures its checksum matches the one on-chain.
  • SIGINTs the current vault process.
  • Runs the new binary. If successful, removes the old binary. Optional: if unsuccessful, runs the old binary and periodically retries the new one.
• Once the vault starts persisting local state with rocksdb, migrations for this would also need to be handled by Vaultvisor.

Additional Context
This solution requires releasing the vault client before a runtime upgrade is merged. JSON argument and rocksdb migrations would require their own script, with its own checksum stored on-chain.

Assumption: Sending a SIGINT signal to the vault process does not cause errors or side-effects.

[sander2]

I am not sure the updater should use subxt - it would become another piece of software that we may have to update if the runtime changes. I think it might be better to use a simple bash script that regularly polls storage using curl

[nud3l]

Related: interlay/interbtc#628

[nud3l]

I think the overall goal is great. I'm wondering though how to best approach this. I've got the following questions:

1. How would you map which vault client version is compatible with which chain version? Currently, we just increment the spec version on the different runtimes but the same tag, say 1.16.0 might be spec version 4 in Interlay, 18 in Kintsugi, etc.

2. How would you read the required version from the chain state? Right now, we refuse to connect the vault client when the spec version changes. The curl approach would work but that would likely call an RPC so would rely on the node to be updated rather than the wasm. This is not ideal since nodes and wasm updates rarely happen at the same time.

[gregdhill]

Closed by #312