sapphire-localnet: ROFL TDX Localnet support

[matevz]

This is epic for improving the devex of the existing ROFL development on Localnet and to add support for ROFL TDX. The flow for both ROFL flavors should be something like:

1. oasis rofl init creates rofl.yaml with prepopulated localnet deployment (because app ID, RPC, the admin user etc. is always constant; this deployment should not be default though, that one is reserved for Testnet)
2. develop your ROFL container(s), write compose.yaml etc.
3. oasis rofl build --deployment localnet
4. docker run -it -p8544-8548:8544-8548 -v .:/rofls ghcr.io/oasisprotocol/sapphire-localnet
5. sapphire-localnet analyzes rofl.yaml, configures ROFL in oasis-node config and spins it up. Then actively monitors changes to rofl.yaml (changed when you rebuild it) and reloads ROFL by calling oasis-node control add-bundle

Ideally, compose.yaml should be unified for any deployment. Just the secrets and env variables are different.

Most of the flow is already there for ROFL SGX without 5 - currently it just relies on the .orc file and developer needs to restart the whole sapphire-localnet manually each time a change to .orc is made.

Considerations, Discussion, Alternatives

Why do we need sapphire-localnet testing? Could we test ROFLs on the host machine without running any chain?

ROFLization has some specifics worth testing them out before deployment:

1. secrets are stored on-chain

Can't really mock that without spinning up sapphire-localnet

2. persistent volume sizes are too small

When spinning up the container, the persistent store may be too small and fetching fails on Oasis node. Can't test this just with podman-compose on the host.

3. appd service (getting app ID, deriving the keys, signing transactions in the name of the ROFL app and submitting them to the chain)

Could we run appd as a 3rd party service/container and let the developer test their containers outside of sapphire-localnet?

• getting the app ID and deriving different kinds of keys should be straight forward (no need to communicate with the chain actually),
• signing transactions with signatures that are successfully authorized by smart contracts in sapphire-localnet requires running the chain anway

4. podman and podman-compose instead of docker and docker compose

May not be relevant anymore if we decide to migrate to docker in the ROFL stage 2 image. Meanwhile, user needs to install those locally and test them out to make sure all features work as they should.

5. ROFL-specific volumes like /run/rofl-appd.sock and storage volume /storage

Could probably be changed to http://localhost/8549 and some local folder on the host. Perhaps using env variables.

[matevz]

TODO:

1. https://github.com/oasisprotocol/oasis-core/blob/master/go/runtime/host/tdx/qemu.go: run QEmu with non-tdx flags
2. rofl-containers: prepare a non-tdx Localnet version, special release made in parallel to ordinary release
3. https://github.com/oasisprotocol/oasis-core/tree/master/runtime/src/common/tdx: add debug-mock-tee flag
4. https://github.com/oasisprotocol/oasis-sdk/blob/main/runtime-sdk/Cargo.toml#L62: add debug-mock-tee flag and probably https://github.com/oasisprotocol/oasis-sdk/blob/main/rofl-containers/Cargo.toml too
5. CLI: support rofl-containers-localnet artifact for Localnet deployments

[matevz]

Today I realized the Localnet testing should not require any access to the internet (e.g. when you're on a plane or you have a slow internet connection). This means that remote images defined in compose.yaml cannot be downloaded, but we can assume they are available on the host.

So for TDX containers, the Localnet docker should register an app, but not run it. Simply expose the storage folder and rofl.d socket to be bind-mounted on the host and then the developer can simply run docker compose up with defined secrets as env variables.