IA1.2.11 Write scripts and documentation for demo (#211)

* Add devnet test for batcher restart

* Check error returns

* Separate op-geth instances for each L2 node

* Build devnet dockers in CI

* Build op-deployer in CI

* Try larger runner

* Increase test outage and recovery time

* Try to speed up transaction verification

* Do not drop batches before we have seen a finalized L1 block

* Remove unnecessary sleep

* Add scripts for demo and documentation

* Build containers in dependency order

* Don't copy config file into Docker image at build time

* Fix syntax

* Checkout submodules in CI

* Don't copy config file into Docker image at build time

* Remove another COPY

* Run devnet test in separate workflow

* Add exposed ports for more nodes in docker-compose

The `verifier`, `caff-node`, and `sequencer` all do not have their ports forwarded to
the host machine.  Having these ports exposed can make testing / debugging easier
by utilizing the optimism RPC API.

This change exposes these ports.

* Fix CAFF environment variables

The CAFF environment variables utilized in the `docker-compose.yml` need to have
a prefix of `OP_NODE_`, otherwise they will not apply.  These not applying causes
the `caff-node` to never make progress.

This change corrects the misconfigured `caff-node` environment variables, and
cleans up an unnecessary one.

* Set `caff.node` to `true` for `caff-node`

The `caff-node` is meant to be running a derivation based on information retrieved
from Espresso's network.  However, it needs to be enabled to do so.  This change
enables the `caff-node`'s mode by explicitly setting `caff.node` to `true` in the
launch configuration.

* Support alias and input check

* Add scripts and instructions for running utility script

In order to showcase the speed and progress of the Espresso solution with the Caff
node, versus the existing sequencer, or the L1 derived verifier, a script has been
added that utilities `watch` and `tmux` to provide a nice visual comparison between
the three nodes.

* Update scripts after separating geth

* Update readme

* tmux in flake.nix

* Update shell being targeted in get_sync_status.sh

The `get_sync_status.sh` script attempts to target `zsh` residing within `/bin/zsh`,
but this isn't a universally available. It is better to target a more commonly available
like `/bin/bash`.

This change modifies the script to target `/bin/bash` instead of `/bin/zsh`.

---------

Co-authored-by: Jeb Bearer <[email protected]>
Co-authored-by: Theodore Schnepper <[email protected]>
Co-authored-by: Philippe Camacho <[email protected]>

Author: 4 people

README_ESPRESSO.md

@@ -338,7 +338,7 @@ docker compose down -v
 docker volume prune -a
 ```
 
-* If you have changed OP contracts, you will have to start the devnet fresh and re-generate 
+* If you have changed OP contracts, you will have to start the devnet fresh and re-generate
   the genesis allocations by running `prepare-allocs.sh`
 
 
@@ -380,3 +380,87 @@ In order to refresh this AMI one needs to:
 1. Create an AWS EC2 instance with the characteristics described in (see `.github/workflows/enclave.yaml` *Launch EC2 Instance* job).
 2. Copy the script `espresso/scrips/enclave-prepare-ami.sh` in the EC2 instance (e.g. using scp) and run it.
 3. [Export the AMI instance](https://docs.aws.amazon.com/toolkit-for-visual-studio/latest/user-guide/tkv-create-ami-from-instance.html).
+
+## Demo to Celo
+For convenience some scripts have been added to make it easier to showcase the
+results, and monitor the progress of the docker compose file.  The primary
+script concerns evaluating `optimism_syncStatus` and displaying the results.
+
+This script requires the commands `tmux`, and `watch` to be installed and
+in the `PATH`. Check to see if you have them, and if you don't, be sure to
+install them using whatever method you deem necessary in order to run the
+script.
+
+After that has been done you should be able to spin up the simple script
+using the following command:
+```console
+./espresso/scripts/demo_tmux_get_sync_status.sh
+```
+
+This will launch a `tmux` session setup with a script to automatically
+query and display the `optimism_syncStatus` result for the `sequencer`,
+`verifier`, and `caff-node`.
+
+It assumes that the `docker-file.yml` is being run with the default values
+and will attempt to connect to them as needed.
+
+If you're not used to `tmux` you should be able to disconnect from the session
+using `<C-b> d`. This only detaches from the session, the session will still
+exist and be running in the background.  You can kill the session using the
+following command:
+```console
+tmux kill-session
+```
+
+Or you can reattach to it using this command instead:
+```console
+tmux attach
+```
+
+If you want to target different RPC endpoints for optimism, if you're not
+running the local demo, and want to target the remote, you can always
+specify environment variables before running the script:
+```console
+OP_RPC_SEQUENCER=http://sequencer.example.com:4545 \
+OP_RPC_VERIFIER=http://verifier.example.com:4545 \
+OP_RPC_CAFF=http://caff.example.com:4545 \
+./espresso/scripts/demo_tmux_get_sync_status.sh
+```
+
+### Prepare for the Demo
+* Go to the scripts directory.
+```console
+cd espresso/scripts
+```
+* Allow access to scripts.
+```console
+chmod +x startup.sh
+chmod +x logs.sh
+chmod +x shutdown.sh
+```
+
+### Prebuild Everything and Start All Services
+Note that `l2-genesis` is expected to take around 2 minutes.
+```console
+./startup.sh
+```
+
+### View Logs
+There are 15 services in total, as listed in `logs.sh`. It is supported to run logs for any
+service, but we may want to show logs selectively, e.g., by running the following commands one by
+one. Note that some service names are replaced by more convenient alias, but it is also suported to
+use their full names.
+```console
+./logs.sh l1-geth
+./logs.sh dev-node
+./logs.sh op-geth-sequencer
+./logs.sh sequencer
+./logs.sh verifier
+./logs.sh caff-node
+./logs.sh batcher
+```
+
+### Shut Down All Services
+```console
+./shutdown.sh
+```

espresso/.env

@@ -24,6 +24,8 @@ L1_HTTP_PORT=8545
 L1_BEACON_PORT=5052
 
 ROLLUP_PORT=9545
+VERIFIER_PORT=9546
+CAFF_PORT=9547
 
 OP_ENGINE_PORT=8552
 OP_HTTP_PORT=8546

espresso/devnet-tests/devnet_tools.go

@@ -96,7 +96,7 @@ func (d *Devnet) Up() (err error) {
 	}()
 
 	// Open RPC clients for the different nodes.
-	d.L2Seq, err = d.serviceClient("op-geth-seq", 8546)
+	d.L2Seq, err = d.serviceClient("op-geth-sequencer", 8546)
 	if err != nil {
 		return err
 	}

espresso/docker-compose.yml

@@ -108,13 +108,13 @@ services:
     environment:
       - MODE=rollup
       - L1_RPC=http://l1-geth:${L1_HTTP_PORT:?err}
-      - OP_RPC=http://op-geth-seq:${OP_HTTP_PORT:?err}
+      - OP_RPC=http://op-geth-sequencer:${OP_HTTP_PORT:?err}
     depends_on:
       l1-geth:
         condition: service_healthy
       l1-genesis:
         condition: service_completed_successfully
-      op-geth-seq:
+      op-geth-sequencer:
         condition: service_healthy
     volumes:
       - ./deployment/l2-config:/config
@@ -136,7 +136,7 @@ services:
       - ./deployment/l2-config:/config
       - ./deployment/deployer:/deployer:ro
 
-  op-geth-seq:
+  op-geth-sequencer:
     extends:
       file: docker-compose-op-geth.yml
       service: op-geth
@@ -166,16 +166,17 @@ services:
     depends_on:
       l2-rollup:
         condition: service_completed_successfully
-      op-geth-seq:
+      op-geth-sequencer:
         condition: service_healthy
       l1-validator:
         condition: service_started
     environment:
-      CAFF_L1_ETH_RPC: http://l1-geth:${L1_HTTP_PORT}
       OP_NODE_L1_ETH_RPC: http://l1-geth:${L1_HTTP_PORT}
       OP_NODE_L1_BEACON: http://l1-beacon:${L1_BEACON_PORT}
-      OP_NODE_L2_ENGINE_RPC: http://op-geth-seq:${OP_ENGINE_PORT}
+      OP_NODE_L2_ENGINE_RPC: http://op-geth-sequencer:${OP_ENGINE_PORT}
       OP_NODE_RPC_PORT: ${ROLLUP_PORT}
+    ports:
+      - "${ROLLUP_PORT}:${ROLLUP_PORT}"
     volumes:
       - ./deployment/l2-config:/config:ro
       - /etc/localtime:/etc/localtime:ro
@@ -208,12 +209,16 @@ services:
       OP_NODE_L1_ETH_RPC: http://l1-geth:${L1_HTTP_PORT}
       OP_NODE_L1_BEACON: http://l1-beacon:${L1_BEACON_PORT}
       OP_NODE_L2_ENGINE_RPC: http://op-geth-verifier:${OP_ENGINE_PORT}
+      OP_NODE_RPC_PORT: ${VERIFIER_PORT}
+    ports:
+      - "${VERIFIER_PORT}:${VERIFIER_PORT}"
     volumes:
       - ./deployment/l2-config:/config:ro
     command:
       - op-node
       - --l2.jwt-secret=/config/jwt.txt
       - --rollup.config=/config/rollup.json
+      - --rpc.addr=0.0.0.0
       - --l1.http-poll-interval=1s
       - --l1.epoch-poll-interval=1s
       - --p2p.disable=true
@@ -232,20 +237,24 @@ services:
       l2-rollup:
         condition: service_completed_successfully
     environment:
-      CAFF_L1_ETH_RPC: http://l1-geth:${L1_HTTP_PORT}
       OP_NODE_L1_ETH_RPC: http://l1-geth:${L1_HTTP_PORT}
       OP_NODE_L1_BEACON: http://l1-beacon:${L1_BEACON_PORT}
       OP_NODE_L2_ENGINE_RPC: http://op-geth-caff-node:${OP_ENGINE_PORT}
-      CAFF_ESPRESSO_LIGHT_CLIENT_ADDR: "0x703848f4c85f18e3acd8196c8ec91eb0b7bd0797"
-      CAFF_HOTSHOT_URLS: http://espresso-dev-node:${ESPRESSO_SEQUENCER_API_PORT},http://espresso-dev-node:${ESPRESSO_SEQUENCER_API_PORT}
+      OP_NODE_CAFF_L1_ETH_RPC: http://l1-geth:${L1_HTTP_PORT}
+      OP_NODE_CAFF_ESPRESSO_LIGHT_CLIENT_ADDR: "0x703848f4c85f18e3acd8196c8ec91eb0b7bd0797"
+      OP_NODE_CAFF_HOTSHOT_URLS: http://espresso-dev-node:${ESPRESSO_SEQUENCER_API_PORT},http://espresso-dev-node:${ESPRESSO_SEQUENCER_API_PORT}
+      OP_NODE_RPC_PORT: ${CAFF_PORT}
+    ports:
+      - "${CAFF_PORT}:${CAFF_PORT}"
     volumes:
       - ./deployment/l2-config:/config:ro
     command:
       - op-node
       - --l2.jwt-secret=/config/jwt.txt
       - --rollup.config=/config/rollup.json
-      - --caff.node=true
+      - --rpc.addr=0.0.0.0
       - --sequencer.enabled=false
+      - --caff.node=true
       - --verifier.l1-confs=0
       - --rollup.load-protocol-versions=false
       - --rollup.halt=none
@@ -269,7 +278,7 @@ services:
     depends_on:
       l1-geth:
         condition: service_healthy
-      op-geth-seq:
+      op-geth-sequencer:
         condition: service_started
       op-node-sequencer:
         condition: service_started
@@ -280,7 +289,7 @@ services:
     environment:
       L1_RPC: http://l1-geth:${L1_HTTP_PORT}
       OP_BATCHER_L1_ETH_RPC: http://l1-geth:${L1_HTTP_PORT}
-      OP_BATCHER_L2_ETH_RPC: http://op-geth-seq:${OP_HTTP_PORT}
+      OP_BATCHER_L2_ETH_RPC: http://op-geth-sequencer:${OP_HTTP_PORT}
       OP_BATCHER_ROLLUP_RPC: http://op-node-sequencer:${ROLLUP_PORT}
       OP_BATCHER_ESPRESSO_URL: http://espresso-dev-node:${ESPRESSO_SEQUENCER_API_PORT},http://espresso-dev-node:${ESPRESSO_SEQUENCER_API_PORT}
     volumes:

espresso/docker/op-geth/op-geth-init.sh

@@ -31,7 +31,7 @@ if [ "$MODE" = "genesis" ]; then
       --data '{"jsonrpc":"2.0","method":"eth_getBlockByNumber","params":["finalized", false],"id":1}' \
       "$L1_RPC" | jq -r '.result.number')
 
-    if [[ -z "$$finalized_block" || "$$finalized_block" == "null" ]]; then
+    if [[ -z "$finalized_block" || "$finalized_block" == "null" ]]; then
       echo "No finalized block yet, waiting..."
       sleep 3
       continue

espresso/scripts/demo_tmux_get_sync_status.sh

@@ -0,0 +1,24 @@
+#!/bin/bash
+
+OP_RPC_SEQUENCER=${OP_RPC_SEQUENCER:-http://localhost:9545}
+OP_RPC_VERIFIER=${OP_RPC_VERIFIER:-http://localhost:9546}
+OP_RPC_CAFF=${OP_RPC_CAFF:-http://localhost:9547}
+
+set -euC pipefail
+
+# Change the current directory to the script's directory
+cd "$(dirname "$0")"
+
+# If the tmux session already exists, we will attach to it.
+if tmux has-session -t '=get_sync_status' 2>/dev/null; then
+  echo "Tmux session 'get_sync_status' already exists. Exiting."
+  tmux kill-session -t get_sync_status || true
+fi
+
+# Create a new tmux session, detached, named "get_sync_status"
+tmux new-session -d -s get_sync_status \; \
+    send-keys "NODE_NAME=sequencer RPC_ADDRESS=$OP_RPC_SEQUENCER watch -p -n 1 -c -d ./get_sync_status.sh" ENTER \; \
+    split-window -h "NODE_NAME=verifier RPC_ADDRESS=$OP_RPC_VERIFIER watch -p -n 1 -c -d ./get_sync_status.sh" \; \
+    split-window -h "NODE_NAME=caff-node RPC_ADDRESS=$OP_RPC_CAFF watch -p -n 1 -c -d ./get_sync_status.sh" \; \
+    select-layout even-horizontal \; \
+    attach

espresso/scripts/get_sync_status.sh

@@ -0,0 +1,36 @@
+#!/bin/bash
+# This is a convenience script to fetch data from the optimism node for
+# "optimism_syncStatus" RPC method.
+
+echo "NODE $NODE_NAME"
+JSON_DATA=$(curl -X POST -H "Content-Type: application/json" --data '{"jsonrpc":"2.0","method":"optimism_syncStatus","params":[],"id":1}' $RPC_ADDRESS 2>/dev/null)
+
+# Make sure the the RPC call was successful
+if [ $? -ne 0 ]; then
+    echo "Failed to connect to $RPC_ADDRESS"
+    exit 1
+fi
+
+
+# Store the results for easier processing
+RESULT=$(echo $JSON_DATA | jq .result)
+
+# Extract and print some fields from the JSON response
+output_block_details() {
+    BLOCK=$(echo $RESULT | jq -r .$1)
+    echo "$1: ($(echo $BLOCK | jq -r .number))"
+    echo "  hash: $(echo $BLOCK | jq -r .hash)"
+    echo "  parentHash: $(echo $BLOCK | jq -r .parentHash)"
+    echo "  timestamp: $(echo $BLOCK | jq -r .timestamp)"
+}
+
+# Output the block details in a simple format
+output_block_details "current_l1"
+output_block_details "current_l1_finalized"
+output_block_details "head_l1"
+output_block_details "safe_l1"
+output_block_details "finalized_l1"
+echo
+output_block_details "unsafe_l2"
+output_block_details "safe_l2"
+output_block_details "finalized_l2"