Add ability to generate recipes in YAML and override (#323)

- New ability to override recipes based on YAML files
- New recipe custom recipes based on base recipes
- New `generate` command to spit out base and custom recipes as YAML
files (along with e.g. `rbuilder.toml` config)
- New `recipes` command to list available base and custom recipes
- `start` command now takes recipe files as argument in addition to
still accepting base recipes
- New `test` command which sends a test tx to EL running locally

TODOs:
- [x] Docs
- [x] Unit tests

Author: canercidam

.gitignore

@@ -6,3 +6,5 @@ e2e-test/
 playground/cache
 playground/utils/op-deployer*
 playground/utils/state.json
+recipe/
+/playground.yaml

README.md

@@ -22,14 +22,49 @@ builder-playground start l1
 builder-playground start opstack --external-builder http://localhost:4444
 ```
 
+## Installation
+
+```
+$ go install github.com/flashbots/builder-playground@latest
+```
+
+or clone the repository and do:
+
+```
+$ go install .
+```
+
+or do `go build .` and run from the repository like `./builder-playground`.
+
 ## Getting started
 
-Clone the repository and use the `start` command to deploy a specific recipe:
+See the available recipes from the `recipes` command:
+
+```
+Base Recipes:
+  l1
+    Deploy a full L1 stack with mev-boost
+    reth, lighthouse-beacon-node, lighthouse-validator-node, mev-boost-relay
+
+... (more base and custom recipes available in the original output)
+```
+
+Use the `start` command to deploy a specific recipe:
+
+```bash
+$ builder-playground start l1
+```
+
+and you can send a test transaction from prefunded accounts by using:
 
 ```bash
-$ builder-playground start <recipe>
+$ builder-playground test
 ```
 
+If you need to modify a recipe or build your own, make sure to check out the [custom recipes](docs/custom_recipes.md) documentation!
+
+Base recipes are also runnable with some flags and you can check out the documentation [here](docs/recipes) do find out.
+
 Currently available recipes:
 
 ### L1 Recipe

custom-recipes/rbuilder/bin.yaml

@@ -0,0 +1,27 @@
+base: l1
+description: Use reth and rbuilder binaries from GitHub releases
+
+recipe:
+  reth:
+    services:
+      el:
+        release:
+          name: "reth"
+          org: "paradigmxyz"
+          version: "v1.10.1"
+
+  builder:
+    services:
+      rbuilder:
+        release:
+          name: "rbuilder"
+          org: "flashbots"
+          version: "v1.3.4"
+          format: "binary"
+        args:
+          - "run"
+          - "rbuilder.toml"
+        files:
+          "rbuilder.toml": "rbuilder.toml"
+        depends_on:
+          - "el:healthy"

custom-recipes/rbuilder/custom.yaml

@@ -0,0 +1,22 @@
+base: l1
+description: Use custom local reth and rbuilder binaries
+
+recipe:
+  reth:
+    services:
+      el:
+        # Use a custom local reth binary - specify relative or absolute path
+        host_path: "reth"
+
+  builder:
+    services:
+      rbuilder:
+        # Use a custom local rbuilder binary - specify relative or absolute path
+        host_path: "rbuilder"
+        args:
+          - "run"
+          - "rbuilder.toml"
+        files:
+          "rbuilder.toml": "rbuilder.toml"
+        depends_on:
+          - "el:healthy"

custom-recipes/rbuilder/rbuilder.toml

@@ -0,0 +1,53 @@
+log_json = false
+log_level = "info,rbuilder=debug"
+redacted_telemetry_server_port = 6061
+redacted_telemetry_server_ip = "0.0.0.0"
+full_telemetry_server_port = 6060
+full_telemetry_server_ip = "0.0.0.0"
+
+# Paths relative to artifacts directory
+chain = "genesis.json"
+reth_datadir = "volume-el-data"
+el_node_ipc_path = "volume-el-data/reth.ipc"
+
+# Ethereum private key for coinbase (receives builder fees)
+# This is the first prefunded account (0xf39Fd6e51aad88F6F4ce6aB8827279cffFb92266)
+coinbase_secret_key = "0xac0974bec39a17e36ba4a6b4d238ff944bacb478cbed5efcae784d7bf4f2ff80"
+
+# BLS secret key for signing relay submissions
+relay_secret_key = "0x25295f0d1d592a90b333e26e85149708208e9f8e8bc18f6c77bd62f8ad7a6866"
+
+cl_node_url = ["http://localhost:3500"]
+jsonrpc_server_port = 8645
+jsonrpc_server_ip = "0.0.0.0"
+extra_data = "Playground Builder ⚡🤖"
+
+ignore_cancellable_orders = true
+
+# Required sparse trie settings
+root_hash_use_sparse_trie = true
+root_hash_compare_sparse_trie = false
+
+# Start bidding immediately
+slot_delta_to_start_bidding_ms = -20000
+
+live_builders = ["mp-ordering"]
+
+enabled_relays = ["playground"]
+
+# Local mev-boost-relay for slot info and block submission
+[[relays]]
+name = "playground"
+url = "http://localhost:5555"
+priority = 0
+use_ssz_for_submit = false
+use_gzip_for_submit = false
+mode = "full"
+
+[[builders]]
+name = "mp-ordering"
+algo = "ordering-builder"
+discard_txs = true
+sorting = "max-profit"
+failed_order_retries = 1
+drop_failed_orders = true

docs/custom_recipes.md

@@ -0,0 +1,124 @@
+# Custom Recipes
+
+> **Important:** This is yet an experimental feature and sometimes might not always work as intended. Please submit issues with your use cases if you find any problems.
+
+Custom recipes let you build on top of existing base recipes (like `l1`, `opstack`, or `buildernet`) by adding, removing, or modifying components and services. Instead of duplicating entire configurations, you write a small YAML file that describes only the changes you want to make.
+
+## How Custom Recipes Work
+
+A custom recipe is a YAML file with two main parts:
+
+- **`base`**: The name of the base recipe to extend (e.g., `l1`)
+- **`recipe`**: A map of components and services to add, modify, or remove
+
+The playground merges your customizations with the base recipe at runtime. You can:
+
+- Change container images and versions
+- Add new services or components
+- Remove existing services or components
+- Override arguments, environment variables, and configuration files
+
+## Quick Start: Writing a Simple Custom Recipe
+
+Let's create a custom recipe that uses a specific version of reth. Create a file called `playground.yaml`:
+
+```yaml
+base: l1
+
+recipe:
+  reth:
+    services:
+      el:
+        tag: v1.9.0
+```
+
+This tells the playground to start with the `l1` recipe but override the `el` service in the `reth` component to use tag `v1.9.0`.
+
+You can also change the image entirely:
+
+```yaml
+base: l1
+
+recipe:
+  reth:
+    services:
+      el:
+        image: ghcr.io/my-org/my-reth-fork-image
+        tag: v1.9.0
+```
+
+## Running Your Custom Recipe
+
+Once you have a `playground.yaml` file, run it with:
+
+```bash
+playground start playground.yaml
+```
+
+The playground will load the base recipe, apply your modifications, and start all the services.
+
+---
+
+# Browsing Available Recipes
+
+To see all available base recipes and pre-built custom recipes:
+
+```bash
+playground recipes
+```
+
+This shows:
+
+- **Base Recipes**: Core recipes like `l1`, `opstack`, `buildernet`
+- **Custom Recipes**: Pre-built configurations like `rbuilder/bin` and `rbuilder/custom`
+
+Each recipe displays a description and the components it includes.
+
+## Running a Pre-Built Custom Recipe
+
+You can run any custom recipe directly by name:
+
+```bash
+playground start rbuilder/bin
+```
+
+This is equivalent to generating the custom recipe files and running them when you don't need to modify the recipe.
+
+## Generating a Custom Recipe
+
+If you want to inspect or modify a pre-built custom recipe, generate it to your current directory:
+
+```bash
+playground generate rbuilder/bin
+```
+
+This creates:
+
+- `playground.yaml` - The recipe configuration
+- Any additional files the recipe needs (e.g., `rbuilder.toml`)
+
+You can then edit these files and run:
+
+```bash
+playground start playground.yaml
+```
+
+---
+
+# Generating a Full Base Recipe
+
+Sometimes you want to see the complete configuration of a base recipe, modify it extensively, and run your own version. You can generate the full YAML representation of any base recipe:
+
+```bash
+playground generate l1
+```
+
+This creates a `playground.yaml` file with the complete `l1` recipe, showing all components, services, images, arguments, and configuration.
+
+Then edit `playground.yaml` to make any changes you need, and run it:
+
+```bash
+playground start playground.yaml
+```
+
+This approach gives you full control over every aspect of the recipe while still benefiting from the playground's orchestration.