Skip to content

Commit 1f7731c

Browse files
kevinnasserynuke-web3
kevinnassery
and
nuke-web3
authored
BM-253: broker README update with setup (github#76)
Co-authored-by: Nuke 🌄 <[email protected]>
1 parent 366be6d commit 1f7731c

File tree

9 files changed

+149
-33
lines changed

9 files changed

+149
-33
lines changed

‎docs/src/SUMMARY.md

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -18,9 +18,11 @@
1818
- [🏋 Prover Manual](./prover-manual/README.md)
1919

2020
- [Bento](./prover-manual/bento/README.md)
21-
- [Running Bento](./prover-manual/bento/running_bento.md)
22-
- [Performance](./prover-manual/bento/performance.md)
21+
- [Running](./prover-manual/bento/running.md)
22+
- [Performance Tuning](./prover-manual/bento/performance.md)
2323
- [Broker](./prover-manual/broker/README.md)
24+
- [Configuration](./prover-manual/broker/configure.md)
25+
- [Operation](./prover-manual/broker/operation.md)
2426

2527
- [🔗 Reference](./reference.md)
2628
- [📑 Glossary](./glossary.md)

‎docs/src/market/deployments.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -22,4 +22,4 @@ NOTE: there is no versioning presently, and these addresses may change at any ti
2222
To interact with Boundless on any network see:
2323

2424
- The [requestor broadcasting guide](../requestor-manual/broadcasting.md#public-networks)
25-
- The [prover running Bento guide](../prover-manual/bento/running_bento.md)
25+
- The [prover running Bento guide](../prover-manual/bento/running.md)

‎docs/src/market/local-development.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -167,7 +167,7 @@ Further instructions for:
167167

168168
[page-bento]: ../prover-manual/bento/README.md
169169
[page-broker]: ../prover-manual/broker/README.md
170-
[page-bento-running]: ../prover-manual/bento/running_bento.md
170+
[page-bento-running]: ../prover-manual/bento/running.md
171171
[page-requestor-broadcast]: ../requestor-manual/broadcasting.md
172172
[page-prover-manual]: ../prover-manual/README.md
173173
[page-market-design]: ./matching.md

‎docs/src/prover-manual/bento/performance.md

Lines changed: 7 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,12 @@
11
# Bento Performance Tuning
22

3-
This guide offers a practical set of steps for optimizing your initial Bento deployment. Though performance tuning ultimately depends on your specific environment, equipment, and requirements... these baseline recommendations will assist your initial deployment.
3+
This guide offers practical techniques for optimizing your initial Bento deployment.
4+
5+
<div class="warning">
6+
7+
Bento performance is tightly couples to the specific environment, equipment, and provider's requirements. Recommendations in this guide should be used as reference rather than a concrete recommendation on configuration.
8+
9+
</div>
410

511
## Recommended tools
612

‎docs/src/prover-manual/bento/running_bento.md renamed to ‎docs/src/prover-manual/bento/running.md

File renamed without changes.

‎docs/src/prover-manual/broker/README.md

Lines changed: 5 additions & 27 deletions
Original file line numberDiff line numberDiff line change
@@ -1,31 +1,9 @@
1-
# Broker Node
1+
# Broker
22

3-
The broker node is a optional addon service that runs within the [Bento][page-bento] docker-compose stack.
3+
The Broker is an optional service that runs within the [Bento][page-bento] docker-compose stack. It is responsible for market interactions including bidding on jobs, locking them, issuing job requests to the bento proving cluster, and submitting proof fulfillments on-chain.
44

5-
# Running
5+
<div class="warning">
66

7-
Before running broker you will need to ensure you have setup and are able to run Bento, the documentation for that can be found in [Running Bento][page-running-bento].
7+
TODO: Write more here about broker design & purpose
88

9-
```sh
10-
docker compose --profile broker --env-file ./.env-compose up --build
11-
```
12-
13-
Optionally if you can't run a Bento stack is is possible to run Broker using the Bonsai proving backend adding the `--bonsai-api-url` and `--bonsai-api-key` flags in the Service CLI config.
14-
15-
# Configuration
16-
17-
There are two layers of configuration for Broker:
18-
19-
- Service daemon config - Setup / wallet keys / IPs
20-
- Live configuration - Market parameters / Prover configs / Batching / Aggregator configs
21-
22-
## Service config
23-
24-
The service can be configured via CLI flags which can be supplied in the `compose.yml` broker->entrypoint. These are mostly for configuration of the service itself and private key material. Many of the vars can be found in `.env-compose` which are passed through to the services
25-
26-
## Live configuration
27-
28-
The docker-compose project bind mounts the `broker.toml` in the root of the project into the container, here you can configure all the different parameters of the market side of the daemon. Additionally this file is dynamically update the config if it is changed on the fly. Most parameters will automatically reload and re-apply async and can be used to adjust broker configurations on the fly without restarting the daemon.
29-
30-
[page-bento]: ../bento/README.md
31-
[page-running-bento]: ../bento/running_bento.md
9+
</div>

‎docs/src/prover-manual/broker/configure.md

Lines changed: 48 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,48 @@
1+
# Broker Configuration
2+
3+
Broker configuration is primarily managed through the `broker.toml` file in the Boundless directory. This file is mounted into the Broker container and is used to configure the Broker daemon. This allows for dynamic configuration of the Broker without needing to restart the daemon as in most cases variables are refreshed. If you have changed a `broker.toml` configuration, and it does not appear to take effect you can restart the Broker service to apply the changes.
4+
5+
## Settings
6+
7+
`broker.toml` contains the following settings for the market:
8+
9+
| setting | initial value | description |
10+
| ---------------------- | ------------- | -------------------------------------------------------------------------------------------------------------- |
11+
| mcycle_price | `".001"` | The price (in native token of target market) of proving 1M cycles. |
12+
| assumption_price | `"0.1"` | Currently unused. |
13+
| peak_prove_khz | `500` | This should correspond to the maximum number of cycles per second (in kHz) your proving backend can operate. |
14+
| min_deadline | `150` | This is a minimum number of blocks before the requested job expiration that Broker will attempt to lock a job. |
15+
| lookback_blocks | `100` | This is used on Broker initialization, and sets the number of blocks to look back for candidate proofs. |
16+
| max_stake | `"0.5"` | The maximum amount used to lock in a job for any single order. |
17+
| skip_preflight_ids | `[]` | A list of `imageID`s that the Broker should skip preflight checks in format `["0xID1","0xID2"]`. |
18+
| max_file_size | `50_000_000` | The maximum guest image size in bytes that the Broker will accept. |
19+
| allow_client_addresses | `[]` | When defined, acts as a firewall to limit proving only to specific client addresses. |
20+
| lockin_priority_gas | `100` | Additional gas to add to the base price when locking in stake on a contract to increase priority. |
21+
22+
<div class="warning">
23+
24+
Pay particular attention to quotation for config values, as they matter in TOML.
25+
26+
</div>
27+
28+
## Increasing Lock-in Rate
29+
30+
The following examples would be methods of making your Broker more competitive in the market, thus more likely to lock-in on orders, either economically or accelerating the bidding process:
31+
32+
1. Decreasing the `mcycle_price` would tune your Broker to bid at lower prices for proofs.
33+
2. Increasing `lockin_priority_gas` would expedite your market operations by consuming more has which could outrun other bidders.
34+
3. Adding known `imageID`s from market observation to `skip_preflight_ids` would reduce the delay of preflight/execution on a binary allow you to beat other Brokers to transmitting a bid.
35+
36+
Before running Broker you will need to ensure you have setup and are able to run Bento, the documentation for that can be found in [Running Bento][page-bento-running].
37+
38+
```bash
39+
docker compose --profile broker --env-file ./.env-compose up --build
40+
```
41+
42+
## Tuning service settings
43+
44+
The `[prover]` settings in `broker.toml` are used to configure the prover service and largely impact the operation of the service rather than the market dynamics.
45+
46+
The most important one to monitor/tune on initial configuration is `txn_timeout`. This is the number of seconds to wait for a transaction to be mined before timing out, if you see timeouts in your logs this can be increased to enable TX operations to chain to finish.
47+
48+
[page-bento-running]: ../bento/running.md

‎docs/src/prover-manual/broker/operation.md

Lines changed: 82 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,82 @@
1+
# Broker Operation
2+
3+
<div class="warning">
4+
5+
Before operation Bento (except in basic [local development][page-local-dev] only), we highly suggest:
6+
7+
1. [Optimizing Bento performance][page-bento-perf] you will connect the Broker to
8+
2. Researching the best [configuration][page-broker-config] options fitting your specific needs
9+
10+
</div>
11+
12+
## Connect a Bento instance
13+
14+
A Broker requires a [running Bento instance][page-bento-run], that can be affirmed with a test proof fulfillment:
15+
16+
```bash
17+
RUST_LOG=info cargo run --bin bento_cli -- -c 32
18+
```
19+
20+
```txt
21+
2024-10-23T14:37:37.364844Z INFO bento_cli: image_id: a0dfc25e54ebde808e4fd8c34b6549bbb91b4928edeea90ceb7d1d8e7e9096c7 | input_id: eccc8f06-488a-426c-ae3d-e5acada9ae22
22+
2024-10-23T14:37:37.368613Z INFO bento_cli: STARK job_id: 0d89e2ca-a1e3-478f-b89d-8ab23b89f51e
23+
2024-10-23T14:37:37.369346Z INFO bento_cli: STARK Job running....
24+
2024-10-23T14:37:39.371331Z INFO bento_cli: STARK Job running....
25+
2024-10-23T14:37:41.373508Z INFO bento_cli: STARK Job running....
26+
2024-10-23T14:37:43.375780Z INFO bento_cli: Job done!
27+
```
28+
29+
## Funding your Broker on the proving market
30+
31+
To have the Broker interact with a [market deployment][page-deployments], an account for the network you wish to operate on is required, with sufficient native token (`ETH` typically) to pay for gas fees.
32+
33+
<div class="warning">
34+
35+
For market v0 Sepolia testnet purposes, we recommend around 1-2 Sepolia ETH.
36+
Gas costs for market operation in future market versions should be significantly less.
37+
38+
</div>
39+
40+
The following process will guide you through setting up a new wallet and funding it with testnet ETH:
41+
42+
1. Set the environment variables `PRIVATE_KEY`, `SET_VERIFIER_ADDR`,`PROOF_MARKET_ADDR` in `.env-compose`:
43+
44+
```bash
45+
# Prover node configs
46+
...
47+
PRIVATE_KEY=0xYOUR_TEST_WALLET_PRIVATE_KEY_HERE
48+
...
49+
PROOF_MARKET_ADDR=0x261D8c5e9742e6f7f1076Fa1F560894524e19cad # This is the address of the market contract on the target chain.
50+
...
51+
RPC_URL="https://rpc.sepolia.org" # This is the RPC URL of the target chain.
52+
...
53+
```
54+
55+
2. Load the `.env-compose` file into the environment:
56+
57+
```bash
58+
source .env-compose
59+
```
60+
61+
3. The Broker needs to have funds deposited on the Boundless market contract to cover lock-in stake on requests. Run the following command to deposit an initial amount of ETH into the market contract.
62+
63+
```bash
64+
# Set amount in ETH denominated units
65+
# 0.5 Sepolia ETH should be OK for basic testing
66+
# BOUNDLESS_DEPOSIT=0.5
67+
68+
RUST_LOG=info,boundless_market=debug cargo run --bin cli -- deposit ${BOUNDLESS_DEPOSIT:?}
69+
```
70+
71+
```txt
72+
2024-10-23T14:29:52.704754Z DEBUG boundless_market::contracts::proof_market: Calling deposit() value: 500000000000000000
73+
2024-10-23T14:29:52.993892Z DEBUG boundless_market::contracts::proof_market: Broadcasting deposit tx 0xfc5c11e75101a9158735ec9e9519f5692b2f64b3337268b7ed999502956cd982
74+
2024-10-23T14:30:07.175952Z DEBUG boundless_market::contracts::proof_market: Submitted deposit 0xfc5c11e75101a9158735ec9e9519f5692b2f64b3337268b7ed999502956cd982
75+
2024-10-23T14:30:07.175994Z INFO cli: Deposited: 500000000000000000
76+
```
77+
78+
[page-broker-config]: ./configure.md
79+
[page-local-dev]: ../../market/local-development.md
80+
[page-bento-perf]: ../bento/performance.md
81+
[page-deployments]: ../../market/deployments.md
82+
[page-bento-run]: ../bento/running.md

‎lychee.toml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,7 @@ exclude_all_private = true
33
# TODO: Pattern based ignores seem to be broken, so running lychee is best done with the following command
44
# git ls-files '*.md' | xargs lychee --base . --cache --
55
# https://github.com/lycheeverse/lychee/issues/470
6-
exclude_path = ["./docs/book","./lib", "./examples/counter/target"]
6+
exclude_path = ["./docs/book","./lib", "./examples/counter/target","./target"]
77
max_cache_age = "10d"
88
max_redirects = 10
99
max_retries = 3

0 commit comments

Comments
 (0)