Skip to content

Commit 0156b04

Browse files
bwbushbwbush
authored
Instructions for building and running simulations
1 parent 66a2eb7 commit 0156b04

File tree

1 file changed

+203
-39
lines changed

1 file changed

+203
-39
lines changed

‎analysis/sims/ReadMe.md‎

Lines changed: 203 additions & 39 deletions
Original file line numberDiff line numberDiff line change
@@ -1,50 +1,169 @@
1-
# Instructions for re-reunning simulation studies
1+
# Simulation studies
22

3-
These instructions are valid for the folders `2025w25/` and onwards.
3+
Contents
44

5+
- [Building the tools](#building-the-tools)
6+
- [Running the simulators](#running-the-simulators)
7+
- [Running the trace processor](#running-the-trace-processor)
8+
- [Index of simulation studies](#index-of-simulation-studies)
9+
- [Re-reunning simulation studies](#re-reunning-simulation-studies)
510

6-
## Recipe
711

8-
### Running the simulation
12+
## Building the tools
913

10-
1. Navigate to the folder for the study, for example `2025w31c/`.
11-
2. Make sure to have the following tools installed: `yaml2json`, `jq` and `pigz`
12-
- *Nix:* use the default dev shell of the `sims/` flake `nix develop ..`
13-
3. Copy or link the Rust simulator `sim-cli` executable to that folder.
14-
- If you want to reproduce the study, check out the git commit listed in the `sim-cli.hash` file.
15-
- Build with `cargo build --release` in the [sim-rs](../../sim-rs/) folder.
16-
- Link with `ln -s ../../../sim-rs/target/release/sim-cli` from the study folder.
17-
4. Copy of link the Haskell `leios-trace-processor` executable to that folder.
18-
- *Cabal:* compile using `cabal build all`.
19-
- *Nix:* compile using `nix build ../..#leios-trace-processor && ln -s result/bin/leios-trace-processor`.
20-
5. You can execute individual scenarios by executing the `run.sh` script in the relevant folder.
21-
- *Nix:* run `./run.sh`, which will automatically fetch the package dependencies.
22-
- *Without Nix:* run `bash run.sh`, but make sure you have the dependent tools already installled, which are listed in the first lines of `run.sh`.
23-
6. The results folder will contain results:
24-
- `sim.log.gz`: the simulation log file, but with a few of the minor message types discarded.
25-
- `lifecycle.csv.gz`: transaction lifecycle results.
26-
- `resoruces.csv.gz`: resource usage results.
27-
- `receipts.csv.gz`: message diffusion results.
28-
- `cpus.csv.gz`: CPU usage results.
29-
- `sizes.csv.gz`: BLock size results.
30-
7. You can also run several simulations at once using the `parallel` command-line tool.
3114

32-
### Combining the results
15+
### Haskell simulator
3316

34-
8. Execute the `combine-results.sh` script to bundle the results from multiple runs into the `results/` folder.
35-
- *Nix:* run `./combine-results.sh`.
36-
- *Without nix:* run `bash combine-results.sh` after installing the required dependencies.
37-
38-
### Plot the results in Jupyter
17+
Instructions for building the Haskell simulator `ols` are availabe in [/simulation/](../../simulator/README.md).
3918

40-
9. Start a Jupter notebook server using the command `nix run #../sims/`.
41-
10. In the web browser, navigate to the relevant experiment folder and open any `.ipynb` files of interest.
19+
20+
### Rust simulator
21+
22+
Instructions for building the Rust simulator `sim-cli` are available in [/sim-rs/](../../sim-rs/README.md).
23+
24+
25+
### Building the trace processor
26+
27+
Instructions for building the trace process `leios-trace-processor` are available in [/analysis/sims/trace-processor/](../../analysis/sims/trace-processor/ReadMe.md).
28+
29+
30+
## Running the simulators
31+
32+
Both the Haskell and the Rust simulators read compatible input files (the [configuration file](#configuration-file) and the [network topology file](#network-topology-file), but their command-line flags differ.
33+
34+
The simulators output events in JSONL or CBOR format. The output schemas of the two simulators differ somewhat but they coincide for the most useful events.
35+
36+
Note that the Rust simulator is multi-threaded and runs much faster than the single-threaded Haskell simulator.
37+
38+
39+
### Configuration file
40+
41+
- Default: [/data/simulation/config.default.yaml](../../data/simulation/config.default.yaml)
42+
- Schema: [/data/simulation/schema.schema.json](../../data/simulation/schema.schema.json)
43+
- Typescript definition: [/data/simulation/config.d.ts](../../data/simulation/config.d.ts)
44+
45+
46+
### Network topology file
47+
48+
- 100-node, stressful topology: [/data/simulation/topo-default-100.yaml](/data/simulation/topo-default-100.yaml)
49+
- Mainnet variants
50+
- Largest, 10k nodes, "pseudo-mainnet": [/data/simulation/pseudo-mainnet/topology-v1.yaml](../../data/simulation/pseudo-mainnet/topology-v1.yaml)
51+
- Most useful, 750 nodes, "mini-mainnet": [/data/simulation/pseudo-mainnet/topology-v2.yaml](../../data/simulation/pseudo-mainnet/topology-v2.yaml)
52+
- Small, for Haskell simulator, 100 nodes, "micro-mainnet": [/data/simulation/pseudo-mainnet/topology-v3.yaml](/data/simulation/pseudo-mainnet/topology-v3.yaml)
53+
54+
A network generation tool is also available.
55+
56+
57+
### Running a Haskell simulation
58+
59+
Supply the path the configuration and topology files and specify the number of slots to run.
60+
61+
```bash
62+
ols --output-seconds 1200 --leios-config-file my-config.yaml --topology-file my-network.yaml --output-file sim.log
63+
```
64+
65+
```console
66+
$ ols sim leios --help
67+
68+
Usage: ols sim leios [--seed NUMBER] [-l|--leios-config-file FILE]
69+
[(-t|--topology-file FILE) |
70+
(--tg|--topology-generation-strategy-file FILE) |
71+
COMMAND] [--log-verbosity NUMBER] [--analize]
72+
[--shared-log-format OUTPUT] [--conformance-events]
73+
74+
Leios simulation.
75+
76+
Available options:
77+
--seed NUMBER The seed for the random number generator.
78+
(default: 0)
79+
-l,--leios-config-file FILE
80+
File containing the configuration values for the
81+
Leios simulation.
82+
-t,--topology-file FILE A topology file describing the world topology.
83+
--tg,--topology-generation-strategy-file FILE
84+
A file describing the topology generation strategy.
85+
--log-verbosity NUMBER 0: no log; 1: major events; 2: debug; 3: all.
86+
(default: 1)
87+
--analize Calculate metrics and statistics.
88+
--shared-log-format OUTPUT
89+
Log format documented in trace.haskell.d.ts. OUTPUT
90+
can be `CBOR` or `JSON`.
91+
--conformance-events Emits `Slot` and `No*Generated` events in the shared
92+
log format.
93+
-h,--help Show this help text
94+
95+
Available topology generation strategies:
96+
close-and-random Pick links to close and random nodes.
97+
98+
Global options:
99+
--output-seconds SECONDS Output N seconds of simulation. (default: 40)
100+
--output-file FILE Output simulation data to file.
101+
--skip-triangle-inequality-check
102+
Do not check the topology's latencies for the
103+
triangle inequality.
104+
```
42105

43106

44-
## Data dictionary
107+
### Running a Rust simulation
45108

109+
Supply the path the configuration and topology files and specify the number of slots to run.
46110

47-
### Lifecycle analysis
111+
```bash
112+
sim-cli --slots 1200 --parameters my-config.yaml my-network.yaml sim.log
113+
```
114+
115+
```console
116+
$ sim-cli --help
117+
Usage: sim-cli [OPTIONS] [TOPOLOGY] [OUTPUT]
118+
119+
Arguments:
120+
[TOPOLOGY]
121+
[OUTPUT]
122+
123+
Options:
124+
-p, --parameters <PARAMETERS>
125+
-t, --timescale <TIMESCALE>
126+
--trace-node <TRACE_NODE>
127+
-s, --slots <SLOTS>
128+
-c, --conformance-events
129+
-a, --aggregate-events
130+
-h, --help Print help
131+
-V, --version Print version
132+
```
133+
134+
135+
## Running the trace processor
136+
137+
The trace processor summarizes key information from the JSONL output logs into convenient CSV tables. Alternatively, tools like `jq` or `mongodb` can be used to query a simulation output file.
138+
139+
```bash
140+
zcat sim.log.gz | leios-trace-processor --lifecycle-file lifecycle.csv --cpu-file cpu.csv --resource-file resource.csv --receipt-file receipt.csv
141+
```
142+
143+
```console
144+
leios-trace-processor --help
145+
Process Leios trace logs into CSV file abstracts
146+
147+
Usage: leios-trace-processor [--trace-file FILE] --lifecycle-file FILE
148+
--cpu-file FILE --resource-file FILE
149+
--receipt-file FILE
150+
151+
Leios trace processor
152+
153+
Available options:
154+
--trace-file FILE Input Leios simulation trace log file
155+
--lifecycle-file FILE Output CSV file for transaction lifecycle data
156+
--cpu-file FILE Output CSV file for CPU data
157+
--resource-file FILE Output CSV file for resource data
158+
--receipt-file FILE Output CSV file for receipt data
159+
-h,--help Show this help text
160+
```
161+
162+
163+
### Data dictionary
164+
165+
166+
#### Lifecycle analysis
48167

49168
| Field | Units | Type | Description
50169
|-------------|-------|------------------|----------------------------------------------------------------------------------------|
@@ -60,7 +179,7 @@ These instructions are valid for the folders `2025w25/` and onwards.
60179
| In RB [s] | s | output | When the transaction was first included in an RB. |
61180

62181

63-
### Resource usage
182+
#### Resource usage
64183

65184
| Field | Units | Type | Description |
66185
|---------------------------|------------|----------|-----------------------------------------------------------------------------------------------|
@@ -72,7 +191,7 @@ These instructions are valid for the folders `2025w25/` and onwards.
72191
| Maximum CPU [s/s] | %/100 | output | The one-second average peak CPU fraction consumed during the simulation. |
73192

74193

75-
### Receipt of messages
194+
#### Receipt of messages
76195

77196
| Field | Units | Type | Description |
78197
|---------------------------|-------|----------|-----------------------------------------------------------------------------------------------|
@@ -87,7 +206,7 @@ These instructions are valid for the folders `2025w25/` and onwards.
87206
| Elapsed [s] | s | output | The time elapsed between when the item was originally sent by the producer and then received. |
88207

89208

90-
### CPU usage
209+
#### CPU usage
91210

92211
| Field | Units | Type | Description |
93212
|---------------------------|-------|----------|-----------------------------------------------------------------------------------------------|
@@ -98,7 +217,7 @@ These instructions are valid for the folders `2025w25/` and onwards.
98217
| Duration [s] | s | output | The number of seconds of vCPU usage for the task. |
99218

100219

101-
### Block size
220+
#### Block size
102221

103222
| Field | Units | Type | Description |
104223
|---------------------------|-------|----------|-----------------------------------------------------------------------------------------------|
@@ -108,3 +227,48 @@ These instructions are valid for the folders `2025w25/` and onwards.
108227
| Generated [s] | s | output | When the block was generated. |
109228
| Transactions | - | output | Number of transactions directly referenced by the block. |
110229
| Endorses | - | output | The unique identifier for the item that the block endorses. |
230+
231+
232+
## Index of simulation studies
233+
234+
The [Simulation Experiments section of the second technical report](../../docs/technical-report-2.md#simulation-experiments) lists each simulation experiment, provides a link to its folder of artifacts, and summarizes its findings.
235+
236+
237+
## Re-reunning simulation studies
238+
239+
These instructions are valid for the folders `2025w25/` and onwards. Earlier folders are not as automated.
240+
241+
### Running the simulation
242+
243+
1. Navigate to the folder for the study, for example `2025w31c/`.
244+
2. Make sure to have the following tools installed: `yaml2json`, `jq` and `pigz`
245+
- *Nix:* use the default dev shell of the `sims/` flake `nix develop ..`
246+
3. Copy or link the Rust simulator `sim-cli` executable to that folder.
247+
- If you want to reproduce the study, check out the git commit listed in the `sim-cli.hash` file.
248+
- Build with `cargo build --release` in the [sim-rs](../../sim-rs/) folder.
249+
- Link with `ln -s ../../../sim-rs/target/release/sim-cli` from the study folder.
250+
4. Copy of link the Haskell `leios-trace-processor` executable to that folder.
251+
- *Cabal:* compile using `cabal build all`.
252+
- *Nix:* compile using `nix build ../..#leios-trace-processor && ln -s result/bin/leios-trace-processor`.
253+
5. You can execute individual scenarios by executing the `run.sh` script in the relevant folder.
254+
- *Nix:* run `./run.sh`, which will automatically fetch the package dependencies.
255+
- *Without Nix:* run `bash run.sh`, but make sure you have the dependent tools already installled, which are listed in the first lines of `run.sh`.
256+
6. The results folder will contain results:
257+
- `sim.log.gz`: the simulation log file, but with a few of the minor message types discarded.
258+
- `lifecycle.csv.gz`: transaction lifecycle results.
259+
- `resoruces.csv.gz`: resource usage results.
260+
- `receipts.csv.gz`: message diffusion results.
261+
- `cpus.csv.gz`: CPU usage results.
262+
- `sizes.csv.gz`: BLock size results.
263+
7. You can also run several simulations at once using the `parallel` command-line tool.
264+
265+
### Combining the results
266+
267+
8. Execute the `combine-results.sh` script to bundle the results from multiple runs into the `results/` folder.
268+
- *Nix:* run `./combine-results.sh`.
269+
- *Without nix:* run `bash combine-results.sh` after installing the required dependencies.
270+
271+
### Plot the results in Jupyter
272+
273+
9. Start a Jupter notebook server using the command `nix run #../sims/`.
274+
10. In the web browser, navigate to the relevant experiment folder and open any `.ipynb` files of interest.

0 commit comments

Comments
 (0)