Skip to content

Commit 97acb7a

Browse files
bwbushbwbush
authored
Linear Leios at 1000 TPS (#468)
* Experimental design for 1000 TPS test * Completed 1000 TPS analysis * Updated logbook
1 parent 87ac4df commit 97acb7a

File tree

18 files changed

+4602
-0
lines changed

18 files changed

+4602
-0
lines changed

‎Logbook.md‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,11 @@
11
# Leios logbook
22

3+
## 2025-07-25
4+
5+
### Linear Leios at 1000 TPS
6+
7+
The Jupyter notebook [analysis/sims/2025w30/analysis.ipynb](analysis/sims/2025w30/analysis.ipynb) provides evidence that Linear Leios with transaction references and Stracciatella can support throughputs of 1000 tx/s at 300 B/tx. Linear Leios with transactions embedded in the EBs cannot support such throughput.
8+
39
## 2025-07-21
410

511
### Generic analysis script for Leios simulator output

‎analysis/sims/2025w30/analysis.ipynb‎

Lines changed: 3329 additions & 0 deletions
Large diffs are not rendered by default.

‎analysis/sims/2025w30/combine-results.sh‎

Lines changed: 41 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,41 @@
1+
#!/usr/bin/env nix-shell
2+
#!nix-shell -i bash -p gnused gzip pigz "rWrapper.override { packages = with rPackages; [ data_table R_utils bit64 ggplot2 magrittr stringr ]; }"
3+
4+
set -e
5+
6+
mkdir -p results/$d
7+
for f in lifecycle resources receipts cpus
8+
do
9+
DIR=$(find linear linear-with-tx-references full-without-ibs -type f -name $f.csv.gz \( -not -empty \) -printf %h\\n -quit)
10+
HL=$(sed -n -e '1p' "$DIR/case.csv")
11+
HR=$(zcat "$DIR/$f.csv.gz" | sed -n -e '1p')
12+
if [[ "$f" == "lifecycle" || "$f" == "resources" ]]
13+
then
14+
FRACT=1.00
15+
else
16+
FRACT=0.33
17+
fi
18+
(
19+
echo "$HL,$HR"
20+
for g in $(find linear linear-with-tx-references full-without-ibs -type f -name $f.csv.gz \( -not -empty \) -printf %h\\n)
21+
do
22+
if [ ! -e "$g/stderr" ]
23+
then
24+
echo "Skipping $g because it has no stderr." >> /dev/stderr
25+
elif [ -s "$g/stderr" ]
26+
then
27+
echo "Skipping $g because its stderr is not empty." >> /dev/stderr
28+
else
29+
BL=$(sed -n -e '2p' "$g/case.csv")
30+
zcat "$g/$f.csv.gz" | gawk 'FNR > 1 && rand() <= '"$FRACT"' { print "'"$BL"'" "," $0}'
31+
fi
32+
done
33+
) | pigz -p 3 -9c > results/$f.csv.gz
34+
R --vanilla << EOI > /dev/null
35+
require(data.table)
36+
sampleSize <- $FRACT
37+
print(sampleSize)
38+
$f <- fread("results/$f.csv.gz", stringsAsFactors=TRUE)
39+
save($f, sampleSize, file="results/$f.Rdata", compression_level=9)
40+
EOI
41+
done

‎analysis/sims/2025w30/full-without-ibs/8,30000000/case.csv‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
Simulator,Variant,Stage length,Max EB size
2+
Rust,full-without-ibs,8 slot/stage,30.0 MB/EB

‎analysis/sims/2025w30/full-without-ibs/8,30000000/config.yaml‎

Lines changed: 225 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,225 @@
1+
# yaml-language-server: $schema=./config.schema.json
2+
3+
################################################################################
4+
# Simulation Configuration File
5+
################################################################################
6+
#
7+
# This file contains the default configuration for running Leios simulations in
8+
# the Haskell simulation (`simulation/`) and the Rust simulation (`sim-rs/`).
9+
#
10+
################################################################################
11+
# Simulation Configuration
12+
################################################################################
13+
14+
relay-strategy: "request-from-first"
15+
tcp-congestion-control: true
16+
multiplex-mini-protocols: true
17+
simulate-transactions: true
18+
treat-blocks-as-full: false
19+
cleanup-policies: ["cleanup-expired-vote"]
20+
timestamp-resolution-ms: 0.1
21+
22+
################################################################################
23+
# Leios Protocol Configuration
24+
################################################################################
25+
26+
leios-variant: full-without-ibs
27+
linear-vote-stage-length-slots: 8
28+
linear-diffuse-stage-length-slots: 8
29+
leios-stage-length-slots: 8
30+
leios-stage-active-voting-slots: 1
31+
leios-vote-send-recv-stages: false
32+
leios-late-ib-inclusion: true
33+
leios-header-diffusion-time-ms: 1000.0
34+
leios-mempool-sampling-strategy: ordered-by-id
35+
leios-mempool-aggressive-pruning: true
36+
# TODO: revise default
37+
praos-chain-quality: 100
38+
praos-fallback-enabled: true
39+
40+
################################################################################
41+
# Transaction Configuration
42+
################################################################################
43+
44+
tx-generation-distribution:
45+
distribution: exp
46+
lambda: 0.85
47+
scale: 1.25
48+
tx-size-bytes-distribution:
49+
distribution: constant
50+
value: 300
51+
tx-conflict-fraction: 0
52+
tx-overcollateralization-factor-distribution:
53+
distribution: constant
54+
value: 0
55+
tx-validation-cpu-time-ms: 0.065
56+
tx-max-size-bytes: 16384
57+
tx-start-time: 0
58+
tx-stop-time: 450
59+
60+
################################################################################
61+
# Ranking Block Configuration
62+
################################################################################
63+
64+
# 1/leios-stage-length-slots, targeting one RB per pipeline.
65+
# Also 20s is current rate of praos blocks.
66+
rb-generation-probability: 5.0e-2
67+
# Eng. team targets 1kB as worst case upper bound.
68+
# Actual size fairly close.
69+
rb-head-size-bytes: 1024
70+
rb-body-max-size-bytes: 90112
71+
# Note: certificate generation/validation is not included in the
72+
# timings here, see cert-* fields.
73+
rb-generation-cpu-time-ms: 1.0
74+
rb-head-validation-cpu-time-ms: 1.0
75+
76+
# On average, no Txs directly embedded in blocks.
77+
rb-body-legacy-praos-payload-avg-size-bytes: 0
78+
rb-body-legacy-praos-payload-validation-cpu-time-ms-constant: 50.0
79+
# the -per-byte component is meant to be using size as a (bad)
80+
# proxy for the complexity of the Txs included.
81+
rb-body-legacy-praos-payload-validation-cpu-time-ms-per-byte: 0.0005
82+
83+
################################################################################
84+
# Input Block Configuration
85+
################################################################################
86+
87+
ib-generation-probability: 2.0
88+
ib-shards: 1
89+
ib-shard-period-length-slots: 1
90+
ib-shard-group-count: 1
91+
92+
# ProducerId 32
93+
# SlotNo 64
94+
# VRF proof 80
95+
# Body hash 32
96+
# RB Ref 32
97+
# Signature 64
98+
# Total 304
99+
#
100+
# NOTE: using a KES Signature (like for Praos headers)
101+
# would instead more than double the total to 668.
102+
# And even 828 including Op Cert.
103+
ib-head-size-bytes: 304
104+
# 98KB to optimize for 3 TCP round trips
105+
ib-body-avg-size-bytes: 98304
106+
ib-body-max-size-bytes: 262144
107+
# Here we also use praos blocks as ballpark estimate.
108+
# Sec 2.3 Forging, of the benchmark cluster report, lists
109+
# * Slot start to announced: 0.12975s
110+
ib-generation-cpu-time-ms: 130.0
111+
ib-head-validation-cpu-time-ms: 1.0
112+
ib-body-validation-cpu-time-ms-constant: 50.0
113+
ib-body-validation-cpu-time-ms-per-byte: 0.0005
114+
ib-diffusion-strategy: "freshest-first"
115+
116+
# Haskell prototype relay mini-protocol parameters.
117+
ib-diffusion-max-bodies-to-request: 1
118+
ib-diffusion-max-headers-to-request: 100
119+
ib-diffusion-max-window-size: 100
120+
121+
################################################################################
122+
# Endorsement Block Configuration
123+
################################################################################
124+
125+
# We want one per pipeline, but not too many.
126+
eb-generation-probability: 1.0
127+
# ProducerId 32
128+
# SlotNo 64
129+
# VRF proof 80
130+
# Signature 64
131+
# Total 240
132+
#
133+
# See Note about signatures on ib-head-size-bytes.
134+
eb-size-bytes-constant: 240
135+
# IB hash
136+
eb-size-bytes-per-ib: 32
137+
# Collecting the IBs to reference and cryptography are the main tasks.
138+
# A comparable task is maybe mempool snapshotting.
139+
# Sec 2.3 Forging, of the benchmark cluster report, lists
140+
# * Mempool snapshotting: 0.07252s
141+
# 75ms then seems a generous estimate for eb generation.
142+
eb-generation-cpu-time-ms: 75.0
143+
# Validating signature and vrf proof, as in other headers.
144+
eb-validation-cpu-time-ms: 1.0
145+
146+
eb-diffusion-strategy: "peer-order"
147+
148+
# Haskell prototype relay mini-protocol parameters.
149+
eb-diffusion-max-bodies-to-request: 1
150+
eb-diffusion-max-headers-to-request: 100
151+
eb-diffusion-max-window-size: 100
152+
153+
# The maximum age of EBs included in RBs.
154+
# A an EB from slot `s` can only be included in RBs
155+
# up to slot `s+eb-max-age-slots`.
156+
# In short leios we expect votes to diffuse within 3 stages lengths of
157+
# EB generation, we allow for 2 more stage lengths to account for
158+
# variance in the interval within RBs.
159+
eb-max-age-slots: 240
160+
161+
# The maximum age of EBs to be relayed.
162+
# An EB from slot `s` will only be relayed
163+
# up to slot `s+eb-max-age-for-relay-slots`.
164+
eb-max-age-for-relay-slots: 40
165+
166+
# The maximum size of transactions (in bytes) which an EB can reference.
167+
# Only relevant when running with the "full-without-ibs" variant.
168+
eb-referenced-txs-max-size-bytes: 30000000
169+
eb-body-avg-size-bytes: 30000000
170+
171+
################################################################################
172+
# Vote Configuration
173+
################################################################################
174+
175+
# Cryptography related values taken from [vote-spec](crypto-benchmarks.rs/Specification.md)
176+
# using weighted averages of 80% persistent and 20% non-persistent.
177+
178+
# vote-spec#Committe and quorum size
179+
#
180+
# Note: this is used as the expected amount of total weight of
181+
# generated votes in the sims.
182+
vote-generation-probability: 500.0
183+
# vote-spec#"Committe and quorum size"
184+
# 60% of `vote-generation-probability`
185+
vote-threshold: 300
186+
# vote-spec#"Generate vote" 0.8*135e-3 + 0.2*280e-3
187+
vote-generation-cpu-time-ms-constant: 164.0e-3
188+
# No benchmark yet.
189+
vote-generation-cpu-time-ms-per-ib: 0
190+
# vote-spec#"Verify vote" 0.8*670e-3 + 0.2*1.4
191+
vote-validation-cpu-time-ms: 816.0e-3
192+
# The `Vote` structure counted in the -per-eb already identifies slot
193+
# (in Eid) and voter. We can assume a vote bundle is all for the same
194+
# voter and slot, so for non-persistent voters we could factor their
195+
# PoolKeyHash (28bytes) here, but that is for 20% of cases.
196+
# More relevant if EB generation is very high.
197+
vote-bundle-size-bytes-constant: 0
198+
# vote-spec#Votes 0.8*90 + 0.2*164
199+
vote-bundle-size-bytes-per-eb: 105
200+
201+
vote-diffusion-strategy: "peer-order"
202+
203+
# Haskell prototype relay mini-protocol parameters.
204+
vote-diffusion-max-bodies-to-request: 1
205+
vote-diffusion-max-headers-to-request: 100
206+
vote-diffusion-max-window-size: 100
207+
208+
################################################################################
209+
# Certificate Configuration
210+
################################################################################
211+
212+
# vote-spec - certificate size plot.
213+
# Realistic stake distributions need about 7 kilobytes for the certificate.
214+
cert-size-bytes-constant: 7168
215+
cert-size-bytes-per-node: 0
216+
217+
# For certificate timings we have bulk figures for realistic scenarios,
218+
# so we do not attempt to give -per-node (i.e. per-voter) timings.
219+
#
220+
# vote-spec#"Generate certificate"
221+
cert-generation-cpu-time-ms-constant: 90.0
222+
cert-generation-cpu-time-ms-per-node: 0
223+
# vote-spec#"Verify certificate"
224+
cert-validation-cpu-time-ms-constant: 130.0
225+
cert-validation-cpu-time-ms-per-node: 0

‎analysis/sims/2025w30/full-without-ibs/8,30000000/run.sh‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
../../linear/8,30000000/run.sh

‎analysis/sims/2025w30/full-without-ibs/8,30000000/summary.txt‎

Lines changed: 78 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,78 @@
1+
INFO praos: sim_cli::events: 438028 transactions(s) were generated in total.
2+
INFO praos: sim_cli::events: 45 naive praos block(s) were published.
3+
INFO praos: sim_cli::events: 705 slot(s) had no naive praos blocks.
4+
INFO praos: sim_cli::events: 412367 transaction(s) (123.71 MB) finalized in a naive praos block.
5+
INFO praos: sim_cli::events: 25661 transaction(s) (7.70 MB) did not reach a naive praos block.
6+
INFO praos: sim_cli::events: Pool 39 published 1 naive praos block(s)
7+
INFO praos: sim_cli::events: Pool 41 published 1 naive praos block(s)
8+
INFO praos: sim_cli::events: Pool 51 published 1 naive praos block(s)
9+
INFO praos: sim_cli::events: Pool 55 published 1 naive praos block(s)
10+
INFO praos: sim_cli::events: Pool 58 published 1 naive praos block(s)
11+
INFO praos: sim_cli::events: Pool 60 published 1 naive praos block(s)
12+
INFO praos: sim_cli::events: Pool 74 published 1 naive praos block(s)
13+
INFO praos: sim_cli::events: Pool 85 published 1 naive praos block(s)
14+
INFO praos: sim_cli::events: Pool 87 published 1 naive praos block(s)
15+
INFO praos: sim_cli::events: Pool 90 published 1 naive praos block(s)
16+
INFO praos: sim_cli::events: Pool 98 published 1 naive praos block(s)
17+
INFO praos: sim_cli::events: Pool 110 published 1 naive praos block(s)
18+
INFO praos: sim_cli::events: Pool 117 published 2 naive praos block(s)
19+
INFO praos: sim_cli::events: Pool 123 published 1 naive praos block(s)
20+
INFO praos: sim_cli::events: Pool 126 published 2 naive praos block(s)
21+
INFO praos: sim_cli::events: Pool 131 published 1 naive praos block(s)
22+
INFO praos: sim_cli::events: Pool 136 published 1 naive praos block(s)
23+
INFO praos: sim_cli::events: Pool 138 published 1 naive praos block(s)
24+
INFO praos: sim_cli::events: Pool 363 published 1 naive praos block(s)
25+
INFO praos: sim_cli::events: Pool 423 published 1 naive praos block(s)
26+
INFO praos: sim_cli::events: Pool 429 published 1 naive praos block(s)
27+
INFO praos: sim_cli::events: Pool 430 published 1 naive praos block(s)
28+
INFO praos: sim_cli::events: Pool 474 published 1 naive praos block(s)
29+
INFO praos: sim_cli::events: Pool 480 published 1 naive praos block(s)
30+
INFO praos: sim_cli::events: Pool 481 published 1 naive praos block(s)
31+
INFO praos: sim_cli::events: Pool 511 published 1 naive praos block(s)
32+
INFO praos: sim_cli::events: Pool 515 published 1 naive praos block(s)
33+
INFO praos: sim_cli::events: Pool 516 published 1 naive praos block(s)
34+
INFO praos: sim_cli::events: Pool 518 published 1 naive praos block(s)
35+
INFO praos: sim_cli::events: Pool 524 published 1 naive praos block(s)
36+
INFO praos: sim_cli::events: Pool 529 published 1 naive praos block(s)
37+
INFO praos: sim_cli::events: Pool 531 published 0 naive praos block(s)
38+
INFO praos: sim_cli::events: Pool 531 failed to publish 1 naive praos block(s) due to slot battles.
39+
INFO praos: sim_cli::events: Pool 534 published 1 naive praos block(s)
40+
INFO praos: sim_cli::events: Pool 537 published 2 naive praos block(s)
41+
INFO praos: sim_cli::events: Pool 539 published 1 naive praos block(s)
42+
INFO praos: sim_cli::events: Pool 551 published 1 naive praos block(s)
43+
INFO praos: sim_cli::events: Pool 553 published 1 naive praos block(s)
44+
INFO praos: sim_cli::events: Pool 556 published 1 naive praos block(s)
45+
INFO praos: sim_cli::events: Pool 557 published 1 naive praos block(s)
46+
INFO praos: sim_cli::events: Pool 558 published 1 naive praos block(s)
47+
INFO praos: sim_cli::events: Pool 562 published 1 naive praos block(s)
48+
INFO praos: sim_cli::events: Pool 746 published 2 naive praos block(s)
49+
INFO leios: sim_cli::events: 0 IB(s) were generated, on average 0.000 IB(s) per slot.
50+
INFO leios: sim_cli::events: 0 out of 438028 transaction(s) were included in at least one IB.
51+
INFO leios: sim_cli::events: The average age of the pending transactions is 395.543s (stddev 7.670).
52+
INFO leios: sim_cli::events: Each transaction was included in an average of NaN IB(s) (stddev NaN).
53+
INFO leios: sim_cli::events: Each IB contained an average of NaN transaction(s) (stddev NaN) and an average of 0 B (stddev 0 B). 0 IB(s) were empty.
54+
INFO leios: sim_cli::events: Each node received an average of 0.000 IB(s) (stddev 0.000).
55+
INFO leios: sim_cli::events: 87 EB(s) were generated; on average there were 0.116 EB(s) per slot.
56+
INFO leios: sim_cli::events: Each EB contained an average of 10808.356 transaction(s) (stddev 10539.621). 1 EB(s) were empty.
57+
INFO leios: sim_cli::events: Each EB contained an average of 0.000 IB(s) (stddev 0.000). 1 EB(s) were empty.
58+
INFO leios: sim_cli::events: Each IB was included in an average of NaN EB(s) (stddev NaN).
59+
INFO leios: sim_cli::events: 0 out of 0 IBs were included in at least one EB.
60+
INFO leios: sim_cli::events: 0 out of 0 IBs expired before they reached an EB.
61+
INFO leios: sim_cli::events: 18 out of 87 EBs expired before an EB from their stage reached an RB.
62+
INFO leios: sim_cli::events: 438028 out of 438028 transaction(s) were included in at least one EB.
63+
INFO leios: sim_cli::events: 60813 total votes were generated.
64+
INFO leios: sim_cli::events: Each stake pool produced an average of 281.542 vote(s) (stddev 61.236).
65+
INFO leios: sim_cli::events: Each EB received an average of 732.687 vote(s) (stddev 385.989).
66+
INFO leios: sim_cli::events: There were 11233 bundle(s) of votes. Each bundle contained 5.414 vote(s) (stddev 6.784).
67+
INFO leios: sim_cli::events: 41 L1 block(s) had a Leios endorsement.
68+
INFO leios: sim_cli::events: 399176 tx(s) (119.75 MB) were referenced by a Leios endorsement.
69+
INFO leios: sim_cli::events: 13191 tx(s) (3.96 MB) were included directly in a Praos block.
70+
INFO leios: sim_cli::events: Spatial efficiency: 119.75 MB/215.39 MB (55.598%) of Leios bytes were unique transactions.
71+
INFO leios: sim_cli::events: 217331 tx(s) (35.252%) referenced by a Leios endorsement were redundant.
72+
INFO leios: sim_cli::events: Each transaction took an average of NaNs (stddev NaN) to be included in an IB.
73+
INFO leios: sim_cli::events: Each transaction took an average of 7.302s (stddev 6.001) to be included in an EB.
74+
INFO leios: sim_cli::events: Each transaction took an average of 93.219s (stddev 100.409) to be included in a block.
75+
INFO network: sim_cli::events: 328082972 TX message(s) were sent. 328082972 of them were received (100.000%).
76+
INFO network: sim_cli::events: 0 IB message(s) were sent. 0 of them were received (NaN%).
77+
INFO network: sim_cli::events: 65163 EB message(s) were sent. 65163 of them were received (100.000%).
78+
INFO network: sim_cli::events: 8413517 Vote message(s) were sent. 8413517 of them were received (100.000%).

‎analysis/sims/2025w30/jobs.list‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
linear-with-tx-references/8,30000000/run.sh
2+
full-without-ibs/8,30000000/run.sh
3+
linear/8,30000000/run.sh

‎analysis/sims/2025w30/linear-with-tx-references/8,30000000/case.csv‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
Simulator,Variant,Stage length,Max EB size
2+
Rust,linear-with-tx-references,8 slot/stage,30.0 MB/EB

0 commit comments

Comments
 (0)