You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: Logbook.md
+108Lines changed: 108 additions & 0 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1,7 +1,91 @@
1
1
# Leios logbook
2
2
3
+
## 2025-02-16
4
+
5
+
### DeltaQ Update
6
+
7
+
The `topology-checker` has been upgraded with a new option to perform a ΔQSD analysis:
8
+
it will extract inter-node latencies from the given topology, classify them into near/far components, and use these to build a parameterised ΔQ model.
9
+
This model is then fitted against the distribution of minimal completion latencies as obtained with the Dijkstra algorithm, averaging over the distributions obtained by using each node as starting point.
10
+
The resulting completion outcome is plotted to the terminal together with the extracted completion latency distribution to give a visual impression of the quality of the model fit.
11
+
12
+
Learnings from this exercise:
13
+
14
+
- latencies within the topologies examined (from the topology generator as well as the “realistic” set from the Rust simulation) very clearly consist of differing near/far components (there’s a “knee” in the graphs)
15
+
- latency-weighted Dijkstra shortest paths are _extremely long_ in terms of hop count, much longer than I expected (mean 4–5, max 8 for the topology-100; min 8, max 20 for the “realistic” topology)
16
+
- composing a ΔQ model as a sequence of some number of near hops (with early exit probabilities) and some number of far hops (also with early exit) yields models that roughly fit the overall shape, which is dominated by the `far` component of latencies, but there always is a significant deviation at low latencies
17
+
- the deviation is that observed completion starts out much slower than the model would predict, so the model goes faster first, then “pauses” for 100ms or so, crossing the observed CDF again, then catching up — thereafter the high latency behaviour works out quite well
18
+
19
+
My hope had been to use a model that can be understood behaviourally, not just statistically, so that the resource usage tracking features of the `delta_q` library could be brought to bear.
20
+
**This has not yet been achieved.**
21
+
While the timeliness graph can be made to match to some degree, doing this results in a ΔQ expression for which at least I no longer understand the load multiplication factors that should be applied — in other words, how many peer connections are supposedly used at each step of the process.
22
+
The remaining part of this week’s plan was to be able to use the fitted model to obtain a formula for creating such models algebraically;
23
+
this has been put on hold because it seems easier to just generate a topology with the desired properties and then use the `topology-checker` to get the corresponding ΔQSD model.
24
+
25
+
In any case, the `topology-checker` now outputs the fitted ΔQSD model in the syntax needed for the `delta_q` web app, so that you can directly play with the results.
26
+
27
+
## 2025-02-14
28
+
29
+
### Formal methods
30
+
31
+
- Added conformance testing client of the executable Short Leios specification that
32
+
is tested against the model using the executable Short Leios specification as well.
33
+
- Merged executable specification for Simplified Leios into main
34
+
35
+
### Haskell simulation
36
+
37
+
- Updated config defaults for block sizes and timings, PR waiting for
38
+
additional reviews by research.
39
+
- Added support for idealized simulation conditions
40
+
- realism features that can be individually dropped:
41
+
- requesting block body from a single peer.
42
+
- tcp congestion window modeling
43
+
- also supports unlimited bandwidth links.
44
+
- mini-protocol multiplexing
45
+
- see data/simulation/config-idealised.yaml
46
+
- Started work on comparison to idealised diffusion report.
47
+
- simulation final output includes `raw` field containing the
48
+
accumulated data and simulation parameters.
49
+
- other stats can be computed from this field.
50
+
- implemented extraction of block diffusion cdf for required
51
+
percentiles.
52
+
- TODO: expose it as a command that takes `raw` field as input
53
+
- small gnuplot script to plot multiple cdfs at once (y axis in logscale).
54
+
55
+
### Rust simulation
56
+
57
+
- Optimized decoding the CBOR stream in the visualization
58
+
- Added total TX count to the visualization's view of blocks
59
+
- Added total CPU time to TaskFinished events
60
+
3
61
## 2025-02-13
4
62
63
+
### Brainstorming succinct schemes for Leios BLS key registration and witnessing
64
+
65
+
Recall that we have the following situation/requirements:
66
+
67
+
- We want to evolve the BLS keys and have forward security.
68
+
- We don't want a registration process (commitment) that involves a big message.
69
+
- We also don't want the proof of possession to involve a large message.
70
+
- Individual votes must be small and contain no redundancy: i.e., we don't want to include a large witness for the proof of possession.
71
+
72
+
Here's a snapshot of one recent proposal, but much discussion is underway.
73
+
74
+
- Every 90 days, as part of its operational certificate each SPO includes a commitment to Leios keys: let's say that Leios keys evolve every epoch or KES period, so we'd need 18 or 60 commitments, respectively. This commitment is only 124 bytes if KZG commitments are used.
75
+
- Before the start of a new key evolution period, each SPOs diffuses a message opening their keys for the new period.
76
+
- This message would have to be 316 bytes:
77
+
- 28 bytes for the pool ID
78
+
- 96 bytes for the public key
79
+
- 96 bytes for the opening
80
+
- 2 * 48 = 96 bytes for the proof of possession
81
+
- 316 bytes/pool * 3000 pools = 948 kB would have to be stored permanently, so that syncing from genesis is possible.
82
+
- This is a lot of data to squeeze into RBs.
83
+
- However, It's not clear if it is safe to put it into IBs or a non-RB block.
84
+
- Instead of storing this on the ledger, would could a single SNARK attest to the following?
85
+
- Input is (ID of key evolution period, pool ID, public key)
86
+
- Output is whether the proof of possession exists.
87
+
- Certificates are really small because we've already recorded the proofs of possession at the start of the key evolution period.
88
+
5
89
### Certificate CPU benchmarks as a function of number of voters
6
90
7
91
In support of the Haskell and Rust simulations, we've benchmarked certificate operations as a function of the number of voters. A realistic distribution of stake is used in these measurements.
@@ -19,6 +103,17 @@ Serialization and deserialization likely also exhibit the same trend.
19
103
20
104
A recipe for parallelizing parts of the certificate operations has been added to the [Specification for BLS certificates](crypto-benchmarks.rs/Specification.md).
21
105
106
+
### Rust simulation
107
+
108
+
Updated event format to more closely match standards:
109
+
110
+
- Timestamps are in seconds instead of nanoseconds.
111
+
- CPU subtasks have a duration attached to the "started" event, and no "finished" event.
112
+
113
+
Started tracking vote bundle sizes to display in visualization.
114
+
115
+
Added support for CBOR output (with identical schema to JSON output).
116
+
22
117
## 2025-02-12
23
118
24
119
### Added BLS crypto to CI
@@ -28,6 +123,12 @@ The CI job [crypto-benchmarks-rs](.github/workflows/crypto-benchmarks-rs.yaml) d
28
123
- Runs the tests for the BLS reference implementation
29
124
- Runs the BLS vote and certificate benchmarks
30
125
126
+
### Rust simulation
127
+
128
+
Minor build fixes (specify a MSRV, use a fixed toolchain version in CI)
129
+
130
+
Visualization now displays a breakdown of the size of each block, as well as total bytes sent/received by each node.
131
+
31
132
## 2025-02-11
32
133
33
134
### Reference implementation and benchmarking BLS certificates
@@ -39,6 +140,7 @@ The [BLS benchmarking Rust code for Leios](crypto-benchmarks.rs/) was overhauled
39
140
- Benchmarks for the inputs to the Leios and Haskell, Rust, and DeltaQ simulations.
40
141
- CBOR serialization and deserialization of Leios messages.
41
142
- Command-line interface (with example) for trying out Leios's cryptography: create and verity votes, certificates, etc.
143
+
42
144
* Document specifying the algorithms and tabulating benchmark results.
43
145
44
146
Note that this BLS scheme is just one viable option for Leios. Ongoing work and ALBA, MUSEN, and SNARKs might result in schemes superior to this BLS approach. The key drawback is the need for periodic registration of ephemeral keys. Overall, this scheme provides the following:
@@ -47,6 +149,12 @@ Note that this BLS scheme is just one viable option for Leios. Ongoing work and
47
149
- Certificate generation and verification in 90 ms and 130 ms, respectively.
48
150
- Votes smaller than 200 bytes.
49
151
152
+
### Rust simulation
153
+
154
+
Added support for `ib-diffusion-strategy` (freshest-first, oldest-first, or peer-order). Unlike the Haskell sim, this doesn't affect EB or vote diffusion; nodes can download an unlimited number of EBs or vote bundles from any given peer.
155
+
156
+
Added support for `relay-strategy`: it affects TXs, IBs, EBs, votes, and RBs.
0 commit comments