Skip to content

Commit 8c1f638

Browse files
authored
Added questions, test cases, metrics, and diagnostics
1 parent fd8d3ce commit 8c1f638

File tree

1 file changed

+35
-2
lines changed

1 file changed

+35
-2
lines changed

docs/protocol-testing.md

Lines changed: 35 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -19,14 +19,47 @@ behavior, it is a red flag.
1919
## Questions
2020

2121
### Input Block (IB) Parameters
22+
2223
- Are smaller-sized, more frequent IBs better or worse than larger-sized, less frequent IBs?
2324
- How does IB generation frequency relate to IB network dissemination time?
24-
- How does this time relate to the time described in the network functionality in the Leios paper?
25-
- What is the induced network capacity?
25+
- How does this time relate to the time described in the network functionality in the Leios paper?
26+
- What is the induced network capacity?
27+
- How rapidly or gradually does efficiency degrade as throughput increases?
28+
- Do RBs propagate back to IB producers in time for the ledger update to be available when the IB is produced?
29+
30+
### Endoser Block (EB) Parameters
31+
- How does EB generation frequency relate to having zero or duplicate EBs as candidates for certification in the RB?
32+
33+
### Resources
34+
35+
- What are the peak CPU resources required as a function of throughput?
36+
- What operations create CPU bottlenecks?
2637

2738
## Test Cases
2839

2940
| No. | Description | [Parameter](../data/simulation/config.d.ts) |
3041
| --- | ---------------------------- | -------------------------------------------------------------------------------- |
3142
| 1 | Impact of increasing IB size | `ib-body-max-size-bytes`, `ib-body-avg-size-bytes`, `tx-size-bytes-distribution` |
3243
| 2 | Vary IB production rate | `ib-generation-probability` |
44+
| 3 | Vary EB production rate | `eb-generation-probability` |
45+
| 4 | Maximize throughput | `ib-body-avg-size-byte` * `ib-generation-probability` |
46+
47+
## Metrics
48+
49+
- Duplicate EBs
50+
- EBs not making into a RB (short Leios only)
51+
- IBs sent or received outside of required window for the pipeline
52+
- EBs sent or received outside of required window for the pipeline
53+
- Votes sent or received outside of required window for the pipeline
54+
- "Expired" EBs
55+
- "Expired" IBs
56+
- RBs arriving after an IB is produced in a new pipeline
57+
- Peak CPU usage
58+
- Average CPU usage
59+
- I/O operations per second
60+
61+
## Diagnostics (i.e., faithfulness to the protocol)
62+
63+
- IB, EB, and vote production matches protocol parameters
64+
- IB, EB, and vote production is proportional to a node's stake
65+
- Propagation times are consistent with DeltaQSD estimates

0 commit comments

Comments
 (0)