Docs/day 3

docs/workshop/day-3-recap.md

@@ -4,9 +4,111 @@ Agenda
 
 [1. First Full Leios Simulation Analysis](https://github.com/input-output-hk/ouroboros-leios/blob/main/analysis/sims/2025w13/analysis.ipynb)
 
-2. Optimistic Ledger State Options
+[2. Optimistic Ledger State References](#2-optimistic-ledger-state-references)
 
 3. Community
 
+## 2. Optimistic Ledger State References
+
+The discussion focused on different approaches for handling optimistic ledger state references in the Leios protocol. The core problem is how to validate Input Blocks (IBs) against a ledger state that is not yet settled in a Ranking Block (RB). This is essential for enabling transaction chaining, where new transactions can build upon the outputs of previous transactions.
+
+### Problem Statement
+
+Validating an Input Block (IB) requires a reference to a ledger state that can be used to verify the validity of its transactions. The choice of this reference state involves a trade-off between security and latency. The most secure approach is to reference the RB from k blocks ago, where k is defined as the stability horizon and represents the length of the volatile chain suffix. This boundary provides perfect security as we can be confident that any older block referenced will be included in all possible futures of the chain. However, this approach introduces significant latency (potentially hours), making it impractical for many use cases that require quick transaction confirmation.
+
+As we move to more recent blocks to reduce latency, we face increasing security challenges. Not every node in the network may have seen the same recent blocks due to network latency or temporary forks. For example, if an IB references the most recent RB, nodes that haven't received that RB yet cannot validate the IB. This creates a coordination problem where we need to ensure that the reference state is available to enough nodes to reach consensus on IB validity.
+
+### Solution Space
+
+Each of the following approaches describes a solution where an Input Block (IB) references a different block variant which provides a ledger reference for validation.
+
+| Reference | Description | Security | Latency | Implementation<br />Complexity | Computational<br />Cost |
+|-----------|-------------|----------|---------|--------------------------|-------------------|
+| [RB](#rb-reference-approach) | IBs reference an older RB | Best | Worst | Best | Min |
+| [EB](#eb-reference-approach) | IBs reference a certified Endorsement Block (EB) | Medium | Medium | Medium | Medium |
+| [IB](#) | IBs reference other IBs | Worst | Best | Worst | Max |
+
 > [!Note]
-> Check again later this week.
+> The choice of referenceable ledger states cannot be arbitrary, not only due to security considerations but also due to practical system constraints. Maintaining too many potential reference states would lead to excessive memory usage and computational overhead as each node would need to track and validate numerous parallel ledger states. We have estimated the associated computation cost in the last column of each approach.
+
+### Data Flow Diagrams
+
+#### IB-to-RB Reference Approach
+
+The simplified diagram below shows respective lower and upper bounds for selecting an RB as ledger state reference for IB validation - each showing the extreme ends of trading off latency for security. Realistically, both are not good choices and some RB such as tip-6 might be more suitable. Note, that even the tip-3 example would introduce on average a delay of 3×20s = 90s before a user could reference outputs from a previously submitted transaction.
+
+![RB Reference Approach](rb-reference.svg)
+
+> [!Note]
+> The parameter k defines the stability horizon, which is the period during which the last k blocks of the chain remain mutable. After k blocks are added, all preceding blocks become immutable or in other words become part of the stable chain prefix.
+
+Thus, we can define a new parameter to define stability for Leios which ranges between k on the upper bound and zero on the lower (representing the tip of the chain).
+
+##### 1. Including EBs
+
+![detailed](./rb-reference-detailed.svg)
+
+This diagram shows the same ledger reference approach - pointing to RBs, but also includes EBs which have been hidden in the previous example for the stake of simplicity.
+
+##### 2. Different IB-to-RB references
+
+![complex](./rb-reference-realistic.svg)
+
+The above diagram displays a more realistic picture of different IBs referencing different RBs as their ledger state reference for validation.
+
+#### EB Reference Approach
+
+The EB reference approach offers a middle ground between security and latency. Certified EBs (those that have received votes from a majority of stake) provide security guarantees with lower latency than the [RB-reference approach](#rb-reference-approach), as they indicate that enough nodes have seen and validated them. Several core variations of the EB reference approach were discussed:
+
+##### 1. **IB-to-EB-to-RB Reference**: IBs directly reference certified EBs, which themselves reference an older RB.
+![EB Reference Approach](eb-reference-01.svg)
+
+Different to the [IB-to-RB referencing](#ib-to-rb-reference-approach), this approach has IBs reference an EB instead which itself references an RB.
+
+We briefly discussed an alternative design choice, in which IBs reference an EB and an RB. However, that design would result in many ledger states that would need to be computed and was therefore dismissed as too expensive.
+
+In this design, one gets a ledger state for each RB which gives a ledger state for each EB to be reused and IBs are validated with respect to that same state. On the contrary, due to EBs referencing an RB, there is still the same trade-off to be made as in the [IB-to-RB reference approach](#ib-to-rb-reference-approach) - having to chose more or less stable RBs for EBs resulting in higher latency or higher loss in EBs.
+
+
+##### 2. **EB Chain/ IB-to-EB(-to-EB)*-to-RB**
+
+In this approach IBs reference an EB which itself may reference another EB, which creates this chain of EBs that anchors on some older RB reference. Thus, EBs may have an RB reference or another EB reference of an EB that has not made it into an RB yet (full Leios variant). RBs on the other hand can only exactly reference one certified EB. IBs reference one of these EBs.
+
+This approach allows for more recent state references while maintaining security through the chain of certified EBs, and handles edge-case scenarios, where multiple EBs have been produced:
+
+- from different pipelines in parallel (see EB ordering below)
+- from the same pipeline
+
+![EB Chain Approach](eb-reference-02.svg)
+
+**Extensions and Implementation Details:**
+
+- **Minimum Age Requirement**: A constraint where EBs must reference an RB that is at least a certain age (e.g., 5 minutes old) to ensure it's widely available in the network.
+
+- **EB Ordering by Slot**: In the Leios protocol, multiple pipelines run in parallel, each ideally producing one RB. With parameters set for 1.5 EBs per stage (with randomness via VRF functions potentially producing more), there may be multiple EBs to choose from for inclusion in an RB.
+
+- **EB State Reuse**: An optimization technique where, when computing ledger states for EBs in a chain, we reuse the ledger state from referenced EBs and only replay the IBs in the current EB, reducing computational overhead, especially important when dealing with multiple pipelines and stages.
+
+#### IB Reference Approach
+
+> [!Warning]
+> We have not discussed this approach further and decided against progressing on it as it does not 
+> meet the security requirements. It also comes with a high implementation complexity and
+> computational overhead. The IB reference approach would require maintaining multiple speculative 
+> ledger states, creating potential for state explosion and significantly increasing validation costs
+> without providing adequate security guarantees.
+
+### Key Considerations
+
+1. **Security vs Latency Trade-off**: Each approach represents a different balance between security guarantees and latency reduction.
+
+2. **State Management**: The certified EB reference approach requires careful management of EB ordering and state reconstruction, but provides a good balance of security and latency.
+
+3. **Certification**: Certified EBs provide stability similar to RBs but with lower latency, making them a promising middle ground.
+
+### Next Steps
+
+Further analysis of the EB reference approach, particularly around:
+   - EB ordering mechanisms
+   - State reconstruction efficiency (computational cost)
+   - Security guarantees

docs/workshop/eb-reference-01.d2

@@ -0,0 +1,146 @@
+...@styles
+
+title: {
+  label: "IB-to-EB-to-RB Reference Approach"
+  near: top-center
+  style.font-size: 24
+  style.bold: true
+  style.fill: "#ffffff"
+  style.stroke: "#ffffff"
+}
+
+volatile: {
+  label: "Volatile chain suffix"
+  class: container
+
+  RBs: {
+    class: rb_stacked
+    label: "[RB]\n(tip-k, tip-3)"
+  }
+
+  RB2: {
+    class: rb
+    label: "RB\n(tip-2)"
+  }
+
+  RB1: {
+    class: rb
+    label: "RB\n(tip-1)"
+  }
+
+  RB: {
+    class: rb
+    label: "RB\n(tip)"
+  }
+}
+
+RB0: {
+  class: rb_unconfirmed
+  label: "RB"
+}
+
+EB1: {
+  class: eb
+  label: "EB"
+}
+
+EB: {
+  class: eb
+  label: "EB"
+}
+
+EB0: {
+  class: eb
+  label: "EB"
+}
+
+IB4: {
+  class: ib
+  label: "IB"
+}
+
+IB3: {
+  class: ib
+  label: "IB"
+}
+
+IB2: {
+  class: ib
+  label: "IB"
+}
+
+IB1: {
+  class: ib
+  label: "IB"
+}
+
+IB: {
+  class: ib
+  label: "IB"
+}
+
+# RB references
+
+RB0 -> volatile.RB: {
+  class: to_rb_arrow
+}
+
+volatile.RB -> volatile.RB1: {
+  class: to_rb_arrow
+}
+volatile.RB1 -> volatile.RB2: {
+  class: to_rb_arrow
+}
+volatile.RB2 -> volatile.RBs: {
+  class: to_rb_arrow
+}
+
+# EB certificates
+volatile.RB1 -> EB1: "Contains\nEB Certificate" {
+  class: eb_cert
+}
+
+volatile.RB -> EB: "Contains\nEB Certificate" {
+  class: eb_cert
+}
+
+RB0 -> EB0: "Contains\nEB Certificate" {
+  class: eb_cert
+}
+
+# EB references
+EB1 -> volatile.RB2: "Ledger Ref" {
+  class: ledger_link
+}
+
+EB -> volatile.RB1: "Ledger Ref" {
+  class: ledger_link
+}
+
+EB0 -> volatile.RB: "Ledger Ref" {
+  class: ledger_link
+}
+
+# IB references
+
+IB4 -> EB1: {
+  class: to_eb_arrow
+}
+
+IB3 -> EB1: {
+  class: to_eb_arrow
+}
+
+IB2 -> EB: {
+  class: to_eb_arrow
+}
+
+IB1 -> EB: {
+  class: to_eb_arrow
+}
+
+IB -> EB0: {
+  class: to_eb_arrow
+}
+
+direction: left