aggregate all block signatures as naive list [devnet-1] by g11tech · Pull Request #67 · leanEthereum/leanSpec

g11tech · 2025-09-25T11:03:40Z

one of the important lean-ing step especially for STF proving considerations as well as to lighten the block network payload

Also the proposer vote isn't casted separate (so validator doesn't emit vote for a validator id if its that validator id's turn to vote)
but is directly bundled with the block because of the OTS scheme limitation

Signed-off-by: Chen Kai <281165273grape@gmail.com>

src/lean_spec/subspecs/containers/state/state.py

Signed-off-by: Chen Kai <281165273grape@gmail.com>

src/lean_spec/subspecs/containers/state/state.py

unnawut

Sorry for missing the Monday's call, and also apologies if any of these comments sound stupid, still trying to catch up.

I'm also trying to grasp all the *Attestation containers and AggregatedSignatures so no thoughts on this part yet other than it seems more complex than the beacon chain.

Should we design this at all for devnet1? Even if we do, I'm thinking it could go as a separate PR so at least the individual signatures can be implemented first. Also would make the discussion scope narrower.

docs/client/chain.md

docs/client/forkchoice.md

src/lean_spec/subspecs/containers/block/block.py

src/lean_spec/subspecs/containers/state/state.py

docs/client/containers.md

Signed-off-by: Chen Kai <281165273grape@gmail.com>

jihoonsong

Can you rebase your PR?

jihoonsong · 2025-10-15T12:17:13Z

docs/client/forkchoice.md

    # sync parent chain if not available before adding block to forkchoice
    assert parent_state is not None, "Parent state not found, sync parent chain first"

+    valid_signatures = validate_signatures(block.signature)


What was the reason that we don't do assert validate_signatures(block.signature) here and make state_transition receive valid_signatures?

Its fine either way imo as this is spec, clients can choose to optimize/parallelize anywhich way they want but its good to be explicit in the STF that signatures have been validated

jihoonsong · 2025-10-15T12:50:28Z

docs/client/containers.md

-## `Vote`
+## `AttestationData`

 Vote is the attestation data that can be aggregated. Although note there is no aggregation yet in `devnet0`.


Nit: there is no Vote anymore.

will look into it if we wanna remove all vote references,

jihoonsong · 2025-10-15T13:08:29Z

docs/client/forkchoice.md

+    latest_known_votes: Dict[ValidatorIndex, SignedValidatorAttestation] = field(default_factory=dict)
+    latest_new_votes: Dict[ValidatorIndex, SignedValidatorAttestation] = field(default_factory=dict)


Why do these have to have SignedValidatorAttestation instead of Checkpoint?

for block construction purposes, you need signed objects to get signatures you want to aggregate

docs/client/forkchoice.md

jihoonsong · 2025-10-15T13:21:28Z

docs/client/containers.md

+    #    ValidatorAttestation created from its ProposerAttestationData.
+    #   till we can actually consume a block signature for re-packing this block's
+    #   proposer vote by some other future proposer
+    signature: List[Vector[byte, 4000], VALIDATOR_REGISTRY_LIMIT]


I probably missing something so I'm asking a question here. When we aggregate signatures, do we want to aggregate all different kinds of signatures? In other words, what's the best way to understand why we want to aggregate attestation signatures along with block signature?

signatures are heavy now, also with OTS constraint proposer signature which is over the block and vote is sort of anyway not "pure" anymore so to take things to logical conclusion a block (and vote) signature will represent all valid objects in the block (and vote)

src/lean_spec/subspecs/containers/block/block.py

jihoonsong · 2025-10-15T14:17:07Z

docs/client/containers.md

 ```

-## `SignedVote`
+## `ProposerAttestationData`


While I understand the constraints that made us bring up this approach, I believe the problem of signing multiple times in a slot remains. If we want to adopt this PG signature effort into a production-level protocol, it's highly likely that a proposer needs to sign more than one message in a slot. If we pack all those messages into one, a proposer has to broadcast all of their messages at once. In other words, there is only one deadline within a slot for a proposer. This is not flexible so I'm not sure if this would be enough to meet all the needs of the protocol.

IMO, we could do either find a proper solution for the problem or use a stopgap solution like this one that is just enough for the upcoming few devnets. If we were to go to the latter, current approach is one of solutions. But it seems it adds complexity, which is might not an ideal given that it's not a long-term solution. So I'd like to encourage to find a better one. Few things I have in my mind are:

Currently, BlockBody has a list of unsigned votes. We can append proposer's attestation at the end of the list and append proposer's signature at the SignedBlock's signature. It's not very clean as there is an exception that proposer's signature is made over a block while others are over attestation. But this is already what we have if I'm not wrong, and this doesn't require to change original Vote and related classes nor to add ProposerAttestationData.

We can add a rule that proposers don't vote. This would make things much simpler.

for

yes we are trying to move closer to aggregation so the list there is just a temporary thing till we have leanVM coming up. also till then its just proposer votes signature, so not even a full block & vote signature since we can't validate such a signature without leanVM

yes can be considered actually, was in my mind, but eventually signatures will be aggregated

infact the signature part can just be totally replaced by full block validity proof including execution

so yes design space is open and we will see how to simplify it if it turns out convoluted

Signed-off-by: Chen Kai <281165273grape@gmail.com>

jihoonsong · 2025-10-23T16:04:26Z

tests/lean_spec/subspecs/forkchoice/test_validator.py


-        block = sample_store.produce_block(slot, validator_idx)
-
+        block_and_signatures = sample_store.produce_block(slot, validator_idx)


Suggested change

block_and_signatures = sample_store.produce_block(slot, validator_idx)

block, signatures = sample_store.produce_block(slot, validator_idx)

You can remove BlockAndSignatures as well.

Signed-off-by: Chen Kai <281165273grape@gmail.com>

unnawut · 2025-10-24T07:37:04Z

src/lean_spec/subspecs/containers/__init__.py

+    "AttestationData",
+    "Attestation",
+    "SignedAttestation",
+    "SignedAggreagtedAttestations",


Suggested change

"SignedAggreagtedAttestations",

"SignedAggregatedAttestations",

also applies to other places

src/lean_spec/subspecs/containers/attestation/attestation.py

unnawut · 2025-10-24T07:42:16Z

src/lean_spec/subspecs/containers/vote/vote.py

+    @property
+    def slot(self) -> Slot:
+        """Return the attested slot."""
+        return self.data.slot
+
+    @property
+    def head(self) -> Checkpoint:
+        """Return the attested head checkpoint."""
+        return self.data.head
+
+    @property
+    def target(self) -> Checkpoint:
+        """Return the attested target checkpoint."""
+        return self.data.target
+
+    @property
+    def source(self) -> Checkpoint:
+        """Return the attested source checkpoint."""
+        return self.data.source


Not sure if we need these accessor functions in the specs, seems more like implementation's decision to make than the specs

Agreed, this makes the spec heavier and we can always come and get them with self.data.source for example

unnawut · 2025-10-24T07:42:53Z

src/lean_spec/subspecs/containers/vote/vote.py

+    @property
+    def data(self) -> Attestation:
+        """Expose the message for backwards compatibility with SignedVote."""
+        return self.message


We don't need backward compatibility right now so this can be removed

unnawut · 2025-10-24T08:04:42Z

src/lean_spec/subspecs/containers/vote/vote.py

+    committee assignments. This structure is defined for future use and is not
+    currently exercised by devnets.


Suggested change

committee assignments. This structure is defined for future use and is not

currently exercised by devnets.

committee assignments.

I think it's needed in devnet1 where aggregation_bits will used to map AggregatedSignatures to their validator id.

unnawut · 2025-10-24T08:17:21Z

src/lean_spec/subspecs/forkchoice/store.py

            # Ensure forkchoice is current before processing gossip
-            time_slots = self.time // INTERVALS_PER_SLOT
-            assert vote.slot <= time_slots, "Attestation from future slot"
+            time_slots = self.time // SECONDS_PER_INTERVAL


Similar to previous comment:

Suggested change

time_slots = self.time // SECONDS_PER_INTERVAL

time_slots = self.time // SLOT_DURATION_MS // 1000

merge issue, it should be INTERVALS_PER_SLOT

unnawut · 2025-10-24T08:21:47Z

src/lean_spec/subspecs/forkchoice/store.py

+        # TODO: Integrate actual aggregated signature verification.
+        return all(self._is_valid_signature(signature) for signature in signatures)
+
+    def process_block(self, signed_block_vote: SignedBlockWithAttestation) -> None:


Since the type name changed

Suggested change

def process_block(self, signed_block_vote: SignedBlockWithAttestation) -> None:

def process_block(self, signed_block_with_attestation: SignedBlockWithAttestation) -> None:

Agreed, we should modify the doc string as well if we change this

unnawut · 2025-10-24T08:37:12Z

src/lean_spec/subspecs/forkchoice/store.py

-            for validator_id, checkpoint in self.latest_known_votes.items():
+            new_attestations: list[Attestation] = []
+            new_signatures: list[Bytes4000] = []
+            for signed in self.latest_known_votes.values():


Suggested change

for signed in self.latest_known_votes.values():

for signed_attestation in self.latest_known_votes.values():

naming ambiguity

src/lean_spec/subspecs/forkchoice/store.py

unnawut · 2025-10-24T08:51:05Z

src/lean_spec/subspecs/forkchoice/store.py

            parent_root=head_root,
            state_root=Bytes32.zero(),  # Will be updated with computed hash
-            body=BlockBody(attestations=Attestations(data=attestations)),
+            body=BlockBody(attestations=Attestations(data=list(attestations))),  # need copy


Not sure what the comment means?

src/lean_spec/subspecs/containers/block/block.py

tcoratger · 2025-10-24T09:41:19Z

src/lean_spec/types/byte_arrays.py

+class Bytes4000(BaseBytes):
+    """Fixed-size byte array of exactly 4000 bytes."""
+
+    LENGTH = 4000


This type is very unusual and will probably be removed in the future.

As this file is about types that should be a bit more stable than the rest of the code, can you put a TODO or something to indicate that this will be removed in the future?

src/lean_spec/subspecs/containers/attestation/attestation.py

tcoratger · 2025-10-24T09:45:09Z

src/lean_spec/subspecs/containers/vote/vote.py

+    @property
+    def slot(self) -> Slot:
+        """Return the attested slot."""
+        return self.data.slot
+
+    @property
+    def head(self) -> Checkpoint:
+        """Return the attested head checkpoint."""
+        return self.data.head
+
+    @property
+    def target(self) -> Checkpoint:
+        """Return the attested target checkpoint."""
+        return self.data.target
+
+    @property
+    def source(self) -> Checkpoint:
+        """Return the attested source checkpoint."""
+        return self.data.source


Agreed, this makes the spec heavier and we can always come and get them with self.data.source for example

src/lean_spec/subspecs/containers/attestation/attestation.py

tcoratger · 2025-10-24T10:03:26Z

src/lean_spec/subspecs/forkchoice/store.py

+@dataclass(slots=True)
+class BlockAndSignatures:
+    """Internal bundle pairing a block with its attestation signatures."""
+
+    block: Block
+    signatures: list[Bytes4000]


Why not BlockAndAttestations here to be consistent with the rest?

src/lean_spec/subspecs/forkchoice/store.py

tcoratger · 2025-10-24T10:07:43Z

src/lean_spec/subspecs/forkchoice/store.py

+        # TODO: Integrate actual aggregated signature verification.
+        return all(self._is_valid_signature(signature) for signature in signatures)
+
+    def process_block(self, signed_block_vote: SignedBlockWithAttestation) -> None:


Agreed, we should modify the doc string as well if we change this

tcoratger · 2025-10-24T10:09:03Z

tests/lean_spec/subspecs/forkchoice/test_validator.py

        validator_idx = ValidatorIndex(2)  # Proposer for slot 2

-        block = sample_store.produce_block(slot, validator_idx)
+        block_and_signatures = sample_store.produce_block(slot, validator_idx)


Suggested change

block_and_signatures = sample_store.produce_block(slot, validator_idx)

block_and_attestations = sample_store.produce_block(slot, validator_idx)

@jihoonsong suggest removing this type and just return multi values, is that preference

Yeah the type is just for the return value and is not necessary. Clients can choose to use such intermediary types, if they prefer.

Signed-off-by: Chen Kai <281165273grape@gmail.com>

Co-authored-by: Thomas Coratger <60488569+tcoratger@users.noreply.github.com>

Signed-off-by: Chen Kai <281165273grape@gmail.com>

GrapeBaBa · 2025-10-24T16:39:41Z

@unnawut @tcoratger fix almost comments

tests/lean_spec/subspecs/containers/test_state.py

Co-authored-by: JihoonSong <jihoonsong@users.noreply.github.com>

jihoonsong

This PR is exactly 1 month old. This can go forever. Let's merge this and iterate fast.

g11tech changed the title ~~aggregate all block siignatures~~ aggregate all block siignatures [devnet-1] Sep 25, 2025

g11tech changed the title ~~aggregate all block siignatures [devnet-1]~~ aggregate all block signatures [devnet-1] Sep 25, 2025

g11tech added 2 commits October 10, 2025 15:04

aggregate all block siignatures

18cfdcc

updated spec

adecd99

g11tech force-pushed the agg-block-sigs branch from 4e0f9f1 to adecd99 Compare October 11, 2025 13:18

g11tech added 6 commits October 11, 2025 18:53

update info

8739640

typo

4ffbfb1

improvs

af89e1c

further updates to forkchoice

505e7ce

improvs

e94357e

apply spec call feedback

84a7f7a

g11tech changed the title ~~aggregate all block signatures [devnet-1]~~ aggregate all block signatures as naive list [devnet-1] Oct 13, 2025

g11tech added 4 commits October 13, 2025 16:50

fix the proposer vote construction and consumption

0636cde

enroll validators in state

603cf87

fix some issues

92967fc

fix the issue in code as well

715653c

g11tech mentioned this pull request Oct 13, 2025

implement devnet1 with signatures [WIP] blockblaz/zeam#314

Closed

unnawut mentioned this pull request Oct 14, 2025

[DRAFT] feat: pqsig specs for devnet1 #73

Closed

16 tasks

feat: init aggregate all block signatures as naive list

3d2de22

Signed-off-by: Chen Kai <281165273grape@gmail.com>

GrapeBaBa force-pushed the agg-block-sigs branch from ac47e76 to 3d2de22 Compare October 15, 2025 01:56

GrapeBaBa added 2 commits October 15, 2025 13:16

fix: fix state attestations processing

855a00b

Signed-off-by: Chen Kai <281165273grape@gmail.com>

fix: fix store process_block arg

8579814

Signed-off-by: Chen Kai <281165273grape@gmail.com>

syjn99 reviewed Oct 15, 2025

View reviewed changes

src/lean_spec/subspecs/containers/state/state.py Outdated Show resolved Hide resolved

fix: fix store produce block

8fdefec

Signed-off-by: Chen Kai <281165273grape@gmail.com>

g11tech commented Oct 15, 2025

View reviewed changes

src/lean_spec/subspecs/containers/state/state.py Outdated Show resolved Hide resolved

unnawut reviewed Oct 15, 2025

View reviewed changes

This was referenced Oct 15, 2025

Update SSZ containers regarding attestation ReamLabs/ream#840

Closed

Update newly added STF logic for Devnet-1 ReamLabs/ream#847

Closed

fix: fix test_state

e712087

Signed-off-by: Chen Kai <281165273grape@gmail.com>

jihoonsong reviewed Oct 15, 2025

View reviewed changes

GrapeBaBa added 3 commits October 23, 2025 23:53

fix; fix review comments

8dd4319

Signed-off-by: Chen Kai <281165273grape@gmail.com>

fix: fix review comment

dde60f7

Signed-off-by: Chen Kai <281165273grape@gmail.com>

Merge branch 'main' into agg-block-sigs

c8cba25

jihoonsong reviewed Oct 23, 2025

View reviewed changes

fix: fix merge issue

50ef62c

Signed-off-by: Chen Kai <281165273grape@gmail.com>

unnawut reviewed Oct 24, 2025

View reviewed changes

unnawut mentioned this pull request Oct 24, 2025

Attestation data is modified during block production #82

Closed

tcoratger reviewed Oct 24, 2025

View reviewed changes

GrapeBaBa and others added 10 commits October 24, 2025 18:12

fix: fix review comment

07a4776

Signed-off-by: Chen Kai <281165273grape@gmail.com>

fix: fix review comments

e1f882a

Signed-off-by: Chen Kai <281165273grape@gmail.com>

Update src/lean_spec/subspecs/containers/block/block.py

341b2f8

Co-authored-by: Thomas Coratger <60488569+tcoratger@users.noreply.github.com>

Update src/lean_spec/subspecs/containers/attestation/attestation.py

190a1ff

Co-authored-by: Thomas Coratger <60488569+tcoratger@users.noreply.github.com>

Update src/lean_spec/subspecs/containers/attestation/attestation.py

79513a9

Co-authored-by: Thomas Coratger <60488569+tcoratger@users.noreply.github.com>

fix: fix review comments

febaaa6

Signed-off-by: Chen Kai <281165273grape@gmail.com>

Merge branch 'main' into agg-block-sigs

ebdf203

fix: merge issue

022e02e

Signed-off-by: Chen Kai <281165273grape@gmail.com>

fix: revert

c30e1af

Signed-off-by: Chen Kai <281165273grape@gmail.com>

fix: lint

cab98d9

Signed-off-by: Chen Kai <281165273grape@gmail.com>

tcoratger approved these changes Oct 25, 2025

View reviewed changes

jihoonsong reviewed Oct 25, 2025

View reviewed changes

tests/lean_spec/subspecs/containers/test_state.py Outdated Show resolved Hide resolved

Update tests/lean_spec/subspecs/containers/test_state.py

06aab37

Co-authored-by: JihoonSong <jihoonsong@users.noreply.github.com>

jihoonsong approved these changes Oct 25, 2025

View reviewed changes

unnawut approved these changes Oct 25, 2025

View reviewed changes

unnawut merged commit ac563e7 into leanEthereum:main Oct 25, 2025
9 checks passed

GrapeBaBa mentioned this pull request Oct 25, 2025

Introducing state.validators should remove state.config.num_validators and replace all its usages #83

Closed

unnawut added the specs Scope: Changes to the specifications label Oct 25, 2025

kamilsa mentioned this pull request Oct 26, 2025

Signatures in block qdrvm/qlean-mini#29

Closed

anshalshukla mentioned this pull request Oct 26, 2025

implement signature aggregation as a naive list for devnet1 (spec PR 67) blockblaz/zeam#331

Merged

unnawut mentioned this pull request Oct 29, 2025

specs: PQ signatures leanEthereum/pm#38

Open

		latest_known_votes: Dict[ValidatorIndex, SignedValidatorAttestation] = field(default_factory=dict)
		latest_new_votes: Dict[ValidatorIndex, SignedValidatorAttestation] = field(default_factory=dict)


		block = sample_store.produce_block(slot, validator_idx)

		block_and_signatures = sample_store.produce_block(slot, validator_idx)

	block_and_signatures = sample_store.produce_block(slot, validator_idx)
	block, signatures = sample_store.produce_block(slot, validator_idx)

	"SignedAggreagtedAttestations",
	"SignedAggregatedAttestations",

		committee assignments. This structure is defined for future use and is not
		currently exercised by devnets.

	committee assignments. This structure is defined for future use and is not
	currently exercised by devnets.
	committee assignments.

	time_slots = self.time // SECONDS_PER_INTERVAL
	time_slots = self.time // SLOT_DURATION_MS // 1000

	def process_block(self, signed_block_vote: SignedBlockWithAttestation) -> None:
	def process_block(self, signed_block_with_attestation: SignedBlockWithAttestation) -> None:

	for signed in self.latest_known_votes.values():
	for signed_attestation in self.latest_known_votes.values():

	block_and_signatures = sample_store.produce_block(slot, validator_idx)
	block_and_attestations = sample_store.produce_block(slot, validator_idx)

Conversation

g11tech commented Sep 25, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

unnawut left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

jihoonsong left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

g11tech Oct 15, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

g11tech commented Sep 25, 2025 •

edited

Loading

g11tech Oct 15, 2025 •

edited

Loading