Skip to content

Commit d33aa8f

Browse files
spencer-tbjsign
andcommitted
feat: block witness checks and test fixes (#857)
* basic witness checks * feat(fw): add verkle chunkify code helper. * feat(fw): add all verkle sub commands. * chore: add chunkify code to init. * chore: remove witness. * chore: remove old witness check. * feat(fw): add witness checks init api. * chore: fix filling tmp. * refactor(fw|evm): use WitnessCheck as temp data struct pre-filling. * chore(fw|evm): fix witness check during fill. * feat(fw): verify witness function. * tests: witness check test_balance. * tests: update witness check test_blockhash_instruction. * tests: update witness check for remaining tests. * chore: rename basic mpt to vkt test as transition. * tests: temp fix test_contract_execution. * chore: fix format witness check function. * feat: add basic data values big-endian encoding. * chore: improve and fix error messages. * chore: more improvements to error messages. * chore: fix code hash for witness check. * chore: more improvements to error messages 2. * chore: more improvements to error messages 3. * chore: temp test fix. * chore: skip blockhash storage slot check temp. * chore: fix eip4762 test calls. * tests: small changes. * verkle: fix test Signed-off-by: Ignacio Hagopian <[email protected]> * verkle: test fix Signed-off-by: Ignacio Hagopian <[email protected]> * verkle: fix selfdestruct tests Signed-off-by: Ignacio Hagopian <[email protected]> * verkle: fix sstore Signed-off-by: Ignacio Hagopian <[email protected]> * verkle: fix extcodesize test Signed-off-by: Ignacio Hagopian <[email protected]> * verkle: fix extcodehash tests Signed-off-by: Ignacio Hagopian <[email protected]> * verkle: contract execution fixes Signed-off-by: Ignacio Hagopian <[email protected]> * fixes Signed-off-by: Ignacio Hagopian <[email protected]> * verkle: many contract execution fixes Signed-off-by: Ignacio Hagopian <[email protected]> * verkle: fix more tests bugs Signed-off-by: Ignacio Hagopian <[email protected]> * verkle: fix create tests Signed-off-by: Ignacio Hagopian <[email protected]> * verkle: add create with insufficient balance value-bearing Signed-off-by: Ignacio Hagopian <[email protected]> * verkle: create with big calldata fixes Signed-off-by: Ignacio Hagopian <[email protected]> * verkle: fix sload test Signed-off-by: Ignacio Hagopian <[email protected]> * verkle: uncomment contract execution subtests Signed-off-by: Ignacio Hagopian <[email protected]> * chore: typing bytes fix. * verkle: more fixes Signed-off-by: Ignacio Hagopian <[email protected]> * verkle: enable all code execution tests Signed-off-by: Ignacio Hagopian <[email protected]> * feat: make witness checks exhaustive. * fix: bug in code chunk. * feat: test_balance exhaustive checks example. * temp-fix: check hist. storage cont. * compilation fixes Signed-off-by: Ignacio Hagopian <[email protected]> * more fixes Signed-off-by: Ignacio Hagopian <[email protected]> * verkle: fixes Signed-off-by: Ignacio Hagopian <[email protected]> * verkle: call warm fixes Signed-off-by: Ignacio Hagopian <[email protected]> * verkle: fixes Signed-off-by: Ignacio Hagopian <[email protected]> * fixes Signed-off-by: Ignacio Hagopian <[email protected]> * simplify Signed-off-by: Ignacio Hagopian <[email protected]> * verkle: fixes Signed-off-by: Ignacio Hagopian <[email protected]> * verkle: codecopy/extcodecopy fixes Signed-off-by: Ignacio Hagopian <[email protected]> * verkle: remove noisy comments Signed-off-by: Ignacio Hagopian <[email protected]> * verkle: fix coinbase assertions Signed-off-by: Ignacio Hagopian <[email protected]> * verkle: fixes Signed-off-by: Ignacio Hagopian <[email protected]> * verkle: fix creates tests Signed-off-by: Ignacio Hagopian <[email protected]> * verkle: fixes Signed-off-by: Ignacio Hagopian <[email protected]> * verkle: extcodesize fixes Signed-off-by: Ignacio Hagopian <[email protected]> * verkle: selfdestruct fixes Signed-off-by: Ignacio Hagopian <[email protected]> * verkle: fix sload Signed-off-by: Ignacio Hagopian <[email protected]> * verkle: sstore fixes Signed-off-by: Ignacio Hagopian <[email protected]> * verkle: fix transfer Signed-off-by: Ignacio Hagopian <[email protected]> * verkle: more calls fixes Signed-off-by: Ignacio Hagopian <[email protected]> * verkle: more fixes Signed-off-by: Ignacio Hagopian <[email protected]> * verkle: more fixes Signed-off-by: Ignacio Hagopian <[email protected]> * verkle: more fixes Signed-off-by: Ignacio Hagopian <[email protected]> * disable temporarily 7709 tests Signed-off-by: Ignacio Hagopian <[email protected]> * verkle: reenable sub-test * verkle: enable sub-tests with partial witness charging * verkle: reenable call subtests Signed-off-by: Ignacio Hagopian <[email protected]> * verkle: add eip-7709 test Signed-off-by: Ignacio Hagopian <[email protected]> * verkle: 7709 fixes Signed-off-by: Ignacio Hagopian <[email protected]> * verkle: update 7709 blockhash values Signed-off-by: Ignacio Hagopian <[email protected]> * fixes Signed-off-by: Ignacio Hagopian <[email protected]> * fixes Signed-off-by: Ignacio Hagopian <[email protected]> * fixes Signed-off-by: Ignacio Hagopian <[email protected]> * verkle: add codecopy test with size bigger than contract Signed-off-by: Ignacio Hagopian <[email protected]> * verkle: add system contract execution test via CALL Signed-off-by: Ignacio Hagopian <[email protected]> --------- Signed-off-by: Ignacio Hagopian <[email protected]> Co-authored-by: Ignacio Hagopian <[email protected]>
1 parent 70d86ce commit d33aa8f

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

41 files changed

+1669
-1070
lines changed

src/ethereum_test_base_types/base_types.py

Lines changed: 11 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -231,18 +231,20 @@ class Sized(cls): # type: ignore
231231

232232
return Sized
233233

234-
def __new__(cls, input: NumberConvertible | N):
234+
def __new__(cls, input: NumberConvertible | N | None):
235235
"""
236236
Creates a new Number object.
237237
"""
238-
i = to_number(input)
239-
if i > cls.max_value:
240-
raise ValueError(f"Value {i} is too large for {cls.byte_length} bytes")
241-
if i < 0:
242-
i += cls.max_value + 1
243-
if i <= 0:
244-
raise ValueError(f"Value {i} is too small for {cls.byte_length} bytes")
245-
return super(FixedSizeHexNumber, cls).__new__(cls, i)
238+
if input is not None:
239+
i = to_number(input)
240+
if i > cls.max_value:
241+
raise ValueError(f"Value {i} is too large for {cls.byte_length} bytes")
242+
if i < 0:
243+
i += cls.max_value + 1
244+
if i <= 0:
245+
raise ValueError(f"Value {i} is too small for {cls.byte_length} bytes")
246+
return super(FixedSizeHexNumber, cls).__new__(cls, i)
247+
return None
246248

247249
def __str__(self) -> str:
248250
"""

src/ethereum_test_forks/forks/forks.py

Lines changed: 21 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -413,7 +413,9 @@ def call_opcodes(
413413
"""
414414
At Homestead, DELEGATECALL opcode was introduced.
415415
"""
416-
return [(Opcodes.DELEGATECALL, EVMCodeType.LEGACY),] + super(
416+
return [
417+
(Opcodes.DELEGATECALL, EVMCodeType.LEGACY),
418+
] + super(
417419
Homestead, cls
418420
).call_opcodes(block_number, timestamp)
419421

@@ -458,7 +460,9 @@ def call_opcodes(
458460
"""
459461
At Byzantium, STATICCALL opcode was introduced.
460462
"""
461-
return [(Opcodes.STATICCALL, EVMCodeType.LEGACY),] + super(
463+
return [
464+
(Opcodes.STATICCALL, EVMCodeType.LEGACY),
465+
] + super(
462466
Byzantium, cls
463467
).call_opcodes(block_number, timestamp)
464468

@@ -492,7 +496,9 @@ def create_opcodes(
492496
"""
493497
At Constantinople, `CREATE2` opcode is added.
494498
"""
495-
return [(Opcodes.CREATE2, EVMCodeType.LEGACY),] + super(
499+
return [
500+
(Opcodes.CREATE2, EVMCodeType.LEGACY),
501+
] + super(
496502
Constantinople, cls
497503
).create_opcodes(block_number, timestamp)
498504

@@ -983,7 +989,7 @@ def pre_allocation_blockchain(cls) -> Mapping:
983989
type tests.
984990
"""
985991
new_allocation = {
986-
0xFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFE: {
992+
Address(0xFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFE): {
987993
"nonce": 1,
988994
"code": (
989995
"0x60203611603157600143035f35116029575f35612000014311602957612000"
@@ -1029,7 +1035,10 @@ def evm_code_types(cls, block_number: int = 0, timestamp: int = 0) -> List[EVMCo
10291035
"""
10301036
EOF V1 is supported starting from this fork.
10311037
"""
1032-
return super(CancunEIP7692, cls,).evm_code_types( # noqa: SC200
1038+
return super(
1039+
CancunEIP7692,
1040+
cls,
1041+
).evm_code_types( # noqa: SC200
10331042
block_number,
10341043
timestamp,
10351044
) + [EVMCodeType.EOF_V1]
@@ -1058,7 +1067,9 @@ def create_opcodes(
10581067
"""
10591068
EOF V1 introduces `EOFCREATE`.
10601069
"""
1061-
return [(Opcodes.EOFCREATE, EVMCodeType.EOF_V1),] + super(
1070+
return [
1071+
(Opcodes.EOFCREATE, EVMCodeType.EOF_V1),
1072+
] + super(
10621073
CancunEIP7692, cls # noqa: SC200
10631074
).create_opcodes(block_number, timestamp)
10641075

@@ -1093,7 +1104,10 @@ def evm_code_types(cls, block_number: int = 0, timestamp: int = 0) -> List[EVMCo
10931104
"""
10941105
EOF V1 is supported starting from this fork.
10951106
"""
1096-
return super(PragueEIP7692, cls,).evm_code_types( # noqa: SC200
1107+
return super(
1108+
PragueEIP7692,
1109+
cls,
1110+
).evm_code_types( # noqa: SC200
10971111
block_number,
10981112
timestamp,
10991113
) + [EVMCodeType.EOF_V1]

src/ethereum_test_specs/blockchain.py

Lines changed: 130 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,7 @@
22
Ethereum blockchain test spec definition and filler.
33
"""
44

5-
from pprint import pprint
5+
from pprint import pformat
66
from typing import Any, Callable, ClassVar, Dict, Generator, List, Optional, Tuple, Type
77

88
import pytest
@@ -54,7 +54,7 @@
5454
Withdrawal,
5555
WithdrawalRequest,
5656
)
57-
from ethereum_test_types.verkle import VerkleTree, Witness
57+
from ethereum_test_types.verkle import StateDiff, VerkleTree, Witness, WitnessCheck
5858
from evm_transition_tool import TransitionTool
5959

6060
from .base import BaseTest, verify_result, verify_transactions
@@ -264,6 +264,10 @@ class Block(Header):
264264
"""
265265
Custom list of requests to embed in this block.
266266
"""
267+
witness_check: WitnessCheck | None = None
268+
"""
269+
Verkle execution witness check for the block.
270+
"""
267271

268272
def set_environment(self, env: Environment) -> Environment:
269273
"""
@@ -458,20 +462,52 @@ def generate_block_data(
458462
debug_output_path=self.get_next_transition_tool_output_path(),
459463
)
460464

461-
try:
465+
try: # General checks for the transition tool output
462466
rejected_txs = verify_transactions(txs, transition_tool_output.result)
463467
verify_result(transition_tool_output.result, env)
464-
# TODO: add verify witness (against vkt)
465-
# verify_witness(transition_tool_output.witness, transition_tool_output.vkt)
468+
if block.witness_check:
469+
if transition_tool_output.result.state_diff is None:
470+
raise Exception(
471+
"no state diff in transition tool output, cannot verify witness"
472+
)
473+
# TODO: hack for now, temp addition to check hist. storage contract
474+
block.witness_check.add_storage_slot(
475+
address=Address(0xFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFE),
476+
storage_slot=env.number - 1,
477+
# value=env.parent_hash,
478+
value=None,
479+
)
480+
self.verify_witness(
481+
t8n=t8n,
482+
state_diff=transition_tool_output.result.state_diff,
483+
witness_check=block.witness_check,
484+
)
466485
except Exception as e:
467486
print_traces(t8n.get_traces())
468-
pprint(transition_tool_output.result)
469-
pprint(previous_alloc)
470-
pprint(transition_tool_output.alloc)
487+
print(
488+
"\nTransition tool output result:\n"
489+
f"{pformat(transition_tool_output.result.model_dump_json())}"
490+
)
491+
print(
492+
"\nPrevious transition tool alloc:\n"
493+
f"{pformat(previous_alloc.model_dump_json())}"
494+
)
495+
if transition_tool_output.alloc is not None:
496+
print(
497+
"\nTransition tool output alloc:\n"
498+
f"{pformat(transition_tool_output.alloc.model_dump_json())}"
499+
)
471500
if transition_tool_output.vkt is not None:
472-
pprint(transition_tool_output.vkt)
473-
if transition_tool_output.witness is not None:
474-
pprint(transition_tool_output.witness)
501+
print(
502+
"\nTransition tools output verkle tree:\n"
503+
f"{pformat(transition_tool_output.vkt.model_dump_json())}"
504+
)
505+
# TODO: t8n has the witness state diff from the result for now
506+
# if transition_tool_output.witness is not None:
507+
# print(
508+
# "\nTransition tools output witness:\n"
509+
# f"{pformat(transition_tool_output.witness.model_dump_json())}"
510+
# )
475511
raise e
476512

477513
if len(rejected_txs) > 0 and block.exception is None:
@@ -544,7 +580,7 @@ def generate_block_data(
544580
)
545581
)
546582
transition_tool_output.alloc = previous_alloc
547-
# TODO: hack for now
583+
# TODO: hack for now, replace with actual witness output once available from t8n
548584
transition_tool_output.witness = Witness(
549585
verkle_proof=transition_tool_output.result.verkle_proof,
550586
state_diff=transition_tool_output.result.state_diff,
@@ -593,6 +629,87 @@ def verify_post_state(
593629
print_traces(t8n.get_traces())
594630
raise e
595631

632+
def verify_witness(
633+
self,
634+
t8n: TransitionTool,
635+
state_diff: StateDiff,
636+
witness_check: WitnessCheck,
637+
) -> None:
638+
"""
639+
Compares the expected witness check allocation account against the values updated
640+
in the block execution witness state diff.
641+
"""
642+
witness_check_state_diff, witness_check_address_mapping = t8n.get_witness_check_mapping(
643+
witness_check
644+
)
645+
print("\nExpected witness check state diff:")
646+
print(witness_check_state_diff.model_dump_json(indent=4))
647+
648+
for stem_state_diff in state_diff.root:
649+
actual_stem = stem_state_diff.stem
650+
address = witness_check_address_mapping.get(actual_stem, None)
651+
print(f"\nChecking witness for stem: {actual_stem} at address: {address}")
652+
# check for stem in the expected witness check
653+
expected_stem_state_diff = next(
654+
(sd for sd in witness_check_state_diff.root if sd.stem == actual_stem), None
655+
)
656+
if not expected_stem_state_diff:
657+
raise ValueError(
658+
"Witness check failed - missing stem not found in expected witness check.\n\n"
659+
+ pformat(
660+
{
661+
"test_account_address": str(address),
662+
"stem": str(actual_stem),
663+
},
664+
indent=4,
665+
)
666+
)
667+
for suffix_diff in stem_state_diff.suffix_diffs:
668+
actual_suffix = suffix_diff.suffix
669+
actual_current_value = suffix_diff.current_value
670+
# check for suffix in the expected witness check
671+
expected_suffix_state_diff = next(
672+
(
673+
sd
674+
for sd in expected_stem_state_diff.suffix_diffs
675+
if sd.suffix == actual_suffix
676+
),
677+
None,
678+
)
679+
if not expected_suffix_state_diff:
680+
raise ValueError(
681+
"Witness check failed - actual suffix not found in expected witness"
682+
" check.\n\n"
683+
+ pformat(
684+
{
685+
"test_account_address": str(address),
686+
"stem": str(actual_stem),
687+
"suffix": actual_suffix,
688+
"value_actual": str(actual_current_value),
689+
"value_expected": "value not found",
690+
},
691+
indent=4,
692+
)
693+
)
694+
# check the current value of the actual suffix state diff matches the expected
695+
if str(actual_current_value) != str(
696+
expected_suffix_state_diff.current_value
697+
): # TODO: temp fix str casting
698+
raise ValueError(
699+
"Witness check failed - current value mismatch. The stem and suffix"
700+
" exist.\n\n"
701+
+ pformat(
702+
{
703+
"test_account_address": str(address),
704+
"stem": str(actual_stem),
705+
"suffix": actual_suffix,
706+
"value_actual": str(actual_current_value),
707+
"value_expected": str(expected_suffix_state_diff.current_value),
708+
},
709+
indent=4,
710+
)
711+
)
712+
596713
def make_fixture(
597714
self,
598715
t8n: TransitionTool,
@@ -848,4 +965,5 @@ def generate(
848965

849966

850967
BlockchainTestSpec = Callable[[str], Generator[BlockchainTest, None, None]]
968+
851969
BlockchainTestFiller = Type[BlockchainTest]

src/ethereum_test_tools/__init__.py

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -46,8 +46,11 @@
4646
Storage,
4747
TestParameterGroup,
4848
Transaction,
49+
VerkleTree,
4950
Withdrawal,
5051
WithdrawalRequest,
52+
Witness,
53+
WitnessCheck,
5154
add_kzg_version,
5255
ceiling_division,
5356
compute_create2_address,
@@ -137,6 +140,9 @@
137140
"TestPrivateKey2",
138141
"Transaction",
139142
"TransactionException",
143+
"VerkleTree",
144+
"Witness",
145+
"WitnessCheck",
140146
"Withdrawal",
141147
"WithdrawalRequest",
142148
"Yul",

src/ethereum_test_types/__init__.py

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -28,11 +28,11 @@
2828
Storage,
2929
Transaction,
3030
TransactionDefaults,
31-
VerkleTree,
3231
Withdrawal,
3332
WithdrawalRequest,
3433
keccak256,
3534
)
35+
from .verkle import VerkleTree, Witness, WitnessCheck
3636

3737
__all__ = (
3838
"AccessList",
@@ -60,6 +60,8 @@
6060
"VerkleTree",
6161
"Withdrawal",
6262
"WithdrawalRequest",
63+
"Witness",
64+
"WitnessCheck",
6365
"ZeroPaddedHexNumber",
6466
"add_kzg_version",
6567
"ceiling_division",

0 commit comments

Comments
 (0)