-
Notifications
You must be signed in to change notification settings - Fork 667
feat: Refactor stateless executor #10629
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Draft
rubo
wants to merge
112
commits into
master
Choose a base branch
from
feature/stateless-zisk
base: master
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
+2,820
−1,430
Draft
Changes from 43 commits
Commits
Show all changes
112 commits
Select commit
Hold shift + click to select a range
2d86ee6
Add initial refactored stateless executor
rubo fcfd95c
Fix a typo
rubo e760d5d
Make paths absolute
rubo 5a07052
Add `modexp` precompile
rubo d6c4906
Add `BLS12_PAIRING_CHECK` precompile
rubo ba1dd7d
Add `BLS12_G2ADD` precompile
rubo b7d02cb
Add `BLS12_G2MSM` precompile
rubo cd466f7
Revise `Makefile`
rubo 143938d
Merge branch 'master' into feature/stateless-zisk
rubo 541baab
Add `BLS12_MAP_FP_TO_G1` precompile
rubo e2160c1
Add `BLS12_MAP_FP2_TO_G2` precompile
rubo 9753bad
Add `BLS12_G1ADD` precompile
rubo c34ad37
Add `BLS12_G1MSM` precompile
rubo d970109
Implement preliminary input serialization
rubo e835363
Merge branch 'master' into feature/stateless-zisk
rubo c101661
Revise input formatting
rubo 1ecab29
Refactor and add SecP256k1 functions
rubo cbaa939
Shut the cspell up
rubo bef6f51
Change chain id to `uint` and add tx sig recovery (disabled)
rubo e0dc870
Reformat
rubo 9265125
Refactor and optimize Zisk I/O
rubo 663c8fe
Minor refactor
rubo 2643987
Replace throwing errors with fail-fasts.
rubo dde4bbb
Simplify the entrypoint
rubo e1afdea
Switch to little-endian
rubo a56075f
Refactor Zisk I/O
rubo ce38d9c
Optimize RLP handling
rubo e7e7874
Add `KZG_POINT_EVALUATION` precompile function
rubo f51fe71
Add `P256VERIFY` precompile function
rubo e06917b
Add Keccak-256 function
rubo aadec1f
Add SHA-256 function
rubo 2670814
Revise execution result handling
rubo 7a3f4ec
Fix `secp256k1_ecdsa_address_recover_c` signature
rubo 72cc51c
Add `BLAKE2F` precompile function
rubo 04f051d
Revise serialization length handling
rubo 4b9fe52
Micro-optimize Zisk I/O
rubo caa49ee
Merge branch 'master' into feature/stateless-zisk
rubo 501c9a6
Replace `zkVM` with `zkEVM`
rubo 7dff8ea
Fix `Witness` API usage
rubo db5e9a1
Fix tester
rubo cda0ba4
Micro-optimize Zisk I/O
rubo 464b222
Make `StatelessExecutor` static
rubo 00df3f3
Merge branch 'master' into feature/stateless-zisk
rubo 650edc5
Improve tester
rubo fc35feb
Fix RLP API
rubo 7835da8
Add testing workflow
rubo 0395776
Remove redundant namespace
rubo 04a2412
Dispose decoded headers
rubo 5b77145
Tighten serialization validation
rubo c234805
Remove redundant stuff
rubo af0d749
Dispose witness
rubo 9f06eaa
Merge branch 'master' into feature/stateless-zisk
rubo 1235eef
Hardcode `nethermindeth/bflat-riscv64` image
rubo be19b0c
Use a NuGet package for Zisk bindings
rubo 8e06802
Remove keys from serialization
rubo 2c99f6d
Fix Zisk testing
rubo e162eb3
Merge branch 'master' into feature/stateless-zisk
rubo 04f0aa4
Remove `StatelessExecution` project
rubo 939ad50
Harden Zisk Dockerfile
rubo e0c6249
Switch to the latest `nethermindeth/bflat-riscv64`
rubo d50c8a3
Implement address recovery
rubo 8361a48
Fix formatting
rubo b6b234f
Add block validation
rubo 3f0cc6b
Merge branch 'master' into feature/stateless-zisk
rubo 16e6ea2
Cancel in-progress test
rubo 316dd42
Remove tx address null check
rubo f05898a
Refactor test workflow
rubo 6ea8960
Add dependencies to solution
rubo 53cd7e9
Reorder usings
rubo 832e48b
Merge branch 'master' into feature/stateless-zisk
rubo bf5af0b
Update Nethermind.ZiskBindings package
rubo d2e2bd2
Remove `Dockerfile.bflat`
rubo 5caee24
Use modern libziskos and implement canonical way to reference precomp…
maximmenshikov e73fc3b
Update `Nethermind.ZiskBindings` package
rubo acd2220
Implement `ECREC` precompile and refactor address recovery
rubo 1fcd952
Implement `BN254` precompiles
rubo 2d3ec98
Merge branch 'master' into feature/stateless-zisk
rubo 31f0a0f
Refactor `EthereumEcdsa` for zkEVM
rubo 320297d
Remove links to `Nethermind.Facade`
rubo a44a374
Improve conditional compilation handling
rubo 0feb4ab
Restore file name
rubo 2eab942
Merge branch 'master' into feature/stateless-zisk
rubo ad98c10
Implement `BLS12_G1ADD` precompile
rubo 8946c71
Switch to Zisk v0.16.0
rubo 37d7a94
Implement `BLS12_G1MSM` precompile
rubo 6a4a2c2
Unify function name
rubo a8c4fbb
Shut cspell up 🤮
rubo a9cf0a6
Merge branch 'master' into feature/stateless-zisk
rubo 10d8b05
Use `ulong` for chain id
rubo 5e99852
Refactor precompiles
rubo 2cab0ae
Merge branch 'master' into feature/stateless-zisk
rubo 9de75da
Revert `TransactionProcessor`
rubo bda1ba6
Implement `BLS12_G2ADD` precompile
rubo 2b6cc7f
Implement `BLS12_G2MSM` precompile
rubo a3c538b
Implement `BLS12_MAP_FP_TO_G1` precompile
rubo 6992f0f
Implement `BLS12_MAP_FP2_TO_G2` precompile
rubo 0461c5c
Update README
rubo c927e91
Merge branch 'master' into feature/stateless-zisk
rubo 060ae92
Implement `BLS12_PAIRING_CHECK` precompile
rubo a9bc3b1
Refactor EIP-2537 precompiles
rubo f30aed1
Merge branch 'master' into feature/stateless-zisk
rubo 6d95550
Update Nethermind.ZiskBindings package
rubo 0fc1fe3
Merge `Tester` project into `ZiskGuest`
rubo 4eee306
Move the project to be alongside the main codebase
rubo 2203263
Merge branch 'master' into feature/stateless-zisk
rubo f7d15b0
Restore `ECRecoverPrecompile` file name
rubo 58ddd96
Implement `MODEXP` precompile
rubo 2b76fb5
Merge branch 'master' into feature/stateless-zisk
rubo 006cb3a
Implement `KZG_POINT_EVALUATION` precompile
rubo a502c86
Bring back `KzgPolynomialCommitments.BlsModulus`
rubo c9a49b9
Implement `P256VERIFY` precompile
rubo 4ecf70c
Implement `SHA256` precompile
rubo File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,6 @@ | ||
| # SPDX-FileCopyrightText: 2026 Demerzel Solutions Limited | ||
| # SPDX-License-Identifier: LGPL-3.0-only | ||
|
|
||
| FROM nethermindeth/bflat-riscv64:latest | ||
|
|
||
| RUN apt-get update && apt-get install -y --no-install-recommends strace gdb file |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,20 @@ | ||
| <Project Sdk="Microsoft.NET.Sdk"> | ||
|
|
||
| <PropertyGroup> | ||
| <AssemblyName>Nethermind.Stateless.Executor</AssemblyName> | ||
| <RootNamespace>Nethermind.Stateless.Execution</RootNamespace> | ||
| </PropertyGroup> | ||
|
|
||
| <ItemGroup> | ||
| <ProjectReference Include="..\..\..\src\Nethermind\Nethermind.Core\Nethermind.Core.csproj" /> | ||
| <ProjectReference Include="..\..\..\src\Nethermind\Nethermind.Facade\Nethermind.Facade.csproj" /> | ||
| <ProjectReference Include="..\..\..\src\Nethermind\Nethermind.Db\Nethermind.Db.csproj" /> | ||
| <ProjectReference Include="..\..\..\src\Nethermind\Nethermind.Trie\Nethermind.Trie.csproj" /> | ||
| <ProjectReference Include="..\..\..\src\Nethermind\Nethermind.Evm\Nethermind.Evm.csproj" /> | ||
| <ProjectReference Include="..\..\..\src\Nethermind\Nethermind.Specs\Nethermind.Specs.csproj" /> | ||
| <ProjectReference Include="..\..\..\src\Nethermind\Nethermind.Consensus\Nethermind.Consensus.csproj" /> | ||
| <ProjectReference Include="..\..\..\src\Nethermind\Nethermind.Logging.NLog\Nethermind.Logging.NLog.csproj" /> | ||
| <ProjectReference Include="..\..\..\src\Nethermind\Nethermind.Serialization.Rlp\Nethermind.Serialization.Rlp.csproj" /> | ||
| </ItemGroup> | ||
|
|
||
| </Project> |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,182 @@ | ||
| // SPDX-FileCopyrightText: 2026 Demerzel Solutions Limited | ||
| // SPDX-License-Identifier: LGPL-3.0-only | ||
|
|
||
| using Nethermind.Consensus.Stateless; | ||
| using Nethermind.Core; | ||
| using Nethermind.Core.Collections; | ||
| using Nethermind.Serialization.Rlp; | ||
| using System.Buffers.Binary; | ||
| using System.Diagnostics; | ||
| using System.Runtime.CompilerServices; | ||
|
|
||
| namespace Nethermind.Stateless.Execution; | ||
|
|
||
| public static class InputSerializer | ||
| { | ||
| public static byte[] Serialize(Block block, Witness witness, uint chainId) | ||
| { | ||
| ArgumentNullException.ThrowIfNull(block); | ||
|
|
||
| IRlpStreamDecoder<Block> blockDecoder = Rlp.GetStreamDecoder<Block>()!; // cannot be null | ||
| var blockLen = blockDecoder.GetLength(block, RlpBehaviors.None); | ||
| var codesLen = GetSerializedLength(witness.Codes); | ||
| var headersLen = GetSerializedLength(witness.Headers); | ||
| var keysLen = GetSerializedLength(witness.Keys); | ||
| var stateLen = GetSerializedLength(witness.State); | ||
| var outputLen = MinSerializedLength + | ||
| blockLen + codesLen + headersLen + keysLen + stateLen; | ||
|
|
||
| byte[] output = GC.AllocateUninitializedArray<byte>(outputLen); | ||
| var offset = 0; | ||
|
|
||
| WriteUInt32(chainId, output, ref offset); | ||
| WriteInt32(blockLen, output, ref offset); | ||
|
|
||
| RlpStream rlpStream = new(output) { Position = offset }; | ||
| blockDecoder.Encode(rlpStream, block, RlpBehaviors.None); | ||
| offset += blockLen; | ||
|
|
||
| Debug.Assert(rlpStream.Position == offset, "Unexpected block RLP length"); | ||
|
|
||
| WriteJaggedArray(witness.Codes, codesLen, output, ref offset); | ||
| WriteJaggedArray(witness.Headers, headersLen, output, ref offset); | ||
| WriteJaggedArray(witness.Keys, keysLen, output, ref offset); | ||
| WriteJaggedArray(witness.State, stateLen, output, ref offset); | ||
|
|
||
| Debug.Assert(offset == outputLen, "Invalid input length"); | ||
|
|
||
| return output; | ||
| } | ||
|
|
||
| public static (Block, Witness, uint) Deserialize(ReadOnlySpan<byte> input) | ||
| { | ||
| ArgumentOutOfRangeException.ThrowIfLessThan(input.Length, MinSerializedLength); | ||
|
|
||
| var offset = 0; | ||
| var chainId = ReadUInt32(input, ref offset); | ||
| var blockLength = ReadInt32(input, ref offset); | ||
|
|
||
| IRlpValueDecoder<Block> blockDecoder = Rlp.GetValueDecoder<Block>()!; // cannot be null | ||
| Rlp.ValueDecoderContext blockContext = new(input.Slice(offset, blockLength)); | ||
| Block block = blockDecoder.Decode(ref blockContext, RlpBehaviors.None); | ||
| blockContext.Check(blockLength); | ||
| offset += blockLength; | ||
|
|
||
| IOwnedReadOnlyList<byte[]> codes = ReadJaggedArray(input, ref offset); | ||
| IOwnedReadOnlyList<byte[]> headers = ReadJaggedArray(input, ref offset); | ||
| IOwnedReadOnlyList<byte[]> keys = ReadJaggedArray(input, ref offset); | ||
| IOwnedReadOnlyList<byte[]> state = ReadJaggedArray(input, ref offset); | ||
|
|
||
| Debug.Assert(offset == input.Length, "Invalid input length"); | ||
|
|
||
| Witness witness = new() | ||
| { | ||
| Codes = codes, | ||
| Headers = headers, | ||
| Keys = keys, | ||
| State = state | ||
| }; | ||
|
|
||
| return (block, witness, chainId); | ||
| } | ||
|
|
||
| [MethodImpl(MethodImplOptions.AggressiveInlining)] | ||
| private static int ReadInt32(ReadOnlySpan<byte> source, ref int offset) | ||
| { | ||
| var value = BinaryPrimitives.ReadInt32LittleEndian(source.Slice(offset, sizeof(int))); | ||
| offset += sizeof(int); | ||
|
|
||
| return value; | ||
| } | ||
|
|
||
| [MethodImpl(MethodImplOptions.AggressiveInlining)] | ||
| private static void WriteInt32(int value, Span<byte> destination, ref int offset) | ||
| { | ||
| BinaryPrimitives.WriteInt32LittleEndian(destination.Slice(offset, sizeof(int)), value); | ||
| offset += sizeof(int); | ||
| } | ||
|
|
||
| [MethodImpl(MethodImplOptions.AggressiveInlining)] | ||
| private static uint ReadUInt32(ReadOnlySpan<byte> source, ref int offset) | ||
| { | ||
| var value = BinaryPrimitives.ReadUInt32LittleEndian(source.Slice(offset, sizeof(uint))); | ||
| offset += sizeof(uint); | ||
|
|
||
| return value; | ||
| } | ||
|
|
||
| [MethodImpl(MethodImplOptions.AggressiveInlining)] | ||
| private static void WriteUInt32(uint value, Span<byte> destination, ref int offset) | ||
| { | ||
| BinaryPrimitives.WriteUInt32LittleEndian(destination.Slice(offset, sizeof(uint)), value); | ||
| offset += sizeof(uint); | ||
| } | ||
|
|
||
| private static ArrayPoolList<byte[]> ReadJaggedArray(ReadOnlySpan<byte> source, ref int offset) | ||
| { | ||
| var sectionLen = ReadInt32(source, ref offset); | ||
|
|
||
| if (sectionLen == 0) | ||
| return ArrayPoolList<byte[]>.Empty(); | ||
|
|
||
| var count = ReadInt32(source, ref offset); | ||
| ArrayPoolList<byte[]> output = new(count, count); | ||
|
|
||
| for (int i = 0; i < count; i++) | ||
| { | ||
| var len = ReadInt32(source, ref offset); | ||
|
|
||
| output[i] = source.Slice(offset, len).ToArray(); | ||
| offset += len; | ||
| } | ||
|
|
||
| return output; | ||
| } | ||
|
|
||
| private static void WriteJaggedArray( | ||
| IOwnedReadOnlyList<byte[]> value, | ||
| int sectionLength, | ||
| Span<byte> destination, | ||
| ref int offset) | ||
| { | ||
| WriteInt32(sectionLength, destination, ref offset); | ||
|
|
||
| if (value.Count == 0) | ||
| return; | ||
|
|
||
| var valueLen = value.Count; | ||
|
|
||
| WriteInt32(valueLen, destination, ref offset); | ||
|
|
||
| for (var i = 0; i < valueLen; i++) | ||
| { | ||
| var len = value[i].Length; | ||
|
|
||
| WriteInt32(len, destination, ref offset); | ||
|
|
||
| value[i].CopyTo(destination.Slice(offset, len)); | ||
| offset += len; | ||
| } | ||
| } | ||
|
|
||
| private static int GetSerializedLength(IOwnedReadOnlyList<byte[]> value) | ||
| { | ||
| if (value.Count == 0) | ||
| return 0; | ||
|
|
||
| var len = sizeof(int); | ||
|
|
||
| for (int i = 0; i < value.Count; i++) | ||
| len += sizeof(int) + value[i].Length; | ||
|
|
||
| return len; | ||
| } | ||
|
|
||
| private static int MinSerializedLength => | ||
| sizeof(uint) + // chain id | ||
| sizeof(int) + // block length | ||
| sizeof(int) + // codes length | ||
| sizeof(int) + // headers length | ||
| sizeof(int) + // keys length | ||
| sizeof(int); // state length | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,77 @@ | ||
| // SPDX-FileCopyrightText: 2026 Demerzel Solutions Limited | ||
| // SPDX-License-Identifier: LGPL-3.0-only | ||
|
|
||
| using Nethermind.Blockchain.Tracing; | ||
| using Nethermind.Consensus.Processing; | ||
| using Nethermind.Consensus.Stateless; | ||
| using Nethermind.Consensus.Validators; | ||
| using Nethermind.Core; | ||
| using Nethermind.Core.Specs; | ||
| using Nethermind.Crypto; | ||
| using Nethermind.Logging; | ||
| using Nethermind.Specs; | ||
|
|
||
| namespace Nethermind.Stateless.Execution; | ||
|
|
||
| public static class StatelessExecutor | ||
| { | ||
| public static bool TryExecute(ReadOnlySpan<byte> data, out Block? processedBlock) | ||
| { | ||
| (Block suggestedBlock, Witness witness, uint chainId) = InputSerializer.Deserialize(data); | ||
|
|
||
| ISpecProvider specProvider = GetSpecProvider(chainId); | ||
| //IReleaseSpec spec = specProvider.GetSpec(suggestedBlock.Header); | ||
| //EthereumEcdsa ecdsa = new(chainId); | ||
|
|
||
| //foreach (Transaction tx in suggestedBlock.Transactions) | ||
| // tx.SenderAddress ??= ecdsa.RecoverAddress(tx, !spec.ValidateChainId); | ||
|
|
||
| return TryExecute(suggestedBlock, witness, specProvider, out processedBlock); | ||
| } | ||
|
|
||
| public static bool TryExecute( | ||
| Block suggestedBlock, Witness witness, ISpecProvider specProvider, out Block? processedBlock) | ||
| { | ||
| processedBlock = null; | ||
| BlockHeader? baseBlock = null; | ||
|
|
||
| foreach (BlockHeader header in witness.DecodeHeaders()) | ||
| { | ||
| if (header.Hash == suggestedBlock.Header.ParentHash) | ||
| baseBlock = header; // no break? | ||
rubo marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
| } | ||
|
|
||
| if (baseBlock is null) | ||
| return false; | ||
|
|
||
| StatelessBlockProcessingEnv blockProcessingEnv = new( | ||
| witness, specProvider, Always.Valid, NullLogManager.Instance); | ||
|
|
||
| using IDisposable scope = blockProcessingEnv.WorldState.BeginScope(baseBlock); | ||
|
|
||
| IBlockProcessor blockProcessor = blockProcessingEnv.BlockProcessor; | ||
|
|
||
| // TODO: Remove once the sender recovery is implemented | ||
| suggestedBlock.Transactions[0].SenderAddress ??= new("0xaa2fbe31e6d774d2e70b1375f3bc791ae487fd50"); | ||
| suggestedBlock.Transactions[1].SenderAddress ??= new("0xa4a59a31360b4ab10d28755f53697b60c796ee03"); | ||
|
|
||
rubo marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
| (processedBlock, TxReceipt[] _) = blockProcessor.ProcessOne( | ||
| suggestedBlock, | ||
| ProcessingOptions.ReadOnlyChain, | ||
| NullBlockTracer.Instance, | ||
| specProvider.GetSpec(suggestedBlock.Header)); | ||
|
|
||
| if (processedBlock.Hash != suggestedBlock.Hash) | ||
| return false; | ||
|
|
||
| return true; | ||
| } | ||
|
|
||
| private static ISpecProvider GetSpecProvider(uint chainId) => chainId switch | ||
| { | ||
| BlockchainIds.Hoodi => HoodiSpecProvider.Instance, | ||
| BlockchainIds.Mainnet => MainnetSpecProvider.Instance, | ||
| BlockchainIds.Sepolia => SepoliaSpecProvider.Instance, | ||
| _ => throw new ArgumentException($"Unsupported chain id: {chainId}") | ||
| }; | ||
| } | ||
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.