|
| 1 | +--- |
| 2 | +sidebar_position: 4 |
| 3 | +description: Learn about parallel transaction execution. |
| 4 | +tags: |
| 5 | +- public networks |
| 6 | +--- |
| 7 | + |
| 8 | +# Parallel transaction execution |
| 9 | + |
| 10 | +Besu supports parallel transaction execution, using an optimistic approach to parallelize |
| 11 | +transactions within a block. |
| 12 | +You can enable this feature when using the [Bonsai Tries](data-storage-formats.md#bonsai-tries) data |
| 13 | +storage format. |
| 14 | +This page provides an [overview of the parallelization mechanism](#parallelization-mechanism-overview), |
| 15 | +and [metrics](#metrics) that highlight Besu's improved performance. |
| 16 | + |
| 17 | +:::warning Important |
| 18 | +Parallel transaction execution is an early access feature. |
| 19 | +You can enable it using the `--Xbonsai-parallel-tx-processing-enabled` option. |
| 20 | +::: |
| 21 | + |
| 22 | +## Parallelization mechanism overview |
| 23 | + |
| 24 | +When parallel transaction execution is enabled, Besu initially executes all transactions within a |
| 25 | +block in parallel, operating under the optimistic assumption that they can all be executed |
| 26 | +concurrently without conflict. |
| 27 | +This parallel execution runs in the background, and Besu proceeds to sequentially process the |
| 28 | +transactions without waiting for the parallel execution to complete. |
| 29 | + |
| 30 | +The following flowchart outlines the transaction execution flow: |
| 31 | + |
| 32 | +<p align="center"> |
| 33 | + |
| 34 | +```mermaid |
| 35 | +graph TD; |
| 36 | + X(Start parallel execution as background process) --> A(Start sequential processing); |
| 37 | + A --> B{{Is transaction completed by background process?}}; |
| 38 | + B --> |Yes| C{{Conflict check}}; |
| 39 | + C --> |No conflict| D(Apply background state modifications); |
| 40 | + C --> |Conflict detected| E(Replay transaction using background cache); |
| 41 | + B --> |No| F(Execute transaction sequentially); |
| 42 | + D --> G(End sequential processing); |
| 43 | + E --> G; |
| 44 | + F --> G; |
| 45 | +``` |
| 46 | + |
| 47 | +</p> |
| 48 | + |
| 49 | +Besu first determines if a transaction has been completed by the background parallel execution: |
| 50 | + |
| 51 | +- **Completed:** If the transaction is completed, Besu examines whether there are any conflicts with |
| 52 | + previously executed transactions. |
| 53 | + - **No conflict:** If no conflict is detected, Besu directly applies the state modifications |
| 54 | + generated in the background to the block, avoiding re-execution. |
| 55 | + - **Conflict detected:** If a conflict is detected, Besu replays the transaction, using a cache of |
| 56 | + background reads to improve efficiency. |
| 57 | +- **Not completed:** If the transaction is not completed, Besu executes it sequentially within the |
| 58 | + block to ensure its completion, independent of the background execution. |
| 59 | + |
| 60 | +### Conflict detection strategy |
| 61 | + |
| 62 | +Besu's conflict detection strategy uses the *accumulator*, a |
| 63 | +[Bonsai Tries](data-storage-formats.md#bonsai-tries) feature that tracks addresses and slots touched |
| 64 | +or modified during block or transaction execution. |
| 65 | + |
| 66 | +:::tip |
| 67 | +You can read more about Bonsai Tries in [Consensys' Guide to Bonsai Tries](https://consensys.io/blog/bonsai-tries-guide). |
| 68 | +::: |
| 69 | + |
| 70 | +If a slot, code, or anything else related to an account is modified, the Bonsai accumulator keeps |
| 71 | +track of this information. |
| 72 | +This strategy leverages Bonsai's storage benefits, only keeping track of block-to-block state diffs |
| 73 | +in Besu storage. |
| 74 | + |
| 75 | +The following flowchart outlines how Besu detects conflicts and imports transactions into the block: |
| 76 | + |
| 77 | +<p align="center"> |
| 78 | + |
| 79 | +```mermaid |
| 80 | +graph TD; |
| 81 | + A(Start block import) --> B(Fetch block's touched addresses); |
| 82 | + B --> C{{For each transaction}}; |
| 83 | + C -->|Next transaction| D(Fetch transaction's touched addresses); |
| 84 | + D --> E{{Compare addresses}}; |
| 85 | + E -->|Conflict detected| F(Replay transaction using cached data); |
| 86 | + E -->|No conflict| G(Apply transaction result directly – no replay); |
| 87 | + F --> H{{Attempt to read from cache}}; |
| 88 | + H -->|Data found in cache| I(Continue replay using cached data); |
| 89 | + H -->|Data not found in cache| J(Fetch data from disk); |
| 90 | + I --> K(Transaction replay complete); |
| 91 | + J --> K; |
| 92 | + K --> L(Apply transaction changes); |
| 93 | + G --> L; |
| 94 | + L --> M{{More transactions?}}; |
| 95 | + M -->|Yes| C; |
| 96 | + M -->|No| N(End block import); |
| 97 | +``` |
| 98 | + |
| 99 | +</p> |
| 100 | + |
| 101 | +Besu takes what the accumulator tracks at the block and transaction level, compares the |
| 102 | +transaction's list of touched addresses to the block's list, and checks for conflicts. |
| 103 | +In particular: |
| 104 | + |
| 105 | +1. Besu identifies conflicts by checking whether a transaction has interacted with accounts modified |
| 106 | + by the block (that is, modified by previous transactions). |
| 107 | +2. If a conflict is detected, Besu replays the transaction using cached data or data fetched from disk. |
| 108 | +3. Each time a transaction is added to the block, Besu incorporates the transaction's tracked list |
| 109 | + into the block's list. |
| 110 | + |
| 111 | +:::info Note |
| 112 | +The following are excluded from the conflict check: |
| 113 | + |
| 114 | +- Unchanged accounts read by the block. |
| 115 | +- Rewards given to the validator coinbase address at the end of each transaction. |
| 116 | + If these were considered, every transaction would conflict with the coinbase address. |
| 117 | + Besu identifies this address as a conflict only if it is accessed for reasons other than receiving |
| 118 | + rewards at the transaction's conclusion. |
| 119 | +::: |
| 120 | + |
| 121 | +The following flowchart outlines how Besu maintains the lists of tracked addresses: |
| 122 | + |
| 123 | +<p align="center"> |
| 124 | + |
| 125 | +```mermaid |
| 126 | +graph TD; |
| 127 | + A(Start) --> B(Fetch block's touched addresses); |
| 128 | + B --> C{{Check each address}}; |
| 129 | + C -->|Unchanged| D(Mark as read); |
| 130 | + C -->|Modified| E(Add to block's tracked addresses); |
| 131 | + D --> F{{Next address}}; |
| 132 | + E --> F; |
| 133 | + F -->|More addresses?| C; |
| 134 | + F -->|No more| G(Fetch transaction's touched addresses); |
| 135 | + G --> H{{For each transaction address}}; |
| 136 | + H -->|From, sender, etc.| I(Add to transaction's tracked addresses); |
| 137 | + I --> J{{Next address}}; |
| 138 | + J -->|More addresses?| H; |
| 139 | + J -->|No more| K{{Compare block and transaction addresses}}; |
| 140 | + K -->|Conflict detected| L(Conflict is detected); |
| 141 | + K -->|No conflict| M(Proceed with transaction); |
| 142 | + L --> N(End); |
| 143 | + M --> N; |
| 144 | +``` |
| 145 | + |
| 146 | +</p> |
| 147 | + |
| 148 | +Besu's conflict detection strategy is intentionally simple to minimize edge cases. |
| 149 | +With this approach to parallel transaction execution, |
| 150 | +[approximately 40% of transactions do not require replay](#metrics). |
| 151 | +In the future, the conflict detection strategy may be refined to reduce false positives. |
| 152 | + |
| 153 | +You can enable parallel transaction execution using the `--Xbonsai-parallel-tx-processing-enabled` option. |
| 154 | + |
| 155 | +## Metrics |
| 156 | + |
| 157 | +Parallel transaction execution uses Besu's resources more efficiently than traditional |
| 158 | +sequential execution, significantly improving performance. |
| 159 | + |
| 160 | +The following metrics were collected on nodes running on Azure VMs (Standard D8as v5 – 8 vCPUs, 32 |
| 161 | +GiB memory), with Teku and Nimbus as consensus layer (CL) clients: |
| 162 | + |
| 163 | +- **Block processing time** - With Teku as CL client, block processing time improves by at least 25%. |
| 164 | + The 50th percentile decreases from 282 ms to 207 ms and the 95th |
| 165 | + percentile decreases from 479 ms to 393 ms. |
| 166 | + |
| 167 | + With Nimbus as CL client, block processing improves by approximately 45%, with the 50th percentile |
| 168 | + at 155 ms, and the 95th percentile at 299 ms. |
| 169 | + Besu running with Nimbus has better performance than with Teku because Nimbus has less overhead on |
| 170 | + Besu, meaning less context switching and fewer cache misses. |
| 171 | + |
| 172 | +- **Execution throughput** - Execution throughput increases, with an average of 96 Mgas/s and peaks |
| 173 | + of up to 250 Mgas/s. |
| 174 | + |
| 175 | +- **Parallel transactions** - Parallel transaction execution introduces two new metrics, which |
| 176 | + indicate that approximately 40% of transactions are parallelized using this feature: |
| 177 | + |
| 178 | + - `besu_block_processing_parallelized_transactions_counter_total` - The number of transactions |
| 179 | + executed in parallel. |
| 180 | + - `besu_block_processing_conflicted_transactions_counter_total` - The number of transactions that |
| 181 | + encountered conflicts and were therefore executed sequentially. |
| 182 | + |
| 183 | +- **Sync time** - Snap synchronization time is approximately 27 hours and 5 minutes, with block import |
| 184 | + time approximately 6 ms on average. |
| 185 | + |
| 186 | +- **CPU profiling** - The new payload call time decreases from 251.68 ms to 172.04 ms on average, |
| 187 | + with notable improvements in SLOAD operation times. |
| 188 | + |
| 189 | +During the faster block processing time, Besu uses more CPU and more disk accesses in parallel |
| 190 | +(higher IOPS). |
| 191 | +However, when these metrics are averaged on different monitoring tools, the resource usage looks the same as |
| 192 | +with sequential execution. |
| 193 | +Overall, parallel transaction execution improves Besu performance with almost no resource usage |
| 194 | +overhead. |
0 commit comments