The DevP2P spec implementation of client MemPool related messages is inconsistent

[1033309821]

We conducted additional testing on the NewPooledTransactionHashes and GetPooledTransactions messages, following the DevP2P spec's design in detail.
NewPooledTransactionHashes: The Ethereum P2P protocol specifies a soft limit of 4096 items for the NewPooledTransactionHashes message. If 5000 transaction hashes are sent, a compliant node should only request the first 4096, not all 5000. We compared the "number sent" with the "number requested" to determine whether the node correctly enforced the soft limit.

The recommended soft limit for this message is 4096 items (~150 KiB).

Client	4096 items	5000 items	8192 items	Status
geth	256 (100%)	256 ( 5%)	256 ( 3%)	⚠ Partial (limit=256)
nethermind	0 (100%)	904 ( 18%)	1078 ( 13%)	⚠ Partial (limit=1078)
reth	1380 (100%)	1380 ( 28%)	1379 ( 17%)	⚠ Partial (limit=1379)
erigon	4096 (100%)	5000 (100%)	8192 (100%)	❌ Not enforced
besu	256 (100%)	256 ( 5%)	256 ( 3%)	⚠ Partial (limit=256)

GetPooledTransactions: Geth and Reth do not implement the soft limit of 256 messages per transaction in GetPooledTransactions.
Besu and Erigon implement this soft limit.
Nethermind will not enter the mempool due to non-compliant nonce. Currently, there is no better way to test the preconditions to verify that this soft limit is met.

The recommended soft limit for GetPooledTransactions requests is 256 hashes (8 KiB).

Env
We use kurtosis's ethereum-package for testing, and the relevant configuration is as follows:

// The ethereum-package network-para.yaml
participants:
  - el_type: geth V1.16.2
    cl_type: lighthouse V7.0.1
  - el_type: nethermind V1.33.0
    cl_type: lighthouse V7.0.1
  - el_type: reth V1.6.0
    cl_type: lighthouse V7.0.1
  - el_type: besu V25.8.0
    cl_type: lighthouse V7.0.1
  - el_type: erigon V3.0.15
    cl_type: lighthouse V7.0.1
additional_services:
  - dora

[WTDuck2255]

Thx

[jackson1511185-stack]

Thank you for bringing this to our attention. Let's address the core issues you've identified with the NewPooledTransactionHashes and GetPooledTransactions messages implementation across different Ethereum clients based on the DevP2P specification.

Observations

1. NewPooledTransactionHashes Message:

   • Geth, Besu: These clients seem to be enforcing a lower soft limit of 256 items, which is far below the recommended 4096 items. This could be an intentional choice or a bug that necessitates further investigation.
   • Nethermind: Shows a variable and inconsistent limit, which warrants an examination of how the logic determines the number of hashes to request.
   • Reth: Follows a consistent pattern but adheres to a limit of 1380 items, indicating potential room for adjustment.
   • Erigon: Does not seem to enforce any limit, as it requests the entire sent amount, including when the number of hashes exceeds the recommended soft limit. This deviation could lead to performance issues and should be corrected.

2. GetPooledTransactions Message:

   • Geth and Reth: These clients do not seem to implement the recommended soft limit of 256 hashes, suggesting a need for adherence to the specification.
   • Besu and Erigon: Properly enforce the soft limit, which aligns with the specifications.
   • Nethermind: Its enforcement could not be effectively tested due to nonce compliance issues, which should ideally be resolved to ensure full testability of the mempool processing logic.

Recommendations

1. Geth and Besu:

   • Examine the rationale behind the 256-item limit for NewPooledTransactionHashes and determine if adjusting to the recommended 4096 is feasible and beneficial. Consider community feedback and performance benchmarks to make an informed decision.

2. Nethermind:

   • Investigate the mechanisms causing variability in the number of requested hashes and aim to establish a consistent soft limit that aligns with specifications.
   • Resolve the nonce compliance issue to test against the mempool soft limit under standardized conditions.

3. Reth:

   • Evaluate if the adjustment of the limit to adhere more closely to the specifications would offer improved performance without incurring significant resource costs.

4. Erigon:

   • Implement the enforcement of the recommended soft limits for both NewPooledTransactionHashes and GetPooledTransactions to align fully with the DevP2P spec and enhance network behavior predictability.

5. Testing and Verification:

   • Continue comprehensive testing with varied transaction sizes and seek insights from other network participants.
   • Engage with the client developer communities for updates on limit enforcement and spec adherence improvements in future releases.

Addressing these inconsistencies will not only improve compliance but should also enhance interoperability and overall network efficiency. If you need further details or assistance, consider reaching out to the respective client development teams or creating follow-up issues for any specific implementation concerns.