Test that we never block on adding a tx to an empty mempool

[amesgen]

Since #49, adding a tx to the mempool is guarded by a lock

ouroboros-consensus/ouroboros-consensus/src/ouroboros-consensus/Ouroboros/Consensus/Mempool/Update.hs

Line 76 in 6a8def9

withMVar allFifo $ \() ->

Therefore, while a tx is waiting to be added to the mempool, no other tx can enter the mempool. In order to avoid a deadlock, it must be guaranteed that adding a tx can not block forever. Such a deadlock is problematic both for the local node (as a restart is necessary), as well as for the entire network, as only empty blocks might otherwise be minted.

This requirement can be deduced from the following two properties:

1. Transactions are removed from the mempool over time due to transactions becoming invalid (most commonly, due to being included in a minted block).
2. Add a transaction to an empty mempool never blocks.

As the mempool will eventually become empty, no transaction can block forever.

The goal of this issue is to test the second property.

---

At the moment (6a8def9), this property is true for the block size capacity due to #21: As the capacity of the mempool is positive, the conditional mempoolsize < mempoolcapacity is always true when adding a tx to an empty mempool.

See #1225 for a recent change restoring this behavior for the ad-hoc reference scripts size limit (added in #1166) after #1168.

As of #1175, this property will be guaranteed by performing the per-tx size checks as part of the check whether to add the tx to the mempool, as the per-tx limit is smaller than the per-block limit, and the mempool capacity encompasses at least one block.

---

One strategy to test this would be to enrich the TestBlock used in Test.Consensus.Mempool with a per-tx size limit, and then add a simple test which tries to add overly large txs to an empty mempool.

[amesgen]

A refinement of this issue would be to check that adding a tx to a nearly empty mempool can not block. Otherwise, minted blocks might not actually make much use of the block capacity despite high tx load. See #1225 (comment) for a concrete example.

#1175 implicitly has this property as the per-tx limits are significantly smaller than the per-block limits.

[nfrisby]

Add a transaction to an empty mempool never block.

adding a tx to a nearly empty mempool can not block

I agree both of those policies are desireable. Which plausible mechanisms are we aware of?

• PR Consolidate TxLimits #1175 enforces a per-tx limit before blocking. As you stated, that's the key ingredient, assuming the per-tx limit << the block limit.

• What other mechanisms are possible? I suppose PR Consolidate TxLimits #1175 does that check explicitly, whereas PR Mempool: reject txs that don't fit in an empty mempool #1225 does the same check slightly indirectly, assuming that the Ledger Rules for applying a tx will fail if it exceeds the per-tx limit. (It seems undesireable to simply begin by validating the tx; that could be wasteful for a valid tx that doesn't fit --- it would be validated again everytime the selection changes without making sufficient space in the mempool... which shouldn't happen more than once or so for some tx? PR Mempool: reject txs that don't fit in an empty mempool #1225 avoids that pitfall by checking the size first and only invoking validation if the size indicates the tx will be rejected.)

[amesgen]

Another mechanism is using the mempoolsize < mempoolcapacity conditional for deciding whether to try adding the tx, as is the case for the tx byte size on current main

ouroboros-consensus/ouroboros-consensus/src/ouroboros-consensus/Ouroboros/Consensus/Mempool/Update.hs

Line 187 in 6a8def9

, curSize < getMempoolCapacityBytes (isCapacity is)