Skip to content

Commit d6e35dc

Browse files
nfrisbynfrisby
authored
Consolidate TxLimits (#1175)
Closes #1177. Beyond that Issue, this PR also changes the mempool's definition of full. The mempool will accepte transactions until the tx chosen to next enter the mempool would cause any component of the capacity vector (byte size, exunit steps, exunit mems, ref script byte size) to exceed its limit. The default capacity is chosen as twice the capacity of a block. Such a capacity can admit more txs than could fit into two back-to-back blocks, but no single component (eg ExUnits) of the mempool's contents' size could exceed what could fit in two blocks. The mempool capacity has two consequences. - It implements _latency hiding_: txs can cover the network a little in advance, so that two blocks on a chain whose slots are very close together could still both be full (if there are enough submitted txs). - It bounds the potential burst of CPU load necessary to fill an empty mempool, which otherwise could be a DoS vector. Also closes #1168 by superseding the "ad-hoc" mempool reference scripts capacity limit. This PR also moves the logic for choosing which prefix of the mempool to put in the block out of the forging functions and into the NodeKernel forging thread --- the mempool snapshot is the perfect place to do that, since tx measurements (each using the correct ledger state) are already determined.
2 parents f7d78f8 + 9c022f3 commit d6e35dc

File tree

47 files changed

+1368
-895
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

47 files changed

+1368
-895
lines changed

‎ouroboros-consensus-cardano/changelog.d/20240711_064942_nick.frisby_consolidate_txlimits.md‎

Lines changed: 23 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,23 @@
1+
<!--
2+
A new scriv changelog fragment.
3+
4+
Uncomment the section that is right (remove the HTML comment wrapper).
5+
-->
6+
7+
### Patch
8+
9+
- Updates for the `TxLimits` mempool consolidation.
10+
11+
### Non-Breaking
12+
13+
- Do not check transaction sizes in the forging functions; simply include all
14+
given transactions.
15+
16+
- Remove the hotfix Babbage mempool checks.
17+
18+
<!--
19+
### Breaking
20+
21+
- A bullet item for the Breaking category.
22+
23+
-->

‎ouroboros-consensus-cardano/ouroboros-consensus-cardano.cabal‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -167,6 +167,7 @@ library
167167
strict-sop-core ^>=0.1,
168168
text,
169169
these ^>=1.2,
170+
validation,
170171
vector-map,
171172

172173
-- GHC 8.10.7 on aarch64-darwin cannot use text-2

‎ouroboros-consensus-cardano/src/byron/Ouroboros/Consensus/Byron/Ledger/Forge.hs‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -48,7 +48,7 @@ forgeByronBlock ::
4848
-> BlockNo -- ^ Current block number
4949
-> SlotNo -- ^ Current slot number
5050
-> TickedLedgerState ByronBlock -- ^ Current ledger
51-
-> [Validated (GenTx ByronBlock)] -- ^ Txs to consider adding in the block
51+
-> [Validated (GenTx ByronBlock)] -- ^ Txs to include
5252
-> PBftIsLeader PBftByronCrypto -- ^ Leader proof ('IsLeader')
5353
-> ByronBlock
5454
forgeByronBlock cfg = forgeRegularBlock (configBlock cfg)
@@ -123,7 +123,7 @@ forgeRegularBlock ::
123123
-> BlockNo -- ^ Current block number
124124
-> SlotNo -- ^ Current slot number
125125
-> TickedLedgerState ByronBlock -- ^ Current ledger
126-
-> [Validated (GenTx ByronBlock)] -- ^ Txs to consider adding in the block
126+
-> [Validated (GenTx ByronBlock)] -- ^ Txs to include
127127
-> PBftIsLeader PBftByronCrypto -- ^ Leader proof ('IsLeader')
128128
-> ByronBlock
129129
forgeRegularBlock cfg bno sno st txs isLeader =
@@ -141,7 +141,7 @@ forgeRegularBlock cfg bno sno st txs isLeader =
141141
foldr
142142
extendBlockPayloads
143143
initBlockPayloads
144-
(takeLargestPrefixThatFits st txs)
144+
txs
145145

146146
txPayload :: CC.UTxO.TxPayload
147147
txPayload = CC.UTxO.mkTxPayload (bpTxs blockPayloads)

‎ouroboros-consensus-cardano/src/byron/Ouroboros/Consensus/Byron/Ledger/Mempool.hs‎

Lines changed: 32 additions & 21 deletions
Original file line numberDiff line numberDiff line change
@@ -48,13 +48,13 @@ import Cardano.Ledger.Binary (ByteSpan, DecoderError (..),
4848
byronProtVer, fromByronCBOR, serialize, slice, toByronCBOR,
4949
unsafeDeserialize)
5050
import Cardano.Ledger.Binary.Plain (enforceSize)
51-
import Cardano.Prelude (cborError)
51+
import Cardano.Prelude (Natural, cborError)
5252
import Codec.CBOR.Decoding (Decoder)
5353
import qualified Codec.CBOR.Decoding as CBOR
5454
import Codec.CBOR.Encoding (Encoding)
5555
import qualified Codec.CBOR.Encoding as CBOR
5656
import Control.Monad (void)
57-
import Control.Monad.Except (Except)
57+
import Control.Monad.Except (Except, throwError)
5858
import Data.ByteString (ByteString)
5959
import qualified Data.ByteString as Strict
6060
import qualified Data.ByteString.Lazy as Lazy
@@ -71,19 +71,9 @@ import Ouroboros.Consensus.Byron.Ledger.Serialisation
7171
(byronBlockEncodingOverhead)
7272
import Ouroboros.Consensus.Ledger.Abstract
7373
import Ouroboros.Consensus.Ledger.SupportsMempool
74-
import Ouroboros.Consensus.Mempool
7574
import Ouroboros.Consensus.Util (ShowProxy (..))
7675
import Ouroboros.Consensus.Util.Condense
7776

78-
{-------------------------------------------------------------------------------
79-
TxLimits
80-
-------------------------------------------------------------------------------}
81-
82-
instance TxLimits ByronBlock where
83-
type TxMeasure ByronBlock = ByteSize
84-
txMeasure _st = ByteSize . txInBlockSize . txForgetValidated
85-
txsBlockCapacity = ByteSize . txsMaxBytes
86-
8777
{-------------------------------------------------------------------------------
8878
Transactions
8979
-------------------------------------------------------------------------------}
@@ -132,18 +122,39 @@ instance LedgerSupportsMempool ByronBlock where
132122
where
133123
validationMode = CC.ValidationMode CC.NoBlockValidation Utxo.TxValidationNoCrypto
134124

135-
txsMaxBytes st =
136-
CC.getMaxBlockSize (tickedByronLedgerState st) - byronBlockEncodingOverhead
125+
txForgetValidated = forgetValidatedByronTx
137126

138-
txInBlockSize =
139-
fromIntegral
140-
. Strict.length
141-
. CC.mempoolPayloadRecoverBytes
142-
. toMempoolPayload
127+
instance TxLimits ByronBlock where
128+
type TxMeasure ByronBlock = IgnoringOverflow ByteSize32
143129

144-
txForgetValidated = forgetValidatedByronTx
130+
blockCapacityTxMeasure _cfg st =
131+
IgnoringOverflow
132+
$ ByteSize32
133+
$ CC.getMaxBlockSize cvs - byronBlockEncodingOverhead
134+
where
135+
cvs = tickedByronLedgerState st
145136

146-
txRefScriptSize _ _ _ = 0
137+
txMeasure _cfg st tx =
138+
if txszNat > maxTxSize then throwError err else
139+
pure $ IgnoringOverflow $ ByteSize32 $ fromIntegral txsz
140+
where
141+
maxTxSize =
142+
Update.ppMaxTxSize
143+
$ CC.adoptedProtocolParameters
144+
$ CC.cvsUpdateState
145+
$ tickedByronLedgerState st
146+
147+
txszNat = fromIntegral txsz :: Natural
148+
149+
txsz =
150+
Strict.length
151+
$ CC.mempoolPayloadRecoverBytes
152+
$ toMempoolPayload tx
153+
154+
err =
155+
CC.MempoolTxErr
156+
$ Utxo.UTxOValidationTxValidationError
157+
$ Utxo.TxValidationTxTooLarge txszNat maxTxSize
147158

148159
data instance TxId (GenTx ByronBlock)
149160
= ByronTxId !Utxo.TxId

‎ouroboros-consensus-cardano/src/ouroboros-consensus-cardano/Ouroboros/Consensus/Cardano/CanHardFork.hs‎

Lines changed: 36 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -2,8 +2,10 @@
22
{-# LANGUAGE DataKinds #-}
33
{-# LANGUAGE DeriveAnyClass #-}
44
{-# LANGUAGE DeriveGeneric #-}
5+
{-# LANGUAGE EmptyCase #-}
56
{-# LANGUAGE FlexibleContexts #-}
67
{-# LANGUAGE FlexibleInstances #-}
8+
{-# LANGUAGE LambdaCase #-}
79
{-# LANGUAGE MultiParamTypeClasses #-}
810
{-# LANGUAGE NamedFieldPuns #-}
911
{-# LANGUAGE OverloadedStrings #-}
@@ -57,7 +59,7 @@ import Data.Maybe (listToMaybe, mapMaybe)
5759
import Data.Proxy
5860
import Data.SOP.BasicFunctors
5961
import Data.SOP.InPairs (RequiringBoth (..), ignoringBoth)
60-
import Data.SOP.Strict (hpure)
62+
import qualified Data.SOP.Strict as SOP
6163
import Data.SOP.Tails (Tails (..))
6264
import qualified Data.SOP.Tails as Tails
6365
import Data.Void
@@ -78,6 +80,8 @@ import Ouroboros.Consensus.HardFork.History (Bound (boundSlot),
7880
addSlots)
7981
import Ouroboros.Consensus.HardFork.Simple
8082
import Ouroboros.Consensus.Ledger.Abstract
83+
import Ouroboros.Consensus.Ledger.SupportsMempool (ByteSize32,
84+
IgnoringOverflow, TxMeasure)
8185
import Ouroboros.Consensus.Ledger.SupportsProtocol
8286
(LedgerSupportsProtocol)
8387
import Ouroboros.Consensus.Protocol.Abstract
@@ -283,6 +287,8 @@ type CardanoHardForkConstraints c =
283287
)
284288

285289
instance CardanoHardForkConstraints c => CanHardFork (CardanoEras c) where
290+
type HardForkTxMeasure (CardanoEras c) = ConwayMeasure
291+
286292
hardForkEraTranslation = EraTranslation {
287293
translateLedgerState =
288294
PCons translateLedgerStateByronToShelleyWrapper
@@ -311,7 +317,7 @@ instance CardanoHardForkConstraints c => CanHardFork (CardanoEras c) where
311317
}
312318
hardForkChainSel =
313319
-- Byron <-> Shelley, ...
314-
TCons (hpure CompareBlockNo)
320+
TCons (SOP.hpure CompareBlockNo)
315321
-- Inter-Shelley-based
316322
$ Tails.hcpure (Proxy @(HasPraosSelectView c)) CompareSameSelectView
317323
hardForkInjectTxs =
@@ -349,6 +355,34 @@ instance CardanoHardForkConstraints c => CanHardFork (CardanoEras c) where
349355
)
350356
$ PNil
351357

358+
hardForkInjTxMeasure =
359+
fromByteSize `o`
360+
fromByteSize `o`
361+
fromByteSize `o`
362+
fromByteSize `o`
363+
fromAlonzo `o`
364+
fromConway `o`
365+
fromConway `o`
366+
nil
367+
where
368+
nil :: SOP.NS f '[] -> a
369+
nil = \case {}
370+
371+
infixr `o`
372+
o ::
373+
(TxMeasure x -> a)
374+
-> (SOP.NS WrapTxMeasure xs -> a)
375+
-> SOP.NS WrapTxMeasure (x : xs)
376+
-> a
377+
o f g = \case
378+
SOP.Z (WrapTxMeasure x) -> f x
379+
SOP.S y -> g y
380+
381+
fromByteSize :: IgnoringOverflow ByteSize32 -> ConwayMeasure
382+
fromByteSize x = fromAlonzo $ AlonzoMeasure x mempty
383+
fromAlonzo x = fromConway $ ConwayMeasure x mempty
384+
fromConway x = x
385+
352386
class (SelectView (BlockProtocol blk) ~ PraosChainSelectView c) => HasPraosSelectView c blk
353387
instance (SelectView (BlockProtocol blk) ~ PraosChainSelectView c) => HasPraosSelectView c blk
354388

0 commit comments

Comments
 (0)