Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
32 changes: 32 additions & 0 deletions docs/website/contents/howtos/benchmarks.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
# Consensus benchmarks

We are in the process of adding component level microbenchmarks for Consensus.

We check for regressions in performance on CI.

## Mempool Benchmark

We started with microbenchmarks for adding transactions to the mempool. The
mempool benchmarks can be run using the following command.

```sh
cabal new-run ouroboros-consensus:mempool-bench
```

## ChainSync Client Benchmark

To aid the refactoring of the ChainSync client, we added a benchmark for it in [PR#823](https://github.com/IntersectMBO/ouroboros-consensus/pull/823). The benchmark could be invoked as follows:

```sh
cabal new-run ouroboros-consensus:ChainSync-client-bench -- 10 10
```

## PerasCertDB Benchmark

We have a microbenchmark for the boosted chain fragment weight calculation, which could be run as follows:

```sh
cabal run ouroboros-consensus:PerasCertDB-bench -- +RTS -T -A32m -RTS
```

We request GHC runtime system statistics with `-T` to get a memory usage estimate, and also request a large nursery with `-A32m` to minimise garbage collection. See `tasty-bench` [documentation](https://github.com/Bodigrim/tasty-bench?tab=readme-ov-file#troubleshooting) for more tips.
13 changes: 13 additions & 0 deletions docs/website/contents/references/glossary.md
Original file line number Diff line number Diff line change
Expand Up @@ -472,6 +472,19 @@ These kinds are maintained by the Networking layer:
- [Public root peers](#public-root-peers).
- [Shared peers](#shared-peers).

## ;Peras ;weight ;boost

Peras is an extension of Praos enabling faster settlement under optimistic conditions.
To this end, Peras can result in a block `B` receiving a *boost*, which means that any chain containing `B` gets additional weight when being compared to other chains.

Consider a chain fragment `F`:

- Its ;*weight boost* is the sum of all boosts received by points on this fragment (excluding the anchor). Note that the same point can be boosted multiple times.

- Its ;*total weight* is its tip block number plus its weight boost.

Note that these notions are always relative to a particular anchor, so different chain fragments must have the same anchor when their total weight is to be compared.

## ;Phases

Byron, Shelley, Goguen (current one as of August 2023), Basho, Voltaire.
Expand Down
118 changes: 118 additions & 0 deletions ouroboros-consensus/bench/PerasCertDB-bench/Main.hs
Original file line number Diff line number Diff line change
@@ -0,0 +1,118 @@
{-# LANGUAGE ImportQualifiedPost #-}
{-# LANGUAGE LambdaCase #-}

-- | This module contains benchmarks for Peras chain weight calculation as
-- implemented by the by the
-- 'Ouroboros.Consensus.Peras.Weight.weightBoostOfFragment' function.
--
-- We benchmark the calculation on a static sequence of chain fragments of
-- increasing length, ranging from 0 to 'fragmentMaxLength', with a step size
-- of 'fragmentLengthStepSize'. The chain fragments are instantiated with
-- 'TestBlock', and every 'boostedBlockGap' blocks there is a booster block
-- with weight 'boostWeight'. All parameters are set in 'benchmarkParams'.
module Main (main) where

import Data.List (iterate')
import Data.Word (Word64)
import Numeric.Natural (Natural)
import Ouroboros.Consensus.Block (PerasWeight (PerasWeight), SlotNo (..))
import Ouroboros.Consensus.Peras.Weight
( PerasWeightSnapshot
, mkPerasWeightSnapshot
, weightBoostOfFragment
)
import Ouroboros.Network.AnchoredFragment qualified as AF
import Test.Ouroboros.Storage.TestBlock (TestBlock (..), TestBody (..), TestHeader (..))
import Test.Ouroboros.Storage.TestBlock qualified as TestBlock
import Test.Tasty.Bench

data BenchmarkParams = BenchmarkParams
{ slotGap :: Word64
-- ^ The slot gap between blocks on the fragments, ie the inverse of the
-- active slot coefficient. Measured in slots.
, fragmentLengthStepSize :: Natural
-- ^ Step size for the fragment lengths between different benchmarks, in
-- blocks.
, fragmentMaxLength :: Natural
-- ^ The maximum length of a fragment, in blocks.
, boostedBlockGap :: Natural
-- ^ How often boosted blocks occur, in blocks.
, boostWeight :: PerasWeight
-- ^ The weight of the boost.
}

benchmarkParams :: BenchmarkParams
benchmarkParams =
BenchmarkParams
{ -- On Cardano mainnet, the active slot coefficient f=1/20, so there are 20
-- slots between blocks on average assuming nominal chain density.
slotGap = 20
, -- Represents a decent balance between the number of benchmarks we run and
-- the granularity at which we can observe results.
fragmentLengthStepSize = 100
, -- This is the maximum size of header fragments while syncing (the current
-- selection (k) plus one forecast window under nominal chain density
-- (3k), where k=2160 on Cardano mainnet).
fragmentMaxLength = 2160 + 3 * 2160
, -- A plausible value for the Peras round length is 90 slots, which means
-- that we expect to see 4-5 blocks per Peras round (and therefore between
-- boosted blocks) on mainnet where the active slot coefficient f=1/20.
boostedBlockGap = 5
, -- This is a plausible mainnet value (the exact value does not impact the
-- benchmark).
boostWeight = PerasWeight 15
}

main :: IO ()
main =
Test.Tasty.Bench.defaultMain $ map benchWeightBoostOfFragment inputs
where
-- NOTE: we do not use the 'env' combinator to set up the test data since
-- it requires 'NFData' for 'AF.AnchoredFragment'. While the necessary
-- instances could be provided, we do not think is necessary for this
-- benchmark, as the input data is rather small.
inputs :: [(Natural, (PerasWeightSnapshot TestBlock, AF.AnchoredFragment TestBlock))]
inputs =
getEveryN (fragmentLengthStepSize benchmarkParams) $
take (fromIntegral $ fragmentMaxLength benchmarkParams) $
zip [0 ..] $
zip (map uniformWeightSnapshot fragments) fragments

benchWeightBoostOfFragment ::
(Natural, (PerasWeightSnapshot TestBlock, AF.AnchoredFragment TestBlock)) -> Benchmark
benchWeightBoostOfFragment (i, (weightSnapshot, fragment)) =
bench ("weightBoostOfFragment of length " <> show i) $
whnf (weightBoostOfFragment weightSnapshot) fragment

-- | An infinite list of chain fragments
fragments :: [AF.AnchoredFragment TestBlock]
fragments = iterate' addSuccessorBlock genesisFragment
where
genesisFragment :: AF.AnchoredFragment TestBlock
genesisFragment = AF.Empty AF.AnchorGenesis

addSuccessorBlock :: AF.AnchoredFragment TestBlock -> AF.AnchoredFragment TestBlock
addSuccessorBlock = \case
AF.Empty _ -> (AF.Empty AF.AnchorGenesis) AF.:> (TestBlock.firstBlock 0 dummyBody)
(xs AF.:> x) ->
let nextBlockSlot = SlotNo (slotGap benchmarkParams) + thSlotNo (testHeader x)
in (xs AF.:> x) AF.:> TestBlock.mkNextBlock x nextBlockSlot dummyBody

dummyBody :: TestBody
dummyBody = TestBody{tbForkNo = 0, tbIsValid = True}

-- | Given a chain fragment, construct a weight snapshot where there's a boosted block every 90 slots
uniformWeightSnapshot :: AF.AnchoredFragment TestBlock -> PerasWeightSnapshot TestBlock
uniformWeightSnapshot fragment =
let pointsToBoost =
map snd
. getEveryN (boostedBlockGap benchmarkParams)
. zip [0 ..]
. map AF.blockPoint
. AF.toOldestFirst
$ fragment
weights = repeat (boostWeight benchmarkParams)
in mkPerasWeightSnapshot $ pointsToBoost `zip` weights

getEveryN :: Natural -> [(Natural, a)] -> [(Natural, a)]
getEveryN n = filter (\(i, _) -> (i `mod` n) == 0)
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
<!--
### Patch

- A bullet item for the Patch category.

-->
<!--
### Non-Breaking

- A bullet item for the Non-Breaking category.

-->
### Breaking

- Introduce `Ouroboros.Consensus.Block.SupportsPeras` with types related to Peras.
- All new types are re-exported through `Ouroboros.Consensus.Block`.
- Introduce `Ouroboros.Consensus.Peras.Weight` with weight computation related types and functions for chains and fragments.
- Introduce a new benchmark suite `PerasCertDB-bench`
- Add property tests and benchmarks for weight computation on chain and fragments
16 changes: 16 additions & 0 deletions ouroboros-consensus/ouroboros-consensus.cabal
Original file line number Diff line number Diff line change
Expand Up @@ -83,6 +83,7 @@ library
Ouroboros.Consensus.Block.RealPoint
Ouroboros.Consensus.Block.SupportsDiffusionPipelining
Ouroboros.Consensus.Block.SupportsMetrics
Ouroboros.Consensus.Block.SupportsPeras
Ouroboros.Consensus.Block.SupportsProtocol
Ouroboros.Consensus.Block.SupportsSanityCheck
Ouroboros.Consensus.BlockchainTime
Expand Down Expand Up @@ -197,6 +198,7 @@ library
Ouroboros.Consensus.Node.Run
Ouroboros.Consensus.Node.Serialisation
Ouroboros.Consensus.NodeId
Ouroboros.Consensus.Peras.Weight
Ouroboros.Consensus.Protocol.Abstract
Ouroboros.Consensus.Protocol.BFT
Ouroboros.Consensus.Protocol.LeaderSchedule
Expand Down Expand Up @@ -596,6 +598,7 @@ test-suite consensus-test
Test.Consensus.MiniProtocol.ChainSync.CSJ
Test.Consensus.MiniProtocol.ChainSync.Client
Test.Consensus.MiniProtocol.LocalStateQuery.Server
Test.Consensus.Peras.WeightSnapshot
Test.Consensus.Util.MonadSTM.NormalForm
Test.Consensus.Util.Versioned

Expand Down Expand Up @@ -827,6 +830,19 @@ benchmark ChainSync-client-bench
unstable-consensus-testlib,
with-utf8,

benchmark PerasCertDB-bench
import: common-bench
type: exitcode-stdio-1.0
hs-source-dirs: bench/PerasCertDB-bench
main-is: Main.hs
other-modules:
build-depends:
base,
ouroboros-consensus,
ouroboros-network-api,
tasty-bench,
unstable-consensus-testlib,

test-suite doctest
import: common-test
main-is: doctest.hs
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -8,5 +8,6 @@ import Ouroboros.Consensus.Block.NestedContent as X
import Ouroboros.Consensus.Block.RealPoint as X
import Ouroboros.Consensus.Block.SupportsDiffusionPipelining as X
import Ouroboros.Consensus.Block.SupportsMetrics as X
import Ouroboros.Consensus.Block.SupportsPeras as X
import Ouroboros.Consensus.Block.SupportsProtocol as X
import Ouroboros.Consensus.Block.SupportsSanityCheck as X
Loading
Loading