From f928427d3c397dbb9773df4c5016eae84025d979 Mon Sep 17 00:00:00 2001 From: KtorZ Date: Tue, 10 Jun 2025 21:11:14 +0200 Subject: [PATCH] leverage lazy bytestring for 'Serialise' CBOR-in-CBOR. The main use case for this type lies within the StateQuery protocol, where it can be particularly useful to obtain a plain CBOR response. This is particularly useful for large query results such as DebugNewEpochState (multiple GB on mainnet). The network library makes a great effort at trying to serialise and deserialise bytes lazily throughout; and these efforts are unfortunately destroyed by this implementation that would here evaluate the entire ByteString when decoding and encoding. So on a machine that would have both a server and client using this library, we would pay twice the cost of fully evaluating in memory the entire response, instead of leverage lazy IO as, I believe, is originally intended. Note that 'decode' and 'encode' here rely on the default implementation for lazy ByteString in Codec.Serialise, which do the right thing: encode lazy bytestrings as indefinite sequences of byte chunks; effectively preserving the laziness in both directions. --- ouroboros-network-api/src/Ouroboros/Network/Block.hs | 7 ++----- 1 file changed, 2 insertions(+), 5 deletions(-) diff --git a/ouroboros-network-api/src/Ouroboros/Network/Block.hs b/ouroboros-network-api/src/Ouroboros/Network/Block.hs index 8e3b25f5458..bcd39c9de75 100644 --- a/ouroboros-network-api/src/Ouroboros/Network/Block.hs +++ b/ouroboros-network-api/src/Ouroboros/Network/Block.hs @@ -473,16 +473,13 @@ fromSerialised dec (Serialised payload) = -- -- TODO: replace with encodeEmbeddedCBOR from cborg-0.2.4 once -- it is available, since that will be faster. --- --- TODO: Avoid converting to a strict ByteString, as that requires copying O(n) --- in case the lazy ByteString consists of more than one chunks. instance Serialise (Serialised a) where encode (Serialised bs) = mconcat [ Enc.encodeTag 24 - , Enc.encodeBytes (Lazy.toStrict bs) + encode bs ] decode = do tag <- Dec.decodeTag when (tag /= 24) $ fail "expected tag 24 (CBOR-in-CBOR)" - Serialised . Lazy.fromStrict <$> Dec.decodeBytes + Serialised <$> decode