@@ -97,10 +97,7 @@ indexSearches !arena !indexes !kopsFiles !ks !rkixs = V.generateM n $ \i -> do
9797 ! k = ks `V.unsafeIndex` kix
9898 ! pspan = Index. search k c
9999 ! size = pageSpanSize pspan
100- -- The current allocation strategy is to allocate a new pinned
101- -- byte array for each 'IOOp'. One optimisation we are planning to
102- -- do is to use a cache of re-usable buffers, in which case we
103- -- decrease the GC load. TODO: re-usable buffers.
100+ -- Acquire a reusable buffer
104101 (! off, ! buf) <- allocateFromArena arena (getNumPages size * 4096 ) 4096
105102 pure $! IOOpRead
106103 h
@@ -111,32 +108,6 @@ indexSearches !arena !indexes !kopsFiles !ks !rkixs = V.generateM n $ \i -> do
111108 where
112109 ! n = VP. length rkixs
113110
114- {-
115- Note [Batched lookups, buffer strategy and restrictions]
116- ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
117-
118- As a first implementation, we use a simple buffering strategy for batched
119- lookups: allocate fresh buffers, and GC them once they are no longer used. In
120- a later phase of development, we will look into more elaborate buffer
121- strategies, for example using a cache of page buffers, which reduces the
122- number of blocks of memory that we allocate/free.
123-
124- When we implement a reusable buffer strategy, we will have to take extra care
125- to copy bytes from raw pages when necessary. 'rawPageLookup' slices serialised
126- values from pages without copying bytes. If the serialised value survives
127- after the reusable buffer is returned to the cache, accessing that serialised
128- value's bytes has undefined behaviour __unless__ we ensure that we copy from
129- the raw page instead of or in addition to slicing.
130-
131- There are currently no alignment constraints on the buffers, but there will be
132- in the future. The plan is to optimise file access by having key\/operation
133- files opened with the @O_DIRECT@ flag, which requires buffers to be aligned to
134- the filesystem block size (typically 4096 bytes). We expect to benefit from
135- this flag in the UTxO use case, because UTxO keys are uniformly distributed
136- hashes, which means that there is little to no spatial locality when
137- performing lookups, and so we can skip looking at the page cache.
138- -}
139-
140111-- | Value resolve function: what to do when resolving two @Mupdate@s
141112type ResolveSerialisedValue = SerialisedValue -> SerialisedValue -> SerialisedValue
142113
0 commit comments