Skip to content

Commit cc9cfb5

Browse files
committed
snaprecv: actually use DataSize in EstimatedDataSize
It was previously using a mix of SST size and data size. I got worried that this might actually regress. Namely, if an SST got finished, the data previously in DataSize would now be reflected in the size of the SST, but compression could result in an overall lower number. This is not something the users of this number would have handled well (we would've passed negative numbers to the pacer, and as far as I can tell there's no check). Simply remove this kind of problem by returning the cumulative data size. As a nice side effect, we also get a taste for both the SST and data size in the datadriven tests.
1 parent 69854cb commit cc9cfb5

File tree

3 files changed

+10
-7
lines changed

3 files changed

+10
-7
lines changed

pkg/kv/kvserver/kvstorage/snaprecv/multi_sst_writer.go

Lines changed: 6 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -185,10 +185,13 @@ func (msstw *MultiSSTWriter) ReadOne(
185185
return nil
186186
}
187187

188-
// EstimatedDataSize returns the approximation of the written bytes to SSTs
189-
// (including currSST).
188+
// EstimatedDataSize returns the sum of lengths of keys and values passed
189+
// to any past or current SST. This is monotonically increasing.
190+
//
191+
// Note that this is not the same as the size of the SSTs themselves, as
192+
// SST files carry additional data but also use compression.
190193
func (msstw *MultiSSTWriter) EstimatedDataSize() int64 {
191-
return msstw.sstSize + msstw.currSST.DataSize
194+
return msstw.dataSize + msstw.currSST.DataSize
192195
}
193196

194197
func (msstw *MultiSSTWriter) emitRangeKey(key rangekey.Span) {

pkg/kv/kvserver/kvstorage/snaprecv/testdata/echotest/TestMultiSSTWriterInitSST_interesting=false

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,7 @@
33
echo
44
----
55
>> finishing msstw
6-
>> sstSize=4623 estDataSize=4623
6+
>> sstSize=4623 estDataSize=126
77
>> sst0:
88
rangedel: /Local/RangeID/100/{r""-s""}
99
rangekeydel: /Local/RangeID/100/{r""-s""}

pkg/kv/kvserver/kvstorage/snaprecv/testdata/echotest/TestMultiSSTWriterInitSST_interesting=true

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -5,11 +5,11 @@ echo
55
>> rangekeyset [/Local/RangeID/100/r/AbortSpan/"00000000-0000-0000-0000-000000000000"-/Local/RangeID/100/r/AbortSpan/"6ba7b810-9dad-11d1-80b4-00c04fd430c8")
66
>> sstSize=0 estDataSize=94
77
>> rangekeyset [/Local/Range"d"/RangeDescriptor-/Local/Range"f"/RangeDescriptor)
8-
>> sstSize=1128 estDataSize=1160
8+
>> sstSize=1128 estDataSize=403
99
>> rangekeyset ["e"-"f")
10-
>> sstSize=4140 estDataSize=4140
10+
>> sstSize=4140 estDataSize=581
1111
>> finishing msstw
12-
>> sstSize=5092 estDataSize=5092
12+
>> sstSize=5092 estDataSize=585
1313
>> sst0:
1414
rangedel: /Local/RangeID/100/r{""-/AbortSpan/"00000000-0000-0000-0000-000000000000"}
1515
rangedel: /Local/RangeID/100/r/AbortSpan/"{00000000-0000-0000-0000-000000000000"-6ba7b810-9dad-11d1-80b4-00c04fd430c8"}

0 commit comments

Comments
 (0)