Skip to content
This repository was archived by the owner on Jul 30, 2025. It is now read-only.

Commit 3c87c1c

Browse files
committed
producer_worker: revert use of ValueGenerator::Generate()
Using the modified `Generate()` function was a lot slower than just simply making an empty payload: ``` Benchmark_random_payload10-32 2445903 485.2 ns/op Benchmark_random_payload100-32 678870 1773 ns/op Benchmark_random_payload1000-32 105667 13700 ns/op Benchmark_random_payload1e4-32 9104 126782 ns/op Benchmark_random_payload1e5-32 1135 1151425 ns/op Benchmark_random_payload1e6-32 100 13046547 ns/op ========================================================== Benchmark_empty_payload10-32 79950942 20.54 ns/op Benchmark_empty_payload100-32 24089101 54.78 ns/op Benchmark_empty_payload1000-32 3789248 366.1 ns/op Benchmark_empty_payload1e4-32 670096 2059 ns/op Benchmark_empty_payload1e5-32 58912 23021 ns/op Benchmark_empty_payload1e6-32 5600 290045 ns/op ``` As can be seen, generating the random payload and ensuring it is UTF-8 valid is orders of magnitude slower than the empty payload. Revert the use of `Generate()` here in place of the empty payload. However, if the user has indicated they want to validate the latest key-value pair produced, generate a `(value-{%018d}, offset)` message in the record. This does mean that message sizes that are less than 24 bytes are not honored if the `validate-latest-values` flag is passed.
1 parent bffac1f commit 3c87c1c

File tree

1 file changed

+17
-2
lines changed

1 file changed

+17
-2
lines changed

pkg/worker/verifier/producer_worker.go

Lines changed: 17 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -121,10 +121,25 @@ func (pw *ProducerWorker) newRecord(producerId int, sequence int64) *kgo.Record
121121
pw.Status.AbortedTransactionMessages += 1
122122
}
123123

124-
payload := pw.config.valueGenerator.Generate()
125-
if payload == nil {
124+
var payload []byte
125+
isTombstone := rand.Float64() < pw.config.valueGenerator.TombstoneProbability
126+
if isTombstone {
127+
payload = nil
126128
pw.Status.TombstonesProduced += 1
129+
} else {
130+
if pw.validateLatestValues {
131+
var value bytes.Buffer
132+
fmt.Fprintf(&value, "value-%018d", sequence)
133+
paddingSize := pw.config.messageSize - value.Len()
134+
if paddingSize > 0 {
135+
value.Write(make([]byte, paddingSize))
136+
}
137+
payload = value.Bytes()
138+
} else {
139+
payload = make([]byte, pw.config.messageSize)
140+
}
127141
}
142+
128143
var r *kgo.Record
129144

130145
if pw.config.keySetCardinality < 0 {

0 commit comments

Comments
 (0)