[Benchmarks] Reduce total runtime #6571
Merged
+3
−3
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Relates to #6567.
Changes
Reduce exporter benchmarks' total runtime by reducing the maximum
NumberOfBatches
value from100
to20
.While running the benchmarks for #6567, I noticed that it took 75 minutes just to get to the warm-up phase of running the OTLP exporter benchmarks for gRPC.
Reducing the value to
20
completes the the three benchmarks in ~50 minutes, so ~100 minutes to do a before and after comparison on the same machine.The results in #6567 show that the time and memory increases linearly with the number of batches, so I don't think there's any need to run 100 batches as that implies a runtime of 400s per iteration for no real benefit of improving accuracy of the benchmarks.
Merge requirement checklist
Unit tests added/updatedAppropriateCHANGELOG.md
files updated for non-trivial changesChanges in public API reviewed (if applicable)