Skip to content

Improve Timer Precision Measurement#19

Open
covertmatthew wants to merge 11 commits intomainfrom
implement_welford_algo_stdlib
Open

Improve Timer Precision Measurement#19
covertmatthew wants to merge 11 commits intomainfrom
implement_welford_algo_stdlib

Conversation

@covertmatthew
Copy link

@covertmatthew covertmatthew commented Dec 11, 2025

What changed:

  • Replaced Boost Accumulators (and all associated headers) with a lightweight, dependency-free implementation using Welford's online algorithm for numerically stable mean and variance calculation.
  • Introduced compile-time clock selection:
    • Prefer std::high_resolution_clock only if it is monotonic (is_steady == true).
    • Fallback to std::steady_clock for guaranteed monotonic behavior.
  • Updated confidence interval calculation:
    • Returns (mean_ns, 95% CI half-width in ns) as typed nanoseconds_double_t durations.
    • Uses z = 1.96 (normal approximation) with Bessel-corrected sample variance.
  • Increased default sample size from 100 → 1000 for better statistical stability.
  • All clock delta samples are accepted as valid — large values on coarse-timer VMs (e.g. 15ms on Hyper-V) are legitimate measurements of that system's clock granularity, not errors.
  • Improved output in p11perftest.cpp: prints "timer granularity (ns): mean ± value (95% confidence interval)".

Why:

  • Eliminates the Boost accumulator dependency for simpler builds and maintenance.
  • Guarantees monotonic clock behavior for reliable interval measurement across all platforms, including VMs.
  • Welford's algorithm is numerically stable and single-pass, avoiding the two-pass variance computation that Boost accumulators perform internally.
  • Correct 95% CI via z = 1.96 with unbiased (Bessel-corrected) variance, replacing the previous rough k = 2 multiplier.
  • Accepting all samples preserves compatibility with coarse-timer environments (KVM, Hyper-V) where valid granularity measurements can exceed 1ms.

Test (not bad for a VM):

[dc-user@ech-10-157-158-249 test]$ for i in {1..10}; do ./precision; sleep 1; done
timer granularity (ns): 69.514 +/- 3.91384
timer granularity (ns): 47.406 +/- 1.22714
timer granularity (ns): 48.027 +/- 1.07027
timer granularity (ns): 38.501 +/- 0.378576
timer granularity (ns): 49.323 +/- 0.501975
timer granularity (ns): 52.987 +/- 0.997222
timer granularity (ns): 40.877 +/- 0.705936
timer granularity (ns): 45.405 +/- 1.26509
timer granularity (ns): 44.501 +/- 0.635019
timer granularity (ns): 93.439 +/- 90.7611

@keldonin keldonin requested a review from Copilot December 12, 2025 01:51
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR modernizes timer precision measurement by removing the Boost Accumulators dependency and implementing Welford's online algorithm for stable statistical calculations. The changes ensure portable, monotonic timing across platforms with improved statistical rigor and better error handling.

Key changes:

  • Replaced Boost Accumulators with a custom implementation using Welford's algorithm for mean/variance calculation
  • Added compile-time clock selection to prefer high_resolution_clock only when monotonic, with fallback to steady_clock
  • Enhanced statistical accuracy by computing proper 95% confidence intervals (z=1.96) and increasing sample size from 100 to 1000

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 3 comments.

File Description
src/timeprecision.hpp Updated function signature to reflect new default sample size (1000) and documented filtering/termination behavior
src/timeprecision.cpp Replaced Boost implementation with Welford's algorithm, added clock selection logic, implemented value filtering with >1ms threshold, and added program termination for insufficient samples
src/p11perftest.cpp Enhanced output message to clarify that the interval represents a 95% confidence interval

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@keldonin
Copy link
Contributor

@covertmatthew can you check if this PR is still needed, and if so, do the necessary cleanup/rebase ?

Thank you!

@covertmatthew
Copy link
Author

@covertmatthew can you check if this PR is still needed, and if so, do the necessary cleanup/rebase ?

Thank you!

@keldonin Yes, I would still like to incorporate these updates as I believe they improve the timer statistics and guards against a non-monotonic clock that could be present when running the performance test on a virtual machine (the original reason for this PR). I'll make the necessary changes.

@sonarqubecloud
Copy link

@covertmatthew
Copy link
Author

@keldonin ready for your review. Note that I bumped the changelog version to 3.16.1, but please let me know if I should bump the minor version instead of the patch.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants