Skip to content

Conversation

@luanpotter
Copy link
Member

@luanpotter luanpotter commented Dec 29, 2024

I've noticed that the current implementation, while a commendable goal, was too delicate in the input size.

For example, the example of a bad PR ran the "bad" version of the code for 6h and then got timed out by GitHub.

Even playing locally with the max operations constant shows me that sometimes a 20% change in the number is the difference between faster than the other benchmark to "infinite" (infinite defined as more time than I had patience to wait).

This is an attempt to make it more linearly dependent on the max size, as well as calibrating the constants, so the total runtime is more manageable.

I will test this by rebasing the "bad PR example" once merged.


That all being said I am very open to suggestions on other alternatives for good benchmarks, this was my idea to try to simulate real life behaviours with concurrent modifications and such, but it has become slightly Frankenstein's monster-esque.

@github-actions
Copy link

Benchmark Results

Package ordered_set:

Benchmarks Current Branch
[luan.better-bench]
Base Branch
[main]
Diff
Iteration Benchmark 249.628 μs 248.661 μs 🔴 +0.389 %
Comprehensive Benchmark 170128.091 μs 11842.107 μs 🔴 +1336.637 %

Benchmarks provided with 💙 by Dart Benchmark Action.

@coveralls
Copy link

Pull Request Test Coverage Report for Build 12532249724

Details

  • 0 of 0 changed or added relevant lines in 0 files are covered.
  • No unchanged relevant lines lost coverage.
  • Overall coverage remained the same at 97.345%

Totals Coverage Status
Change from base Build 12470143696: 0.0%
Covered Lines: 110
Relevant Lines: 113

💛 - Coveralls

@luanpotter luanpotter marked this pull request as ready for review December 29, 2024 03:53
@luanpotter luanpotter requested a review from spydon December 29, 2024 03:53
@luanpotter luanpotter merged commit d62771e into main Dec 29, 2024
7 checks passed
@luanpotter luanpotter deleted the luan.better-bench branch December 29, 2024 15:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants