Split/make benchee_test.exs run faster #480
Description
benchee_test.exs right now takes a whopping ~15 seconds to run - so even if it's running async, it still slows us down.
There are probably individual tests where the times are set wrong and hence run too long, but also we may be at the time where it'd do well to split that file up (as the file itself isn't run in parallel):
mix test --slowest-modules 10
...
Top 10 slowest (24.3s), 99.0% of total time:
BencheeTest (15807.8ms)
[test/benchee_test.exs]
Benchee.ProfileTest (5178.1ms)
[test/benchee/profile_test.exs]
Benchee.Benchmark.RunnerTest (2998.8ms)
[test/benchee/benchmark/runner_test.exs]
Benchee.SystemTest (185.1ms)
[test/benchee/system_test.exs]
Benchee.Utility.RepeatNTest (100.6ms)
[test/benchee/utility/repeat_n_test.exs]
Benchee.Collect.MemoryTest (51.6ms)
[test/benchee/benchmark/collect/memory_test.exs]
Benchee.Utility.FileCreationIntegrationTest (3.8ms)
[test/benchee/utility/file_creation_integration_test.exs]
Benchee.Formatters.Console.MemoryTest (1.8ms)
[test/benchee/formatters/console/memory_test.exs]
Benchee.ScenarioLoaderTest (1.5ms)
[test/benchee/scenario_loader_test.exs]
Benchee.Formatters.TaggedSaveTest (1.2ms)
[test/benchee/formatters/tagged_save_test.exs]