Skip to content

Conversation

not-matthias
Copy link
Member

@not-matthias not-matthias commented Mar 11, 2025

With CODSPEED_MODE:

Checked: examples/google_benchmark/main.cpp::BM_takes_args::int_string_test
Checked: examples/google_benchmark/main.cpp::BM_takes_args::int_test
Checked: examples/google_benchmark/main.cpp::BM_VectorPushBack[int]
Checked: examples/google_benchmark/main.cpp::BM_VectorPushBack[std::string]
Checked: examples/google_benchmark/main.cpp::BM_Template1[int]
Checked: examples/google_benchmark/main.cpp::BM_Template2[int,double]
Checked: examples/google_benchmark/main.cpp::BM_takes_args::int_string_test
Checked: examples/google_benchmark/main.cpp::BM_takes_args[int,double]::two_type_test
Checked: examples/google_benchmark/main.cpp::BM_rand_vector
Checked: examples/google_benchmark/main.cpp::BM_StringCopy
Checked: examples/google_benchmark/main.cpp::BM_memcpy[8]
Checked: examples/google_benchmark/main.cpp::BM_memcpy[64]
Checked: examples/google_benchmark/main.cpp::BM_memcpy[512]
Checked: examples/google_benchmark/main.cpp::BM_memcpy[4096]
Checked: examples/google_benchmark/main.cpp::BM_memcpy[8192]

Without CODSPEED_MODE:

BM_takes_args/int_string_test                0.000 ns        0.000 ns   1000000000000
BM_takes_args/int_test                       0.000 ns        0.000 ns   1000000000000
BM_VectorPushBack<int>                        3.60 ns         3.59 ns    163322322
BM_VectorPushBack<std::string>                20.8 ns         20.7 ns     28019743
BM_Template1<int>                            0.217 ns        0.217 ns   3203998488
BM_Template2<int,double>                     0.111 ns        0.111 ns   6262381175
BM_takes_args<void>/int_string_test          0.000 ns        0.000 ns   1000000000000
BM_takes_args<int,double>/two_type_test      0.000 ns        0.000 ns   1000000000000
BM_rand_vector                               0.000 ns        0.000 ns   1000000000000
BM_StringCopy                                 1.52 ns         1.51 ns    462118237
BM_memcpy/8                                  0.000 ns        0.000 ns   1000000000000
BM_memcpy/64                                 0.000 ns        0.000 ns   1000000000000
BM_memcpy/512                                0.000 ns        0.000 ns   1000000000000
BM_memcpy/4096                               0.000 ns        0.000 ns   1000000000000
BM_memcpy/8192                               0.000 ns        0.000 ns   1000000000000

Copy link

linear bot commented Mar 11, 2025

Copy link

codspeed-hq bot commented Mar 11, 2025

CodSpeed Instrumentation Performance Report

Merging #5 will not alter performance

Comparing cod-657-invalid-uri-when-using-benchmark_capture (4686610) with main (c41090c)

Summary

✅ 7 untouched benchmarks
🆕 8 new benchmarks

Benchmarks breakdown

Benchmark BASE HEAD Change
🆕 BM_Capture[int_string_test] N/A 8.4 µs N/A
🆕 BM_Capture[int_test] N/A 7 µs N/A
🆕 BM_Template1[int] N/A 6.2 µs N/A
🆕 BM_Template1_Capture[int_string_test] N/A 8.3 µs N/A
🆕 BM_Template1_Capture[two_type_test, int, double] N/A 8.4 µs N/A
🆕 BM_Template2[int, double] N/A 6.3 µs N/A
🆕 BM_Template[int] N/A 9 µs N/A
🆕 BM_Template[std\:\:string] N/A 9.4 µs N/A

Copy link

codspeed-hq bot commented Mar 11, 2025

CodSpeed Walltime Performance Report

Merging #5 will not alter performance

Comparing cod-657-invalid-uri-when-using-benchmark_capture (4686610) with main (c41090c)

Summary

✅ 7 untouched benchmarks
🆕 8 new benchmarks

Benchmarks breakdown

Benchmark BASE HEAD Change
🆕 BM_Capture[int_string_test] N/A 3.8 ns N/A
🆕 BM_Capture[int_test] N/A 3.8 ns N/A
🆕 BM_Template1[int] N/A 4.5 ns N/A
🆕 BM_Template1_Capture[int_string_test] N/A 3.7 ns N/A
🆕 BM_Template1_Capture[two_type_test, int, double] N/A 3.7 ns N/A
🆕 BM_Template2[int, double] N/A 5.6 ns N/A
🆕 BM_Template[int] N/A 61.2 ns N/A
🆕 BM_Template[std\:\:string] N/A 225.8 ns N/A

Copy link
Contributor

@GuillaumeLagrange GuillaumeLagrange left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall, the rule to build URIs should be

the base name (i.e the part after the last ::) of the benchmark should be the name of either the function of the template

the arguments should be in order

  • the name of the test case
  • the template type argument
  • the function extra arguments

@not-matthias
Copy link
Member Author

not-matthias commented Mar 12, 2025

New URI:

Checked: examples/google_benchmark/template_bench.hpp::BM_Template[int]
Checked: examples/google_benchmark/template_bench.hpp::BM_Template[std::string]
Checked: examples/google_benchmark/template_bench.hpp::BM_Template1[int]
Checked: examples/google_benchmark/template_bench.hpp::BM_Template2[int, double]
Checked: examples/google_benchmark/template_bench.hpp::BM_Template1_Capture[int_string_test]
Checked: examples/google_benchmark/template_bench.hpp::BM_Template1_Capture[two_type_test, int, double]
Checked: examples/google_benchmark/main.cpp::BM_Capture[int_string_test]
Checked: examples/google_benchmark/main.cpp::BM_Capture[int_test]
Checked: examples/google_benchmark/main.cpp::BM_rand_vector
Checked: examples/google_benchmark/main.cpp::BM_StringCopy
Checked: examples/google_benchmark/main.cpp::BM_memcpy[8]
Checked: examples/google_benchmark/main.cpp::BM_memcpy[64]
Checked: examples/google_benchmark/main.cpp::BM_memcpy[512]
Checked: examples/google_benchmark/main.cpp::BM_memcpy[4096]
Checked: examples/google_benchmark/main.cpp::BM_memcpy[8192]

Original URI:

BM_Template<int>                                     2.46 ns         2.46 ns    253312128
BM_Template<std::string>                             19.4 ns         19.3 ns     28216095
BM_Template1<int>                                   0.482 ns        0.480 ns   2791500751
BM_Template2<int,double>                            0.241 ns        0.240 ns   2918141994
BM_Template1_Capture<void>/int_string_test          0.000 ns        0.000 ns   1000000000000
BM_Template1_Capture<int,double>/two_type_test      0.000 ns        0.000 ns   1000000000000
BM_Capture/int_string_test                          0.000 ns        0.000 ns   1000000000000
BM_Capture/int_test                                 0.000 ns        0.000 ns   1000000000000
BM_rand_vector                                      0.000 ns        0.000 ns   1000000000000
BM_StringCopy                                        3.19 ns         3.18 ns    220803906
BM_memcpy/8                                         0.000 ns        0.000 ns   1000000000000
BM_memcpy/64                                        0.000 ns        0.000 ns   1000000000000
BM_memcpy/512                                       0.000 ns        0.000 ns   1000000000000
BM_memcpy/4096                                      0.000 ns        0.000 ns   1000000000000
BM_memcpy/8192                                      0.000 ns        0.000 ns   1000000000000

NOTE: There also seems to be an issue in this one: Checked: examples/google_benchmark/main.cpp::BM_VectorPushBack[std::string]. The benchmark name is incorrectly resolved to string] (as seen above). The easiest fix would be using a more unique separator (e.g. ::: or ;; or :;). Or use a heuristic to detect the file ending or last /.

@GuillaumeLagrange GuillaumeLagrange force-pushed the cod-657-invalid-uri-when-using-benchmark_capture branch from 7be070f to 6a7b505 Compare March 13, 2025 10:13
@GuillaumeLagrange GuillaumeLagrange force-pushed the cod-657-invalid-uri-when-using-benchmark_capture branch from 891ae51 to 891d999 Compare March 13, 2025 10:21
Copy link
Contributor

@GuillaumeLagrange GuillaumeLagrange left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@GuillaumeLagrange GuillaumeLagrange force-pushed the cod-657-invalid-uri-when-using-benchmark_capture branch from d4a0cf2 to 4686610 Compare March 13, 2025 14:33
@GuillaumeLagrange GuillaumeLagrange merged commit 4686610 into main Mar 13, 2025
5 checks passed
@not-matthias not-matthias deleted the cod-657-invalid-uri-when-using-benchmark_capture branch March 13, 2025 14:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants