Skip to content

Conversation

@ldionne
Copy link
Member

@ldionne ldionne commented Feb 15, 2025

This patch adds benchmarks for all the remaining algorithms in [alg.modifying.operations] that we didn't already have a benchmark for.

@ldionne ldionne requested a review from a team as a code owner February 15, 2025 23:42
@llvmbot llvmbot added the libc++ libc++ C++ Standard Library. Not GNU libstdc++. Not libc++abi. label Feb 15, 2025
@llvmbot
Copy link
Member

llvmbot commented Feb 15, 2025

@llvm/pr-subscribers-libcxx

Author: Louis Dionne (ldionne)

Changes

This patch adds benchmarks for all the remaining algorithms in [alg.modifying.operations] that we didn't already have a benchmark for.


Patch is 93.48 KiB, truncated to 20.00 KiB below, full version: https://github.com/llvm/llvm-project/pull/127354.diff

28 Files Affected:

  • (modified) libcxx/include/module.modulemap (+2-1)
  • (removed) libcxx/test/benchmarks/algorithms/fill.bench.cpp (-51)
  • (added) libcxx/test/benchmarks/algorithms/modifying/fill.bench.cpp (+86)
  • (added) libcxx/test/benchmarks/algorithms/modifying/fill_n.bench.cpp (+86)
  • (added) libcxx/test/benchmarks/algorithms/modifying/generate.bench.cpp (+59)
  • (added) libcxx/test/benchmarks/algorithms/modifying/generate_n.bench.cpp (+59)
  • (added) libcxx/test/benchmarks/algorithms/modifying/remove.bench.cpp (+133)
  • (added) libcxx/test/benchmarks/algorithms/modifying/remove_copy.bench.cpp (+109)
  • (added) libcxx/test/benchmarks/algorithms/modifying/remove_copy_if.bench.cpp (+119)
  • (added) libcxx/test/benchmarks/algorithms/modifying/remove_if.bench.cpp (+143)
  • (added) libcxx/test/benchmarks/algorithms/modifying/replace.bench.cpp (+107)
  • (added) libcxx/test/benchmarks/algorithms/modifying/replace_if.bench.cpp (+117)
  • (added) libcxx/test/benchmarks/algorithms/modifying/reverse.bench.cpp (+57)
  • (added) libcxx/test/benchmarks/algorithms/modifying/reverse_copy.bench.cpp (+61)
  • (added) libcxx/test/benchmarks/algorithms/modifying/rotate.bench.cpp (+59)
  • (added) libcxx/test/benchmarks/algorithms/modifying/rotate_copy.bench.cpp (+65)
  • (added) libcxx/test/benchmarks/algorithms/modifying/sample.bench.cpp (+68)
  • (added) libcxx/test/benchmarks/algorithms/modifying/shift_left.bench.cpp (+56)
  • (added) libcxx/test/benchmarks/algorithms/modifying/shift_right.bench.cpp (+56)
  • (added) libcxx/test/benchmarks/algorithms/modifying/shuffle.bench.cpp (+56)
  • (added) libcxx/test/benchmarks/algorithms/modifying/swap_ranges.bench.cpp (+64)
  • (added) libcxx/test/benchmarks/algorithms/modifying/transform.binary.bench.cpp (+73)
  • (added) libcxx/test/benchmarks/algorithms/modifying/transform.unary.bench.cpp (+68)
  • (added) libcxx/test/benchmarks/algorithms/modifying/unique.bench.cpp (+132)
  • (added) libcxx/test/benchmarks/algorithms/modifying/unique_copy.bench.cpp (+105)
  • (added) libcxx/test/benchmarks/algorithms/modifying/unique_copy_pred.bench.cpp (+121)
  • (added) libcxx/test/benchmarks/algorithms/modifying/unique_pred.bench.cpp (+144)
  • (removed) libcxx/test/benchmarks/algorithms/reverse.bench.cpp (-48)
diff --git a/libcxx/include/module.modulemap b/libcxx/include/module.modulemap
index b0720703bd0de..6af6e96af7d75 100644
--- a/libcxx/include/module.modulemap
+++ b/libcxx/include/module.modulemap
@@ -700,6 +700,7 @@ module std [system] {
     }
     module ranges_remove {
       header "__algorithm/ranges_remove.h"
+      export std.ranges.subrange // return type
     }
     module ranges_replace_copy_if {
       header "__algorithm/ranges_replace_copy_if.h"
@@ -724,7 +725,7 @@ module std [system] {
     }
     module ranges_rotate_copy {
       header "__algorithm/ranges_rotate_copy.h"
-      export std.algorithm.in_out_result
+      export std.ranges.subrange // return type
     }
     module ranges_rotate                          { header "__algorithm/ranges_rotate.h" }
     module ranges_sample                          { header "__algorithm/ranges_sample.h" }
diff --git a/libcxx/test/benchmarks/algorithms/fill.bench.cpp b/libcxx/test/benchmarks/algorithms/fill.bench.cpp
deleted file mode 100644
index 6a48b25b7eb63..0000000000000
--- a/libcxx/test/benchmarks/algorithms/fill.bench.cpp
+++ /dev/null
@@ -1,51 +0,0 @@
-//===----------------------------------------------------------------------===//
-//
-// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
-// See https://llvm.org/LICENSE.txt for license information.
-// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
-//
-//===----------------------------------------------------------------------===//
-
-// UNSUPPORTED: c++03, c++11, c++14, c++17, c++20
-
-#include <algorithm>
-#include <benchmark/benchmark.h>
-#include <vector>
-
-static void bm_fill_n_vector_bool(benchmark::State& state) {
-  std::vector<bool> vec1(state.range());
-  for (auto _ : state) {
-    benchmark::DoNotOptimize(vec1);
-    benchmark::DoNotOptimize(std::fill_n(vec1.begin(), vec1.size(), false));
-  }
-}
-BENCHMARK(bm_fill_n_vector_bool)->DenseRange(1, 8)->Range(16, 1 << 20);
-
-static void bm_ranges_fill_n_vector_bool(benchmark::State& state) {
-  std::vector<bool> vec1(state.range());
-  for (auto _ : state) {
-    benchmark::DoNotOptimize(vec1);
-    benchmark::DoNotOptimize(std::ranges::fill_n(vec1.begin(), vec1.size(), false));
-  }
-}
-BENCHMARK(bm_ranges_fill_n_vector_bool)->DenseRange(1, 8)->Range(16, 1 << 20);
-
-static void bm_fill_vector_bool(benchmark::State& state) {
-  std::vector<bool> vec1(state.range());
-  for (auto _ : state) {
-    benchmark::DoNotOptimize(vec1);
-    std::fill(vec1.begin(), vec1.end(), false);
-  }
-}
-BENCHMARK(bm_fill_vector_bool)->DenseRange(1, 8)->Range(16, 1 << 20);
-
-static void bm_ranges_fill_vector_bool(benchmark::State& state) {
-  std::vector<bool> vec1(state.range());
-  for (auto _ : state) {
-    benchmark::DoNotOptimize(vec1);
-    benchmark::DoNotOptimize(std::ranges::fill(vec1, false));
-  }
-}
-BENCHMARK(bm_ranges_fill_vector_bool)->DenseRange(1, 8)->Range(16, 1 << 20);
-
-BENCHMARK_MAIN();
diff --git a/libcxx/test/benchmarks/algorithms/modifying/fill.bench.cpp b/libcxx/test/benchmarks/algorithms/modifying/fill.bench.cpp
new file mode 100644
index 0000000000000..64c7364be6549
--- /dev/null
+++ b/libcxx/test/benchmarks/algorithms/modifying/fill.bench.cpp
@@ -0,0 +1,86 @@
+//===----------------------------------------------------------------------===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+
+// UNSUPPORTED: c++03, c++11, c++14, c++17
+
+#include <algorithm>
+#include <cstddef>
+#include <deque>
+#include <iterator>
+#include <list>
+#include <string>
+#include <vector>
+
+#include "benchmark/benchmark.h"
+#include "../../GenerateInput.h"
+#include "test_macros.h"
+
+template <class Container, class Operation>
+void bm(std::string operation_name, Operation fill) {
+  auto bench = [fill](auto& st) {
+    std::size_t const size = st.range(0);
+    using ValueType        = typename Container::value_type;
+    ValueType x            = Generate<ValueType>::random();
+    ValueType y            = Generate<ValueType>::random();
+    Container c(size, y);
+
+    for ([[maybe_unused]] auto _ : st) {
+      fill(c.begin(), c.end(), x);
+      std::swap(x, y);
+      benchmark::DoNotOptimize(c);
+      benchmark::DoNotOptimize(x);
+      benchmark::DoNotOptimize(y);
+      benchmark::ClobberMemory();
+    }
+  };
+  benchmark::RegisterBenchmark(operation_name, bench)->Arg(32)->Arg(1024)->Arg(8192);
+}
+
+template <class Operation>
+void bm_vector_bool(std::string operation_name, Operation fill) {
+  auto bench = [fill](auto& st) {
+    std::size_t const size = st.range(0);
+    bool x                 = true;
+    bool y                 = false;
+    std::vector<bool> c(size, y);
+
+    for ([[maybe_unused]] auto _ : st) {
+      fill(c.begin(), c.end(), x);
+      std::swap(x, y);
+      benchmark::DoNotOptimize(c);
+      benchmark::DoNotOptimize(x);
+      benchmark::DoNotOptimize(y);
+      benchmark::ClobberMemory();
+    }
+  };
+  benchmark::RegisterBenchmark(operation_name, bench)->Arg(32)->Arg(1024)->Arg(8192);
+}
+
+int main(int argc, char** argv) {
+  auto std_fill    = [](auto first, auto last, auto const& value) { return std::fill(first, last, value); };
+  auto ranges_fill = [](auto first, auto last, auto const& value) { return std::ranges::fill(first, last, value); };
+
+  // std::fill
+  bm<std::vector<int>>("std::fill(vector<int>)", std_fill);
+  bm<std::deque<int>>("std::fill(deque<int>)", std_fill);
+  bm<std::list<int>>("std::fill(list<int>)", std_fill);
+  bm_vector_bool("std::fill(vector<bool>)", std_fill);
+
+  // ranges::fill
+  bm<std::vector<int>>("ranges::fill(vector<int>)", ranges_fill);
+  bm<std::deque<int>>("ranges::fill(deque<int>)", ranges_fill);
+  bm<std::list<int>>("ranges::fill(list<int>)", ranges_fill);
+#if TEST_STD_VER >= 23 // vector<bool>::iterator is not an output_iterator before C++23
+  bm_vector_bool("ranges::fill(vector<bool>)", ranges_fill);
+#endif
+
+  benchmark::Initialize(&argc, argv);
+  benchmark::RunSpecifiedBenchmarks();
+  benchmark::Shutdown();
+  return 0;
+}
diff --git a/libcxx/test/benchmarks/algorithms/modifying/fill_n.bench.cpp b/libcxx/test/benchmarks/algorithms/modifying/fill_n.bench.cpp
new file mode 100644
index 0000000000000..aaeb0982956f9
--- /dev/null
+++ b/libcxx/test/benchmarks/algorithms/modifying/fill_n.bench.cpp
@@ -0,0 +1,86 @@
+//===----------------------------------------------------------------------===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+
+// UNSUPPORTED: c++03, c++11, c++14, c++17
+
+#include <algorithm>
+#include <cstddef>
+#include <deque>
+#include <iterator>
+#include <list>
+#include <string>
+#include <vector>
+
+#include "benchmark/benchmark.h"
+#include "../../GenerateInput.h"
+#include "test_macros.h"
+
+template <class Container, class Operation>
+void bm(std::string operation_name, Operation fill_n) {
+  auto bench = [fill_n](auto& st) {
+    std::size_t const size = st.range(0);
+    using ValueType        = typename Container::value_type;
+    ValueType x            = Generate<ValueType>::random();
+    ValueType y            = Generate<ValueType>::random();
+    Container c(size, y);
+
+    for ([[maybe_unused]] auto _ : st) {
+      fill_n(c.begin(), size, x);
+      std::swap(x, y);
+      benchmark::DoNotOptimize(c);
+      benchmark::DoNotOptimize(x);
+      benchmark::DoNotOptimize(y);
+      benchmark::ClobberMemory();
+    }
+  };
+  benchmark::RegisterBenchmark(operation_name, bench)->Arg(32)->Arg(1024)->Arg(8192);
+}
+
+template <class Operation>
+void bm_vector_bool(std::string operation_name, Operation fill_n) {
+  auto bench = [fill_n](auto& st) {
+    std::size_t const size = st.range(0);
+    bool x                 = true;
+    bool y                 = false;
+    std::vector<bool> c(size, y);
+
+    for ([[maybe_unused]] auto _ : st) {
+      fill_n(c.begin(), size, x);
+      std::swap(x, y);
+      benchmark::DoNotOptimize(c);
+      benchmark::DoNotOptimize(x);
+      benchmark::DoNotOptimize(y);
+      benchmark::ClobberMemory();
+    }
+  };
+  benchmark::RegisterBenchmark(operation_name, bench)->Arg(32)->Arg(1024)->Arg(8192);
+}
+
+int main(int argc, char** argv) {
+  auto std_fill_n    = [](auto out, auto n, auto const& value) { return std::fill_n(out, n, value); };
+  auto ranges_fill_n = [](auto out, auto n, auto const& value) { return std::ranges::fill_n(out, n, value); };
+
+  // std::fill_n
+  bm<std::vector<int>>("std::fill_n(vector<int>)", std_fill_n);
+  bm<std::deque<int>>("std::fill_n(deque<int>)", std_fill_n);
+  bm<std::list<int>>("std::fill_n(list<int>)", std_fill_n);
+  bm_vector_bool("std::fill_n(vector<bool>)", std_fill_n);
+
+  // ranges::fill_n
+  bm<std::vector<int>>("ranges::fill_n(vector<int>)", ranges_fill_n);
+  bm<std::deque<int>>("ranges::fill_n(deque<int>)", ranges_fill_n);
+  bm<std::list<int>>("ranges::fill_n(list<int>)", ranges_fill_n);
+#if TEST_STD_VER >= 23 // vector<bool>::iterator is not an output_iterator before C++23
+  bm_vector_bool("ranges::fill_n(vector<bool>)", ranges_fill_n);
+#endif
+
+  benchmark::Initialize(&argc, argv);
+  benchmark::RunSpecifiedBenchmarks();
+  benchmark::Shutdown();
+  return 0;
+}
diff --git a/libcxx/test/benchmarks/algorithms/modifying/generate.bench.cpp b/libcxx/test/benchmarks/algorithms/modifying/generate.bench.cpp
new file mode 100644
index 0000000000000..15b039a4d3009
--- /dev/null
+++ b/libcxx/test/benchmarks/algorithms/modifying/generate.bench.cpp
@@ -0,0 +1,59 @@
+//===----------------------------------------------------------------------===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+
+// UNSUPPORTED: c++03, c++11, c++14, c++17
+
+#include <algorithm>
+#include <cstddef>
+#include <deque>
+#include <iterator>
+#include <list>
+#include <string>
+#include <vector>
+
+#include "benchmark/benchmark.h"
+#include "../../GenerateInput.h"
+
+template <class Container, class Operation>
+void bm(std::string operation_name, Operation generate) {
+  auto bench = [generate](auto& st) {
+    std::size_t const size = st.range(0);
+    Container c(size);
+    using ValueType = typename Container::value_type;
+    ValueType x     = Generate<ValueType>::random();
+
+    for ([[maybe_unused]] auto _ : st) {
+      auto f = [&x] { return x; };
+      generate(c.begin(), c.end(), f);
+      benchmark::DoNotOptimize(c);
+      benchmark::DoNotOptimize(x);
+      benchmark::ClobberMemory();
+    }
+  };
+  benchmark::RegisterBenchmark(operation_name, bench)->Arg(32)->Arg(1024)->Arg(8192);
+}
+
+int main(int argc, char** argv) {
+  auto std_generate    = [](auto first, auto last, auto f) { return std::generate(first, last, f); };
+  auto ranges_generate = [](auto first, auto last, auto f) { return std::ranges::generate(first, last, f); };
+
+  // std::generate
+  bm<std::vector<int>>("std::generate(vector<int>)", std_generate);
+  bm<std::deque<int>>("std::generate(deque<int>)", std_generate);
+  bm<std::list<int>>("std::generate(list<int>)", std_generate);
+
+  // ranges::generate
+  bm<std::vector<int>>("ranges::generate(vector<int>)", ranges_generate);
+  bm<std::deque<int>>("ranges::generate(deque<int>)", ranges_generate);
+  bm<std::list<int>>("ranges::generate(list<int>)", ranges_generate);
+
+  benchmark::Initialize(&argc, argv);
+  benchmark::RunSpecifiedBenchmarks();
+  benchmark::Shutdown();
+  return 0;
+}
diff --git a/libcxx/test/benchmarks/algorithms/modifying/generate_n.bench.cpp b/libcxx/test/benchmarks/algorithms/modifying/generate_n.bench.cpp
new file mode 100644
index 0000000000000..75b088411810f
--- /dev/null
+++ b/libcxx/test/benchmarks/algorithms/modifying/generate_n.bench.cpp
@@ -0,0 +1,59 @@
+//===----------------------------------------------------------------------===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+
+// UNSUPPORTED: c++03, c++11, c++14, c++17
+
+#include <algorithm>
+#include <cstddef>
+#include <deque>
+#include <iterator>
+#include <list>
+#include <string>
+#include <vector>
+
+#include "benchmark/benchmark.h"
+#include "../../GenerateInput.h"
+
+template <class Container, class Operation>
+void bm(std::string operation_name, Operation generate_n) {
+  auto bench = [generate_n](auto& st) {
+    std::size_t const size = st.range(0);
+    Container c(size);
+    using ValueType = typename Container::value_type;
+    ValueType x     = Generate<ValueType>::random();
+
+    for ([[maybe_unused]] auto _ : st) {
+      auto f = [&x] { return x; };
+      generate_n(c.begin(), size, f);
+      benchmark::DoNotOptimize(c);
+      benchmark::DoNotOptimize(x);
+      benchmark::ClobberMemory();
+    }
+  };
+  benchmark::RegisterBenchmark(operation_name, bench)->Arg(32)->Arg(1024)->Arg(8192);
+}
+
+int main(int argc, char** argv) {
+  auto std_generate_n    = [](auto out, auto n, auto f) { return std::generate_n(out, n, f); };
+  auto ranges_generate_n = [](auto out, auto n, auto f) { return std::ranges::generate_n(out, n, f); };
+
+  // std::generate_n
+  bm<std::vector<int>>("std::generate_n(vector<int>)", std_generate_n);
+  bm<std::deque<int>>("std::generate_n(deque<int>)", std_generate_n);
+  bm<std::list<int>>("std::generate_n(list<int>)", std_generate_n);
+
+  // ranges::generate_n
+  bm<std::vector<int>>("ranges::generate_n(vector<int>)", ranges_generate_n);
+  bm<std::deque<int>>("ranges::generate_n(deque<int>)", ranges_generate_n);
+  bm<std::list<int>>("ranges::generate_n(list<int>)", ranges_generate_n);
+
+  benchmark::Initialize(&argc, argv);
+  benchmark::RunSpecifiedBenchmarks();
+  benchmark::Shutdown();
+  return 0;
+}
diff --git a/libcxx/test/benchmarks/algorithms/modifying/remove.bench.cpp b/libcxx/test/benchmarks/algorithms/modifying/remove.bench.cpp
new file mode 100644
index 0000000000000..764dfe73ab70c
--- /dev/null
+++ b/libcxx/test/benchmarks/algorithms/modifying/remove.bench.cpp
@@ -0,0 +1,133 @@
+//===----------------------------------------------------------------------===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+
+// UNSUPPORTED: c++03, c++11, c++14, c++17
+
+#include <algorithm>
+#include <cstddef>
+#include <deque>
+#include <iterator>
+#include <list>
+#include <string>
+#include <vector>
+
+#include "benchmark/benchmark.h"
+#include "../../GenerateInput.h"
+
+// Create a sequence of the form xxxxxxxxxxyyyyyyyyyy and remove
+// the prefix of x's from it.
+//
+// We perform this benchmark in a batch because we need to restore the
+// state of the container after the operation.
+template <class Container, class Operation>
+void bm_prefix(std::string operation_name, Operation remove) {
+  auto bench = [remove](auto& st) {
+    std::size_t const size          = st.range(0);
+    constexpr std::size_t BatchSize = 10;
+    using ValueType                 = typename Container::value_type;
+    Container c[BatchSize];
+    ValueType x = Generate<ValueType>::random();
+    ValueType y = Generate<ValueType>::random();
+    for (std::size_t i = 0; i != BatchSize; ++i) {
+      c[i]      = Container(size);
+      auto half = size / 2;
+      std::fill_n(std::fill_n(c[i].begin(), half, x), half, y);
+    }
+
+    while (st.KeepRunningBatch(BatchSize)) {
+      for (std::size_t i = 0; i != BatchSize; ++i) {
+        auto result = remove(c[i].begin(), c[i].end(), x);
+        benchmark::DoNotOptimize(result);
+        benchmark::DoNotOptimize(c[i]);
+        benchmark::DoNotOptimize(x);
+        benchmark::ClobberMemory();
+      }
+
+      st.PauseTiming();
+      for (std::size_t i = 0; i != BatchSize; ++i) {
+        auto half = size / 2;
+        std::fill_n(std::fill_n(c[i].begin(), half, x), half, y);
+      }
+      st.ResumeTiming();
+    }
+  };
+  benchmark::RegisterBenchmark(operation_name, bench)->Arg(32)->Arg(1024)->Arg(8192);
+}
+
+// Create a sequence of the form xyxyxyxyxyxyxyxyxyxy and remove
+// the x's from it.
+//
+// We perform this benchmark in a batch because we need to restore the
+// state of the container after the operation.
+template <class Container, class Operation>
+void bm_sprinkled(std::string operation_name, Operation remove) {
+  auto bench = [remove](auto& st) {
+    std::size_t const size          = st.range(0);
+    constexpr std::size_t BatchSize = 10;
+    using ValueType                 = typename Container::value_type;
+    Container c[BatchSize];
+    ValueType x    = Generate<ValueType>::random();
+    ValueType y    = Generate<ValueType>::random();
+    auto alternate = [&](auto out, auto n) {
+      for (std::size_t i = 0; i != n; ++i) {
+        *out++ = (i % 2 == 0 ? x : y);
+      }
+    };
+    for (std::size_t i = 0; i != BatchSize; ++i) {
+      c[i] = Container(size);
+      alternate(c[i].begin(), size);
+    }
+
+    while (st.KeepRunningBatch(BatchSize)) {
+      for (std::size_t i = 0; i != BatchSize; ++i) {
+        auto result = remove(c[i].begin(), c[i].end(), x);
+        benchmark::DoNotOptimize(result);
+        benchmark::DoNotOptimize(c[i]);
+        benchmark::DoNotOptimize(x);
+        benchmark::ClobberMemory();
+      }
+
+      st.PauseTiming();
+      for (std::size_t i = 0; i != BatchSize; ++i) {
+        alternate(c[i].begin(), size);
+      }
+      st.ResumeTiming();
+    }
+  };
+  benchmark::RegisterBenchmark(operation_name, bench)->Arg(32)->Arg(1024)->Arg(8192);
+}
+
+int main(int argc, char** argv) {
+  auto std_remove    = [](auto first, auto last, auto const& value) { return std::remove(first, last, value); };
+  auto ranges_remove = [](auto first, auto last, auto const& value) { return std::ranges::remove(first, last, value); };
+
+  // std::remove
+  bm_prefix<std::vector<int>>("std::remove(vector<int>) (prefix)", std_remove);
+  bm_sprinkled<std::vector<int>>("std::remove(vector<int>) (sprinkled)", std_remove);
+
+  bm_prefix<std::deque<int>>("std::remove(deque<int>) (prefix)", std_remove);
+  bm_sprinkled<std::deque<int>>("std::remove(deque<int>) (sprinkled)", std_remove);
+
+  bm_prefix<std::list<int>>("std::remove(list<int>) (prefix)", std_remove);
+  bm_sprinkled<std::list<int>>("std::remove(list<int>) (sprinkled)", std_remove);
+
+  // ranges::remove
+  bm_prefix<std::vector<int>>("ranges::remove(vector<int>) (prefix)", ranges_remove);
+  bm_sprinkled<std::vector<int>>("ranges::remove(vector<int>) (sprinkled)", ranges_remove);
+
+  bm_prefix<std::deque<int>>("ranges::remove(deque<int>) (prefix)", ranges_remove);
+  bm_sprinkled<std::deque<int>>("ranges::remove(deque<int>) (sprinkled)", ranges_remove);
+
+  bm_prefix<std::list<int>>("ranges::remove(list<int>) (prefix)", ranges_remove);
+  bm_sprinkled<std::list<int>>("ranges::remove(list<int>) (sprinkled)", ranges_remove);
+
+  benchmark::Initialize(&argc, argv);
+  benchmark::RunSpecifiedBenchmarks();
+  benchmark::Shutdown();
+  return 0;
+}
diff --git a/libcxx/test/benchmarks/algorithms/modifying/remove_copy.bench.cpp b/libcxx/test/benchmarks/algorithms/modifying/remove_copy.bench.cpp
new file mode 100644
index 0000000000000..88c05384eb7bd
--- /dev/null
+++ b/libcxx/test/benchmarks/algorithms/modifying/remove_copy.bench.cpp
@@ -0,0 +1,109 @@
+//===----------------------------------------------------------------------===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-...
[truncated]

@siya100
Copy link
Contributor

siya100 commented Feb 18, 2025

can i work on this issue

@ldionne ldionne force-pushed the review/benchmark-add-all-modifying branch from 7d00280 to 55c1ebe Compare February 19, 2025 21:37
Copy link
Member

@mordante mordante left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! LGTM!

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For the copy family of algorithms I think it would be interesting to test some non-trivial type. Specifically, that the compiler would not be able to see through the operation. That could be achieved by an out-of-line definition or a [[gnu::noinline]] on the function. Maybe just an optimization barrier would work to.

Copy link
Member Author

@ldionne ldionne Feb 25, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here too, I would like to defer to a separate patch where I can focus on doing just that for the copy and the move algorithms.

#128737

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should also add benchmarks for cases where the output is a segmented iterator (i.e. output to a deque) for the _copy versions of algorithms.

Copy link
Member Author

@ldionne ldionne Feb 25, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree, but after consideration I would like to tackle that as a separate patch. Indeed, adding this variation of the benchmark requires writing a new batched benchmark since I can't just alternate between the source and the destination range. I'd like to do that in a separate patch that also tackles the benchmarks for std::copy (which have already landed).

#128733

@ldionne ldionne force-pushed the review/benchmark-add-all-modifying branch from 5c563b5 to fd313e5 Compare March 17, 2025 13:36
@ldionne
Copy link
Member Author

ldionne commented Mar 17, 2025

I'm going to land this now since I applied all of @philnik777 's comments except #128733 and #128737. We can do another pass once he's back from vacation, but landing this now will at least ensure we don't have merge conflicts.

@ldionne ldionne merged commit 24e88b0 into llvm:main Mar 17, 2025
85 checks passed
@ldionne ldionne deleted the review/benchmark-add-all-modifying branch March 17, 2025 19:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

libc++ libc++ C++ Standard Library. Not GNU libstdc++. Not libc++abi.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants