You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The environment variable `NODEJS_BENCHMARK_ZERO_ALLOWED` is necessary
731
-
if the test runs so fast that it can result in errors or flaky tests.
732
-
Setting this environment variable will make the benchmark ignore these issues.
733
-
734
-
The way tests will be executed depends on the environment variable `NODE_RUN_ALL_BENCH_TESTS`.
735
-
If it is set to **true**, benchmarks run with minimal iterations (`n=1`, `rounds=1`).
736
-
The idea here is to ignore performance analysis and just check if it can run without failing.
737
-
Despite running the minimal amount, it still consumes too much time to run
738
-
because it needs to run all configurations.
739
-
740
-
Otherwise, when `NODE_RUN_ALL_BENCH_TESTS` isn't set,
741
-
only one configuration per benchmark is executed.
742
-
This significantly reduces execution time but offers limited coverage
743
-
and cannot ensure all configurations work correctly.
730
+
The environment variable `NODEJS_BENCHMARK_ZERO_ALLOWED` is required
731
+
when tests execute so quickly that they may produce errors or inconsistent results.
732
+
Setting this variable instructs the benchmark to disregard such issues.
733
+
734
+
Test execution behavior depends on the `NODE_RUN_ALL_BENCH_TESTS` environment variable.
735
+
When set to **true**, benchmarks run with minimal iterations (`n=1`, `rounds=1`).
736
+
This approach bypasses performance analysis to verify that tests can complete without failures.
737
+
Despite the minimal iterations, execution remains time-consuming
738
+
as all configurations must be tested.
739
+
740
+
When `NODE_RUN_ALL_BENCH_TESTS` is not set,
741
+
only a single configuration per benchmark executes.
742
+
While this dramatically reduces execution time, it provides limited coverage
743
+
and cannot guarantee that all configurations function properly.
744
+
745
+
This PR introduce the usage of new env `NODE_RUN_ALL_BENCH_TESTS`, that can be set to all becnhmark configs run in tests to try coverage more scenarios when benchmarks can fails.
746
+
this PR as well document how to write benchmark tests, and give more details about the envs
747
+
748
+
- NODE_RUN_ALL_BENCH_TESTS
749
+
- NODEJS_BENCHMARK_ZERO_ALLOWED
750
+
751
+
Some benchmarks tests was introduced as well for benchmark groups
0 commit comments