Skip to content

Commit 241b50f

Browse files
committed
fixup! doc: add more context about benchmark tests and envs used
1 parent e3d6691 commit 241b50f

File tree

2 files changed

+38
-15
lines changed

2 files changed

+38
-15
lines changed

doc/contributing/writing-and-running-benchmarks.md

Lines changed: 37 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -727,18 +727,41 @@ const runBenchmark = require('../common/benchmark');
727727
runBenchmark('buffers', { NODEJS_BENCHMARK_ZERO_ALLOWED: 1 });
728728
```
729729

730-
The environment variable `NODEJS_BENCHMARK_ZERO_ALLOWED` is necessary
731-
if the test runs so fast that it can result in errors or flaky tests.
732-
Setting this environment variable will make the benchmark ignore these issues.
733-
734-
The way tests will be executed depends on the environment variable `NODE_RUN_ALL_BENCH_TESTS`.
735-
If it is set to **true**, benchmarks run with minimal iterations (`n=1`, `rounds=1`).
736-
The idea here is to ignore performance analysis and just check if it can run without failing.
737-
Despite running the minimal amount, it still consumes too much time to run
738-
because it needs to run all configurations.
739-
740-
Otherwise, when `NODE_RUN_ALL_BENCH_TESTS` isn't set,
741-
only one configuration per benchmark is executed.
742-
This significantly reduces execution time but offers limited coverage
743-
and cannot ensure all configurations work correctly.
730+
The environment variable `NODEJS_BENCHMARK_ZERO_ALLOWED` is required
731+
when tests execute so quickly that they may produce errors or inconsistent results.
732+
Setting this variable instructs the benchmark to disregard such issues.
733+
734+
Test execution behavior depends on the `NODE_RUN_ALL_BENCH_TESTS` environment variable.
735+
When set to **true**, benchmarks run with minimal iterations (`n=1`, `rounds=1`).
736+
This approach bypasses performance analysis to verify that tests can complete without failures.
737+
Despite the minimal iterations, execution remains time-consuming
738+
as all configurations must be tested.
739+
740+
When `NODE_RUN_ALL_BENCH_TESTS` is not set,
741+
only a single configuration per benchmark executes.
742+
While this dramatically reduces execution time, it provides limited coverage
743+
and cannot guarantee that all configurations function properly.
744+
745+
This PR introduce the usage of new env `NODE_RUN_ALL_BENCH_TESTS`, that can be set to all becnhmark configs run in tests to try coverage more scenarios when benchmarks can fails.
746+
this PR as well document how to write benchmark tests, and give more details about the envs
747+
748+
- NODE_RUN_ALL_BENCH_TESTS
749+
- NODEJS_BENCHMARK_ZERO_ALLOWED
750+
751+
Some benchmarks tests was introduced as well for benchmark groups
752+
753+
- abort_controller
754+
- error
755+
- https
756+
- perf_hooks
757+
- permission
758+
- sqlite
759+
- test_runner
760+
- websocket
761+
762+
and some incosistent test files was renamed
763+
764+
test/benchmark/test-benchmark-async-hooks.js → test/benchmark/test-benchmark-async_hooks.js
765+
test/benchmark/test-benchmark-child-process.js → test/benchmark/test-benchmark-child_process.js
766+
744767

test/benchmark/test-benchmark-test_runner.js

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -4,4 +4,4 @@ require('../common');
44

55
const runBenchmark = require('../common/benchmark');
66

7-
runBenchmark('test_runner')
7+
runBenchmark('test_runner');

0 commit comments

Comments
 (0)