Skip to content

Commit e3d6691

Browse files
committed
doc: add more context about benchmark tests and envs used
1 parent a14e056 commit e3d6691

File tree

1 file changed

+35
-0
lines changed

1 file changed

+35
-0
lines changed

doc/contributing/writing-and-running-benchmarks.md

Lines changed: 35 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -707,3 +707,38 @@ Supported options keys are:
707707
[node-benchmark-compare]: https://github.com/targos/node-benchmark-compare
708708
[t-test]: https://en.wikipedia.org/wiki/Student%27s_t-test#Equal_or_unequal_sample_sizes%2C_unequal_variances_%28sX1_%3E_2sX2_or_sX2_%3E_2sX1%29
709709
[wrk]: https://github.com/wg/wrk
710+
711+
### Creating Benchmark Tests
712+
713+
It is recommended to create a new test file when a new benchmark is introduced
714+
so it can be easily made creating the new test file in `test/benchmark`.
715+
716+
When calling the `runBenchmark`, provide the benchmark group name
717+
(which is the folder name in the `benchmark/` folder) as the first parameter,
718+
and optionally pass environment variables as the second parameter.
719+
720+
```js
721+
'use strict';
722+
723+
require('../common'); // Import the common module - required for all benchmark files
724+
725+
const runBenchmark = require('../common/benchmark');
726+
727+
runBenchmark('buffers', { NODEJS_BENCHMARK_ZERO_ALLOWED: 1 });
728+
```
729+
730+
The environment variable `NODEJS_BENCHMARK_ZERO_ALLOWED` is necessary
731+
if the test runs so fast that it can result in errors or flaky tests.
732+
Setting this environment variable will make the benchmark ignore these issues.
733+
734+
The way tests will be executed depends on the environment variable `NODE_RUN_ALL_BENCH_TESTS`.
735+
If it is set to **true**, benchmarks run with minimal iterations (`n=1`, `rounds=1`).
736+
The idea here is to ignore performance analysis and just check if it can run without failing.
737+
Despite running the minimal amount, it still consumes too much time to run
738+
because it needs to run all configurations.
739+
740+
Otherwise, when `NODE_RUN_ALL_BENCH_TESTS` isn't set,
741+
only one configuration per benchmark is executed.
742+
This significantly reduces execution time but offers limited coverage
743+
and cannot ensure all configurations work correctly.
744+

0 commit comments

Comments
 (0)