Skip to content

Commit 57faf66

Browse files
changkunchangkun
committed
all: add initial implementation
1 parent 35d507a commit 57faf66

35 files changed

+4653
-1
lines changed

‎README.md‎

Lines changed: 123 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,2 +1,124 @@
11
# bench
2-
refined performance measurement in Go
2+
3+
Reliable performance measurement for Go programs. All in one design.
4+
5+
```
6+
$ go get golang.design/x/bench
7+
```
8+
9+
## Features
10+
11+
- Combine [benchstat](https://pkg.go.dev/golang.org/x/perf/cmd/benchstat), [perflock](https://github.com/aclements/perflock) and more...
12+
- Short command and only run benchmarks
13+
- Automatic performance locking for benchmarks
14+
- Automatic statistic analysis for benchmark results
15+
- Color indications for benchmark results
16+
17+
## Usage
18+
19+
### Enable `bench` Daemon (optional, Linux only)
20+
21+
```sh
22+
$ cd $GOPATH/src/golang.design/x/bench
23+
$ ./install.bash
24+
```
25+
26+
If your init system is supported, this will also configure `bench` to start automatically on boot.
27+
28+
Or you can install and run `bench` daemon manually:
29+
30+
```sh
31+
$ sudo install $GOPATH/bin/bench /usr/bin/bench
32+
$ sudo -b bench -daemon
33+
```
34+
35+
### Default Behavior
36+
37+
```sh
38+
$ bench
39+
```
40+
41+
The detault behavior of `bench` run benchmarks under your current
42+
working directory, and each benchmark will be ran 10 times for further
43+
statistical analysis. It will also try to acquire performance lock from
44+
`bench` daemon to gain more stable results. Furthermore, the benchmark
45+
results are saved as a text file to the working directory and named as
46+
`<timestamp>.txt`.
47+
48+
Example:
49+
50+
```
51+
$ cd example
52+
$ bench
53+
bench: run benchmarks under 90% cpufreq...
54+
bench: go test -run=^$ -bench=. -count=10
55+
goos: linux
56+
goarch: amd64
57+
pkg: golang.design/x/bench/example
58+
BenchmarkDemo-16 21114 57340 ns/op
59+
...
60+
BenchmarkDemo-16 21004 57097 ns/op
61+
PASS
62+
ok golang.design/x/bench/example 17.791s
63+
bench: results are saved to file: ./bench-2020-11-07-19:59:51.txt
64+
65+
name time/op
66+
Demo-16 57.0µs ±1%
67+
68+
$ # ... do some changes to the benchmark ...
69+
70+
$ bench
71+
bench: run benchmarks under 90% cpufreq...
72+
bench: go test -run=^$ -bench=. -count=10
73+
goos: linux
74+
goarch: amd64
75+
pkg: golang.design/x/bench/example
76+
BenchmarkDemo-16 213145 5625 ns/op
77+
...
78+
BenchmarkDemo-16 212959 5632 ns/op
79+
PASS
80+
ok golang.design/x/bench/example 12.536s
81+
bench: results are saved to file: ./bench-2020-11-07-20:00:16.txt
82+
83+
name time/op
84+
Demo-16 5.63µs ±0%
85+
86+
$ bench bench-2020-11-07-19:59:51.txt bench-2020-11-07-20:00:16.txt
87+
name old time/op new time/op delta
88+
Demo-16 57.0µs ±1% 5.6µs ±0% -90.13% (p=0.000 n=10+8)
89+
```
90+
91+
### Options
92+
93+
Options for checking daemon status:
94+
95+
```sh
96+
bench -list
97+
```
98+
99+
Options for statistic tests:
100+
101+
```sh
102+
bench old.txt [new.txt] # same from benchstat
103+
bench -delta-test
104+
bench -alpha
105+
bench -geomean
106+
bench -split
107+
bench -sort
108+
```
109+
110+
Options for running benchmarks:
111+
112+
```sh
113+
bench -v # enable verbose outputs
114+
bench -shared # enable shared execution
115+
bench -cpufreq 90 # cpu frequency (default: 90)
116+
bench -name BenchmarkXXX # go test `-bench` flag (default: .)
117+
bench -count 20 # go test `-count` flag (default: 10)
118+
bench -time 100x # go test `-benchtime` flag (default: unset)
119+
bench -cpuproc 1,2,4,8,16,32,128 # go test `-cpu` flag (default: unset)
120+
```
121+
122+
## License
123+
124+
&copy; 2020 The golang.design Authors

‎example/bench-2020-11-07-19:59:51.txt‎

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,15 @@
1+
goos: linux
2+
goarch: amd64
3+
pkg: golang.design/x/bench/example
4+
BenchmarkDemo-16 21114 57340 ns/op
5+
BenchmarkDemo-16 21109 56897 ns/op
6+
BenchmarkDemo-16 21091 56833 ns/op
7+
BenchmarkDemo-16 21121 56821 ns/op
8+
BenchmarkDemo-16 21105 56953 ns/op
9+
BenchmarkDemo-16 21112 56830 ns/op
10+
BenchmarkDemo-16 20836 57499 ns/op
11+
BenchmarkDemo-16 21084 56885 ns/op
12+
BenchmarkDemo-16 20990 57245 ns/op
13+
BenchmarkDemo-16 21004 57097 ns/op
14+
PASS
15+
ok golang.design/x/bench/example 17.791s

‎example/bench-2020-11-07-20:00:16.txt‎

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,15 @@
1+
goos: linux
2+
goarch: amd64
3+
pkg: golang.design/x/bench/example
4+
BenchmarkDemo-16 213145 5625 ns/op
5+
BenchmarkDemo-16 211988 5629 ns/op
6+
BenchmarkDemo-16 211838 5629 ns/op
7+
BenchmarkDemo-16 212151 5627 ns/op
8+
BenchmarkDemo-16 212577 5627 ns/op
9+
BenchmarkDemo-16 210150 5635 ns/op
10+
BenchmarkDemo-16 212650 5637 ns/op
11+
BenchmarkDemo-16 210063 5670 ns/op
12+
BenchmarkDemo-16 212568 5665 ns/op
13+
BenchmarkDemo-16 212959 5632 ns/op
14+
PASS
15+
ok golang.design/x/bench/example 12.536s

‎example/bench_test.go‎

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,19 @@
1+
// Copyright 2020 The golang.design Initiative Authors.
2+
// All rights reserved. Use of this source code is governed
3+
// by a GNU GPLv3 license that can be found in the LICENSE file.
4+
5+
package example
6+
7+
import "testing"
8+
9+
func grow(n int) {
10+
if n > 0 {
11+
grow(n - 1)
12+
}
13+
}
14+
15+
func BenchmarkDemo(b *testing.B) {
16+
for i := 0; i < b.N; i++ {
17+
grow(10000) // try change this and re-run `bench`
18+
}
19+
}

‎go.mod‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
module golang.design/x/bench
2+
3+
go 1.15

‎init/systemd/README.md‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
To configure systemd to run `bench`, run
2+
3+
```
4+
$ sudo install -m 0644 bench.service /etc/systemd/system
5+
$ sudo systemctl enable --now bench.service
6+
```

‎init/systemd/bench.service‎

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,10 @@
1+
[Unit]
2+
Description=Bench daemon
3+
4+
[Service]
5+
Type=simple
6+
ExecStart=/usr/bin/bench -daemon
7+
Restart=on-failure
8+
9+
[Install]
10+
WantedBy=multi-user.target

‎init/upstart/README.md‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
To configure Upstart to run `bench`, copy `bench.conf` into `/etc/init/` and run sudo start `bench`. E.g.,
2+
3+
```sh
4+
$ sudo install -m 0644 bench.conf /etc/init/
5+
$ sudo start bench
6+
```

‎init/upstart/bench.conf‎

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
1+
# bench - Benchmark performance locking service
2+
3+
description "bench daemon"
4+
5+
start on runlevel [2345]
6+
stop on runlevel [!2345]
7+
respawn
8+
9+
console log
10+
11+
exec bench -daemon

‎install.sh‎

Lines changed: 40 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,40 @@
1+
#!/bin/bash
2+
3+
set -e
4+
5+
BINPATH="$(go env GOBIN)"
6+
if [[ -z "$BINPATH" ]]; then
7+
BINPATH="$(go env GOPATH)/bin"
8+
fi
9+
BIN="$BINPATH/bench"
10+
if [[ ! -x "$BIN" ]]; then
11+
echo "bench binary $BIN does not exist." 2>&1
12+
echo "Please run go install golang.design/x/bench" 2>&1
13+
exit 1
14+
fi
15+
16+
echo "Installing $BIN to /usr/bin" 1>&2
17+
sudo install "$BIN" /usr/bin/bench
18+
19+
start="-b /usr/bin/bench -daemon"
20+
starttype=
21+
if [[ -d /etc/init ]]; then
22+
echo "Installing init script for Upstart" 1>&2
23+
sudo install -m 0644 init/upstart/bench.conf /etc/init/
24+
start="service bench start"
25+
starttype=" (using Upstart)"
26+
fi
27+
if [[ -d /etc/systemd ]]; then
28+
echo "Installing service for systemd" 1>&2
29+
sudo install -m 0644 init/systemd/bench.service /etc/systemd/system
30+
sudo systemctl enable --quiet bench.service
31+
start="systemctl start bench.service"
32+
starttype=" (using systemd)"
33+
fi
34+
35+
if /usr/bin/bench -list >/dev/null 2>&1; then
36+
echo "Not starting bench daemon (already running)" 1>&2
37+
else
38+
echo "Starting bench daemon$starttype" 1>&2
39+
sudo $start
40+
fi

0 commit comments

Comments
 (0)