Skip to content

Conversation

ldez
Copy link
Member

@ldez ldez commented May 27, 2024

Adds 2 targets to benchmark a linter.

LINTER=gosec VERSION=v1.59.0 make bench_local
LINTER=gosec VERSION_OLD=v1.58.2 VERSION_NEW=v1.59.0 make bench_version

It uses Hyperfine.

It's not really possible to automate that because it will take too much time to run, but at least we will be able to benchmark manually a PR.

Demo
$ LINTER=misspell VERSION_OLD=v1.57.2 VERSION_NEW=v1.59.0 make bench_version
golangci/golangci-lint info checking GitHub for tag 'v1.57.2'
golangci/golangci-lint info found version: 1.57.2 for v1.57.2/linux/amd64
golangci/golangci-lint info installed ./temp-v1.57.2/golangci-lint
golangci/golangci-lint info checking GitHub for tag 'v1.59.0'
golangci/golangci-lint info found version: 1.59.0 for v1.59.0/linux/amd64
golangci/golangci-lint info installed ./temp-v1.59.0/golangci-lint
Benchmark 1: ./golangci-lint-v1.57.2 run --issues-exit-code 0 --print-issued-lines=false --enable-only misspell
  Time (mean ± σ):     630.3 ms ±  26.9 ms    [User: 2246.2 ms, System: 1483.3 ms]
  Range (min … max):   591.8 ms … 668.5 ms    10 runs
 
Benchmark 2: ./golangci-lint-v1.59.0 run --issues-exit-code 0 --print-issued-lines=false --enable-only misspell
  Time (mean ± σ):     379.5 ms ±  24.7 ms    [User: 1055.4 ms, System: 479.7 ms]
  Range (min … max):   349.4 ms … 430.7 ms    10 runs
 
Summary
  ./golangci-lint-v1.59.0 run --issues-exit-code 0 --print-issued-lines=false --enable-only misspell ran
    1.66 ± 0.13 times faster than ./golangci-lint-v1.57.2 run --issues-exit-code 0 --print-issued-lines=false --enable-only misspell
Clean binaries

Related to #1152

@ldez ldez added enhancement New feature or improvement area: ci PR that update CI labels May 27, 2024
@ldez ldez added this to the next milestone May 27, 2024
@ldez ldez force-pushed the feat/benchmark-script branch 2 times, most recently from b01ae84 to 74751bd Compare May 27, 2024 22:07
@ldez ldez force-pushed the feat/benchmark-script branch from 15e1017 to 34a0e3e Compare May 28, 2024 13:29
@ldez ldez added area: benchmark and removed area: ci PR that update CI labels May 28, 2024
ldez and others added 2 commits May 28, 2024 16:29
@ldez ldez merged commit aea73fb into golangci:master May 28, 2024
@ldez ldez deleted the feat/benchmark-script branch May 28, 2024 17:39
@ldez ldez modified the milestones: next, v1.59 Jun 9, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area: benchmark enhancement New feature or improvement
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants