Skip to content

Commit 2f7bad3

Browse files
eightbitraptoreightbitraptor
authored
Merge pull request #375 from ruby/ractor-harness-luke
Introduce Ractor benchmarks and harness
2 parents 6254610 + 83f1331 commit 2f7bad3

File tree

58 files changed

+121007
-120220
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

58 files changed

+121007
-120220
lines changed

‎.github/workflows/test.yml‎

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -37,6 +37,22 @@ jobs:
3737
MIN_BENCH_TIME: '0'
3838
continue-on-error: ${{ matrix.continue-on-error || false }}
3939

40+
- name: Test run_benchmarks.rb - Ractors
41+
run: ./run_benchmarks.rb --category=ractor
42+
env:
43+
WARMUP_ITRS: '1'
44+
MIN_BENCH_ITRS: '1'
45+
MIN_BENCH_TIME: '0'
46+
continue-on-error: ${{ matrix.continue-on-error || false }}
47+
48+
- name: Test run_benchmarks.rb - Ractor Only
49+
run: ./run_benchmarks.rb --category=ractor-only
50+
env:
51+
WARMUP_ITRS: '1'
52+
MIN_BENCH_ITRS: '1'
53+
MIN_BENCH_TIME: '0'
54+
continue-on-error: ${{ matrix.continue-on-error || false }}
55+
4056
- name: Test run_benchmarks.rb --graph
4157
run: |
4258
sudo apt-get update

‎.gitignore‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -3,6 +3,7 @@
33
*.csv
44
*.dump
55
logs*
6+
benchmarks/*/data/
67

78
__pycache__
89
/benchmarks/discourse

‎README.md‎

Lines changed: 37 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -101,6 +101,43 @@ It's also convenient for profiling, debugging, etc, especially since all benchma
101101
ruby benchmarks/some_benchmark.rb
102102
```
103103

104+
## Ractor Benchmarks
105+
106+
ruby-bench supports Ractor-specific benchmarking with dedicated categories and benchmark directories.
107+
108+
### Ractor Categories
109+
110+
There are two Ractor-related categories:
111+
112+
* **`--category ractor`** - Runs both regular benchmarks marked with `ractor:
113+
true` in `benchmarks.yml` AND all benchmarks from the `benchmarks-ractor`
114+
directory. The `harness-ractor` harness is used for both types of benchmark.
115+
116+
* **`--category ractor-only`** - Runs ONLY benchmarks from the
117+
`benchmarks-ractor` directory, ignoring regular benchmarks even if they are
118+
marked with `ractor: true`. This category also automatically uses the
119+
`harness-ractor` harness.
120+
121+
### Directory Structure
122+
123+
The `benchmarks-ractor/` directory sits at the same level as the main
124+
`benchmarks` directory, and contains Ractor-specific benchmark
125+
implementations that are designed to test Ractor functionality. They are not
126+
intended to be used with any harness except `harness-ractor`.
127+
128+
### Usage Examples
129+
130+
```bash
131+
# Run all Ractor-capable benchmarks (both regular and Ractor-specific)
132+
./run_benchmarks.rb --category ractor
133+
134+
# Run only dedicated Ractor benchmarks from benchmarks-ractor directory
135+
./run_benchmarks.rb --category ractor-only
136+
```
137+
138+
Note: The `harness-ractor` harness is automatically selected when using these
139+
categories, so there's no need to specify `--harness` manually.
140+
104141
## Ruby options
105142

106143
By default, ruby-bench benchmarks the Ruby used for `run_benchmarks.rb`.

‎benchmarks-ractor/gvl_release_acquire/benchmark.rb‎

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,9 @@
1+
require_relative "../../harness/loader"
2+
3+
run_benchmark(5) do |num_rs, ractor_args|
4+
output = File.open("/dev/null", "wb")
5+
input = File.open("/dev/zero", "rb")
6+
100_000.times do
7+
output.write(input.read(10))
8+
end
9+
end

‎benchmarks-ractor/json_parse_float/Gemfile‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
source "https://rubygems.org"
2+
gem "json", "2.13.2"

‎benchmarks-ractor/json_parse_float/Gemfile.lock‎

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,14 @@
1+
GEM
2+
remote: https://rubygems.org/
3+
specs:
4+
json (2.13.2)
5+
6+
PLATFORMS
7+
arm64-darwin-23
8+
ruby
9+
10+
DEPENDENCIES
11+
json (= 2.13.2)
12+
13+
BUNDLED WITH
14+
2.7.0

‎benchmarks-ractor/json_parse_float/benchmark.rb‎

Lines changed: 31 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,31 @@
1+
require_relative "../../harness/loader"
2+
3+
Dir.chdir(__dir__)
4+
use_gemfile
5+
require "json"
6+
puts "json v#{JSON::VERSION}"
7+
8+
ELEMENTS = 100_000
9+
list = ELEMENTS.times.map do
10+
[
11+
rand, rand, rand, rand, rand, rand, rand, rand, rand, rand,
12+
rand, rand, rand, rand, rand, rand, rand, rand, rand, rand,
13+
].to_json
14+
end
15+
Ractor.make_shareable(list)
16+
17+
# Work is divided between ractors
18+
run_benchmark(5, ractor_args: [list]) do |num_rs, list|
19+
# num_rs: 1,list: 100_000
20+
# num_rs: 2 list: 50_000
21+
# num_rs: 4 list: 25_000
22+
if num_rs.zero?
23+
num = list.size
24+
else
25+
num = list.size / num_rs
26+
end
27+
list.each_with_index do |json, idx|
28+
break if idx >= num
29+
JSON.parse(json)
30+
end
31+
end

‎benchmarks-ractor/json_parse_string/Gemfile‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
source "https://rubygems.org"
2+
gem "json", "2.13.2"

‎benchmarks-ractor/json_parse_string/Gemfile.lock‎

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,14 @@
1+
GEM
2+
remote: https://rubygems.org/
3+
specs:
4+
json (2.13.2)
5+
6+
PLATFORMS
7+
arm64-darwin-23
8+
ruby
9+
10+
DEPENDENCIES
11+
json (= 2.13.2)
12+
13+
BUNDLED WITH
14+
2.7.0

‎benchmarks-ractor/json_parse_string/benchmark.rb‎

Lines changed: 49 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,49 @@
1+
require_relative "../../harness/loader"
2+
3+
Dir.chdir(__dir__)
4+
use_gemfile
5+
require "json"
6+
puts "json v#{JSON::VERSION}"
7+
8+
ELEMENTS = 300_000
9+
list = ELEMENTS.times.map do |i|
10+
{
11+
"string #{i}" => "value #{i}",
12+
"string #{i}" => "value #{i}",
13+
"string #{i}" => "value #{i}",
14+
"string #{i}" => "value #{i}",
15+
"string #{i}" => "value #{i}",
16+
"string #{i}" => "value #{i}",
17+
"string #{i}" => "value #{i}",
18+
"string #{i}" => "value #{i}",
19+
"string #{i}" => "value #{i}",
20+
"string #{i}" => "value #{i}",
21+
"string #{i}" => "value #{i}",
22+
"string #{i}" => "value #{i}",
23+
"string #{i}" => "value #{i}",
24+
"string #{i}" => "value #{i}",
25+
"string #{i}" => "value #{i}",
26+
"string #{i}" => "value #{i}",
27+
"string #{i}" => "value #{i}",
28+
"string #{i}" => "value #{i}",
29+
"string #{i}" => "value #{i}",
30+
"string #{i}" => "value #{i}",
31+
}.to_json
32+
end
33+
Ractor.make_shareable(list)
34+
35+
# Work is divided between ractors
36+
run_benchmark(5, ractor_args: [list]) do |num_rs, list|
37+
# num_rs: 1,list: 100_000
38+
# num_rs: 2 list: 50_000
39+
# num_rs: 4 list: 25_000
40+
if num_rs.zero?
41+
num = list.size
42+
else
43+
num = list.size / num_rs
44+
end
45+
list.each_with_index do |json, idx|
46+
break if idx >= num
47+
JSON.parse(json)
48+
end
49+
end

0 commit comments

Comments
 (0)