Skip to content

Commit dfb5aaa

Browse files
committed
Speed up integration tests
The integration tests can be quite slow to run. This is because we are actually creating a test program as a string, dumping that string into a temporary file, shelling out to another instance of Ruby/RSpec, running the file inside of that process, then performing a test on the output. We have a lot of integration tests, and running the test suite on Travis can take around 20 minutes. In my opinion, that's way too long for a gem that isn't terribly large. This commit attempts to alleviate this pain point by introducing a development dependency on Zeus. [Zeus] is a "code preloader" that was primarily designed to speed up development within a Rails app, as it can preload your Rails environment so that your tests will run faster. While Zeus has largely been superceded by Spring, it is still a valuable tool and can also be used for non-Rails applications. How does Zeus work? The Zeus server is a persistent process that establishes an environment by running Ruby code of your choosing, then awaits a connection from the Zeus client, which will then run some code within that environment. Zeus employs forking to keep the environment code in memory so that it does not need to be re-run. (This is not the best description, but there is more information on the website if you want to learn more.) The way that we use Zeus is that we create an environment where `super_diff` is already required and — for tests that need ActiveRecord — where ActiveRecord is already required too. The integration tests will then re-use this environment (the code that's run in this environment is specified in the test itself). After all is said and done, this ends up speeding up all integration tests by a little over 2x. How do you get this goodness? I've added a `bin/start-dev` script which will run the persistent Zeus server by way of `zeus start`. You will want to run this command in a separate tab. After this, you can then run `bin/rspec spec/integration/...` and you should see the speed bump. [Zeus]: https://github.com/burke/zeus
1 parent 15cf285 commit dfb5aaa

32 files changed

+794
-500
lines changed

.gitignore

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -3,3 +3,5 @@ Gemfile.lock
33
pkg
44
spec/examples.txt
55
tmp
6+
zeus.server.log
7+
zeus.server-start.log

.rubocop.yml

Lines changed: 13 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,7 @@ AllCops:
33
Exclude:
44
- db/schema.rb
55
TargetRubyVersion: 2.4
6-
Layout/AlignParameters:
6+
Layout/ParameterAlignment:
77
EnforcedStyle: with_fixed_indentation
88
Layout/ClassStructure:
99
Enabled: true
@@ -13,11 +13,11 @@ Layout/CommentIndentation:
1313
Enabled: false
1414
Layout/InitialIndentation:
1515
Enabled: false
16-
Layout/IndentFirstArrayElement:
16+
Layout/FirstArrayElementIndentation:
1717
EnforcedStyle: "consistent"
18-
Layout/IndentFirstHashElement:
18+
Layout/FirstHashElementIndentation:
1919
EnforcedStyle: "consistent"
20-
Layout/IndentHeredoc:
20+
Layout/HeredocIndentation:
2121
Enabled: false
2222
Layout/MultilineOperationIndentation:
2323
EnforcedStyle: indented
@@ -29,7 +29,7 @@ Layout/SpaceInsideStringInterpolation:
2929
Enabled: false
3030
Lint/AssignmentInCondition:
3131
Enabled: false
32-
Lint/HandleExceptions:
32+
Lint/SuppressedException:
3333
Enabled: false
3434
Lint/RequireParentheses:
3535
Enabled: false
@@ -56,7 +56,7 @@ Metrics/ClassLength:
5656
Enabled: false
5757
Metrics/CyclomaticComplexity:
5858
Max: 15
59-
Metrics/LineLength:
59+
Layout/LineLength:
6060
Max: 80
6161
IgnoredPatterns:
6262
- "^\\s*RSpec.describe"
@@ -77,11 +77,11 @@ Naming/AccessorMethodName:
7777
Naming/MemoizedInstanceVariableName:
7878
EnforcedStyleForLeadingUnderscores: required
7979
Naming/PredicateName:
80-
NamePrefixBlacklist:
80+
ForbiddenPrefixes:
8181
- is_
8282
Exclude:
8383
- spec/**/*
84-
Naming/UncommunicativeMethodParamName:
84+
Naming/MethodParameterName:
8585
Enabled: false
8686
Style/AccessModifierDeclarations:
8787
Enabled: false
@@ -91,6 +91,8 @@ Style/CollectionMethods:
9191
Enabled: true
9292
Style/Documentation:
9393
Enabled: false
94+
Style/DoubleNegation:
95+
Enabled: false
9496
Style/EachWithObject:
9597
Enabled: false
9698
Style/EmptyElse:
@@ -99,6 +101,8 @@ Style/EmptyMethod:
99101
EnforcedStyle: expanded
100102
Style/FormatString:
101103
EnforcedStyle: percent
104+
Style/FormatStringToken:
105+
Enabled: false
102106
Style/FrozenStringLiteralComment:
103107
Enabled: false
104108
Style/GuardClause:
@@ -153,7 +157,7 @@ Style/TrailingCommaInArrayLiteral:
153157
EnforcedStyleForMultiline: comma
154158
Style/TrailingCommaInHashLiteral:
155159
EnforcedStyleForMultiline: comma
156-
Style/UnneededCondition:
160+
Style/RedundantCondition:
157161
Enabled: false
158162
Style/WhileUntilModifier:
159163
Enabled: false

.travis.yml

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,12 @@
11
language: ruby
22
before_install:
33
- "gem install bundler -v '~> 2.0'"
4+
- "gem install zeus"
5+
- "bundle config set deployment true"
6+
- "unset RAILS_ENV"
7+
install: "bundle install --jobs=3 --retry=3 --path=${BUNDLE_PATH:-vendor/bundle}"
8+
before_script:
9+
- "bin/start-dev --background --debug"
410
cache:
511
bundler: true
612
env:

CONTRIBUTING.md

Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -36,6 +36,26 @@ I may have suggestions about code style or your approach,
3636
but hopefully everything looks good and your changes get merged!
3737
Now you're a contributor! 🎉
3838

39+
## Speeding up the integration tests
40+
41+
The integration tests,
42+
located in `spec/integration`,
43+
can be quite slow to run.
44+
If you'd like to speed them up,
45+
run the following command in a separate tab:
46+
47+
```
48+
zeus start
49+
```
50+
51+
Now the next time you run an integration test by saying
52+
53+
```
54+
bin/rspec spec/integration/...
55+
```
56+
57+
it should run twice as fast.
58+
3959
## Understanding the codebase
4060

4161
If you want to make a change

Gemfile

Lines changed: 1 addition & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -9,10 +9,7 @@ gem "childprocess"
99
gem "pry-byebug", platform: :mri
1010
gem "pry-nav", platform: :jruby
1111
gem "rake"
12-
gem "rspec-core", github: "rspec/rspec-core"
13-
gem "rspec-expectations", github: "rspec/rspec-expectations"
14-
gem "rspec-mocks", github: "rspec/rspec-mocks"
15-
gem "rspec-support", github: "rspec/rspec-support"
12+
gem "rspec"
1613
gem "rubocop"
1714

1815
gemspec

bin/rspec

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -4,6 +4,6 @@ require_relative "../support/current_bundle"
44

55
current_bundle = SuperDiff::CurrentBundle.instance
66

7-
ENV["BUNDLE_GEMFILE"] ||= current_bundle.latest_appraisal.gemfile_path
7+
ENV["BUNDLE_GEMFILE"] ||= current_bundle.latest_appraisal.gemfile_path.to_s
88

99
exec("bundle", "exec", "rspec", *ARGV)

bin/start-dev

Lines changed: 79 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,79 @@
1+
#!/usr/bin/env bash
2+
3+
set -euo pipefail
4+
5+
success() {
6+
echo -e "\033[32m$@\033[0m"
7+
}
8+
9+
error() {
10+
echo -e "\033[31m$@\033[0m"
11+
}
12+
13+
print-help() {
14+
echo "USAGE: $0 [--debug]"
15+
echo "This script runs Zeus, for speeding up integration tests."
16+
}
17+
18+
debug=0
19+
background=0
20+
21+
while [[ ${1:-} ]]; do
22+
case "$1" in
23+
--debug)
24+
debug=1
25+
shift
26+
;;
27+
-h | --help)
28+
print-help
29+
exit
30+
;;
31+
--background)
32+
background=1
33+
shift
34+
;;
35+
*)
36+
error "Invalid option $1."
37+
print-help
38+
exit 1
39+
;;
40+
esac
41+
done
42+
43+
if [[ -S .zeus.sock ]]; then
44+
error "It looks like Zeus is already running."
45+
echo "Try running 'pkill zeus' to stop it, then run this again."
46+
exit 1
47+
fi
48+
49+
if [[ $background -eq 1 ]]; then
50+
echo -n "Starting Zeus..."
51+
52+
if [[ $debug -eq 1 ]]; then
53+
rm -f zeus.server.log
54+
zeus --log zeus.server.log start &>zeus.server-start.log &
55+
else
56+
zeus start &>zeus.server-start.log &
57+
fi
58+
pid="$!"
59+
echo " ($pid)"
60+
61+
echo "Confirming that Zeus is started..."
62+
63+
sleep 3
64+
65+
if [[ $debug -eq 1 ]]; then
66+
cat zeus.server-start.log
67+
if [[ -f zeus.server.log ]]; then
68+
cat zeus.server.log
69+
fi
70+
fi
71+
72+
exec zeus confirm_started
73+
else
74+
if [[ $debug -eq 1 ]]; then
75+
exec zeus --log zeus.server.log start
76+
else
77+
exec zeus start
78+
fi
79+
fi

config/zeus_plan.rb

Lines changed: 29 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,29 @@
1+
require "zeus"
2+
require "zeus/plan"
3+
require "forwardable"
4+
5+
require_relative "../support/test_plan"
6+
7+
class CustomZeusPlan < Zeus::Plan
8+
extend Forwardable
9+
10+
def_delegators(
11+
:@test_plan,
12+
:after_fork,
13+
:boot,
14+
:boot_active_record,
15+
:run_plain_test,
16+
:run_rspec_active_record_test,
17+
:run_rspec_rails_test,
18+
:confirm_started,
19+
)
20+
21+
def initialize(using_outside_of_zeus: false, color_enabled: false)
22+
@test_plan = TestPlan.new(
23+
using_outside_of_zeus: using_outside_of_zeus,
24+
color_enabled: color_enabled,
25+
)
26+
end
27+
end
28+
29+
Zeus.plan = CustomZeusPlan.new

gemfiles/no_rails.gemfile

Lines changed: 1 addition & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -7,10 +7,7 @@ gem "childprocess"
77
gem "pry-byebug", platform: :mri
88
gem "pry-nav", platform: :jruby
99
gem "rake"
10-
gem "rspec-core", git: "https://github.com/rspec/rspec-core"
11-
gem "rspec-expectations", git: "https://github.com/rspec/rspec-expectations"
12-
gem "rspec-mocks", git: "https://github.com/rspec/rspec-mocks"
13-
gem "rspec-support", git: "https://github.com/rspec/rspec-support"
10+
gem "rspec"
1411
gem "rubocop"
1512

1613
gemspec path: "../"

gemfiles/no_rails.gemfile.lock

Lines changed: 28 additions & 50 deletions
Original file line numberDiff line numberDiff line change
@@ -1,32 +1,3 @@
1-
GIT
2-
remote: https://github.com/rspec/rspec-core
3-
revision: 1174cc5e58ae72fb8cdc689a785089889bfdb740
4-
specs:
5-
rspec-core (3.10.0.pre)
6-
rspec-support (= 3.10.0.pre)
7-
8-
GIT
9-
remote: https://github.com/rspec/rspec-expectations
10-
revision: c085c5a6f0678ae7f88a489996e391e6b661d8a9
11-
specs:
12-
rspec-expectations (3.10.0.pre)
13-
diff-lcs (>= 1.2.0, < 2.0)
14-
rspec-support (= 3.10.0.pre)
15-
16-
GIT
17-
remote: https://github.com/rspec/rspec-mocks
18-
revision: e6057a61d8a15c4638dec116f68579d83fdd5ad5
19-
specs:
20-
rspec-mocks (3.10.0.pre)
21-
diff-lcs (>= 1.2.0, < 2.0)
22-
rspec-support (= 3.10.0.pre)
23-
24-
GIT
25-
remote: https://github.com/rspec/rspec-support
26-
revision: b83b4cecb3bb56300addae89cb0c437f1af906fb
27-
specs:
28-
rspec-support (3.10.0.pre)
29-
301
PATH
312
remote: ..
323
specs:
@@ -43,37 +14,50 @@ GEM
4314
rake
4415
thor (>= 0.14.0)
4516
ast (2.4.0)
46-
attr_extras (6.2.2)
47-
byebug (11.0.1)
17+
attr_extras (6.2.3)
18+
byebug (11.1.3)
4819
childprocess (3.0.0)
4920
coderay (1.1.2)
5021
diff-lcs (1.3)
51-
jaro_winkler (1.5.4)
52-
method_source (0.9.2)
22+
method_source (1.0.0)
5323
parallel (1.19.1)
54-
parser (2.7.0.2)
24+
parser (2.7.1.2)
5525
ast (~> 2.4.0)
5626
patience_diff (1.1.0)
5727
trollop (~> 1.16)
58-
pry (0.12.2)
59-
coderay (~> 1.1.0)
60-
method_source (~> 0.9.0)
61-
pry-byebug (3.7.0)
28+
pry (0.13.1)
29+
coderay (~> 1.1)
30+
method_source (~> 1.0)
31+
pry-byebug (3.9.0)
6232
byebug (~> 11.0)
63-
pry (~> 0.10)
33+
pry (~> 0.13.0)
6434
rainbow (3.0.0)
6535
rake (13.0.1)
66-
rubocop (0.79.0)
67-
jaro_winkler (~> 1.5.1)
36+
rexml (3.2.4)
37+
rspec (3.9.0)
38+
rspec-core (~> 3.9.0)
39+
rspec-expectations (~> 3.9.0)
40+
rspec-mocks (~> 3.9.0)
41+
rspec-core (3.9.2)
42+
rspec-support (~> 3.9.3)
43+
rspec-expectations (3.9.2)
44+
diff-lcs (>= 1.2.0, < 2.0)
45+
rspec-support (~> 3.9.0)
46+
rspec-mocks (3.9.1)
47+
diff-lcs (>= 1.2.0, < 2.0)
48+
rspec-support (~> 3.9.0)
49+
rspec-support (3.9.3)
50+
rubocop (0.83.0)
6851
parallel (~> 1.10)
6952
parser (>= 2.7.0.1)
7053
rainbow (>= 2.2.2, < 4.0)
54+
rexml
7155
ruby-progressbar (~> 1.7)
72-
unicode-display_width (>= 1.4.0, < 1.7)
56+
unicode-display_width (>= 1.4.0, < 2.0)
7357
ruby-progressbar (1.10.1)
7458
thor (1.0.1)
7559
trollop (1.16.2)
76-
unicode-display_width (1.6.0)
60+
unicode-display_width (1.7.0)
7761

7862
PLATFORMS
7963
ruby
@@ -84,15 +68,9 @@ DEPENDENCIES
8468
pry-byebug
8569
pry-nav
8670
rake
87-
rspec-core!
88-
rspec-expectations!
89-
rspec-mocks!
90-
rspec-support!
71+
rspec
9172
rubocop
9273
super_diff!
9374

94-
RUBY VERSION
95-
ruby 2.7.1p83
96-
9775
BUNDLED WITH
9876
2.1.4

0 commit comments

Comments
 (0)