Skip to content

Commit 857f9a1

Browse files
committed
(PUP-11755) Avoid slow, buggy pattern for enum functions
The each/filter/map functions, for some dispatches, use patterns that are slow and/or buggy when run on jruby. The correct pattern for enumeration is ``` object.each do |value| yield(value) end ``` Two alternatives in use are ``` enum = object.each object.size.times do yield(enum.next) end enum = object.each begin loop do yield(enum.next) end rescue StopIteration end ``` The first pattern is incorrect because it never triggers StopIteration on the enumerator (by attempting to call `next` one last time). On MRI this is apparently harmless, but on JRuby it leaves the Fiber associated with the Enumerator around. Since Fibers are implemented via native threads on JRuby, that leaves around a native thread. That can deplete the available threads and interferes with garbage collection, as every native thread functions as a GC root (leaving objects referenced but functionally inaccessible). The second pattern is technically correct but extremely inefficient on JRuby, presumably due to the overhead associated with using a Fiber for enumeration. It's also no more correct than simply calling the enum method directly, so we can simply change it. In benchmarks, the difference on MRI is ~10x and on JRuby it's ~150x.
1 parent fd50746 commit 857f9a1

File tree

4 files changed

+26
-38
lines changed

4 files changed

+26
-38
lines changed

lib/puppet/functions/each.rb

Lines changed: 11 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -116,10 +116,9 @@
116116
end
117117

118118
def foreach_Hash_1(hash)
119-
enumerator = hash.each_pair
120119
begin
121-
hash.size.times do
122-
yield(enumerator.next)
120+
hash.each_pair do |pair|
121+
yield(pair)
123122
end
124123
rescue StopIteration
125124
end
@@ -128,10 +127,9 @@ def foreach_Hash_1(hash)
128127
end
129128

130129
def foreach_Hash_2(hash)
131-
enumerator = hash.each_pair
132130
begin
133-
hash.size.times do
134-
yield(*enumerator.next)
131+
hash.each_pair do |pair|
132+
yield(*pair)
135133
end
136134
rescue StopIteration
137135
end
@@ -141,10 +139,12 @@ def foreach_Hash_2(hash)
141139

142140
def foreach_Enumerable_1(enumerable)
143141
enum = Puppet::Pops::Types::Iterable.asserted_iterable(self, enumerable)
144-
begin
145-
loop { yield(enum.next) }
146-
rescue StopIteration
142+
begin
143+
enum.each do |value|
144+
yield value
147145
end
146+
rescue StopIteration
147+
end
148148
# produces the receiver
149149
enumerable
150150
end
@@ -155,10 +155,8 @@ def foreach_Enumerable_2(enumerable)
155155
enum.each { |entry| yield(*entry) }
156156
else
157157
begin
158-
index = 0
159-
loop do
160-
yield(index, enum.next)
161-
index += 1
158+
enum.each_with_index do |value, index|
159+
yield(index, value)
162160
end
163161
rescue StopIteration
164162
end

lib/puppet/functions/filter.rb

Lines changed: 5 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -109,11 +109,8 @@ def filter_Enumerable_1(enumerable)
109109
result = []
110110
enum = Puppet::Pops::Types::Iterable.asserted_iterable(self, enumerable)
111111
begin
112-
loop do
113-
it = enum.next
114-
if yield(it)
115-
result << it
116-
end
112+
enum.each do |value|
113+
result << value if yield(value)
117114
end
118115
rescue StopIteration
119116
end
@@ -124,18 +121,13 @@ def filter_Enumerable_2(enumerable)
124121
enum = Puppet::Pops::Types::Iterable.asserted_iterable(self, enumerable)
125122
if enum.hash_style?
126123
result = {}
127-
enum.each { |k, v| result[k] = v if yield(k, v) }
124+
enum.each {| k, v| result[k] = v if yield(k, v) }
128125
result
129126
else
130127
result = []
131128
begin
132-
index = 0
133-
loop do
134-
it = enum.next
135-
if yield(index, it)
136-
result << it
137-
end
138-
index += 1
129+
enum.each_with_index do |value, index|
130+
result << value if yield(index, value)
139131
end
140132
rescue StopIteration
141133
end

lib/puppet/functions/map.rb

Lines changed: 7 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -90,7 +90,7 @@
9090
def map_Hash_1(hash)
9191
result = []
9292
begin
93-
hash.map {|x, y| result << yield([x, y]) }
93+
hash.each {|x, y| result << yield([x, y]) }
9494
rescue StopIteration
9595
end
9696
result
@@ -99,7 +99,7 @@ def map_Hash_1(hash)
9999
def map_Hash_2(hash)
100100
result = []
101101
begin
102-
hash.map {|x, y| result << yield(x, y) }
102+
hash.each {|x, y| result << yield(x, y) }
103103
rescue StopIteration
104104
end
105105
result
@@ -109,7 +109,9 @@ def map_Enumerable_1(enumerable)
109109
result = []
110110
enum = Puppet::Pops::Types::Iterable.asserted_iterable(self, enumerable)
111111
begin
112-
loop { result << yield(enum.next) }
112+
enum.each do |val|
113+
result << yield(val)
114+
end
113115
rescue StopIteration
114116
end
115117
result
@@ -122,14 +124,11 @@ def map_Enumerable_2(enumerable)
122124
else
123125
result = []
124126
begin
125-
index = 0
126-
loop do
127-
result << yield(index, enum.next)
128-
index = index + 1
127+
enum.each_with_index do |val, index|
128+
result << yield(index, val)
129129
end
130130
rescue StopIteration
131131
end
132-
133132
result
134133
end
135134
end

lib/puppet/pops/lookup/context.rb

Lines changed: 3 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -92,12 +92,11 @@ def cached_value(key)
9292

9393
def cached_entries(&block)
9494
if block_given?
95-
enumerator = @cache.each_pair
96-
@cache.size.times do
95+
@cache.each_pair do |pair|
9796
if block.arity == 2
98-
yield(*enumerator.next)
97+
yield(*pair)
9998
else
100-
yield(enumerator.next)
99+
yield(pair)
101100
end
102101
end
103102
nil

0 commit comments

Comments
 (0)