Skip to content

Commit 1cd4d2a

Browse files
authored
Restart workers periodically to prevent memory accumulation (#4666)
I'm pretty sure the reference test generation working functions are accumulating memory and this is why the runner is running out of memory. I tested locally and found that this did make a difference. I will merge this & see if it solves our issue. I think switching to pytest might solve the underlying issue, not sure. I believe this issue popped up now because etan's PR added many new SSZ tests and that pushed it over the limit.
1 parent 38ec2b2 commit 1cd4d2a

File tree

1 file changed

+2
-1
lines changed

1 file changed

+2
-1
lines changed

tests/core/pyspec/eth2spec/gen_helpers/gen_base/gen_runner.py

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -284,7 +284,8 @@ def display_active_tests(active_tests, total_tasks, completed, skipped, width):
284284
skipped.value += 1
285285
completed.value += 1
286286
else:
287-
pool = Pool(processes=args.threads)
287+
# Restart workers periodically to prevent memory accumulation
288+
pool = Pool(processes=args.threads, maxtasksperchild=100)
288289
try:
289290
for result in pool.uimap(worker_function, inputs):
290291
if result == "skipped":

0 commit comments

Comments
 (0)