Skip to content

Commit 6ff7e64

Browse files
committed
RF: use new cached append for multi-seq read
Use the new cached append to replace the coroutines for the read across multiple array sequences in ``create_arraysequences_from_generator``. Before this change (using coroutines): ``` Old: Loaded 5,000 streamlines in 2.85 New: Loaded 5,000 streamlines in 3.6 Speedup of 0.79 Old: Loaded 5,000 streamlines with scalars in 5.16 New: Loaded 5,000 streamlines with scalars in 7.13 Speedup of 0.723703 ``` After this change (using cached append): ``` Old: Loaded 5,000 streamlines in 3.21 New: Loaded 5,000 streamlines in 3.9 Speedup of 0.82 Old: Loaded 5,000 streamlines with scalars in 5.21 New: Loaded 5,000 streamlines with scalars in 7.16 Speedup of 0.727654 ``` This seems to be well within run-to-run measurement error.
1 parent e3b4db5 commit 6ff7e64

File tree

1 file changed

+4
-10
lines changed

1 file changed

+4
-10
lines changed

nibabel/streamlines/array_sequence.py

Lines changed: 4 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -444,17 +444,11 @@ def create_arraysequences_from_generator(gen, n):
444444
Number of :class:`ArraySequences` object to create.
445445
"""
446446
seqs = [ArraySequence() for _ in range(n)]
447-
coroutines = [seq._extend_using_coroutine() for seq in seqs]
448-
449-
for coroutine in coroutines:
450-
coroutine.send(None)
451-
452447
for data in gen:
453-
for i, coroutine in enumerate(coroutines):
448+
for i, seq in enumerate(seqs):
454449
if data[i].nbytes > 0:
455-
coroutine.send(data[i])
456-
457-
for coroutine in coroutines:
458-
coroutine.close()
450+
seq.append(data[i], cache_build=True)
459451

452+
for seq in seqs:
453+
seq.finalize_append()
460454
return seqs

0 commit comments

Comments
 (0)