-
Notifications
You must be signed in to change notification settings - Fork 8
Open
Description
One may write pseudocode for foldxt
and foldxd
as something like:
function foldxt(step, xf, xs)
parts = split_into_parts(xs)
output = init_output()
@threads for part ∈ parts
insert!(output, foldxl(step, xf, part))
end
combine(xf, output)
end
This is fairly intuitive, however currently Transducers.jl doesn't really expose any of these intermediate steps, and the internals of it are such that it might not be all that obvious what's going on until close inspection (at least it wasn't to me).
I propose that we create a public API for each of these pieces such that a user could easily write its own foldxt
. As I see it, some benefits of this would be:
- The semantics of
foldxt
might be somewhat more obvious. This is important because thecombine
step can cause it to behave quite differently fromfoldxl
. - Users could create custom
combine
. Currently users have to rely on Transducers to providecombine
functionality. - Users could more easily create custom splitting strategies.
On the other hand, exposing these details could make it significantly easier to write unsafe code, though note that a custom combine
needn't rely on parallelism at all.
Metadata
Metadata
Assignees
Labels
No labels