Releases: TimelyDataflow/timely-dataflow
Releases · TimelyDataflow/timely-dataflow
timely_logging-v0.24.0
chore: Release package timely_logging version 0.24.0
timely_container-v0.24.0
chore: Release package timely_container version 0.24.0
timely_communication-v0.24.0
chore: Release package timely_communication version 0.24.0
timely_bytes-v0.24.0
chore: Release package timely_bytes version 0.24.0
timely-v0.24.0
This version of Timely has some exciting new features.
- The
Distributortrait offers a generalization of theExchangetype. It allows users to define custom distribution
strategies for routing data across workers. TheExchangetype requires a row-by-row distribution and uses
container builders to form batches of data, which limits its flexibility. TheDistributortrait, on the other hand,
allows users to define their own behavior for distributing data, and does not need to work row-by-row.
It only needs to uphold progress tracking requirements, which are that the amount of records on the input
matches those on the output. - We split the
Containertrait into multiple traits. TheAccountabletrait indicates the number of records
in a container for progress tracking and is the only required trait for a container.
Theiteranddrainfunctions are part of the newIterContainerandDrainContainertraits.
We do not require all containers to provide all of these traits, and it is mostly high-level operators that require
a container to implement them. We alos removed theclearfunction from containers as it
is not well-defined in all cases, for example for immutable data. - The
Maptrait offers aflat_map_builderfunction to accumulate flatmap logic.
Other
timely_logging-v0.23.0
chore: Release package timely_logging version 0.23.0
timely_container-v0.23.0
chore: Release package timely_container version 0.23.0
timely_communication-v0.23.0
chore: Release package timely_communication version 0.23.0
timely_bytes-v0.23.0
chore: Release package timely_bytes version 0.23.0
timely-v0.23.0
Other
- Remove some quadratic behavior from propagate_all
- Remove T, C parameters from LogPuller/Pusher