Skip to content

Conversation

@frankmcsherry
Copy link
Member

This PR demonstrates the potential use of columnar containers, which use the crate of the same name to represent data. It's mostly a demonstration, and isn't meant to be merged.

cc: @antiguru

@frankmcsherry frankmcsherry force-pushed the columnar_demo branch 2 times, most recently from 5e1d128 to cc7db71 Compare September 20, 2024 23:18
@frankmcsherry frankmcsherry force-pushed the columnar_demo branch 2 times, most recently from 6e30f16 to 3bbb127 Compare November 25, 2024 00:26
impl<T: Timestamp, C, H: 'static> ParallelizationContract<T, C> for ExchangeCore<C, H>
where
C: ExchangeData + PushPartitioned + crate::dataflow::channels::ContainerBytes,
C: Data + Send + PushPartitioned + crate::dataflow::channels::ContainerBytes,
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removes the requirement that containers implement Serializable + for<'a> Deserializable<'a>.

@frankmcsherry frankmcsherry marked this pull request as ready for review November 27, 2024 02:13
@frankmcsherry
Copy link
Member Author

This is not complete yet, but other than the timely change relaxing ExchangeCore constraints, it is all localized to examples/columnar.rs. Merging on that basis!

@frankmcsherry frankmcsherry merged commit 97ac85e into TimelyDataflow:master Nov 27, 2024
7 checks passed
@github-actions github-actions bot mentioned this pull request Nov 25, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant