Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
588 commits
Select commit Hold shift + click to select a range
5a17779
Book edits, fix #183 (#190)
jhellerstein Sep 19, 2022
8b68c64
Update `recv_stream` to handle all `Stream`s instead of just `tokio::…
MingweiSamuel Sep 19, 2022
4936198
Enable partitioned output on hydroflow_parser!
MingweiSamuel Sep 27, 2022
7ba4321
Add surface diamond tests
MingweiSamuel Sep 27, 2022
3559fbf
Surface syntax fix handling of wildcard linear chains which might cau…
MingweiSamuel Sep 27, 2022
3a0ab8a
Fix handling of empty `merge()`/`tee()`, add tests
MingweiSamuel Sep 27, 2022
302b213
Remove internal runtime, use tokio::spawn mechanism (requires tokio c…
MingweiSamuel Sep 23, 2022
9362edf
Add tcp echo example test
MingweiSamuel Sep 23, 2022
666d14e
Rename `send_async` -> `write_async` to match trait names
MingweiSamuel Sep 26, 2022
4a36e1b
Restructor operator constraints into single `write_fn`
MingweiSamuel Sep 26, 2022
bba512f
Emit better, more local, more useful error messages in surface syntax
MingweiSamuel Sep 27, 2022
19424cf
Implement and add test for `sink_async`
MingweiSamuel Sep 27, 2022
1bf8642
Add `#[allow(clippy::map_identity)]` to some tests
MingweiSamuel Sep 27, 2022
c2fc4b2
mechanical merge of fork (#197)
jhellerstein Sep 29, 2022
c9529c5
Add UDP networking & test
MingweiSamuel Sep 28, 2022
84d7406
Remove awkward testing side-effects from surface syntax async tests
MingweiSamuel Sep 29, 2022
dfc22e8
Add assertiosn to surface tcp test
MingweiSamuel Sep 29, 2022
c241c05
Fix surface syntax port ordering bug
MingweiSamuel Oct 3, 2022
83ab8a5
Use Rust Sitter release from crates.io (#204)
shadaj Oct 3, 2022
fa20c12
Fix #201 `run_async()`, wait when ALL strata have no work
MingweiSamuel Oct 3, 2022
7854971
Add apache-2.0 license
MingweiSamuel Oct 3, 2022
09953f7
Add type guards, better spans to surface syntax codegen
MingweiSamuel Oct 3, 2022
6d9616e
Update datalog snapshots
MingweiSamuel Oct 4, 2022
c526f9a
Add cross join surface syntax operator, update tests, fix #200 (#211)
MingweiSamuel Oct 14, 2022
7797c6c
Implement `inspect()` surface syntax operator, fix #208
MingweiSamuel Oct 13, 2022
f802b95
add flatten op to surface syntax (#213)
jhellerstein Oct 18, 2022
01ff0c6
start porting examples to surface syntax (#207)
jhellerstein Oct 19, 2022
ea99408
use sequential numbers to index tee output
jhellerstein Oct 20, 2022
f141609
Fix example graph printing (#222)
MingweiSamuel Oct 21, 2022
7b75f5e
Remove surface API, fix #224
MingweiSamuel Oct 26, 2022
625e9a0
Pick random ports for udp/tcp tests to prevent hanging
MingweiSamuel Oct 26, 2022
af550ca
Remove Surface API from book
MingweiSamuel Oct 26, 2022
bd8313c
Add parsing of named ports (WIP, compiling)
MingweiSamuel Oct 26, 2022
879e977
Implement named ports in operators
MingweiSamuel Oct 28, 2022
2346dc8
Update dependencies for `cargo audit`
MingweiSamuel Nov 2, 2022
adbf8e1
Add cargo audit to gh-actions
MingweiSamuel Nov 2, 2022
008425b
Refactor out surface syntax diagnostics (error messages)
MingweiSamuel Nov 2, 2022
b8394d8
Add testing of surface syntax errors (and warnings) (#230)
MingweiSamuel Nov 4, 2022
3b7c998
Move type list code into `type_list` subpackage (#231)
MingweiSamuel Nov 4, 2022
b235746
Add surface graph snapshot tests for datalog. (#223)
tylerhou Nov 7, 2022
016b971
Add split and switch pusherators (#233)
MingweiSamuel Nov 10, 2022
b90c546
deadlock detector example (#234)
jhellerstein Nov 15, 2022
39764c5
remove FlowGraph and related graphing facilities (#235)
jhellerstein Nov 15, 2022
e3e8db2
add unique operator to remove duplicates (#236)
jhellerstein Nov 16, 2022
e52054f
Allow chat clients to connect to non-localhost servers (#238)
alexras Nov 16, 2022
2230af8
Add a driver for the chat example (#237)
alexras Nov 16, 2022
c85a19d
first version of groupby with test and example (#239)
jhellerstein Nov 17, 2022
3b79280
Extract parts of `expand_join_plan` into new functions. (#232)
tylerhou Nov 17, 2022
5d56aaf
Implement `hydroflow_internalmacro::operator_docgen` for surface op d…
MingweiSamuel Nov 18, 2022
694571b
Move operators into individual files, use `#[hydroflow_internalmacro:…
MingweiSamuel Nov 18, 2022
a5de404
Add `hydroflow_macr/build.rs` to autogen operator book docs
MingweiSamuel Nov 18, 2022
26e4cfe
book docs for ops (#245)
jhellerstein Nov 21, 2022
e6bb7d1
Multiplatform example builds (#241)
alexras Nov 22, 2022
851d97d
Separate surface doctests by operator
MingweiSamuel Nov 22, 2022
43e32ee
Add/update more operator docs
MingweiSamuel Nov 22, 2022
f5d141d
Fix for new clippy lifetime lint
MingweiSamuel Nov 23, 2022
05a05bb
Limit `null()` to up to one input and/or output. (#250)
MingweiSamuel Nov 23, 2022
bf5e7dd
Refining the examples container's definition (#248)
alexras Nov 23, 2022
dd3d9e1
Allow the chat example to bind and connect to DNS names (#249)
alexras Nov 23, 2022
4182371
Update book references to surface_ops (#256)
zzlk Nov 28, 2022
6515bbd
Add `mdbook-linkcheck` to prevent dead book links
MingweiSamuel Nov 28, 2022
e4bb6dd
Update mdbook dependency versions
MingweiSamuel Nov 28, 2022
978e282
CI workflow for multiplatform example container builds (#252)
alexras Nov 29, 2022
458aa0b
initial kvs example (#262)
jhellerstein Dec 2, 2022
d07e5c1
Add demux operator (#261)
MingweiSamuel Dec 2, 2022
17be5dd
Allow `clippy::uninlined-format-args` in `.cargo/config.toml`
MingweiSamuel Dec 4, 2022
45f76a8
use demux rather than tee and filter_map (#268)
jhellerstein Dec 5, 2022
7341f87
Add operator-specific diagnostics, use in `demux(..)`, fix #265 (#278)
MingweiSamuel Dec 6, 2022
a98c745
improvements to book (#277)
jhellerstein Dec 6, 2022
9d9a7ec
Echo Server example (#279)
jhellerstein Dec 6, 2022
1534e1a
Clarify handling of ports/`Ends` naming in `FlatGraph`
MingweiSamuel Dec 8, 2022
696eb32
More `indices` -> `ports` renames
MingweiSamuel Dec 8, 2022
7db1357
Disallow overwriting names in surface syntax (preps for #158)
MingweiSamuel Dec 8, 2022
a7933eb
Cargo.lock update
MingweiSamuel Dec 8, 2022
57403cc
simplify boilerplate with new helpers, ops (#282)
jhellerstein Dec 9, 2022
d9541b2
Re-export `serde`, `serde_json`
MingweiSamuel Dec 9, 2022
b7074eb
Rename source and dest surface syntax operators, fix #216 #276 (#284)
MingweiSamuel Dec 9, 2022
293c37c
Rename split->unzip, implement surface op (#259)
MingweiSamuel Dec 9, 2022
9b29c6f
Update README.md
jhellerstein Dec 9, 2022
36708ab
`demux` use `Pusherator` automatically, fix #267 (#285)
MingweiSamuel Dec 10, 2022
d7dcd88
book edits: echoserver example (#288)
jhellerstein Dec 13, 2022
50e7361
Rename pkg `type_list` -> `variadics`
MingweiSamuel Dec 12, 2022
91d37b0
Rename variadics/tuple_list macros
MingweiSamuel Dec 13, 2022
419bc30
add chat server example to book (#294)
jhellerstein Dec 14, 2022
96f1481
explicit serde example, resolve #214 (#295)
jhellerstein Dec 14, 2022
a2316df
make ipv4_resolve return a Result for use in clap (#296)
jhellerstein Dec 14, 2022
05c990f
Add example usage code to `dest_sink`, `dest_asyncwrite`, #216
MingweiSamuel Dec 12, 2022
5418ea4
Remove `dest_asyncwrite`, consolidate using codecs, now in `hydroflow…
MingweiSamuel Dec 14, 2022
32fddfe
Rename `recv_into` -> `collect_ready`
MingweiSamuel Dec 14, 2022
e6b1ec5
better names/structure for serde helper functions, get UdpSocket back…
jhellerstein Dec 15, 2022
328026d
align book with cleaned up examples for echo and chat (#300)
jhellerstein Dec 15, 2022
57be9a2
Format `hydroflow_datalog` snaps w/ `prettyplease` (#302)
MingweiSamuel Dec 15, 2022
b5d6f60
add sort_by, rename groupby to group_by (#301)
jhellerstein Dec 16, 2022
58ef72d
Add more "badtypes" compile-fail tests for upcoming changes #263
MingweiSamuel Dec 14, 2022
6413fa4
Improve `Iterator`/`Pusherator` typeguards by erasing types, using lo…
MingweiSamuel Dec 14, 2022
53e62cd
Improve spanning of handoffs #263
MingweiSamuel Dec 16, 2022
58668bd
Improve spanning of write context `make_ident(..)` #263
MingweiSamuel Dec 16, 2022
3ee9d33
Add very good type guard to `join` op #263
MingweiSamuel Dec 14, 2022
c6510da
Emit type guards inline, configurable #263
MingweiSamuel Dec 15, 2022
6db3f60
Add type guard to `merge` #263
MingweiSamuel Dec 15, 2022
c215e8c
Add type guard before `Pivot` #263
MingweiSamuel Dec 15, 2022
6aa4d41
Simplify `dest_sink`, add type guards #263
MingweiSamuel Dec 15, 2022
3fcfb46
Add type guard to `group_by` #263
MingweiSamuel Dec 16, 2022
496a7a1
Type guard for `source_iter`, `repeat_iter` #263
MingweiSamuel Dec 16, 2022
a44f08c
Ingore `surface_dest_sink_baditem` due to `rustc` inconsistency
MingweiSamuel Dec 16, 2022
d4be35b
Fix rare bug in `dest_sink` doctest
MingweiSamuel Dec 16, 2022
552264c
add `cargo-generate` to README.md
jhellerstein Dec 17, 2022
77b37b5
Update README.md
jhellerstein Dec 17, 2022
91cddfd
Ignore inconsistent compile-fail tests (nix vs windows full paths)
MingweiSamuel Dec 19, 2022
5de534b
use `cargo generate` template in book (#307)
jhellerstein Dec 21, 2022
1bc6134
Todo items for book (#308)
jhellerstein Dec 21, 2022
f4ad527
`epoch` --> `tick` replace (#309)
jhellerstein Dec 21, 2022
8193409
sort ops while generating the op doc in the book (#310)
jhellerstein Dec 21, 2022
2cbd3e7
better autogen of input/output specs for ops docs (#311)
jhellerstein Dec 22, 2022
29e4398
redo docs gen and docs PR (#315, #316, #317)
jhellerstein Dec 23, 2022
7d1cb83
Rearrange README.md
MingweiSamuel Dec 23, 2022
9530fd7
reintroduce code to generate streaming/blocking annotations in docs (…
jhellerstein Dec 23, 2022
68cc316
additions to book (#319)
jhellerstein Dec 24, 2022
f2ee139
better mermaid graphs (#320)
jhellerstein Dec 27, 2022
876fb31
better graphs for both mermaid and dot (#321)
jhellerstein Dec 29, 2022
3e788ad
update mermaid diagrams in book (#322)
jhellerstein Dec 29, 2022
501aeba
cleanup and reorg book (#323)
jhellerstein Dec 30, 2022
49271ec
Update example_7_echo_server.md
jhellerstein Dec 31, 2022
d4b5547
Update example_8_chat_server.md
jhellerstein Dec 31, 2022
2886647
Update example_8_chat_server.md
jhellerstein Dec 31, 2022
15259a4
Update example_8_chat_server.md
jhellerstein Dec 31, 2022
57600a4
Update surface_flows.md
jhellerstein Dec 31, 2022
2a2e8ff
Update surface_data.md
jhellerstein Dec 31, 2022
47b2941
Add persistence lifetimes to join #272
MingweiSamuel Dec 7, 2022
df13190
Add persistence spec to `group_by` #272
MingweiSamuel Dec 23, 2022
75f152e
Add generic type arguments for `group_by` when inference fails #272
MingweiSamuel Dec 24, 2022
274b77f
CI test benchmarks with [ci-bench] in PR title or body
MingweiSamuel Jan 5, 2023
4c811e4
update criterion, add html_reports feature as suggested by criterion …
zzlk Jan 5, 2023
8b5faa7
Implement `'tick`/`'static` lifetimes for `cross_join` #272 (#334)
MingweiSamuel Jan 6, 2023
ffacc15
Implement `'tick`/`'static` for `sort()` #272 (#337)
MingweiSamuel Jan 6, 2023
fe79c6d
implement `'tick`/`'static` for `unique()` #272 (#338)
MingweiSamuel Jan 6, 2023
e9049ad
CI create gh-pages branch if it doesn't exist
MingweiSamuel Jan 9, 2023
3905cf2
Add surface versions of existing micro benchmarks (#341)
zzlk Jan 12, 2023
06ec89e
Micro benchmarks (#343)
zzlk Jan 12, 2023
ce071d4
text crdt example: RGA (#344)
jhellerstein Jan 13, 2023
d8d833c
Fix `run_tick()` semantics, fix `unique`'s `'static` (#350)
MingweiSamuel Jan 17, 2023
861fd94
Add `is_external_input` field to `OperatorConstraints`
MingweiSamuel Jan 20, 2023
9a746a0
Add post-partitioning step to break source operators into stratum 0, …
MingweiSamuel Jan 23, 2023
22704cd
symmetric hash join improvements (#358)
zzlk Jan 25, 2023
5ca4bd3
add cargo profile for code profiling (#359)
zzlk Jan 25, 2023
e7acecc
Preserve varnames info, display in mermaid, fix #327 (#360)
MingweiSamuel Jan 30, 2023
51e5e92
Update trybuild stderr output for new nightly (#369)
MingweiSamuel Feb 1, 2023
946d1a2
Get `hydroflow_lang` to compile on WASM (#331)
shadaj Feb 1, 2023
cdc83b6
Ops specify persistence/type arg counts, handle separately in `partit…
MingweiSamuel Jan 31, 2023
8431060
Replace old references to `'epoch` with `'static`
MingweiSamuel Feb 1, 2023
bef2435
Get Datalog compiler to build on WASM (#371)
shadaj Feb 1, 2023
c259bea
Document surface syntax `context` object, cleanup internal usage (#363)
MingweiSamuel Feb 1, 2023
1283da5
Add persistence lifetimes to `fold`
MingweiSamuel Jan 31, 2023
050cadf
Add persistence lifetimes to `reduce`
MingweiSamuel Jan 31, 2023
403e5e0
Pin Rust toolchain version to nightly-2023-01-31 (#370)
tylerhou Feb 1, 2023
7462cc3
Add filtering expressions to Dedalus rules. #178 (#209)
tylerhou Feb 2, 2023
efd6733
Provide `&mut Context` alongside handoffs to subgraphs, surface synta…
MingweiSamuel Feb 6, 2023
5f58f31
Support Dedalus rules that send results to the next tick (#374)
shadaj Feb 7, 2023
962d903
Joins in Dedalus should only have the 'tick lifetime (#376)
shadaj Feb 8, 2023
eb7ff56
Only receive external events at the start of a tick, before stratum 0
MingweiSamuel Feb 8, 2023
13a51e5
Remove unnecessary `mut` from `repeat_iter`
MingweiSamuel Feb 8, 2023
e5f46df
`repeat_iter` now repeats via self-scheduling #143 #364
MingweiSamuel Feb 8, 2023
a2078f7
`fold::<'static>` now replays #143 #364
MingweiSamuel Feb 8, 2023
bc3d12f
`reduce::<'static>` now replays #143 #364
MingweiSamuel Feb 8, 2023
62fcfb1
`group_by::<'static>` now replays #143 #364
MingweiSamuel Feb 8, 2023
f77a0a4
Add timeouts to CI (#378)
MingweiSamuel Feb 8, 2023
546b9e0
Fix `run_async()` not yielding with replay (stateful) operators (#381)
MingweiSamuel Feb 10, 2023
da1047c
fixup! Add `DiMulGraph`, use in `FlatGraph` (not compiling)
MingweiSamuel Feb 11, 2023
54bcbaa
Add `anti_join` operator (#382)
shadaj Feb 13, 2023
9d17b4d
Allow alias name assignment without any arrow in surface syntax, clos…
MingweiSamuel Feb 14, 2023
dc870e8
Add support for negated relations to Dedalus (#388)
shadaj Feb 15, 2023
b00c909
Add surface syntax syntactical error tests
MingweiSamuel Feb 15, 2023
bfe9a90
Improve parsing handling/error messages
MingweiSamuel Feb 16, 2023
52aa6e0
Introduce initial Hydro CLI architecture (#390)
shadaj Feb 17, 2023
572a43c
Don't build the `hydro_cli` crate when preparing Docker images (#394)
shadaj Feb 18, 2023
8cc479e
Implement forward name references in surface syntax, closes #158
MingweiSamuel Feb 11, 2023
398cff6
Update examples to use forward name references
MingweiSamuel Feb 18, 2023
9dd3bd9
Refactor `FlatGraph` assembly into separate `FlatGraphBuilder`
MingweiSamuel Feb 18, 2023
4113ec5
placeholder commit for adding more tests
MingweiSamuel Feb 19, 2023
00d5f63
Detect name cycles sooner, memoize resolution, better error messages
MingweiSamuel Feb 19, 2023
6570401
Refactor `OpConstraints` fns, use lookup helper (#399)
MingweiSamuel Feb 22, 2023
8406905
fixup! Update examples to use forward name references
MingweiSamuel Feb 22, 2023
608e65b
Fix op docs "blocking" to check elided port names, fix #400 (#404)
MingweiSamuel Feb 22, 2023
f80754e
Implement simple aggregations for Dedalus (#403)
shadaj Feb 23, 2023
89cab62
Make `DiMulGraph` fields private for encapsulation (#405)
MingweiSamuel Feb 23, 2023
dbdad61
Add basic support for connecting services with Unix/TCP sockets (#397)
shadaj Feb 24, 2023
74979ca
Drop unnecessary merge() and tee() in CLI examples (#409)
shadaj Feb 24, 2023
644980b
Add more generics compile-fail tests
MingweiSamuel Feb 23, 2023
c883fd4
Build `OperatorInstance` data in `FlatGraph`
MingweiSamuel Feb 23, 2023
215f74a
Comment out inconsistent compile-fail test
MingweiSamuel Feb 24, 2023
fff4d0a
Additional cleanups for PR #407
MingweiSamuel Feb 24, 2023
5dac7e4
Fix non-unix (windows) build referencing unix sockets (#411)
MingweiSamuel Feb 27, 2023
3d46bde
Implement async rules in Dedalus by deferring to external sender (#413)
shadaj Feb 27, 2023
f10a54f
Initial support for GCP deployments (#410)
shadaj Feb 27, 2023
9ead3f7
Add monotonicity properties to operators (currently unused) (#412)
MingweiSamuel Feb 28, 2023
8695b5d
fixup! Initial support for GCP deployments (#410)
MingweiSamuel Feb 28, 2023
4be709f
Update clap (#420)
MingweiSamuel Feb 28, 2023
24354d2
Get hydroflow to compile to WASM (#329)
tylerhou Feb 22, 2023
d91bd8f
Remove errant dep info file (#423)
tylerhou Mar 1, 2023
b757201
Bump pinned cargo toolchain version (#429)
tylerhou Mar 2, 2023
2fb8871
Add API for defining custom services in deployment (#417)
shadaj Mar 2, 2023
d29bc63
Turn on WASM tests
tylerhou Feb 27, 2023
394d182
Go back to `profile.release` `opt-level = 3` (#427)
MingweiSamuel Mar 2, 2023
2eec0da
Add hydroflow WASM tests to CI
tylerhou Mar 1, 2023
6f0c29a
Fix `FlatGraph::write_surface_syntax` (#425)
MingweiSamuel Mar 2, 2023
68f9bde
Make `unique()` streaming and dedup Dedalus facts (#431)
shadaj Mar 2, 2023
fceaea5
Encapsulate `FlatGraph`, separate `FlatGraphBuilder` (#419)
MingweiSamuel Mar 6, 2023
b952257
Package CLI as a Python wheel to simplify distribution (#433)
shadaj Mar 6, 2023
2a6ddd5
Add `try_build` to flat graph to expose diagnostics (#434)
shadaj Mar 7, 2023
4853c0f
vector clock example (#435)
jhellerstein Mar 7, 2023
f40009c
Support passing through extra arguments to deployment scripts (#436)
shadaj Mar 7, 2023
44cce72
Extract common logic for establishing CLI-configured connections (#437)
shadaj Mar 7, 2023
b107c47
Require users to specify Hydroflow pipelines at the edges of a Dedalu…
shadaj Mar 8, 2023
748242c
Add `.async` Dedalus directive to specify pipeline for inter-Dedalus …
shadaj Mar 8, 2023
58a2438
Add snapshot testing of graph visualizations (mermaid and dot) (#444)
MingweiSamuel Mar 8, 2023
4c7ec11
Only skipped pinned Rust builds if nothing has changed when merging a…
shadaj Mar 8, 2023
892b1c3
Use `prettyplease` in relalg tests to avoid `rustfmt` dependency (#449)
shadaj Mar 9, 2023
886d00f
Add `demux` operator to Hydro CLI to map node IDs to connections (#445)
shadaj Mar 9, 2023
4b3233a
Support running example deployment script without CLI (#446)
shadaj Mar 9, 2023
74c8d3d
Enable local deployments on non-Linux hosts (#451)
shadaj Mar 10, 2023
f09227b
Add `source_stream(...)` type guard (#455)
MingweiSamuel Mar 10, 2023
742ca19
Avoid spinning on internal state replay, fix #380 (#448 1/2)
MingweiSamuel Mar 8, 2023
28cf07b
Add no-hang tests (#448 2/2)
MingweiSamuel Mar 9, 2023
3e33d0c
Build CLI wheels in CI and minimize CLI dependencies (#452)
shadaj Mar 10, 2023
ae7eef2
Add CI config for cross-building CLI to ARM Linux (#453)
shadaj Mar 10, 2023
743554d
Add CI config for building CLI for Windows (#454)
shadaj Mar 10, 2023
ee54d91
Add `--no-fail-fast` to ci `cargo test`s (#456)
MingweiSamuel Mar 10, 2023
b382477
Fix cross-build Linux and Windows not uploading wheels (#457)
shadaj Mar 10, 2023
86d5623
Fix coloring (pull vs push) error in serdegraph, recompute colors rat…
MingweiSamuel Mar 10, 2023
b977e95
Move `find_barrier_crossers`, coloring, subgraph-making into builder …
MingweiSamuel Mar 4, 2023
9dcaea8
Move `find_subgraph_strata()` into builder (#441 2/14)
MingweiSamuel Mar 6, 2023
dcceaf1
Move `separate_external_inputs()` into builder (#441 3/14)
MingweiSamuel Mar 6, 2023
7e90818
`helper_find_subgraph_handoffs()` (does not compile) (#441 4/14)
MingweiSamuel Mar 7, 2023
733b00c
working, moved internal handoffs (#441 5/14)
MingweiSamuel Mar 7, 2023
c0c00b3
Remove builder (didn't do much really) (#441 6/14)
MingweiSamuel Mar 7, 2023
f37c025
Regenerate colors in `SerdeGraph`, remove from `PartitionedGraph` (#4…
MingweiSamuel Mar 7, 2023
eb8f0e4
Encapsulate subgraph insertion (#441 8/14)
MingweiSamuel Mar 9, 2023
577071a
Update subgraph handoff algorithm (#441 9/14)
MingweiSamuel Mar 10, 2023
a1efedc
Remove `subgraph_send/recv_handoffs` from `PartitionedGraph`, compute…
MingweiSamuel Mar 10, 2023
b640b53
Remove `FlatGraph`, unify under `PartitionedGraph` (#441 11/14)
MingweiSamuel Mar 10, 2023
f95b325
Rename `PartitionedGraph` -> `HydroflowGraph` (#441 12/14)
MingweiSamuel Mar 10, 2023
3ddb10a
Make `HydroflowGraph` fields private (#441 13/14)
MingweiSamuel Mar 10, 2023
09d3b57
Cleanup graph docs, organize method names (#441 14/14)
MingweiSamuel Mar 10, 2023
3575fd3
Allow specifying args to launch `HydroflowCrate` with (#460)
shadaj Mar 13, 2023
09dd190
Add generic arg to `identity()`, add tests, close #392 (#465)
MingweiSamuel Mar 13, 2023
0ec6d88
Support networking topologies that mix local and cloud through SSH tu…
shadaj Mar 13, 2023
b94413a
Directly expose Rust bindings as Python APIs (#462)
shadaj Mar 14, 2023
c2203a1
Add APIs for sending data to a Hydroflow service from Python (#466)
shadaj Mar 14, 2023
688026b
Add scalar `persist()` operator, #438 (#468)
MingweiSamuel Mar 14, 2023
82fb79a
benchmark to test how expensive mapping into/out of joins is
zzlk Mar 16, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
6 changes: 6 additions & 0 deletions .cargo/config.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
[build]
rustflags = [
"-Zproc-macro-backtrace",
# TODO(mingwei): Need rust-analyzer support:
"-Aclippy::uninlined-format-args",
]
1 change: 1 addition & 0 deletions .dockerignore
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
target/
7 changes: 7 additions & 0 deletions .github/gh-pages/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
/debug
/release
/tmp
/CACHEDIR.tag
/.lock
/.rustc_info.json
/.rustdoc_fingerprint.json
8 changes: 8 additions & 0 deletions .github/gh-pages/index.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
---
---
- [The Hydroflow Book]({{ "/book/" | prepend: site.github.url | replace: 'http://', '//' }})
- [Rust Docs]({{ "/doc/hydroflow/" | prepend: site.github.url | replace: 'http://', '//' }})
- [Design Docs]({{ "/design_docs/" | prepend: site.github.url | replace: 'http://', '//' }})
- [Benchmark History]({{ "/bench/" | prepend: site.github.url | replace: 'http://', '//' }})
- [Latest Benchmarks]({{ "/criterion/report/" | prepend: site.github.url | replace: 'http://', '//' }})
- [Github Repository 🔗](https://github.com/hydro-project/hydroflow)
14 changes: 14 additions & 0 deletions .github/workflows/audit.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
name: Security audit
on:
push:
paths:
- '**/Cargo.toml'
- '**/Cargo.lock'
jobs:
security_audit:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v1
- uses: actions-rs/audit-check@v1
with:
token: ${{ secrets.GITHUB_TOKEN }}
146 changes: 146 additions & 0 deletions .github/workflows/build-cli.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,146 @@
name: Build CLI

on:
push:
branches: [main]

env:
PACKAGE_NAME: hydro_cli
PYTHON_VERSION: "3.7" # to build abi3 wheels

# based on Ruff's CI
jobs:
pre_job:
runs-on: ubuntu-latest
outputs:
should_skip: ${{ steps.skip_check.outputs.should_skip }}
steps:
- id: skip_check
uses: fkirc/[email protected]
with:
cancel_others: "true"

macos-universal:
if: ${{ needs.pre_job.outputs.should_skip != 'true' }}
runs-on: macos-latest
steps:
- uses: actions/checkout@v3
- uses: actions/setup-python@v4
with:
python-version: ${{ env.PYTHON_VERSION }}
architecture: x64
- name: "Build wheels - universal2"
uses: PyO3/maturin-action@v1
with:
working-directory: hydro_cli
args: --release --universal2 --out dist
- name: "Install built wheel - universal2"
run: |
pip install hydro_cli/dist/${{ env.PACKAGE_NAME }}-*universal2.whl --force-reinstall
- name: "Upload wheels"
uses: actions/upload-artifact@v3
with:
name: wheels
path: hydro_cli/dist

linux:
if: ${{ needs.pre_job.outputs.should_skip != 'true' }}
runs-on: ubuntu-latest
strategy:
matrix:
target:
- x86_64-unknown-linux-gnu
- i686-unknown-linux-gnu
steps:
- uses: actions/checkout@v3
- uses: actions/setup-python@v4
with:
python-version: ${{ env.PYTHON_VERSION }}
architecture: x64
- name: "Build wheels"
uses: PyO3/maturin-action@v1
with:
working-directory: hydro_cli
target: ${{ matrix.target }}
manylinux: auto
args: --release --out dist
- name: "Install built wheel"
if: ${{ startsWith(matrix.target, 'x86_64') }}
run: |
pip install hydro_cli/dist/${{ env.PACKAGE_NAME }}-*.whl --force-reinstall
- name: "Upload wheels"
uses: actions/upload-artifact@v3
with:
name: wheels
path: hydro_cli/dist

musllinux-cross:
if: ${{ needs.pre_job.outputs.should_skip != 'true' }}
runs-on: ubuntu-latest
strategy:
matrix:
platform:
- target: aarch64-unknown-linux-musl
arch: aarch64
- target: armv7-unknown-linux-musleabihf
arch: armv7
steps:
- uses: actions/checkout@v3
- uses: actions/setup-python@v4
with:
python-version: ${{ env.PYTHON_VERSION }}
- name: "Build wheels"
uses: PyO3/maturin-action@v1
with:
working-directory: hydro_cli
target: ${{ matrix.platform.target }}
manylinux: musllinux_1_2
args: --release --out dist
- uses: uraimo/run-on-arch-action@master
name: Install built wheel
with:
arch: ${{ matrix.platform.arch }}
distro: alpine_latest
install: |
apk add py3-pip
run: |
pip3 install ${{ env.PACKAGE_NAME }} --no-index --find-links hydro_cli/dist/ --force-reinstall
- name: "Upload wheels"
uses: actions/upload-artifact@v3
with:
name: wheels
path: hydro_cli/dist

windows:
runs-on: windows-latest
strategy:
matrix:
platform:
- target: x86_64-pc-windows-msvc
arch: x64
- target: i686-pc-windows-msvc
arch: x86
- target: aarch64-pc-windows-msvc
arch: x64
steps:
- uses: actions/checkout@v3
- uses: actions/setup-python@v4
with:
python-version: ${{ env.PYTHON_VERSION }}
architecture: ${{ matrix.platform.arch }}
- name: "Build wheels"
uses: PyO3/maturin-action@v1
with:
working-directory: hydro_cli
target: ${{ matrix.platform.target }}
args: --release --out dist
- name: "Install built wheel"
if: ${{ !startsWith(matrix.platform.target, 'aarch64') }}
shell: bash
run: |
python -m pip install hydro_cli/dist/${{ env.PACKAGE_NAME }}-*.whl --force-reinstall
- name: "Upload wheels"
uses: actions/upload-artifact@v3
with:
name: wheels
path: hydro_cli/dist
Loading