Materialize query pipeline to a store and query it #6
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
@KyleAMathews, some quick notes on this:
This is a work in progress draft of how we could do an in-memory "store", materialise to them, and allow then to be queryable with D2.
The fruit-processed.ts example has been updated to use it.
Storeis aMaplike store, but that emits change events.You can materialise any keyed D2 pipeline to a Store with
Store.materialize, this is a static method on the store rather than the S2 pipeline builder so that it is optional and not included if a user doesn't need it. (they may be materialising to anything)A Store, or multiple, can be queried with the
Store.queryAllstatic method. You pass a list of stores, and a callback, which receives a D2 pipeline builder.Mutations to a store can be grouped together with
store.transaction, the changes are only emitted at the end of the transaction.A few (initial) things that I want to change:
store.queryinstance method on a store without having to use thequeryAllstatic methodThis is really targeted at front end processing, rather than server side processing.
An Electric ShapeStream could easily be materialised to a Store.