-
Notifications
You must be signed in to change notification settings - Fork 8
Add virtual concatenation of LHDataStores #36
Copy link
Copy link
Open
Description
We want to concatenates multiple LH5 files in a virtual/lazy fashion, so that read operations see arrays and tables that are concatenated along all files. Reading, broadcasting, etc. should use Julia processes to work in parallel.
The implementation should open and close files dynamically to both limit the number of active file descriptors and prevent too frequent opening and closing of files.
User-facing, this could simply look like lh5open(vector_of_filenames).
Initially only reading needs to be supported, but longer term we'll probably want to support writing as well.
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
No labels