Skip to content

Commit 2dd0516

Browse files
authored
feat: Build out Device and Data module
# Summary This PR adds a centralized `Buffer` object which allows transposition from an Array of Structures (AOS) given by `Vec<Sample>` to a Structure of Array (SOA) given by an `AlignedWindow` struct. In doing so, it exposes three means of reading, two of which have not been tested and are bolded below: 1. Pull-based read - reads the most recent $N$ samples on request 2. **Cursor-based read** (`reader.rs`) - reads the next $N$ samples, keeping track of the location based on an iterator 3. **Push-based read** (`subscription.rs`) - reads the most recent $N$ samples as sent by a `SubscriptionManager` This also includes a cascade of changes, such as: - Moving the CLI arg parser from [getopts](https://github.com/rust-lang/getopts) to [clap](https://github.com/clap-rs/clap) - Moving from [crossterm](https://github.com/crossterm-rs/crossterm) to [ratatui](https://github.com/ratatui/ratatui) for TUI tools (e.g., tio-monitor, the new tio-health) - Exposing a `DeviceTree` to avoid instantiating multiple `Device`s (using `subtree_full()`) - Support for `HDF5` export from a binary log file - Elevating thread priority on macOS and Windows using `objc2` and `MMCSS` respectively (to minimizing dropped `Sample`s) ## New Files ### Device Module > `tree.rs`, `rpc_registry.rs`, `reader.rs`, `subscription.rs`, `util.rs` - `tree.rs` copies over the functionality of `device.rs` but adds the `DeviceRoute` as an argument - `rpc_registry.rs` is for future functionality, as many tools would like to be exposed to the tree hierarchy of the RPC structure without having to reconstruct it themselves through a manual rpc.listinfo and building the entire query - `util.rs` hosts the RPC parsing that was originally found exclusively in `tio-tool` that can be reused in other programs ### Data Module > `buffer.rs`, `export.rs`, `filter.rs`, `util.rs` The main area for future change is the developer API for the `Buffer` object. In supporting multiple use cases (monitoring, real-time signal processing, offline processing), there are possibly redundant structures. Thus far, the API is as follows: - `OverflowPolicy` which determines whether to act like a ring buffer or, effectively, a batched streaming buffer - `BufferEvent` structures derived from characteristics of the stream of `Sample`s (e.g. `SamplesSkipped`). It is sent over a crossbeam channel and, for now, overloads the batched streaming from `OverflowPolicy` in the form of a `DataChunk` and has an optional override to (copy) and forward all samples in a `BufferEvent::Samples`. **It should be possible to move away from this `DataChunk` and just use the proper subscription based `push` module.** - `ReadError` a run-time error which determines if read queries are impossible, such as if the `Cursor` is pointing to a spot out of the `Buffer`, if it requested too many samples, or a column that doesn't exist The whole idea is that a "run" attempts to denote a maximal contiguous region of samples for a given (route, stream_id) that share: - `session_id` - effective sampling rate (sampling_rate / decimation) - time reference epoch (MetadataEpoch) and do not exhibit large gaps or backward jumps in sample number / timestamp. ### OS Module - `mod.rs` This only has the thread priority functionality. In the future it would be appropriate to host any other OS dependent code here. # Changelog ## Changes - Update `tio-monitor` to support RPC execution, basic time-series and periodogram plotting, and multiple devices using [ratatui](https://github.com/ratatui/ratatui) - Update all existing tools (`tio-proxy`, `tio-tool`, `tio-text-proxy`) to use [clap](https://github.com/clap-rs/clap) parser - Change `lib.rs` inside `twinleaf-tools` to use [clap](https://github.com/clap-rs/clap) parser ## Added - Add `buffer.rs` for multi device data transposition and alignment - Thread priority elevation - Add `tio-health.rs` for viewing the sample rate and statistics of all incoming samples - Add `HDF5` support in `tio-tool` using `export.rs` and a flushing `Buffer` - Add `BufferType` enum inside `proto.rs` to map to typed vector more easily ## Fixed - Allow negative args in tio-tool rpc, solving #9
2 parents 173721a + 1b6249b commit 2dd0516

35 files changed

+7215
-1030
lines changed

.gitignore

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,2 +1,3 @@
11
*~
22
/target/
3+
/.vscode/

0 commit comments

Comments
 (0)