Allow for merging recordings within the viewer

[pweids]

Problem statement

Sometimes data is spread between multiple recordings. These can be recordings at the same time and same event but to different places on disk (say if it's distributed, or if a recording was created from a post-process routine like in creation of image embeddings). These could also be the same event that got split between files (for efficiency or because of interruption). Combining these recordings into one brings all the data together, making it play better with visualization and data set creation.

The problems, more generally:

• data was recorded in different places
• data recording was interrupted
• data files are cycled for efficiency reasons
• post-processed data needs to be added to the original recording (embeddings, e.g.)
• compare two runs side-by-side

Goal / value

Ship an in-viewer affordance that lets a user pick 2+ files and create a (virtual?) merged partition.

Scope

There are two potential modes:

1. Concatenate — keep original timestamps, simply interleave/append records.
2. Align — resample all recordings to the same timeline. Helpful especially in comparing episodes side-by-side.

For this ticket, let's first validate with concat mode. Align could be a more advanced use case, so needs some better validation.

Merging, also, should not overwrite the existing recordings but create a new one. If technically feasible, this new one could be virtual in that data is not actually copied, just references/pointers to the data are created.

User Journey

An example user journey for this feature could look like:

• In my partition table, I multi-select (shift + click, ctrl + click, or checkboxes) the partitions I want to merge
• Right-click on a selected item and "Merge partitions…" becomes active
• A dialogue appears asking me to name the new partition.
  • Eventually, it'll also ask if we want to reindex to a particular partition. Needs source partition and timeline.
• My screen scrolls to and selects the newly created partition.
• I click "Open" and see that it's been merged
• speculative: If virtual, I want to save this, so choose "export to RRD" from some place

Open questions

1. Should merged views persist in the workspace history or be transient?

See also

rosbag merge 🔗

mcap merge 🔗

[teh-cmc]

See also

rerun rrd merge 🔗

rerun rrd route 🔗

😛