Memory Store Spec #1188
Description
WIP - will write actual type specs soon
still specifiying this
Memory store
on every dimos run, automatically store all sensor data as binary blobs + metadata in appropriate format
optional post-run cleanup/deletion, settings for max space used etc like rerun has
efficient storage and indexing based on time/space/and embeddings
- pluggable backends for diff data (psql/sqlite/pickle implemented)
- sqlite vector extensions seem very good for multidim search/embeddings
Easily add new programatic overlays for temporal/spatial/semantic data
so we don't just store sensor streams, but maybe LLM thoughts, some sort of reprocessing, even debug data,
also be able to dynamically create a new reprocessed data streams, suddenly we care about bottles last 2 weeks, we can generate
replaces our save/replay system completely
you like your run? it's already stored, just name it, mark it as permanently available so it doesn't get auto cleaned up
run traditional search algorithms on top of this data
I can ask "give me a list of most bottle like things, within this physical space, with this kind of temporal bias"
when searching for a bottle, since we are doing a search in mutlidimensional space (time, embeddings, space as search heuristics) we can utilize traditional search algos, simulated anneling etc, heuristics can be pluggable
then I can (easily!) run VLM query on top, potentially output the same data type again - TemporalData augmented/biased with VLM, run more queries on top "give me latest with score above 0.7", give me a spatial heatmap to overlay on my occupancygrid, give me temporal graph of semantic similarity to this third query etc
corelate tempral stores
"give me lidar frame closest in time to this image frame" - so I can project something into space