Skip to content

Refactor to go fully in-memory first, and persistent second #99

@matko

Description

@matko

A lot of the data structures are currently written around the idea that we're persisting layers directly to some persistent storage. The memory store is then implemented as a special kind of persistence which just saves things in a hashmap.

This is kinda weird and the wrong way around. We actually almost always want to build up stuff in memory first, and only then (possibly after some more checks, like schema checking) persist to disk.

Therefore, we should consider refactoring terminusdb-store to be in-memory first. That means that none of the data structures, nor the layer structures, should have to know about an underlying storage method (FileLoad, FileStore, etc), but instead directly work on Bytes. Builders likewise should not have to know anything about storage, but build directly into Bytes.

Doing all this will also reduce the complexity of the typing in store, as a lot of that is now complicated by the fact that in many places we want to work with builders without knowing this underlying storage method. If all actual persisting logic is moved into the various LayerStores, we probably don't have to include FileLoad or FileStore in any of the typing anymore.

Metadata

Metadata

Assignees

No one assigned

    Labels

    triageissue to be triage

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions