[Feature] Persistent storage for state recovery

[i5hi]

Hey! Thank you so much for sharing this repo!!

I've been playing around with it and wondering what your thoughts are on adding a module to keep writing the orderbook state to persistent storage; so that incase of a crash the orderbook state can be recovered.

I am currently working on a project that requires a simple orderbook and I am using this repo to learn and possibly use in production. More than happy to help extend it or work on known bugs.

Cheers!

[dgtony]

Hi, and thanks for your interest!
This project was started just for fun, and frankly I had no intentions of its further development. But your idea seems to be a cool low-hanging fruit, so why not :)

By design this orderbook is a self-sufficient pure in-memory implementation. Nevertheless persistence could be pretty easily achieved with a log of incoming order requests stored somewhere else. Having such a log one can replay it on a fresh instance of orderbook and get exactly the same final state.

Surely, it's unpractical to replay entire history on every restart of orderbook, so we need to bolt on some snapshotting here, e.g. with a couple of new methods on the orderbook structure: dump and restore. dump should serialize current state of the queues and the sequence value, and return it as a snapshot. Snapshot is just a binary-serialized orderbook state prepended with a fixed-length header that contains version and checksum for data integrity. And restore is the opposite operation: read the snapshot, validate and deserialize it into the orderbook structure.

[i5hi]

Cheers for the response!

Makes sense to offload persistence to the user so the focus is on order matching.

Will give the implementation a shot and submit a PR. Just recently wrapped my head around how to create (solid) interfaces in Rust so the project can have a generic storage interface which will allow users to plug in any backend. The default can be sled.

I'm also using this project for learning so this should be a fun little exercise. I found another lib called lobster and comparing the two to get a better understanding of how orderbooks work. This storage interface should be generic enough to be implemented in both libs as easily.