Full node <-> light wallet

[benjaminvanrenterghem]

I have shared my thoughts on full node and light wallet congruency here: Open-DB/full-node-lightwallet-sync.md

This issue serves as its place for permanent discussions on this particular matter.
A database choice was not mentioned, perhaps someone can chime on something efficient which can handle such a large volume of data efficiently, and which doesn't have a complicated set-up procedure, mongodb comes to mind, but I have little experience with it when it comes to large volumes of data.

[EggPool]

Thanks for the feedback.
I realize some important info is missing from the repo.
I'll add more asap.

We already have a fixed and common database structure, used by 2 different feeders implementations.
Since the structure is stable, a document oriented database does not add meaningful features or ease of dev.
An old school relational db - mysql/mariadb is currently used and allows for precise index definitions and complex queries, while keeping storage need to a minimum.

Other implementations could use postgressql for instance with the same schema,
and we could even have another one feeding a mongodb, always with the same schema.
(losing some joins queries and features along the way probably).

Going nosql/document db from the start makes no sense when the data - at core - is relational and follows a rigid structure.

I'll add a few .md in the repo to summarize current state and specs as well as current working implementations.
As for API / Light wallet, I'll also begin a minimal draft.
Idea there would be to follow calls and naming from bitcoin daemon as much as possible, rather than spec a new vocabulary, endpoints and parameters.

[EggPool]

Hopefully newest commits should shred some light on what was done - and is working - so far, and the scope of each "open nyzo" project.
Commands and Transports (both WIP) do more specifically answer to your light wallet API comments, while trying to adhere to existing bitcoin standards as much as possible.