Future work on blueboxes

[beechord]

Out of discussions in #timelords on keybase and with @fchirica for #7175. The goal is to make the chain smaller and quicker to sync by compacting all VDF proofs on the chain, and to do so sustainably and ideally in a decentralized fashion.

Overview:

There are not a lot of design documents to go by, so I'll summarize: Blueboxes are a type of timelord that compresses the chain instead of extending it. Basically, they clean up the mess that regular timelords leave, and in fact have to leave for technical/cryptographical reasons. They do this by:

• scanning the local copy of the blockchain for not-yet-fully-compacted (uncompact) blocks, each of which contains 1 to n proofs to be compacted. (broadcast_uncompact_blocks)
• after having found (usually) 1000 uncompact proofs, they pick some at random and crunch away, usually for 10-20 minutes per proof on 2-3 cores. (_manage_discriminant_queue_sanitizer)
• they then send out those proofs (respond_compact_proof_of_time) via new_compact_vdf, and other nodes incorporate them into their chains and relay them if new.

The Numbers

Currently, timelords produce 4608 blocks per day and on a good day, blueboxes compact about 70% of that, so they're not yet catching up. The percentage of weight-proof-relevant proofs that's already compacted (is_fully_compactified) is about 30%. The overall percentage is lower, as there are about as many proofs again that are not needed for weight proofs but still count towards chain size.
One compacted proof saves about 300 bytes, at 5+ proofs per block and a chain length of about 600000, that's not quite yet ~1GB of a chain file that's currently at ~8GB. Modest, but it's work that has to be done only once for a network-wide benefit. I haven't profiled how much time this would save during a sync.
According to my estimates, about 120 bluebox processes running on reasonably recent CPUs would be needed to keep up with weight proofs, and 300 for all proofs, but this varies with chain height and number of iterations required. My bluebox currently hosts 20 processes, so right now all we'd need is a couple repurposed plotters (but it's not plug-and-play yet - ask in #timelords for how to set one up properly).

Future Work

My rough parameters here are: Make it possible for random network participants to donate cores to the network by entering a number in the UI. There will be no reward, but there will be a percentage of compacted blocks, and who doesn't like watching a number go up. Eventually, ASIC timelords will upset the balance, but I suspect those will also make excellent blueboxes.
An alternative approach would be for timelords that are neither fastest nor second-fastest to switch to bluebox mode automatically, but blueboxing and timelording are very different workloads and I feel like this comes too close to messing with important security properties anyway.
Needed (in order of decreasing priority):

• A catchup protocol. Detailed proposal further down.
• There should be exactly one bluebox tunable, and it should be the number of bluebox processes.
  • In order to do this, switch blueboxes to a pull-work model (currently push-work). Greatly simplifies the logic involved and reduces load on the fullnode. Requires a bluebox protocol change though.
  • If the code is solid enough by then, this tunable could be made accessible via GUI.
• Further coordination improvements:
  • Work could be sampled from a wider area and not just the bottom 1000.
  • Abort compactions if the network comes up with the compact proof before our node does. About halves work wasted in collisions.
• a VDF implementation optimized for throughput - blueboxes are not in a hurry, but they only have a limited amount of cores to donate.
• Compacting non-weight proofs.

I personally am currently playing with running a Xeon Phi bluebox - mostly because it's great fun and because Xeon Phis have a ton of vector units for small money. A second gen Xeon Phi might run chiavdf unmodified, a first gen (which I have) is probably more of a challenge. Upside of this approach is, if it works as well as I think it does, a handful of Xeon Phis might provide for the whole network. Still, not the best decentralization properties.

Catchup Protocol

There's a lot of blocks out there that could be compacted but aren't, usually because their nodes were offline when that block was compacted, or were doing a full sync. This is a blocker to just conscripting random plotters, because their work will be wasted for weeks if their chains miss a couple ten thousand compacts (this happens a lot - you can copy chains from other nodes, but finding a good one requires fiddling around with sqlite3). Rough outline:

• Every node (bluebox or not) will occasionally ask a random peer for a compact version of its oldest non-orphan uncompact block.
• If it receives one in answer, it broadcasts it to its other peers, which then treat it like any other new_compact_vdf.
• In order not to swamp the network, it scales back its demands depending on how many compact VDFs are already flying. This should ensure a smooth network transition even if tens of thousands of nodes with this code are rolled out due to a software release.
• Blueboxes specifically only start compacting once their peers stop answering with new compacts.

That's about it.

[beechord]

Bonus Content

This touches on stuff that's not up to me to decide but - querying the db for any block contents is awkward. A database field needs to be created and indexed, database upgrade code must be written, and care must be taken to not get out of sync with block contents. Two properties that would be interesting to query over are is_orphan and is_space_compactified, where the former is really more of an in-context property and therefore tricky, and the latter is like is_fully_compactified, except it includes really all the proofs in a block. One way to do this would be to leverage sqlite's app-defined SQL functions (https://www.sqlite.org/appfunc.html) and then to create indexes over those, though that comes with its own problems - you'll want to be really careful about calling into stateful stuff, or just code organisation in general (there are no private functions here).

[SlugFiller]

Any space save in the chain is nice, but a 10% save on something that went to 8gb in under 5 months leaves something to be desired.

I'm more interested in why the wallet's db is 3gb as well. You'd think if it's just a wallet, it needs to store a lot less than a full node. For farmers, and on desktop, it's fine as is. But for mobile users, a true lightweight wallet will be in need soon.

[turbotas]

I may be wrong - it happens a lot - but the BB timelords produce proofs that are mathematically compact - any space saving is incidental.