[3.0] Optimize deserialization for new schema

[bchess]

#120 was a proof-of-concept that then led to several of schema changes in 3.0. But we need to actually do something like what's done in that PR and merge it in the 3.0 branch.