[CAP 66] In-memory Read Resource and Automatic Restoration

[SirTyson]

Discussion for CAP 66 which introduces automatic archived entry restoration and a new read resource type for in-memory reads.

[SirTyson]

I think there are two logistical questions I have with this CAP.

First, are we fine with using a vector of uint8_t for evictedSorobanEntries? This is the vector of indices that differentiate live soroban state with archived state. There was initially some concern that this only supports read-write footprints with up to 255 keys. XDR doesn't support 16 bit ints, so we could either manually manage the 16 bit values via a vector of uint8_t, or go up to32 bits. Bit packing is a little hacky, and with a 32 bit vector, we might as well just have a vector of key hashes instead of indexes. I think 255 keys is a very reasonable upper bound, and to protect ourselves I can add an upper bound check to the txMaxWriteLedgerEntries config upgrade with a comment explaining why the bound is there. I the future if we need to go beyond 255, we can switch to bit packing uin16_t without changing XDR.

2nd issue, should RestoreFootprintOp be deprecated? If we allow automatic restoration via footprint, there's no reason to have both ExtendFootprintTTLOp and RestoreFootprintOp, as ExtendFootprintTTLOp with a TTL value of 0 would be functionally identical to restore op. We might want to change the name to something like ManageEntryLifetime that applies to both TTL and restoration. This might be more work than it's worth though, as it is a minor issue and we'd have to maintain the older versions for catchup anyway.

[anupsdf]

Using sorobanStateTargetSizeBytes, which is a smaller number than classic target bucket list size, we should look into the impact it would have to the fluctuation of write fees due to bucket merges. Classic target BL size was in GBs while Soroban state size is in MBs so the formula might have different variance. This is mostly for contract data and TTL entries that get updated more often. Contact code would rarely get updated.
fyi, @dmkozh

[dmkozh]

Yeah, we need to update this, but before that we need to clarify how exactly the state size is computed.

[anupsdf]

Just saw #1663. Since, sorobanStateTargetSizeBytes will be tracking in-memory state instead of on-disk there won't be any fluctuations from bucket list merges.

[dmkozh]

Yeah, I think we'll need to create a new core build that supports in-memory module cache metering and measure the size of the live state in memory. Then we can use the live state size for re-calibrating the setting values in order to produce roughly the same write fee, as we have now. After the upgrade we can monitor the state size and update the fee computations if necessary.