Update state v2 api for gloas

[nflaig]

I think we need some clarification on the /eth/v2/debug/beacon/states/{state_id} api too.
With ePBS we gonna have 2 state updates per slot, first when the block is processed and second when the payload is processed (if revealed).
It is currently not specified which state is returned by the endpoint, so I guess the response changes once the payload is revealed?

Originally posted by @pk910 in #552 (comment)

[rolfyone]

the state hash would change, but that'd be the only way you'd get to any transition after the payload is applied...

So i think pre-state (slot change but no block applied) you'd use hash for state_id,
states/ would be the post state of consensus block applied
payload applied would be via hash also

so its just a permutation of what we're already doing... but i do think documenting that is sensible on the api

I'm not sure where the best place to document this would be, it'll be basically the same thing for every accessor using /{state_id}/

[pk910]

I meant the state by slot number & "head" / "finalized" labels with my comment, where it was not clear to me which state is returned.
I think serving the post-consensus-block state makes sense. This matches the state root that is linked via the block header.

However, it seems to be tricky to even request the post-payload state as the state_root of that "intermediate" state is not accessible anywhere? (might be fine tho)

[rolfyone]

so pre-state and post payload state would need to be retrieved using the hash, and the 'post state' is what would be a number OR hash...
We have this limitation today, I've not looked at getting a pre-state for a while because i generally just wind it forward from previous slot, but in theory its do-able...

maybe what we need is at least a way to determine all the states in a slot or the pre-state hash of a given slot and payload-applied hash of a given slot or something... I'm not sure how complicated that would be to implement :) but at least then you'd have the hash to lookup...

I do agree we'd want to define whats returned at a given label, I think it's never been consistent for 'finalized', but i could be wrong...

[ensi321]

It is currently not specified which state is returned by the endpoint, so I guess the response changes once the payload is revealed?

I guess we also need to gather PTC votes and attestation votes to see if they support the fact that payload is available.

I think a more correct approach would be to consult our fork choice, see if the current chain that we are tracking has payload at slot, if so then we return the state after payload processing.

[rolfyone]

I think a more correct approach would be to consult our fork choice, see if the current chain that we are tracking has payload at slot, if so then we return the state after payload processing.

so basically the 'best state' at that slot?

eg. if we've got the block at slot N but not payload, then our best state is the post state, but if we've got the payload it'd be the state with payload applied? would we want payload status returned in metadata or something if we start doing that?

i guess definitively we're talking about the state when we're referencing state_id as the slot number, and it being defined by fork choice depending on what our best response we can give is, and you'd need to ask for a specific hash if you wanted something else?

i guess by the same theory /finalized, /justified and /head should be the best state at that given slot

[potuz]

I'll leave here my preferences on this endpoint so that we do not have to hack this behavior locally. And I believe these are the right options anyway:

• For identifiers like blockroot I think the consensus post-state is the right option. The reason being there may be an actual CL block for that blockroot and no payload.
• For identifiers like slot I also think that the consensus post-state is the right option. The reason being that there may be a CL block at that slot and no payload for that slot. This situation is minimally more ambiguous than the previous one when during that slot the block is missed and the last state was full, in this case one could argue that we could return the head state before that slot advanced to the slot. But this complexity is rather useless, it does not save the caller any time to understand what they are getting. So sticking to the simple algorithm: "find out what's the latest block root before or equal than that slot, call the previous helper for that blockroot, is the simplest and most elegant option".
• For identifiers like finalized and justified: I think that the consensus post-state is the right option. Again the reason is that at the given blockroot/slot/start of the epoch there may be a block without a payload. Same reasoning applies for missed slots as above.
• Finally for "head" I could see an argument of someone stupid enough to DOS their node constantly to get head information from the state and actually wanted a full state every few milliseconds. This case could really be the current headstate as cached by clients, which could mean either full or empty depending on the situation. However, I still think that making this particular identifier return differently forces the caller to add code to parse the received state. I thus think that in this case we should also return post CL info.

This way the server is simpler. We could add a separate endpoint for post-payload states if needed, but I suspect it's overkill.

[twoeths]

I'd suggest:

• "head": should be the head state of the beacon-node. It has the payload status semantic in fork-choice
• "genesis": not a concern
• "finalized", "justified": the respective finalized and justified states in beacon-node. In lodestar implementation post-gloas, a finalized/justified checkpoint comes with payload status too so it's pretty clear for us.
• "slot": this one is tricky because we may have consensus state and optionally a payload state for that slot
  • for eth/v2/debug/beacon/states/${state_id}: I found it useful to add a payloadStatus query param for debugging purpose
  • for other apis, since it queries "beacon" data, most of the times returned data would be the same for consensus state vs payload state
  • hence best option could be having payloadStatus as an optional param? use consensus state by default
• state_root with 0x prefix: use the exact state of that root inside block state cache of the beacon-node

For identifiers like blockroot I think the consensus post-state is the right option. The reason being there may be an actual CL block for that blockroot and no payload.

I don't know if the block_root is one of identifiers for state apis? But I'd go with having an optional payload status param, similar to the slot identifier, if we have it