Do we actually need the IsValid flag?

[nfrisby]

The IsValid flag is currently present in Praos blocks. If the txs on a script don't agree with the flag, then the block is invalid.

https://github.com/IntersectMBO/cardano-ledger/blob/8ea8d1e665951bb474a861c8fad5b4b2818e3fe2/eras/alonzo/impl/src/Cardano/Ledger/Alonzo/Tx.hs#L135

Something I never questioned until recently: why is the IsValid flag included in the block itself? If you're trusting that the block is valid (eg in your tool that processes blocks without validating them), then you would necessarily be trusting whoever gave you the block. And so they could just annotate it with the correct IsValid flags, instead of those IsValid flags actually being part of the block.

---

The reason this question occurred to me is that maybe there's a useful Leios ledger design where IBs/RBs can't necessarily predict the IsValid flag. However, it's more plausible to me that at least RBs must include the IsValid flag. The only weirdness then is that the corresponding tx might be dead (aka skipped) in the RB, and so its IsValid flag actually can't be validated. Maybe the RB should be able to specify a third option: dead, as opposed to the Boolean IsValid.

[bwbush]

In Praos, IsValid (actually, invalid_transactions) seems to be a convenience that is redundant.

It doesn't look like IsValid is part of the serialized tx on the ledger: it is just in the serialized CBOR for a transaction, but is outside of the signed part. It just signals the node to include it in the memory pool regardless of its invalidity.

[bwbush]

The invalid_transactions list in the RB's CBOR might get quite large at high TPS. For example, at 1000 TPS, the size of this array could be larger than allowed in an RB. This validity information would have to be stored in EBs instead, but they might also suffer from bloat at high TPS.

[ch1bo]

This reminds me of the idea @Quantumplation had on having a conflicting_transactions list in EBs and RBs? I think he called that variant raspberry.

I'm not sure about the IsValid flags necessity just yet, but having a list of invalid or conflicting transactions sounds useful?

[Quantumplation]

Yes, the (rough sketch of an) idea there was to make the EB producer responsible for identifying the conflicts, so that we can minimize the work we do to reconstruct the ledger state. Most people optimistically assume that list is honest, and a small subset is chosen (by lottery, or the RB producer, etc) to double check and report it if it's wrong, thereby slashing the en producers bond.

[WhatisRT]

IsValid is purely and optimization for revalidation (or other purposes where you trust the contents of the blocks). We could remove it, but then you'd always have to run scripts to know how to update the ledger state, whereas now if you trust the validity of a block (which implies correctness of the IsValid flags) you can recompute the ledger state without running any scripts.

So it's completely optional, and it is an optimization that could also be used for dealing with conflicting transactions. But I feel like these are relatively small details that are probably better worked out later.

[nfrisby]

Thanks for the reply, Andre.

My "proposed alternative" is for the IsValid flag to be provided by nodes alongside the block rather than part of the block. IE the flags are not "on the block chain", but there'd be some kind of provenance chain between someone "trusting the contents of the blocks" and a node that actually ran the scripts.

And so they could just annotate it with the correct IsValid flags, instead of those IsValid flags actually being part of the block.

Ah ha. I realize now a major difference. Having the IsValid flag be part of the block itself, ie involved in the hash, signatures, etc, means that certificates etc (eg Mithril) could justify trusting the IsValid flags, regardless of where you got the block itself from. My preconception that trusted blocks must be acquired from a trusted source (and so could be accompanied by trusted IsValid flag annotations) was wrong.

[WhatisRT]

Indeed, all you need to trust for the correctness of IsValid flags is that some honest node validated the block. The only way to be sure is to do it yourself, but as you mention Mithril might be another way. Or you fetch the history from a bunch of random nodes and see it in enough histories you can gain confidence as well.

You could provide this information outside of the block, but then you need the block to contain a hash of this information to maintain the same trust properties. And we need a lot of transactions for this to be more space efficient (at least with sufficiently efficient encodings, I don't know why the current encoding isn't just a list of bits).