need to clarify how BIP141 fields are handled on serialized coinbases

[plebhash]

on stratum-mining/stratum#1399 we realized that SRI and Braiins were following different standards with regards to BIP141 fields on the coinbase (marker, flag, witness count, witness length and witness reserved value)

this is meant for discussions about how BIP141 fields are handled on serialized coinbases and propose conventions on the spec for the purpose of interoperability

[plebhash]

Mining Protocol

under the Mining Protocol, this is relevant for NewExtendedMiningJob, which handles a serialized coinbase as coinbase_tx_prefix and coinbase_tx_suffix

before stratum-mining/stratum#1832, SRI would create Extended Jobs where coinbase_tx_prefix and coinbase_tx_suffix contained all BIP141 fields, such that hashing the serialized coinbase transaction would lead to a wtxid (instead of a txid)

while this is a valid approach, it puts the burden on the message receiver to be aware of this and strip those bytes before using the coinbase's txid for Merkle Root calculation... unaware clients would generate shares that would be rejected

meanwhile, Braiins sends Extended Jobs that do strip coinbase_tx_prefix and coinbase_tx_suffix from BIP141 fields, which is closer to what is done on Sv1, and doesn't require for the message receiver to strip anything before calculating txid for the Merkle Root

the fact that there are two different valid approaches for creating Extended Jobs should be acknowledged in the spec, so that implementers are able to be interoperable without unpleasant surprises

[Fi3]

mmm not sure I just took a very shallow look at it. I think that if we are going to have them in the DeclareMiningJob could make sense to have them also in the NewExtendJob to avoid confusion.

[plebhash]

I totally agree that this would help avoiding confusion, and that has been my stance up until recently

but unfortunately after reporting stratum-mining/stratum#1399, Braiins has been completely unresponsive on the multiple times we reached out to discuss this with them, which at this point seems deliberate and I take as a sign they're not willing to change how they're dealing with this

so my initial suggestion here was to allow some flexibility with the "SHOULD strip" terminology and only force the Client to figure out whether it was stripped or not... that would allow old-SRI/DMND (no stripping) and Braiins (stripping) approaches to co-exist

but @TheBlueMatt is suggesting above that "MUST strip" is a better path

---

as mentioned above, I agree that having different conventions for NewExtendedMiningJob vs DeclareMiningJob has some potential for confusion

personally speaking, I'm fine with all possibilities for NewExtendedMiningJob:

• MUST NOT strip
• MUST strip
• SHOULD strip
• MAY strip

but the main point here is that we need some convention with clear wording, rather than no convention which gives room to broken interoperability like we saw in stratum-mining/stratum#1399

[TheBlueMatt]

IMO its really weird for a miner doing normal non-extended channels to even be aware of the concept of segwit. The point of non-extended channels is to be similar to Sv1 where its just dead-simple to connect, get work, and start hashing it, without really having to think too hard about Bitcoiny things.

[plebhash]

hmm we're discussing Extended Jobs tho?

Standard Job messages (which I guess could be referred as non-Extended) have fixed merkle roots, and have no message field that could potentially carry BIP141

Extended Job messages have coinbase_tx_prefix and coinbase_tx_suffix, which are the fields we're debating whether should carry BIP141 or not

anyways, pedantic corrections aside, I get the point of Extended Jobs being aligned with Sv1 and given that we need to establish some convention, I'm more than fine with going with the one that forces everyone to always strip BIP141

I just want to make sure @Fi3 is also ok with this, because it differs from what DMND currently does

[Fi3]

yep is ok with me just make sure that is very clear that while normal job must strip bip141 bytes declared job must not strip them. Sincerly I would add a paragraph somewhere on the spec about bip141 bytes.

[plebhash]

Job Declaration Protocol

under the Job Declaration Protocol, this is relevant for DeclareMiningJob, which handles a serialized coinbase as coinbase_tx_prefix and coinbase_tx_suffix

(note: the following analysis assumes the original coinbase is created as a SegWit transaction... in the (rare) case it's not, there will be no BIP141 fields anyways)

differently from Mining Protocol, we should NOT allow two different approaches for BIP141

when JDS receives a PushSolution message, it reconstructs the coinbase (based on coinbase_tx_prefix plus coinbase_tx_suffix from DeclareMiningJob) and uses it to propage the newfound block

if the BIP141 fields are stripped and JDS tries to construct a non-SegWit coinbase, the propagated block will be rejected by its peers with similar logs:

2025-03-05T01:56:02Z [error] AcceptBlock: bad-witness-nonce-size, CheckWitnessMalleation : invalid witness reserved value size
2025-03-05T01:56:02Z [error] ProcessNewBlock: AcceptBlock FAILED (bad-witness-nonce-size, CheckWitnessMalleation : invalid witness reserved value size)

Here's an analysis of a scenario where this happened with ViaBTC: https://b10c.me/observations/10-viabtc-blocks-without-witness-data/

One could argue that JDS could simply do some heuristics while trying to cater for both BIP141 stripped and non-stripped coinbases. For BIP141 stripped, the most likely heuristics would infer the commonly used values for those fields:

• marker: 0x00 (indicates SegWit)
• flag: 0x01 (current P2P protocol, could change in the future)
• witness reserved value: 0x0000000000000000000000000000000000000000000000000000000000000000 (consensus critical and TIPICALLY 0s, but could be anything and it's actually reserved for future use)

while this heuristics could work fine today, flag and witness reserved value could eventually cause problems

BIP141 specifies that some soft fork could repurpose the witness reserved value to contain the hash of some consensus-critical data

and then JDS would not be able to guess those values and be unable to propagate blocks with SegWit coinbases

[plebhash]

therefore, I feel we should add some words saying something along these lines to the description of DeclareMiningJob:

If the original coinbase is a SegWit transaction, coinbase_tx_prefix and coinbase_tx_suffix MUST contain all BIP141 fields, which MUST NOT be optionally stripped like in NewExtendedMiningJob

[TheBlueMatt]

We should either always include it or never. IMO doing it sometimes is just weird. The witness reserved vale is definitely consensus-critical - its committed to in the segwit commitment.

[plebhash]

We should either always include it or never. IMO doing it sometimes is just weird. The witness reserved vale is definitely consensus-critical - its committed to in the segwit commitment.

never including is not ideal for the reasons I mentioned above

if BIP141 data is stripped from DeclareMiningJob, right now JDS can kinda infer values for flag and witness reserved value that will be ok most of the time, but whenever a soft fork changes things there we would have to upgrade the protocol, because otherwise JDS would not be able to infer those values after receiving PushSolution, which would make it unable to propagate the block, which then would make PushSolution pretty much useless

so always including it makes things future proof

btw, that's what SRI already does (and I think it's safe to assume DMND as well), so adding the phrase proposed above to the description of DeclareMiningJob would simply be formalizing it as a convention

[plebhash]

The witness reserved vale is definitely consensus-critical

sorry that's right, it's concatenated with witness root hash and hashed into the witness commitment

but the main point is that today it could (and tomorrow it definitely will) be something other than 0x0000000000000000000000000000000000000000000000000000000000000000

so if it was stripped from DeclareMiningJob:

• today: JDS still has the 0s as a safe-ish guess, although it could be something different, so it could potentially cause problems
• tomorrow (with a hypothetical soft-fork): 0s would no longer be a viable guess, so PushSolution would be pretty much useless

[TheBlueMatt]

Right, we're on the same page. I think I put this comment in the wrong place, but, yes, it should definitely always be included.

[plebhash]

Template Distribution Protocol

under Template Distribution Protocol, this is relevant for SubmitSolution, which carries a serialized coinbase as coinbase_tx

assuming the original coinbase is a SegWit transaction, and SubmitSolution.coinbase_tx is stripped of BIP141, the block propagation would fail with the same reasons as described above in the Job Declaration Protocol

one could make the same argument about heuristics, but the same rationale around future soft forks would apply

[plebhash]

therefore, I feel we should add some words saying something along these lines to the description of SubmitSolution:

If the original coinbase is a SegWit transaction, SubmitSolution.coinbase_tx MUST contain all BIP141 fields, which MUST NOT be optionally stripped like in NewExtendedMiningJob

[Sjors]

I would like to keep the TP implementation simple, so yes please don't strip things :-)

[GitGab19]

As we already discussed during our last meetings, I feel this is the best direction for everyone, and I completely ACK all your proposals aforementioned! 💯