Merge pull request bitcoin#441 from jl2012/bip146nullfail

BIP146: title and content changed

Author: luke-jr

README.mediawiki

@@ -513,8 +513,8 @@ Those proposing changes should consider that ultimately consent may rest with th
 | Draft
 |-
 | [[bip-0146.mediawiki|146]]
-| Dealing with signature malleability
-| Pieter Wuille, Johnson Lau
+| Dealing with signature encoding malleability
+| Johnson Lau, Pieter Wuille
 | Standard
 | Draft
 |-

bip-0146.mediawiki

@@ -1,67 +1,123 @@
 <pre>
   BIP: 146
-  Title: Dealing with signature malleability
-  Author: Pieter Wuille <[email protected]>
-          Johnson Lau <[email protected]>
+  Title: Dealing with signature encoding malleability
+  Author: Johnson Lau <[email protected]>
+          Pieter Wuille <[email protected]>
   Status: Draft
   Type: Standards Track
   Created: 2016-08-16
 </pre>
 
 ==Abstract==
 
-This document specifies proposed changes to the Bitcoin transaction validity rules to fix signature malleability for common transaction types.
+This document specifies proposed changes to the Bitcoin transaction validity rules to fix signature malleability related to ECDSA signature encoding.
 
 
 ==Motivation==
 
 Signature malleability refers to the ability of any relay node on the network to transform the signature in transactions, with no access to the relevant private keys required. For non-segregated witness transactions, signature malleability will change the <code>txid</code> and invalidate any unconfirmed child transactions. Although the <code>txid</code> of segregated witness ([https://github.com/bitcoin/bips/blob/master/bip-0141.mediawiki BIP141]) transactions is not third party malleable, this malleability vector will change the <code>wtxid</code> and may reduce the efficiency of compact block relay ([https://github.com/bitcoin/bips/blob/master/bip-0152.mediawiki BIP152]).
 
-Since the enforcement of Strict DER signatures ([https://github.com/bitcoin/bips/blob/master/bip-0066.mediawiki BIP66]), there are 2 remaining known sources of malleability in the signature passed to ECDSA verification opcodes:
+Since the enforcement of Strict DER signatures ([https://github.com/bitcoin/bips/blob/master/bip-0066.mediawiki BIP66]), there are 2 remaining known sources of malleability in ECDSA signatures:
 
 # '''Inherent ECDSA signature malleability''': ECDSA signatures are inherently malleable as taking the negative of the number S inside (modulo the curve order) does not invalidate it.
 
-# '''Inputs ignored by scripts''': The (unnecessary) extra stack element consumed by <code>OP_CHECKMULTISIG</code> and <code>OP_CHECKMULTISIGVERIFY</code> is not inspected in any manner, and could be replaced with any value.
+# '''Malleability of failing signature''': If a signature failed to validate in <code>OP_CHECKSIG</code> or <code>OP_CHECKMULTISIG</code>, a <code>FALSE</code> would be returned to the stack and the script evaluation would continue. The failing signature may take any value, as long as it follows all the rules described in BIP66.
 
 This document specifies new rules to fix the aforesaid signature malleability.
 
 
 ==Specification==
 
-To fix signature malleability, the following new rules are applied:
+To fix signature encoding malleability, the following new rules are applied to pre-segregated witness and segregated witness scripts:
 
 
 ===LOW_S===
 
 We require that the S value inside ECDSA signatures is at most the curve order divided by 2 (essentially restricting this value to its lower half range). Every signature passed to <code>OP_CHECKSIG</code><ref>Including pay-to-witness-public-key-hash (P2WPKH) described in BIP141</ref>, <code>OP_CHECKSIGVERIFY</code>, <code>OP_CHECKMULTISIG</code>, or <code>OP_CHECKMULTISIGVERIFY</code>, to which ECDSA verification is applied, MUST use a S value between <code>0x1</code> and <code>0x7FFFFFFF FFFFFFFF FFFFFFFF FFFFFFFF 5D576E73 57A4501D DFE92F46 681B20A0</code> (inclusive) with strict DER encoding (see [https://github.com/bitcoin/bips/blob/master/bip-0066.mediawiki BIP66]).
 
-These operators all perform ECDSA verifications on pubkey/signature pairs, iterating from the top of the stack backwards. For each such verification, if the signature does not pass the Low S value check, the entire script evaluates to false immediately. If the signature is valid DER with low S value, but does not pass ECDSA verification, opcode execution continues as it used to, causing opcode execution to stop and push false on the stack (but not immediately fail the script) in some cases, which potentially skips further signatures (and thus does not subject them to Low S value check).
+If a signature passing to ECDSA verification does not pass the Low S value check and is not an empty byte array, the entire script evaluates to false immediately.
 
 A high S value in signature could be trivially replaced by <code>S' = 0xFFFFFFFF FFFFFFFF FFFFFFFF FFFFFFFE BAAEDCE6 AF48A03B BFD25E8C D0364141 - S</code>.
 
 
-===NULLDUMMY===
+===NULLFAIL===
 
-The extra stack element consumed by <code>OP_CHECKMULTISIG</code> and <code>OP_CHECKMULTISIGVERIFY</code> MUST be the empty byte array (the result of <code>OP_0</code>). Anything else makes the script evaluate to false immediately.
+If an <code>OP_CHECKSIG</code> is trying to return a <code>FALSE</code> value to the stack, we require that the relevant signature must be an empty byte array.
+
+If an <code>OP_CHECKMULTISIG</code> is trying to return a <code>FALSE</code> value to the stack, we require that all signatures passing to this <code>OP_CHECKMULTISIG</code> must be empty byte arrays, even the processing of some signatures might have been skipped due to early termination of the signature verification.
+
+Otherwise, the entire script evaluates to false immediately.
+
+
+==Examples==
+
+The following examples are the combined results of the LOW_S and NULLFAIL rules.<ref>Please note that due to implementation details in reference client v0.13.1, some signatures with S value higher than the half curve order might pass the LOW_S test. However, such signatures are certainly invalid, and will fail later due to NULLFAIL test.</ref>
+
+Notation:
+
+  CO       : curve order = 0xFFFFFFFF FFFFFFFF FFFFFFFF FFFFFFFE BAAEDCE6 AF48A03B BFD25E8C D0364141
+  HCO      : half curve order = CO / 2 = 0x7FFFFFFF FFFFFFFF FFFFFFFF FFFFFFFF 5D576E73 57A4501D DFE92F46 681B20A0
+  P1, P2   : valid, serialized, public keys
+  S1L, S2L : valid low S value signatures using respective keys P1 and P2 (1 ≤ S ≤ HCO)
+  S1H, S2H : signatures with high S value (otherwise valid) using respective keys P1 and P2 (HCO < S < CO)
+  F        : any BIP66-compliant non-empty byte array but not a valid signature
+
+These scripts will return a <code>TRUE</code> to the stack as before:
+
+  S1L P1 CHECKSIG
+  0 S1L S2L 2 P1 P2 2 CHECKMULTISIG
+
+These scripts will return a <code>FALSE</code> to the stack as before:
+
+  0 P1 CHECKSIG
+  0 0 0 2 P1 P2 2 CHECKMULTISIG
+
+These previously <code>TRUE</code> scripts will fail immediately under the new rules:
+
+  S1H P1 CHECKSIG
+  0 S1H S2L 2 P1 P2 2 CHECKMULTISIG
+  0 S1L S2H 2 P1 P2 2 CHECKMULTISIG
+  0 S1H S2H 2 P1 P2 2 CHECKMULTISIG
+
+These previously <code>FALSE</code> scripts will fail immediately under the new rules:
+
+  F P1 CHECKSIG
+  0 S2L S1L 2 P1 P2 2 CHECKMULTISIG
+  0 S1L F   2 P1 P2 2 CHECKMULTISIG
+  0 F   S2L 2 P1 P2 2 CHECKMULTISIG
+  0 S1L 0   2 P1 P2 2 CHECKMULTISIG
+  0 0   S2L 2 P1 P2 2 CHECKMULTISIG
+  0 F   0   2 P1 P2 2 CHECKMULTISIG
+  0 0   F   2 P1 P2 2 CHECKMULTISIG
 
 
 ==Deployment==
 
-This BIP will be deployed by "version bits" [https://github.com/bitcoin/bips/blob/master/bip-0009.mediawiki BIP9] using the same parameters for BIP141 and BIP143, with the name "segwit" and using bit 1.
+This BIP will be deployed by "version bits" [https://github.com/bitcoin/bips/blob/master/bip-0009.mediawiki BIP9]. Details TBD.
 
 For Bitcoin mainnet, the BIP9 starttime will be midnight TBD UTC (Epoch timestamp TBD) and BIP9 timeout will be midnight TBD UTC (Epoch timestamp TBD).
 
-For Bitcoin testnet, the BIP9 starttime will be midnight 1 May 2016 UTC (Epoch timestamp 1462060800) and BIP9 timeout will be midnight 1 May 2017 UTC (Epoch timestamp 1493596800).
+For Bitcoin testnet, the BIP9 starttime will be midnight TBD UTC (Epoch timestamp TBD) and BIP9 timeout will be midnight TBD UTC (Epoch timestamp TBD).
 
 
 ==Compatibility==
 
-The reference client has produced compatible signatures since v0.9.0, and NULLDUMMY and LOW_S have been enforced as relay policy by the reference client since v0.10.0 and v0.11.1 respectively. As of August 2016, very few transactions violating the requirement are being added to the chain. In addition, every non-compliant signature can trivially be converted into a compliant one, so there is no loss of functionality by this requirement.
+The reference client has produced LOW_S compatible signatures since v0.9.0, and the LOW_S rule has been enforced as relay policy by the reference client since v0.11.1. As of August 2016, very few transactions violating the requirement are being added to the chain. For all scriptPubKey types in actual use, non-compliant signatures can trivially be converted into compliant ones, so there is no loss of functionality by these requirements.
+
+Scripts with failing <code>OP_CHECKSIG</code> or <code>OP_CHECKMULTISIG</code> rarely happen on the chain. The NULLFAIL rule has been enforced as relay policy by the reference client since v0.13.1.
+
+Users MUST pay extra attention to these new rules when designing exotic scripts.
 
 
 ==Implementation==
 
-An implementation for the reference client is available at https://github.com/bitcoin/bitcoin/pull/8533
+Implementations for the reference client is available at:
+
+https://github.com/bitcoin/bitcoin/blob/35fe0393f216aa6020fc929272118eade5628636/src/script/interpreter.cpp#L185
+
+and
+
+https://github.com/bitcoin/bitcoin/pull/8634
 
 
 ==Footnotes==