BIP-322 updates to fix verification and other fixes

Author: kallewoof

bip-0322.mediawiki

@@ -34,7 +34,7 @@ Two actions "Sign" and "Verify" are defined along with one ''purpose'', "SignMes
 !Name
 !Comment
 |-
-|Uint32||4||flags||standard flags (1-to-1 with standard flags in Bitcoin Core)
+|Uint32||4||version||BIP322 version format; must be equal to 1; if > 1, verifier must abort the verification process
 |-
 |Uint8||1||entries||number of proof entries<ref><strong>Why support multiple proofs?</strong> It is non-trivial to check a large number of individual proofs for duplicates. Software could be written to do so, but it seems more efficient to build this check into the specification itself.</ref>
 |}
@@ -70,7 +70,7 @@ A verification call will return a result code according to the table below.
 |-
 |INCOMPLETE||One or several of the given challenges had an empty proof. The prover may need some other entity to complete the proof.
 |-
-|INCONCLUSIVE||One or several of the given proofs used unknown opcodes or the scriptPubKey had an unknown witness version, perhaps due to the verifying node being outdated.
+|INCONCLUSIVE||One or several of the given proofs was consensus-valid but policy-invalid.
 |-
 |VALID||All proofs were deemed valid.
 |-
@@ -81,7 +81,7 @@ A verification call will return a result code according to the table below.
 
 == Signing and Verifying ==
 
-If the challenge consists of a single address and the address is in the P2PK(H) (legacy) format, sign using the legacy format (further information below). Otherwise continue as stated below.
+If the challenge consists of a single address and the address is in the P2PKH (legacy) format, sign using the legacy format (further information below). Otherwise continue as stated below.
 
 Let there be an empty set <code>inputs</code> which is populated and tested at each call to one of the actions below.
 
@@ -90,7 +90,7 @@ Let there be an empty set <code>inputs</code> which is populated and tested at e
 The "SignMessage" purpose generates a sighash based on a scriptPubKey and a message. It emits a VALID verification result code unless otherwise stated.
 
 # Return INVALID if scriptPubKey already exists in <code>inputs</code> set, otherwise insert it<ref><strong>Why track duplicates?</strong> Because a 3-entry proof is not proving 3 entries unless they are all distinct</ref>
-# Define the message pre-image as the sequence "Bitcoin Message:" concatenated with the message, encoded in UTF-8 using Normalization Form Compatibility Decomposition (NFKD)
+# Define the message pre-image as the sequence "Bitcoin Signed Message:\n" concatenated with the message, encoded in UTF-8 using Normalization Form Compatibility Decomposition (NFKD)
 # Let sighash = sha256(sha256(scriptPubKey || pre-image))
 
 A private key may be used directly to sign a message. In this case, its P2WPKH bech32 address shall be derived, and used as the input.
@@ -111,9 +111,10 @@ The "Verify" action takes as input a standard flags value, a script sig, an opti
 It emits one of INCONCLUSIVE, VALID, INVALID, or ERROR.
 
 # Obtain the sighash and scriptPubKey from the purpose; pass on result code if not VALID
-# If one or more of the standard flags are unknown, return INCONCLUSIVE
-# Verify Script with flags=standard flags, scriptSig=script sig, scriptPubKey=scriptPubKey, witness=witness, and sighash=sighash
-# Return VALID if verify succeeds, otherwise return INVALID
+# Verify Script with flags=consensus flags (currently P2SH, DERSIG, NULLDUMMY, CLTV, CSV, WITNESS), scriptSig=script sig, scriptPubKey=scriptPubKey, witness=witness, and sighash=sighash
+# Return INVALID if verification fails
+# Verify Script with flags=standard flags (above plus STRICTENC, MINIMALDATA, etc.), scriptSig=script sig, scriptPubKey=scriptPubKey, witness=witness, and sighash=sighash
+# Return VALID if verification succeeds, otherwise return INCONCLUSIVE
 
 === Multiple Proofs ===
 
@@ -130,29 +131,30 @@ Note that the order of the entries in the proof must match the order of the entr
 
 == Legacy format ==
 
-The legacy format is restricted to the legacy P2PK(H) address format, and restricted to one single challenge (address).
+The legacy format is restricted to the legacy P2PKH address format, and restricted to one single challenge (address).
 
-Any other input (e.g. multiple addresses, or non-P2PK(H) address format(s)) must be signed using the new format described above.
+Any other input (e.g. multiple addresses, or non-P2PKH address format(s)) must be signed using the new format described above.
 
 === Signing ===
 
-Given the P2PK(H) address <code>a</code> and the message <code>m</code>:
-# let <code>p</code> be the pubkey-hash contained in <code>a</code>
-# let <code>x</code> be the private key associated with <code>p</code>
+Given the P2PKH address <code>a</code> and the message <code>m</code>, and the pubkey-hash function <code>pkh(P) = ripemd160(sha256(P))</code>:
+
+# let <code>p</code> be the pubkey-hash <code>pkh(P)</code> for the pubkey <code>P</code>, contained in <code>a</code>
+# let <code>x</code> be the private key associated with <code>P</code> so that <code>pkh(xG) = p</code>
 # let <code>digest</code> be <code>SHA56d("Bitcoin Signed Message:\n"||m)</code>
 # create a compact signature <code>sig</code> (aka "recoverable ECDSA signature") using <code>x</code> on <code>digest</code>
 
 The resulting proof is <code>sig</code>, serialized using the base64 encoding.
 
 === Verifying ===
 
-Given the P2PK(H) address <code>a</code>, the message <code>m</code>, and the compact signature <code>sig</code>:
+Given the P2PKH address <code>a</code>, the message <code>m</code>, the compact signature <code>sig</code>, and the pubkey-hash function <code>pkh(P) = ripemd160(sha256(P))</code>:
 
-# let <code>p</code> be the pubkey-hash contained in <code>a</code>
+# let <code>p</code> be the pubkey-hash <code>pkh(P)</code> for the pubkey <code>P</code>, contained in <code>a</code>
 # let <code>digest</code> be <code>SHA56d("Bitcoin Signed Message:\n"||m)</code>
 # attempt pubkey recovery for <code>digest</code> using the signature <code>sig</code> and store the resulting pubkey into <code>Q</code>
 ## fail verification if pubkey recovery above fails
-# let <code>q</code> be the pubkey-hash of <code>Q</code>
+# let <code>q</code> be the pubkey-hash <code>pkh(Q)</code> for the pubkey <code>Q</code>
 # if <code>p == q</code>, the proof is valid, otherwise it is invalid
 
 == Compatibility ==
@@ -179,17 +181,50 @@ Thanks to David Harding, Jim Posen, Kalle Rosenbaum, Pieter Wuille, and many oth
 
 This document is licensed under the Creative Commons CC0 1.0 Universal license.
 
+== Consensus and standard flags ==
+
+Each flag is associated with some type of enforced rule (most often a soft fork). There are two sets of flags: consensus flags (which result in a block being rejected, if violated), and policy flags (which result in a transaction being accepted only if it is contained within an actual block, and rejected otherwise, if violated). The policy flags are a super-set of the consensus flags.
+
+BIP322 specifies that a proof that validates for both rulesets is valid, a proof that validates for consensus rules, but not for policy rules, is "inconclusive", and a proof that does not validate for consensus rules is "invalid" (regardless of policy rule validation).
+
+The ruleset sometimes changes. This BIP does not intend to be complete, nor does it indicate enforcement of rules, it simply lists the rules as they stand at the point of writing.
+
+=== Consensus rules ===
+
+* P2SH: evaluate P2SH ([https://github.com/bitcoin/bips/blob/master/bip-0016.mediawiki BIP16]) subscripts
+* DERSIG: enforce strict DER ([https://github.com/bitcoin/bips/blob/master/bip-0066.mediawiki BIP66]) compliance
+* NULLDUMMY: enforce NULLDUMMY ([https://github.com/bitcoin/bips/blob/master/bip-0147.mediawiki BIP147])
+* CHECKLOCKTIMEVERIFY: enable CHECKLOCKTIMEVERIFY ([https://github.com/bitcoin/bips/blob/master/bip-0065.mediawiki BIP65])
+* CHECKSEQUENCEVERIFY: enable CHECKSEQUENCEVERIFY ([https://github.com/bitcoin/bips/blob/master/bip-0112.mediawiki BIP112])
+* WITNESS: enable WITNESS ([https://github.com/bitcoin/bips/blob/master/bip-0141.mediawiki BIP141])
+
+=== Policy rules ===
+
+All of the above, plus (subject to change):
+
+* STRICTENC: non-strict DER signature or undefined hashtype
+* MINIMALDATA: require minimal encodings for all push operations
+* DISCOURAGE_UPGRADABLE_NOPS: discourage use of NOPs reserved for upgrades
+* CLEANSTACK: require that only a single stack element remains after evaluation
+* MINIMALIF: Segwit script only: require the argument of OP_IF/NOTIF to be exactly 0x01 or empty vector
+* NULLFAIL: signature(s) must be empty vector if a CHECK(MULTI)SIG operation failed
+* LOW_S: signature with S > order/2 in a checksig operation
+* DISCOURAGE_UPGRADABLE_WITNESS_PROGRAM: v1-16 witness programs are non-standard (i.e. forbidden)
+* WITNESS_PUBKEYTYPE: public keys in segregated witness scripts must be compressed
+* CONST_SCRIPTCODE: OP_CODESEPARATOR and FindAndDelete fail any non-segwit scripts
+
 == Test vectors ==
 
-* <code>STANDARD_SCRIPT_VERIFY_FLAGS = 01ffdf (131039)</code>
-* <code>address = 2MsnqGxX7Abtn4b379MEpkDaD3VbNKQosd8</code>
-* <code>message = "hello world"</code>
-* <code>sighash = 7b66a1861b4e179e1dbab4702e26bcefeabf1cada7cccc97b6ebaec89a035d84</code> (<code>sha256d("Bitcoin Message:hello world")</code>)
+== Native segwit test vector ==
+
+* <code>address = bcrt1qe7nte4zk4ayly5tc53dtdjupgkz0lr8azx3rzz</code>
+* <code>message = hello</code>
+* <code>sighash = 790eef86c204f0bff969ff822121317aa34eff0215dbd30ccf031e7b2f3f0cc1</code> (<code>sha256d("Bitcoin Signed Message:\n:hello")</code>)
 
 A possible proof is:
 
-* HEX: <code>dfff01000117160014689bbb5d76774321c652832ea209958fa1770b330247304402204368b119399d33b9bc9beef06d713becefd3ac508dc95ff62d1859d4912960c7022063d88ddc648faed710b3f870b7a839fdc1b3bfc3c3bd065df51bbbd8c386c81c012102b4e4c6d5021576a5c0bc4535890c3f17e1ff23a94eac87beb0a5e8747c42d920</code>
-* Base64: <code>3/8BAAEXFgAUaJu7XXZ3QyHGUoMuogmVj6F3CzMCRzBEAiBDaLEZOZ0zubyb7vBtcTvs79OsUI3JX/YtGFnUkSlgxwIgY9iN3GSPrtcQs/hwt6g5/cGzv8PDvQZd9Ru72MOGyBwBIQK05MbVAhV2pcC8RTWJDD8X4f8jqU6sh76wpeh0fELZIA==</code>
+* HEX: <code>01000000010002473044022075b4fb40421d55c55462879cb352a85eeb3af2138d3f02902c9143f12870f5f70220119c2995c1661138142f3899c1fd6d1af7e790e0e081be72db9ce7bf5b5b932901210290beccd02b73eca57467b2b6f1e47161a9b76a5e67586e7c1dee9ea6e2dcd869</code>
+* Base64: <code>AQAAAAEAAkcwRAIgdbT7QEIdVcVUYoecs1KoXus68hONPwKQLJFD8Shw9fcCIBGcKZXBZhE4FC84mcH9bRr355Dg4IG+ctuc579bW5MpASECkL7M0Ctz7KV0Z7K28eRxYam3al5nWG58He6epuLc2Gk=</code>
 
 Split into components:
 
@@ -201,24 +236,24 @@ Split into components:
 !Value
 !Comment
 |-
-|Uint32||4||flags||<code>dfff0100</code>||standard flags used in signing
+|Uint32||4||flags||<code>01000000</code>||proof format version
 |-
 |Uint8||1||entries||<code>01</code>||1 entry
 |-
-|VarInt||1-8||scriptsiglen||<code>17</code>||23 byte scriptsig
-|-
-|Uint8[32]||32||scriptsig||<code>160014689bbb5d76774321c652832ea209958fa1770b33</code>||ScriptSig data
+|VarInt||1-8||scriptsiglen||<code>00</code>||0 byte scriptsig
 |-
-|VarInt||1-8||witlen||<code>02</code>||2 entries in witness stack
+|VarInt||1-8||wit entries||<code>02</code>||2 witness stack entries
 |-
 |VarInt||1-8||entry1len||<code>47</code>||71 byte entry
 |-
-|Uint8[71]||71||entry1||<code>304402204368b119399d33b9bc9beef06d713becefd3ac50
-8dc95ff62d1859d4912960c7022063d88ddc648faed710b3
-f870b7a839fdc1b3bfc3c3bd065df51bbbd8c386c81c01</code>||Witness stack item 1
+|Uint8[71]||71||entry1||<code>3044022075b4fb40421d55c55462879cb352a85eeb3af213
+8d3f02902c9143f12870f5f70220119c2995c1661138142f
+3899c1fd6d1af7e790e0e081be72db9ce7bf5b5b932901</code>||Witness stack item 1
 |-
 |VarInt||1-8||entry2len||<code>21</code>||33 byte entry
 |-
-|Uint8[33]||33||entry2||<code>02b4e4c6d5021576a5c0bc4535890c3f17e1ff23a94eac87
-beb0a5e8747c42d920</code>||Witness stack item 2
+|Uint8[33]||33||entry2||<code>0290beccd02b73eca57467b2b6f1e47161a9b76a5e67586e
+7c1dee9ea6e2dcd869</code>||Witness stack item 2
 |}
+
+The above test vector is for a bech32 P2WPKH (native segwit) address. (Once BIP solidifies, will add test vector for other types.)