BIP8: replace FAILING with MUST_SIGNAL

This removes the FAILING state and adds compulsory signalling during a
new MUST_SIGNAL phase during the last retarget period prior to the
timeout height.

This ensures that if a deployment occurs using bip8 with
lockinontimeout=false and timeoutheight=N, that a later deployment using
bip8 with lockinontimeout=true and timeoutheight=K, where K<N that any
chain where LOCKED_IN is reached prior to height K, will be accepted as
valid by nodes using either set of deployment parameters.

It also ensures that the soft-fork's changed rules are only enforced
on chain a retarget period after signalling indicates enforcement is
expected (which was not previously the case if the FAILING to ACTIVE
transition took place).

Author: ajtowns

bip-0008.mediawiki

@@ -37,8 +37,8 @@ Each soft fork deployment is specified by the following per-chain parameters (fu
 # The '''name''' specifies a very brief description of the soft fork, reasonable for use as an identifier. For deployments described in a single BIP, it is recommended to use the name "bipN" where N is the appropriate BIP number.
 # The '''bit''' determines which bit in the nVersion field of the block is to be used to signal the soft fork lock-in and activation. It is chosen from the set {0,1,2,...,28}.
 # The '''startheight''' specifies the height of the first block at which the bit gains its meaning.
-# The '''timeoutheight''' specifies a block height at which the miner signalling ends. Once this height has been reached, if the soft fork has not yet locked in (excluding this block's bit state), the deployment is either considered failed on all descendants of the block (but see the exception during '''FAILING''' state), or, if '''lockinontimeout'' is true, transitions to the '''LOCKED_IN''' state.
-# The '''lockinontimeout''' boolean if set to true, will transition state to '''LOCKED_IN''' at timeoutheight if not already '''LOCKED_IN''' or '''ACTIVE'''.
+# The '''timeoutheight''' specifies a block height at which the miner signalling ends. Once this height has been reached, if the soft fork has not yet locked in (excluding this block's bit state), the deployment is considered failed on all descendants of the block.
+# The '''lockinontimeout''' boolean if set to true, blocks are required to signal in the final period, ensuring the soft fork has locked in by timeoutheight.
 
 ===Selection guidelines===
 
@@ -59,10 +59,10 @@ With each block and soft fork, we associate a deployment state. The possible sta
 
 # '''DEFINED''' is the first state that each soft fork starts out as. The genesis block is by definition in this state for each deployment.
 # '''STARTED''' for blocks at or beyond the startheight.
-# '''LOCKED_IN''' for one retarget period after the first retarget period with STARTED blocks of which at least threshold have the associated bit set in nVersion, or for one retarget period after the timeout when '''lockinontimeout''' is true.
+# '''MUST_SIGNAL''' for one retarget period prior to the timeout, if LOCKED_IN was not reached and '''lockinontimeout''' is true.
+# '''LOCKED_IN''' for one retarget period after the first retarget period with STARTED (or MUST_SIGNAL) blocks of which at least threshold have the associated bit set in nVersion.
 # '''ACTIVE''' for all blocks after the LOCKED_IN retarget period.
-# '''FAILING''' for one retarget period after the timeout, if LOCKED_IN was not reached and '''lockinontimeout''' is false.
-# '''FAILED''' for all blocks after the FAILING retarget period.
+# '''FAILED''' for all blocks after the timeoutheight if LOCKED_IN is not reached.
 
 ===Bit flags===
 
@@ -77,14 +77,13 @@ for the purposes of this proposal, and support two future upgrades for different
 When a block nVersion does not have top bits 001, it is treated as if all
 bits are 0 for the purposes of deployments.
 
-Miners must continue setting the bit in LOCKED_IN phase so uptake is visible and acknowledged.
-Blocks without the applicable bit set are invalid during this period.
-For flexibility, this rule does NOT require the top 3 bits to be set any particular way.
-
 ===New consensus rules===
 
 The new consensus rules for each soft fork are enforced for each block that has ACTIVE state.
 
+During the MUST_SIGNAL and LOCKED_IN phases, blocks that fail to signal are invalid.
+For flexibility, during the LOCKED_IN phase only, this rule does NOT require the top 3 bits to be set any particular way.
+
 ===State transitions===
 
 <img src="bip-0008/states.png" align="middle"></img>
@@ -121,7 +120,8 @@ We remain in the initial state until we reach the start block height.
 After a period in the STARTED state, we tally the bits set,
 and transition to LOCKED_IN if a sufficient number of blocks in the past period set the deployment bit in their
 version numbers. The threshold is ≥1916 blocks (95% of 2016), or ≥1512 for testnet (75% of 2016).
-If the threshold hasn't been met, and we reach the timeout, then we either transition to LOCKED_IN state anyway (if lockinontimeout is true), or we transition to FAILING.
+If the threshold hasn't been met, lockinontimeout is true, and we are at the last period before the timeout, then we transition to MUST_SIGNAL.
+If the threshold hasn't been met and we reach the timeout, we transition directly to FAILED.
 
 Note that a block's state never depends on its own nVersion; only on that of its ancestors.
 
@@ -131,28 +131,22 @@ Note that a block's state never depends on its own nVersion; only on that of its
             for (i = 0; i < 2016; i++) {
                 walk = walk.parent;
                 if (walk.nVersion & 0xE0000000 == 0x20000000 && (walk.nVersion >> bit) & 1 == 1) {
-                    count++;
+                    ++count;
                 }
             }
             if (count >= threshold) {
                 return LOCKED_IN;
+            } else if (lockinontimeout && block.height + 2016 >= timeoutheight) {
+                return MUST_SIGNAL;
             } else if (block.height >= timeoutheight) {
-                return (lockinontimeout == true) ? LOCKED_IN : FAILING;
+                return FAILED;
             }
             return STARTED;
 
-If the deployment is not LOCKED_IN by the timeout (or '''lockinontimeout'''), it has a single retarget period during which it may still become active, only by unanimous signalling in every block.
-This state exists such that if '''lockinontimeout''' is set to true later, it remains compatible with the original deployment.
+If we have finished a period of MUST_SIGNAL, we transition directly to LOCKED_IN.
 
-        case FAILING:
-            walk = block;
-            for (i = 0; i < 2016; i++) {
-                walk = walk.parent;
-                if (walk.nVersion & 0xE0000000 == 0x20000000 && ((walk.nVersion >> bit) & 1) != 1) {
-                    return FAILED;
-                }
-            }
-            return ACTIVE;
+        case MUST_SIGNAL:
+            return LOCKED_IN;
 
 After a retarget period of LOCKED_IN, we automatically transition to ACTIVE.
 
@@ -178,6 +172,23 @@ current retarget period (i.e. up to and including its ancestor with height block
 it is possible to implement the mechanism above efficiently and safely by caching the resulting state of every multiple-of-2016
 block, indexed by its parent.
 
+===Mandatory signalling===
+
+Blocks received while in the MUST_SIGNAL and LOCKED_IN phases must be checked to ensure that they signal. For example:
+
+    if (GetStateForBlock(block) == MUST_SIGNAL) {
+        if ((block.nVersion & 0xE0000000) != 0x20000000 || ((block.nVersion >> bit) & 1) != 1) {
+            return state.Invalid(BlockValidationResult::RECENT_CONSENSUS_CHANGE, "bad-version-bip8-must-signal");
+        }
+    }
+    if (GetStateForBlock(block) == LOCKED_IN) {
+        if (((block.nVersion >> bit) & 1) != 1) {
+            return state.Invalid(BlockValidationResult::RECENT_CONSENSUS_CHANGE, "bad-version-bip8-locked-in");
+        }
+    }
+
+Implementations should be careful not to ban peers that send blocks that are invalid due to not signalling (or blocks that build on those blocks), as that would allow an incompatible chain that is only briefly longer than the compliant chain to cause a split of the p2p network. If that occurred, nodes that have not set ''lockinontimeout'' may not see new blocks in the compliant chain, and thus not reorg to it at the point when it has more work, and would thus not be following the valid chain with the most work.
+
 ===Warning mechanism===
 
 To support upgrade warnings, an extra "unknown upgrade" is tracked, using the "implicit bit" mask = (block.nVersion & ~expectedVersion) != 0. Mask will be non-zero whenever an unexpected bit is set in nVersion.  Whenever LOCKED_IN for the unknown upgrade is detected, the software should warn loudly about the upcoming soft fork. It should warn even more loudly after the next retarget period (when the unknown upgrade is in the ACTIVE state).
@@ -211,7 +222,7 @@ The template Object is also extended:
 The "version" key of the template is retained, and used to indicate the server's preference of deployments.
 If versionbits is being used, "version" MUST be within the versionbits range of [0x20000000...0x3FFFFFFF].
 Miners MAY clear or set bits in the block version WITHOUT any special "mutable" key, provided they are listed among the template's "vbavailable" and (when clearing is desired) NOT included as a bit in "vbrequired".
-Servers MUST set bits in "vbrequired" for deployments in LOCKED_IN state, to ensure blocks produced are valid.
+Servers MUST set bits in "vbrequired" for deployments in MUST_SIGNAL and LOCKED_IN states, to ensure blocks produced are valid.
 
 Softfork deployment names listed in "rules" or as keys in "vbavailable" may be prefixed by a '!' character.
 Without this prefix, GBT clients may assume the rule will not impact usage of the template as-is; typical examples of this would be when previously valid transactions cease to be valid, such as BIPs 16, 65, 66, 68, 112, and 113.
@@ -225,9 +236,8 @@ https://github.com/bitcoin/bitcoin/compare/master...luke-jr:bip8
 
 ==Contrasted with BIP 9==
 
-* The '''lockinontimeout''' flag is added. BIP 9 would only transition to the FAILED state when timeout was reached.
+* The '''lockinontimeout''' flag is added, providing a way to guarantee transition to LOCKED_IN.
 * Block heights are used for the deployment monotonic clock, rather than median-time-past.
-* The last-ditch effort during a new FAILING state is added to allow '''lockinontimeout''' to be safely set after the initial deployment.
 
 ==Backwards compatibility==
 

bip-0008/states.dot

@@ -1,34 +1,34 @@
 digraph {
   rankdir=TD;
 
-  node  [style="rounded,filled,bold", shape=box, fixedsize=true, width=1.3, fontname="Arial"];
+  node [style="rounded,filled,bold", shape=box, fixedsize=true, width=1.5, fontname="Arial"];
 
   edge [weight = 100];
   "DEFINED" -> "STARTED" [label="height >= start_height"];
-  "STARTED" -> "FAILING" [label="height >= timeoutheight AND NOT lockinontimeout"];
-  "STARTED" -> "LOCKED_IN" [label="(height < timeoutheight AND threshold reached)\nOR\n(height >= timeoutheight AND lockinontimeout)"];
+  "STARTED" -> "MUST_SIGNAL" [label="height + 2016 >= timeoutheight AND lockinontimeout"];
+  "STARTED" -> "FAILED" [label="height >= timeoutheight\nAND\nNOT lockinontimeout"];
   "LOCKED_IN" -> "ACTIVE" [label="always"];
-  "FAILING" -> "FAILED" [label="NOT all blocks signal"];
+  "MUST_SIGNAL" -> "LOCKED_IN" [label="always"];
 
   edge [weight = 1];
-  "FAILING" -> "ACTIVE" [label="all blocks signal"];
+  "STARTED" -> "LOCKED_IN" [label="height < timeoutheight\nAND\nthreshold reached"];
+
+  "FAILED" -> "LOCKED_IN" [style=invis];
 
   "DEFINED":sw -> "DEFINED":nw;
   "STARTED":sw -> "STARTED":nw;
   "ACTIVE":sw -> "ACTIVE":nw;
   "FAILED":sw -> "FAILED":nw;
 
   "STARTED" [fillcolor="#a0a0ff"];
-
-  "FAILING"   [fillcolor="#ffffa0"];
+  "MUST_SIGNAL" [fillcolor="#a0a0ff"];
   "LOCKED_IN" [fillcolor="#ffffa0"];
-  "ACTIVE" [fillcolor="#a0ffa0", shape=box];
-  "FAILED" [fillcolor="#ffa0a0", shape=box];
-
-  "ACTIVE" -> "FAILED" [style=invis];
+  "ACTIVE" [fillcolor="#a0ffa0"];
+  "FAILED" [fillcolor="#ffa0a0"];
 
-  { rank=same; "STARTED" "FAILING" }
-  { rank=sink; "ACTIVE" "FAILED" }
+  { rank=same; "STARTED" "MUST_SIGNAL" }
+  { rank=same; "FAILED" "LOCKED_IN" }
+  { rank=sink; "ACTIVE" }
 }