Upgrade & Recovery Instructions for lotus v1.4.0 #5232

momack2 · 2020-12-19T22:06:45Z

momack2
Dec 19, 2020

Shut down your nodes
Upgrade your nodes and miners to v1.4.0 and restart the daemon.
Run lotus chain sethead --epoch=336458
Shut down your daemon
Start your daemons (and miners if applicable)
Miners only: start your lotus miner with the --nosync flag --> lotus-miner run --nosync

Upgrading Notes:

If you continue to have sync issues after running the above steps, please repeat them with an earlier epoch in step 3 (Run lotus chain sethead --epoch=336400).
You do NOT need to delete your Lotus repo - just get the latest code and build afresh! (See Upgrade in Place for instructions on how to upgrade safely.)
Note: this release is built on top of v1.3.0 and the v.1.3.0 upgrade will still be happening as scheduled at 343200.

The network will catch up for all missed epochs in the last few hours. During this period lotus nodes will mine blocks faster than usual. This means the chain will be forky and quality will be low.
Once the chain catches up to the current time/epoch, normal operation will resume. We expect catchup to take around 1-2 hours. The chain should stabilize by epoch 337200.
If your node hasn’t synced properly with the chain at that epoch, you may want to re-sync (sethead again, and let your node catch up).

What happened? You can read about the bug that caused this chain halt in the release notes.
Follow the Status Page. You can follow the status page at https://filecoin.statuspage.io/

Syncing to the chain

If your miner is offline because lotus won't let it start until it syncs
- Make sure to start your lotus miner with the --nosync flag (e.g.lotus-miner run --nosync)
If your node does not sync with the main chain, or does not progress:
- Make sure you have the right version:
  - lotus version output should 1.4.0 for both client and server
- Run lotus chain sethead --epoch=336400
- Restart your node

Missing ProveCommits

As the chain catches up, some miners may miss ProveCommit deadlines (ProveCommits must be submitted within ~24h from PreCommit), which will lead to PreCommitDeposit being lost.

Missing WindowPoSts

As the chain catches up, some miners may miss WindowPoSts. Note that there is no penalty for the first missed WindowPoSt, so most miners should be fine if they recover tomorrow. Only miners who had missed the same exact deadlines yesterday should face fees.
Miners may lose some power as a result of missing WindowPoSt during catch-up, but this should recover as normal when WindowPoSts are submitted tomorrow for the missed deadlines.

Deals

BaseFee

We expect BaseFee to be volatile as the network catches up since chain capacity will be lower and new blocks may be over-subscribed with delayed messages.