Fix failing blockchain tests

[holgerd77]

Just ran the blockchain tests after a long time (see also associated issue #337 on a revived BlockchainTests test run strategy).

Currently 66 tests are failing:

1..956
# tests 956
# pass  890
# fail  66

I will list all here so we have a basis to fix without the need (for the initial fixes) to re-run the tests again and again:

RPC_API_Test
GasLimitHigherThan2p63m1
ChainAtoChainB_BlockHash
randomStatetest218BC
randomStatetest224BC
randomStatetest234BC
randomStatetest240BC
randomStatetest255BC
randomStatetest331BC
randomStatetest34BC
randomStatetest35BC
randomStatetest403BC
randomStatetest431BC
randomStatetest432BC
randomStatetest529BC
randomStatetest598BC
randomStatetest65BC
BLOCKHASH_Bounds
OOGStateCopyContainingDeletedContract
blockhashNonConstArg
blockhashTests
suicideCoinbase
ExtraData32
RecallSuicidedContractInOneBlock
log1_correct
timeDiff0
timeDiff12
timeDiff13
timeDiff14
wallet2outOf3txs2
wallet2outOf3txsRevoke
wallet2outOf3txsRevokeAndConfirmAgain
walletReorganizeOwners

Test runs on the listed files can be triggered with node tests/tester.js -b --file='blockhashTests'.

Extra note: I think atm blockchain tests are not updated to take the HF setting into account and explicitly instantiate the VM with it (like on the state tests). Atm this shouldn't be a (the) problem, but this should be updated anyhow.

[holgerd77]

Blockchain tests are now running here: #341

[holgerd77]

On-top-note: we are running low on CircleCI usage capacity (at 91%). If we want to keep the CircleCI switch (which I would assume regarding the advantages) we would have to upgrade rather sooner than later. Think spending an additional 50-100 $ here per month should be in the budget.

[holgerd77]

Started with sending a mail to the CircleCI support if we qualify for the OSS package.

[holgerd77]

Update on CircleCI usage capacity (cc @axic): had some very nice support exchange with Circle, OSS package is project/repo-wise. Seems that we are already on the OSS package with the VM, but it is not displayed on Circle (bug!), the capacity message seems to be another (sigh 😄) bug in the Circle UI and this should not apply for us and we should reach back if we have problems.

[holgerd77]

First note on this:

Many failures preceed the following error stack output:

Invalid block error: Error: Error: Error: invalid receiptTrie invalid gasUsed invalid block stateRoot
    at /Users/hdrewes/Documents/DEV/EthereumJS/ethereumjs-vm/dist/runBlock.js:159:19
    at /Users/hdrewes/Documents/DEV/EthereumJS/ethereumjs-vm/dist/cache.js:106:7
    at /Users/hdrewes/Documents/DEV/EthereumJS/ethereumjs-vm/node_modules/async/dist/async.js:473:16
    at replenish (/Users/hdrewes/Documents/DEV/EthereumJS/ethereumjs-vm/node_modules/async/dist/async.js:1006:25)
    at /Users/hdrewes/Documents/DEV/EthereumJS/ethereumjs-vm/node_modules/async/dist/async.js:1016:9
    at eachLimit$1 (/Users/hdrewes/Documents/DEV/EthereumJS/ethereumjs-vm/node_modules/async/dist/async.js:3196:24)
    at Object.<anonymous> (/Users/hdrewes/Documents/DEV/EthereumJS/ethereumjs-vm/node_modules/async/dist/async.js:1046:16)
    at /Users/hdrewes/Documents/DEV/EthereumJS/ethereumjs-vm/dist/cache.js:102:11
    at /Users/hdrewes/Documents/DEV/EthereumJS/ethereumjs-vm/node_modules/async/dist/async.js:969:16
    at next (/Users/hdrewes/Documents/DEV/EthereumJS/ethereumjs-vm/node_modules/async/dist/async.js:5225:18)
    at /Users/hdrewes/Documents/DEV/EthereumJS/ethereumjs-vm/dist/cache.js:99:7
    at next (/Users/hdrewes/Documents/DEV/EthereumJS/ethereumjs-vm/node_modules/async/dist/async.js:5223:28)
    at /Users/hdrewes/Documents/DEV/EthereumJS/ethereumjs-vm/dist/cache.js:99:7
    at next (/Users/hdrewes/Documents/DEV/EthereumJS/ethereumjs-vm/node_modules/async/dist/async.js:5223:28)
    at /Users/hdrewes/Documents/DEV/EthereumJS/ethereumjs-vm/dist/cache.js:94:9
    at /Users/hdrewes/Documents/DEV/EthereumJS/ethereumjs-vm/node_modules/merkle-patricia-tree/util.js:51:36
not ok 571 last block hash
  ---
    operator: equal
    expected: |-
      '2bda3f5684b144a955a28a1ffdc2dae25e51603f685aae03d0fcc885184d568f'
    actual: |-
      'f7666415a06fe86f39a614be3e707683426ca0cec5a92ccce6664bebbe2ba55c'
    at: self.blockDb.get (/Users/hdrewes/Documents/DEV/EthereumJS/ethereumjs-vm/node_modules/ethereumjs-blockchain/index.js:331:7)

[holgerd77]

Same level as before (66 failing) also with updated blockchain and block libraries.

[danjm]

Below is a script that can tell us the last commit at which as test was passing and the commit at which it started failing. The code is at bottom.

Example: the output for the script when running ruby commit-test-check.rb "tests/tester.js -b --file='RPC_API_Test'":

Once the failing blockchain tests are the top priority, this script can be expanded to iterate over all of the files listed above (#338 (comment)) and output a summary of the commits at which each test started failing. Which should be a good starting point for debugging.

commit-test-check.rb

# To get this script to work
#   - Save this file to the root of the `ethereumjs-vm` directory as
#     `commit-test-check.rb` (or whatever file name you like, but
#     note the name in the example below)
#   - Install ruby: https://www.ruby-lang.org/en/documentation/installation/
#   - Install the JSON gem for Ruby: `gem install json`
#   - Globally install tap-mocha-report: npm install -g tap-mocha-reporter
#   - `git checkout master`
#   - In your terminal, run `ruby commit-test-check.rb` followed by the command
#     you wish to check (which should be contained in double quotes). Example:
#     `ruby commit-test-check.rb "tests/tester.js -b --file='RPC_API_Test'"`
#
# Running the above example should produce the following output:
# ```
# The command `tests/tester.js -b --file='RPC_API_Test'` was passing at commit
# 941329898ad9a1dfce3dc9af0c800486ff71277a
# The stats for that test run were: {"suites"=>0, "tests"=>4, "passes"=>4,
# "pending"=>0, "failures"=>0, "start"=>"2018-09-28T14:46:09.830Z",
# "end"=>"2018-09-28T14:46:15.780Z", "duration"=>5950}
#
# It started failing at 914dfd610a68f6685f1179ee1de3c4bb389741e4
# The stats for that test run were: {"suites"=>0, "tests"=>4, "passes"=>2,
# "pending"=>0, "failures"=>2, "start"=>"2018-09-28T14:45:59.361Z",
# "end"=>"2018-09-28T14:46:05.386Z", "duration"=>6025}
# ```


require 'rubygems'
require 'json'

test_command = ARGV[0]

tests_pass = false
current_commit = `git rev-parse HEAD`
previous_commit = current_commit
test_stats = ''
prev_test_stats = ''

while !tests_pass
  previous_commit = current_commit
  checkout_result = `git checkout HEAD~1`
  # puts checkout_result[/(HEAD\sis\now.+)/]
  checkout_result[/(HEAD\sis\now.+)/]
  current_commit = `git rev-parse HEAD`
  `npm install`
  test_result = `node #{test_command} | tap-mocha-reporter json`
  parsed = JSON.parse(test_result)
  prev_test_stats = test_stats
  test_stats = parsed['stats']
  puts current_commit
  puts test_stats
  tests_pass = test_stats['failures'] === 0
  # puts tests_pass
  puts "-------------------------------------"
end

puts "===================================="
puts ""
puts "The command \`#{test_command}\` was passing at commit #{current_commit}"
puts "The stats for that test run were: #{test_stats}"
puts ""
puts "It started failing at #{previous_commit}"
puts "The stats for that test run were: #{prev_test_stats}"