Understanding and debugging Nimbus EVM JSON tests

Introduction

Nimbus JSON tests suite is taken from the official Ethereum tests. As of July 2018, only the VMTests are used out of the following list:

• ABITests
• BasicTests
• BlockchainTests
• GeneralStateTests
• GenesisTests
• KeyStoreTests
• PoWTests
• RLPTests
• RPCTests
• TransactionTests
• TrieTests
• VMTests

The full documentation for the official Ethereum tests is available here.

VMTests

JSON structure

A JSON EVM test has the following structure (example is add0.json).

{
    "add0" : {
        "_info" : {
            "comment" : "",
            "filledwith" : "cpp-1.3.0+commit.6e0ce939.Linux.g++",
            "lllcversion" : "Version: 0.4.18-develop.2017.9.25+commit.a72237f2.Linux.g++",
            "source" : "src/VMTestsFiller/vmArithmeticTest/add0Filler.json",
            "sourceHash" : "dcc7fc8aebdc2d7334440cfe6c63172941b4164c1ba8c32897318ca0cdfb7a1c"
        },
        "callcreates" : [
        ],
        "env" : {
            "currentCoinbase" : "0x2adc25665018aa1fe0e6bc666dac8fc2697ff9ba",
            "currentDifficulty" : "0x0100",
            "currentGasLimit" : "0x0f4240",
            "currentNumber" : "0x00",
            "currentTimestamp" : "0x01"
        },
        "exec" : {
            "address" : "0x0f572e5295c57f15886f9b263e2f6d2d6c7b5ec6",
            "caller" : "0xcd1722f2947def4cf144679da39c4c32bdc35681",
            "code" : "0x7fffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff7fffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff01600055",
            "data" : "0x",
            "gas" : "0x0186a0",
            "gasPrice" : "0x5af3107a4000",
            "origin" : "0xcd1722f2947def4cf144679da39c4c32bdc35681",
            "value" : "0x0de0b6b3a7640000"
        },
        "gas" : "0x013874",
        "logs" : "0x1dcc4de8dec75d7aab85b567b6ccd41ad312451b948a7413f0a142fd40d49347",
        "out" : "0x",
        "post" : {
            "0x0f572e5295c57f15886f9b263e2f6d2d6c7b5ec6" : {
                "balance" : "0x0de0b6b3a7640000",
                "code" : "0x7fffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff7fffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff01600055",
                "nonce" : "0x00",
                "storage" : {
                    "0x00" : "0xfffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffe"
                }
            }
        },
        "pre" : {
            "0x0f572e5295c57f15886f9b263e2f6d2d6c7b5ec6" : {
                "balance" : "0x0de0b6b3a7640000",
                "code" : "0x7fffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff7fffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff01600055",
                "nonce" : "0x00",
                "storage" : {
                }
            }
        }
    }
}

Important: As of July 2018, VMTests always use Homestead fork for opcode implementations (notably CALL) and gas prices. We should not rely on currentNumber.

The important fields are:

• exec section:

  • code: the code being executed in isolation. (In production this is part of a transaction).
  • gas: usually 0x0186a0 (100000) the starting gas

• gas field: the remaining gas after code execution. Here 0x013874 (79988)

• pre section: the state before execution of the code by the VM

  • storage: if the account already stores data that we can retrieve with SLOAD or overwrite/reset with SSTORE

• post section: the state after execution of the code by the VM. If there is no post section, the code is supposed to throw an EVM exception. After an EVM exception, gas is consumed but state is reverted.

  Implementation-wise computation.isError returns true and an error message is available in computation.error field (implementation as of July 2018)

Decompiling bytecode.

You can decompile EVM bytecode using Etherscan: https://etherscan.io/opcode-tool Pasting 0x6003600202600055 gives:

[1] PUSH1 0x03 
[3] PUSH1 0x02 
[4] MUL 
[6] PUSH1 0x00 
[7] SSTORE 

Alternatively you can use the following:

import ../nimbus/vm/code_stream, strformat

var c = newCodeStreamFromUnescaped("0x6003600202600055")

let opcodes = c.decompile()
for op in opcodes:
  echo &"[{op[0]}]\t{op[1]}\t{op[2]}"

# [1]     PUSH1   0x03
# [3]     PUSH1   0x02
# [4]     MUL
# [6]     PUSH1   0x00
# [7]     SSTORE
# [-1]    STOP

The number in bracket refers to the position after reading the opcode and its arguments (i.e. the value of the program counter).

add0 (0x7fffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff7fffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff01600055) decompiles to:

[32] PUSH32 0xffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff 
[65] PUSH32 0xffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff 
[66] ADD 
[68] PUSH1 0x00 
[69] SSTORE