provider.getBlock does not output all the block details

[zemse]

I am working on a block explorer project with ethers.js, experiencing that some data is not being outputted (specifically transaction and state roots). I tried the same with web3.js and could see these details.

getBlock from web3.js (https://web3js.readthedocs.io/en/v1.2.4/web3-eth.html#id51)

{ author: '0x08d85bd1004e3e674042eaddf81fb3beb4853a22',
  difficulty: '340282366920938463463374607431768211452',
  extraData:
   '0xde830207008f5061726974792d457468657265756d86312e33392e30826c69',
  gasLimit: 6041126,
  gasUsed: 21000,
  hash:
   '0xf532be00c9cc1429c9fe11aeebb1b2541b98700c56b75ccca529b29918691739',
  logsBloom:
   '0x00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000',
  miner: '0x08D85Bd1004E3e674042EAddF81Fb3beb4853a22',
  number: 7,
  parentHash:
   '0x473125e4191781ee52242a2f88309bc5f1aeddf51b2c565e57f919f48e542f8d',
  receiptsRoot:
   '0x056b23fbba480696b65fe5a59b8f2148a1299103c4f57df839233af2cf4ca2d2',
  sealFields:
   [ '0x8412c58a07',
     '0xb841ebebece16b24a9d9bf9c25c74d4371a2262669ce7bfaa70f7ee7f21614cfee922eddb6b03828ae81b8f5b2ef9cbb635d498810f93adb63c3d5996ba4ba032f7c01' ],
  sha3Uncles:
   '0x1dcc4de8dec75d7aab85b567b6ccd41ad312451b948a7413f0a142fd40d49347',
  signature:
   'ebebece16b24a9d9bf9c25c74d4371a2262669ce7bfaa70f7ee7f21614cfee922eddb6b03828ae81b8f5b2ef9cbb635d498810f93adb63c3d5996ba4ba032f7c01',
  size: 700,
  stateRoot:
   '0x8ee6d1d71c090e51ba7b153f7d41472e417b44c8b66d31e473f588561bb4f949',
  step: '314935815',
  timestamp: 1574679079,
  totalDifficulty: '2381976568446569244243622252022062675442',
  transactions:
   [ '0xd30e18e0f2d56e23ef85390222ba72e990e2480072d6d948698ed17b1e929894' ],
  transactionsRoot:
   '0x52a43eca1b06b7901b20429f45d687ac3a7dac95fe383d8d1223a1a19290078e',
  uncles: [] }

The ethers.js getBlock gives the following output

{ hash:
   '0xf532be00c9cc1429c9fe11aeebb1b2541b98700c56b75ccca529b29918691739',
  parentHash:
   '0x473125e4191781ee52242a2f88309bc5f1aeddf51b2c565e57f919f48e542f8d',
  number: 7,
  timestamp: 1574679079,
  difficulty: null,
  gasLimit: BigNumber { _hex: '0x5c2e26' },
  gasUsed: BigNumber { _hex: '0x5208' },
  miner: '0x08D85Bd1004E3e674042EAddF81Fb3beb4853a22',
  extraData:
   '0xde830207008f5061726974792d457468657265756d86312e33392e30826c69',
  transactions:
   [ '0xd30e18e0f2d56e23ef85390222ba72e990e2480072d6d948698ed17b1e929894' ] }

I also saw #276 which is closed, so I created a new one.

[zemse]

Just figured out a way to do this with ethers js:
const block = await provider.send('eth_getBlockByNumber', [ethers.utils.bigNumberify(i)._hex, true]);

[ricmoo]

I wasn't planning to add these into the standard Block object, since they are very rarely used and I'm not sure all backends support them.

What are you using them for? I could certainly have my mind changed, I've just never found a use for them.

Also, it is not safe to use the _hex, for 2 reasons, a) _hex is an internal property and may no longer exist in the future (you could use .toHexString()), but more importantly b) the JSON-RPC spec requires that block tags be bare, so you should use ethers.utils.hexValue(i), assuming you are using v5.

The hexValue (in v5) will ensure there are no leading-zeros, unless the value is exactly 0. Some backends will return an error if you use a block tag of 0x0123 for example; you are likely just lucky right now that the blocks are in a even-length-in-hex part of the number line. :)

[zemse]

What are you using them for? I could certainly have my mind changed, I've just never found a use for them.

I'm working on a project in which I need to verify a transaction hash included in a transaction root of a block. I couldn't find transactionsRoot key which intuitively should be there in a block so I started finding for a workaround to get it.

I wasn't planning to add these into the standard Block object since they are very rarely used and I'm not sure all backends support them.

Sure that it is not used at all most of the time. But I think we can add instructions in the docs for those developers who might need that data, how can they get the info they need depending on node whether they'll get it or not.

Also, it is not safe to use the _hex, for 2 reasons, a) _hex is an internal property and may no longer exist in the future (you could use .toHexString()), but more importantly b) the JSON-RPC spec requires that block tags be bare, so you should use ethers.utils.hexValue(i), assuming you are using v5.

Oh thanks for pointing this wrong practice, I was facing this issue with blockTag but couldn't figure it out earlier.

[paouvrard]

@ricmoo Would it be possible to re-open this issue ?
Fields like receiptsRoot, stateRoot and transactionsRoot are really important for doing data verification in dapps: for example building light client proofs.
Would love to use ethers.js but not having the full block data makes it impossible.

[zemse]

I think this could be given a second thought. In the rpc specs they do mention these fields. Though I agree that the use case for these projects is very very niche (like maybe doing some L2 stuff), they are anyway present in response whenever a get block request is done. Though I don't think this is a blocker or urgent at all, but it could be nice to have. Opening so that we won't miss to reconsider it.

not having the full block data makes it impossible.

@paouvrard You may go by the hacky way mentioned in #667 (comment)

[paouvrard]

Thank you @zemse this worked well.

const fullBlock = await provider.send(
    'eth_getBlockByNumber',
    [ethers.utils.hexValue(blockNumber), true]
)

[ricmoo]

I may add some of these back in for v6, but likely won't make this change for v5.

As an extra note, you can do something akin to:

const rawBlock = await provider.send(
    'eth_getBlockByNumber',
    [ethers.utils.hexValue(blockNumber), true]
);
const fullBlock = provider.formatter.blockWithTransactions(rawBlock);
fullBlock.stateRoot = provider.formatter.hash(rawBlock.stateRoot);
// etc.

This will help keep the block largely normalized for the other properties.

[LndnMdiaLb]

hi! this is slightly of topic sorry. I found my way here researching why not all properties are returned and using send is a staright forward workaround. I can't find any information about the Formatter class and what it is useful for ex provider.formatter.blockWithTransactions. I have found formatter.ts but can't find any documentation

[zemse]

Formatter class and what it is useful for

It is useful to convert values from the node into workable types like gasLimit, gasUsed, difficulty into BigNumber and number, timestamp into a JS number (instead of keeping them as raw hex strings). There are formatters for tx response, receipt, log as well.