cryptonote_core: support RandomX V2 and commitments in v17 #10038
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,52 @@ | ||
| # Block hashing | ||
|
|
||
| ## Background | ||
|
|
||
| In Bitcoin, block IDs and Proof-of-Work (PoW) hashes are calculated the same way, which is why on-chain block IDs always start | ||
| with many 0s. However, in Monero, block IDs and PoW hashes are calculated using different hash functions and slightly different | ||
| inputs. This is because the hash functions used for PoW (CryptoNight and RandomX) are much slower to run than other | ||
| cryptographic hash functions, specifically Keccak256, which Monero uses for block IDs. By contrast, Bitcoin uses SHA-256 for | ||
| both block IDs and PoW. The reason that CryptoNight and RandomX were chosen for PoW in Monero is to protect against | ||
| application-specific integrated circuits (ASICs) from dominanting the network hashrate like in Bitcoin. In theory, this | ||
| would restore the principle of "1 CPU, 1 vote" outlined in the Satoshi whitepaper, and bring greater decentralization to the | ||
| Monero exosystem. This document aims to describe exactly how block IDs and PoW hashes are calculated in the Monero reference | ||
| codebase. | ||
|
|
||
| ## CryptoNight epoch | ||
|
|
||
| To be reductive, the underlying design of Cryptonight is to 1) fill a 2MB buffer of memory using a memory-hard function, 2) | ||
| perform over a millon random reads on that buffer to permutate some state, and then 3) do a final hash on the state using | ||
| a random choice between four independent hash fuctions. There are 4 variants of CryptoNight: v0, v1, v2, and v4. This is the | ||
| data flow for how block hashing in the CryptoNight epoch (Monero fork versions v1-v11) is performed: | ||
|
|
||
|  | ||
|
|
||
| ## RandomX epoch | ||
|
|
||
| Like, CryptoNight, RandomX also uses a memory hard function to fill a large space of memory called a "cache". However, this | ||
| space is 256MB, not 2MB. It can be further expanded to 2GB to trade memory usage for compute speed. Unlike CryptoNight's cache, | ||
| the RandomX cache is computed only once every 2048 blocks, not once per block. Monero uses a "seed" block ID of a previous block | ||
| in the chain to fill the memory space. And to make ASICs even harder, RandomX uses random instruction execution as well. This | ||
| is the data flow for how block hashing in the RandomX epoch (Monero fork versions v12-present) is performed: | ||
|
|
||
|  | ||
|
|
||
| ## RandomX, with commitments, epoch | ||
|
|
||
| Because RandomX can be slow to run, especially while building the cache (nearly 1/2 a second on my machine), it inadvertantly | ||
| can introduce a Denial-of-Service (DoS) vector. For this reason, RandomX hash commitments can be added, so the PoW can be | ||
| partially verified using only Blake2B, a much lighter hash function. For simplicity's sake, we can also remove the number of | ||
| transactions in the block from the block hashing blob. This is the data flow for how block hashing works with these changes: | ||
|
|
||
|  | ||
|
|
||
| The way that the hashes are calculated allows us to do partial PoW verification like so: | ||
|
|
||
|  | ||
|
|
||
| A node can be given simply the block header, block content hash, and intermediate PoW hash, and verify that some PoW was done | ||
| using only `randomx_calculate_commitment()` (Blake2B underneath). Passing this partial verification requires PoW to be done using | ||
| Blake2B up to the relevant difficulty. While doing so is much easier than doing the full RandomX PoW, it still requires | ||
| significant work. It is very important that we can calculate the block ID from the same information (or a subset thereof) | ||
| that we calculate PoW from, since each block header contains the previous block ID. Binding the block headers as such makes | ||
| faking intermediate PoW on chains of blocks exponentially harder the longer the chain is. |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -1438,7 +1438,8 @@ namespace cryptonote | |
| blobdata blob = t_serializable_object_to_blob(static_cast<block_header>(b)); | ||
| crypto::hash tree_root_hash = get_tx_tree_hash(b); | ||
| blob.append(reinterpret_cast<const char*>(&tree_root_hash), sizeof(tree_root_hash)); | ||
| if (b.major_version < HF_VERSION_POW_COMMITMENT) | ||
|
Contributor
There was a problem hiding this comment. Why do you stop including the number of transactions into the hashing blob? This field is used by XMRig, for display purposes. More importantly, it's a piece of data that gets erased by including only the tree root hash. Less data that goes into hashing is worse than more data.
Contributor
Author
There was a problem hiding this comment. I removed it because it isn't necessary for consensus and bloats the hashing blob size. If we chose to implement header-only syncing, the varint per hashing blob would impose a non-trivial cost. It shouldn't affect security AFAIK because Keccak isn't susceptible to length extension attacks, but please correct me if wrong.
Contributor
There was a problem hiding this comment. Hashing blob is 75 bytes (76-77 with the tx count). It's not a big difference, and apart for using it to display tx count in XMRig, it's also useful for comparing different hashing blobs between different Monero pools. Things like "which pools use the same Monero node, how many different Monero nodes does a pool use, is it mining empty blocks or not, and so on".
Contributor
There was a problem hiding this comment. Oh, and one more important thing - I'm pretty sure the "76 bytes minimum" is hard-coded in countless pool implementations in many different places, so it's better to not touch it.
Contributor
Author
There was a problem hiding this comment.
I don't think that the number of txs is a very good metric because it can be easily faked. The miner can determine that there's 25 txs in the tx hash list, but that doesn't mean that there's 25 real txs in the hash list; they could all be generated by the pool itself, and of course the fee goes back to the miner.
Why can't they change it to a 75 byte minimum?
Contributor
There was a problem hiding this comment. https://github.com/SChernykh/p2pool/blob/master/src/block_template.cpp#L1303 These were easy to find and will be easy to fix, but how many constants like these are out there and will have to be fixed one by one after they fail? As for fake/real txs, this varint is not about fake/real. It's about how many txs are in the block template which is an indicator of what the pool is doing. If it says 25, there are 25 transactions of some kind in the block. |
||
| blob.append(tools::get_varint_data(b.tx_hashes.size()+1)); | ||
| return blob; | ||
| } | ||
| //--------------------------------------------------------------- | ||
|
|
@@ -1604,10 +1605,16 @@ namespace cryptonote | |
| return get_tx_tree_hash(txs_ids); | ||
| } | ||
| //--------------------------------------------------------------- | ||
| int get_randomx_variant_for_hf_version(const std::uint8_t hf_version) | ||
| { | ||
| return (hf_version >= HF_VERSION_RANDOMX_V2) ? RX_VARIANT_2 : RX_VARIANT_1; | ||
| } | ||
| //--------------------------------------------------------------- | ||
| crypto::hash get_block_longhash(const blobdata_ref block_hashing_blob, | ||
| const uint64_t height, | ||
| const uint8_t major_version, | ||
| const crypto::hash &seed_hash, | ||
| crypto::hash &intermediate_hash_out) | ||
| { | ||
| crypto::hash res; | ||
|
|
||
|
|
@@ -1618,7 +1625,12 @@ namespace cryptonote | |
| } | ||
| else if (major_version >= RX_BLOCK_VERSION) // RandomX | ||
| { | ||
| const int rx_variant = get_randomx_variant_for_hf_version(major_version); | ||
| crypto::rx_slow_hash(seed_hash.data, rx_variant, | ||
| block_hashing_blob.data(), block_hashing_blob.size(), res.data); | ||
| intermediate_hash_out = res; | ||
| if (major_version >= HF_VERSION_POW_COMMITMENT) | ||
| crypto::rx_commitment(block_hashing_blob.data(), block_hashing_blob.size(), res.data, res.data); | ||
| } | ||
| else // CryptoNight | ||
| { | ||
|
|
||
Uh oh!
There was an error while loading. Please reload this page.