Update readmes with debugging info (#3663)

am1r021 · acolytec3 · holgerd77 · web-flow · commit ffc2c24b5fc8 · 2024-09-16T12:26:21.000+02:00
* blockchain: Update debugging section of README * devp2p: Update debugging section of README * client: Update debugging section of README * evm: Update debugging section of README * Rename evm journal debug symbol * Update statemanager DEVELOPER document with debuging documentation * Update statemanager debug symbol names * verkle: Update debugging section of README * Add information about guard rail being necessary to all package READMEs that provide logging but do not include it * Add initial sentence with link to debug library in package readmes * Remove sentence about exporting DEBUG env variable * statemanager: Make namespaces all lowercase * client: Make debug namespaces all lowercase * client: Add list of debug namespace options to debug docs * Make debug usage examples all use lowercase namespacing * Move info about guardrail to bottom of debugging section in docs and make example more concrete * Revert "Make debug usage examples all use lowercase namespacing" This reverts commit 30164f1. * Add trienode to list of correct words for cspell * Some doc updates and fixes --------- Co-authored-by: acolytec3 <17355484+acolytec3@users.noreply.github.com> Co-authored-by: Holger Drewes <Holger.Drewes@gmail.com>
diff --git a/config/cspell-md.json b/config/cspell-md.json
@@ -2,6 +2,7 @@
   "language": "en-US",
   "ignoreRegExpList": ["/0x[0-9A-Fa-f]+/"],
   "words": [
+    "trienode",
     "t8ntool",
     "calldatasize",
     "Dencun",
diff --git a/packages/blockchain/README.md b/packages/blockchain/README.md
@@ -239,9 +239,9 @@ The `Blockchain` class has a public property `events` which contains an `EventEm
 | ------------------------ | ------------------------------------------- |
 | `deletedCanonicalBlocks` | Emitted when blocks are reorged and deleted |
 
-## Developer
+## Debugging
 
-For debugging blockchain control flows the [debug](https://github.com/visionmedia/debug) library is used and can be activated on the CL with `DEBUG=[Logger Selection] node [Your Script to Run].js`.
+This library uses the [debug](https://github.com/visionmedia/debug) debugging utility package.
 
 The following initial logger is currently available:
 
@@ -257,6 +257,11 @@ Run with the clique logger:
 DEBUG=ethjs,blockchain:clique tsx test.ts
 ```
 
+`ethjs` **must** be included in the `DEBUG` environment variables to enable **any** logs.
+Additional log selections can be added with a comma separated list (no spaces). Logs with extensions can be enabled with a colon `:`, and `*` can be used to include all extensions (currently do not apply for blockchain debugging, example taken from another library).
+
+`DEBUG=ethjs,statemanager:cache:*,trie,statemanager:merkle npx vitest test/statemanager.spec.ts`
+
 ## EthereumJS
 
 See our organizational [documentation](https://ethereumjs.readthedocs.io) for an introduction to `EthereumJS` as well as information on current standards and best practices. If you want to join for work or carry out improvements on the libraries, please review our [contribution guidelines](https://ethereumjs.readthedocs.io/en/latest/contributing.html) first.
diff --git a/packages/client/README.md b/packages/client/README.md
@@ -457,12 +457,28 @@ For more in-depth debugging on networking the underlying [devp2p](../devp2p) lib
 DEBUG=ethjs,*,-babel [CLIENT_START_COMMAND]
 ```
 
-The above command outputs the log messages from all `devp2p` debug loggers available. For a more targeted logging the different loggers can also be activated separately, e.g.:
+The above command outputs the log messages from all `devp2p` debug loggers available. For more targeted logging, the different loggers can also be activated separately, e.g.:
 
 ```shell
 DEBUG=ethjs,devp2p:rlpx,devp2p:eth,-babel [CLIENT_START_COMMAND]
+
+The following options are available:
+
+| Logger              | Description                                      |
+| ------------------- | ------------------------------------------------ |
+| `client:fetcher`            | This option enables logging for all fetchers      |
+| `client:fetcher:bytecode`   | This option enables logging for the snap sync bytecode fetcher         |
+| `client:fetcher:storage`    | This option enables logging for the snap sync storage fetcher  |
+| `client:fetcher:trienode`   | This option enables logging for the snap sync trienode fetcher      |
+| `client:fetcher:account`    | This option enables logging for the snap sync account fetcher      |
+
 ```
 
+`ethjs` **must** be included in the `DEBUG` environment variables to enable **any** logs.
+Additional log selections can be added with a comma separated list (no spaces). Logs with extensions can be enabled with a colon `:`, and `*` can be used to include all extensions.
+
+`DEBUG=ethjs,client:fetcher:*,devp2p:eth npm run client:start`
+
 #### Hive testing
 
 See [DEVELOPER.md](./DEVELOPER.md)
diff --git a/packages/client/src/sync/fetcher/accountfetcher.ts b/packages/client/src/sync/fetcher/accountfetcher.ts
@@ -99,7 +99,7 @@ export class AccountFetcher extends Fetcher<JobTask, AccountData[], AccountData>
     this.stateManager = options.stateManager ?? new MerkleStateManager()
     this.accountTrie = this.stateManager['_getAccountTrie']()
 
-    this.debug = debugDefault('client:AccountFetcher')
+    this.debug = debugDefault('client:fetcher:account')
 
     this.storageFetcher = new StorageFetcher({
       config: this.config,
diff --git a/packages/client/src/sync/fetcher/bytecodefetcher.ts b/packages/client/src/sync/fetcher/bytecodefetcher.ts
@@ -61,7 +61,7 @@ export class ByteCodeFetcher extends Fetcher<JobTask, Uint8Array[], Uint8Array>
 
     this.keccakFunction = this.config.chainCommon.customCrypto.keccak256 ?? keccak256
 
-    this.debug = debug('client:ByteCodeFetcher')
+    this.debug = debug('client:fetcher:bytecode')
     if (this.hashes.length > 0) {
       const fullJob = { task: { hashes: this.hashes } } as Job<JobTask, Uint8Array[], Uint8Array>
       this.DEBUG &&
diff --git a/packages/client/src/sync/fetcher/storagefetcher.ts b/packages/client/src/sync/fetcher/storagefetcher.ts
@@ -94,7 +94,7 @@ export class StorageFetcher extends Fetcher<JobTask, StorageData[][], StorageDat
     this.fetcherDoneFlags.storageFetcher.count = BigInt(this.storageRequests.length)
 
     this.accountToHighestKnownHash = new Map<String, Uint8Array>()
-    this.debug = debugDefault('client:StorageFetcher')
+    this.debug = debugDefault('client:fetcher:storage')
     if (this.storageRequests.length > 0) {
       const fullJob = {
         task: { storageRequests: this.storageRequests },
diff --git a/packages/client/src/sync/fetcher/trienodefetcher.ts b/packages/client/src/sync/fetcher/trienodefetcher.ts
@@ -110,7 +110,7 @@ export class TrieNodeFetcher extends Fetcher<JobTask, Uint8Array[], Uint8Array>
     this.codeDB = this.stateManager['_getCodeDB']()
 
     this.nodeCount = 0
-    this.debug = debug('client:TrieNodeFetcher')
+    this.debug = debug('client:fetcher:trienode')
 
     this.keccakFunction = this.config.chainCommon.customCrypto.keccak256 ?? keccak256
 
diff --git a/packages/devp2p/README.md b/packages/devp2p/README.md
@@ -436,10 +436,7 @@ Events emitted:
 
 This library uses the [debug](https://github.com/visionmedia/debug) debugging utility package.
 
-For the debugging output to show up, set the `DEBUG` environment variable (e.g. in Linux/Mac OS:
-`export DEBUG=ethjs,*,-babel`).
-
-Use the `DEBUG` environment variable to active the logger output you are interested in, e.g.:
+Use the `DEBUG` environment variable to activate the logger output you are interested in, e.g.:
 
 ```shell
 DEBUG=ethjs,devp2p:dpt:\*,devp2p:eth node -r tsx/register [YOUR_SCRIPT_TO_RUN.ts]
@@ -458,14 +455,19 @@ The following loggers are available:
 | `devp2p:eth`          | ETH protocol message logging (`STATUS`, `GET_BLOCK_HEADER`, `TRANSACTIONS`,... messages) |
 | `devp2p:les`          | LES protocol message logging (`STATUS`, `GET_BLOCK_HEADER`, `GET_PROOFS`,... messages)   |
 
+`ethjs` **must** be included in the `DEBUG` environment variables to enable **any** logs.
+Additional log selections can be added with a comma separated list (no spaces). Logs with extensions can be enabled with a colon `:`, and `*` can be used to include all extensions.
+
+`DEBUG=ethjs,devp2p:dns:dns,devp2p:dpt:*,devp2p:rlpx:peer npx vitest test/dns.spec.ts`
+
 ### Debug Verbosity
 
 For more verbose output on logging (e.g. to output the entire msg payload) use the `verbose` logger
 in addition:
 
 DEBUG=ethjs,devp2p:dpt:\*,devp2p:eth,verbose node -r tsx/register [YOUR_SCRIPT_TO_RUN.ts]
 
-Exemplary logging output:
+Example logging output:
 
 ```shell
 Add peer: 52.3.158.184:30303 Geth/v1.7.3-unstable-479aa61f/linux-amd64/go1.9 (eth63) (total: 2)
@@ -495,7 +497,7 @@ on two message names along `ETH` protocol debugging:
 DEBUG=ethjs,devp2p:eth:GET_BLOCK_HEADERS,devp2p:eth:BLOCK_HEADERS -r tsx/register [YOUR_SCRIPT_TO_RUN.ts]
 ```
 
-Exemplary logging output:
+Example logging output:
 
 ```shell
 devp2p:eth:GET_BLOCK_HEADERS Received GET_BLOCK_HEADERS message from 207.154.201.177:30303: d188659b37d8e321bc52c782198181c08080 +50ms
@@ -519,7 +521,7 @@ DEBUG=ethjs,devp2p:3.209.45.79 -r tsx/register [YOUR_SCRIPT_TO_RUN.ts]
 
 #### First Connected
 
-Logging can be limited to the peer the first successful subprotocol (e.g. `ETH`) connection could be established:
+Logging can be limited to the peer with which the first successful subprotocol (e.g. `ETH`) connection could be established:
 
 ```shell
 DEBUG=ethjs,devp2p:FIRST_PEER -r tsx/register [YOUR_SCRIPT_TO_RUN.ts]
diff --git a/packages/ethereum-tests b/packages/ethereum-tests
@@ -1 +1 @@
-Subproject commit 9201075490807f58811078e9bb5ec895b4ac01a5
+Subproject commit e46e1db503ee2711ad02e1f5b3ea45d43e9cd8cb
diff --git a/packages/evm/README.md b/packages/evm/README.md
@@ -382,9 +382,10 @@ The following loggers are currently available:
 
 | Logger                             | Description                                         |
 | ---------------------------------- | --------------------------------------------------- |
-| `evm`                              |  EVM control flow, CALL or CREATE message execution |
+| `evm:evm`                          |  EVM control flow, CALL or CREATE message execution |
 | `evm:gas`                          |  EVM gas logger                                     |
-| `evm:eei:gas`                      |  EEI gas logger                                     |
+| `evm:precompiles`                  |  EVM precompiles logger                             |
+| `evm:journal`                      |  EVM journal logger                                 |
 | `evm:ops`                          |  Opcode traces                                      |
 | `evm:ops:[Lower-case opcode name]` | Traces on a specific opcode                         |
 
@@ -420,6 +421,11 @@ Run some specific loggers including a logger specifically logging the `SSTORE` e
 DEBUG=ethjs,evm,evm:ops:sstore,evm:*:gas tsx test.ts
 ```
 
+`ethjs` **must** be included in the `DEBUG` environment variables to enable **any** logs.
+Additional log selections can be added with a comma separated list (no spaces). Logs with extensions can be enabled with a colon `:`, and `*` can be used to include all extensions.
+
+`DEBUG=ethjs,evm:journal,evm:ops:* npx vitest test/runCall.spec.ts`
+
 ### Internal Structure
 
 The EVM processes state changes at many levels.
diff --git a/packages/evm/src/journal.ts b/packages/evm/src/journal.ts
@@ -53,7 +53,7 @@ export class Journal {
     this.DEBUG =
       typeof window === 'undefined' ? (process?.env?.DEBUG?.includes('ethjs') ?? false) : false
 
-    this._debug = debugDefault('statemanager:statemanager')
+    this._debug = debugDefault('evm:journal')
 
     // TODO maybe call into this.clearJournal
     this.cleanJournal()
diff --git a/packages/statemanager/DEVELOPER.md b/packages/statemanager/DEVELOPER.md
@@ -10,3 +10,34 @@ Tests can be found in the `tests` directory. The StateManager can be tested as:
 ### CI Test Integration
 
 Tests and checks are run in CI using [Github Actions](https://github.com/ethereumjs/ethereumjs-monorepo/actions). The configuration can be found in `.github/workflows`.
+
+## Debugging
+
+This library uses the [debug](https://github.com/visionmedia/debug) debugging utility package.
+
+The following initial logger is currently available:
+
+| Logger                          | Description                                              |
+| ------------------------------- | -------------------------------------------------------- |
+| `statemanager:merkle`           | Operations happening on the `MerkleStateManager`         |
+| `statemanager:rpc`              | Operations happening on the `RPCStateManager`            |
+| `statemanager:verkle:stateful`  | Operations happening on the `StatefulVerkleStateManager` |
+| `statemanager:verkle:stateless` | Operations accessing verkle witnesses                    |
+| `statemanager:verkle:aw`        | Operations accessing verkle witnesses                    |
+| `statemanager:cache`            | Operations accessing statemanager caches                 |
+| `statemanager:cache:code`       | Operations accessing statemanager code cache             |
+| `statemanager:cache:account`    | Operations accessing statemanager account cache          |
+| `statemanager:cache:storage`    | Operations accessing statemanager storage cache          |
+
+The following is an example for a logger run:
+
+Run with the clique logger:
+
+```shell
+DEBUG=ethjs,statemanager:merkle,statemanager:cache:* tsx test.ts
+```
+
+`ethjs` **must** be included in the `DEBUG` environment variables to enable **any** logs.
+Additional log selections can be added with a comma separated list (no spaces). Logs with extensions can be enabled with a colon `:`, and `*` can be used to include all extensions.
+
+`DEBUG=ethjs,statemanager:cache:*,trie,statemanager:merkle npx vitest test/statemanager.spec.ts`
diff --git a/packages/statemanager/src/merkleStateManager.ts b/packages/statemanager/src/merkleStateManager.ts
@@ -110,7 +110,7 @@ export class MerkleStateManager implements StateManagerInterface {
     this.DEBUG =
       typeof window === 'undefined' ? (process?.env?.DEBUG?.includes('ethjs') ?? false) : false
 
-    this._debug = debugDefault('statemanager:statemanager')
+    this._debug = debugDefault('statemanager:merkle')
 
     this.common = opts.common ?? new Common({ chain: Mainnet })
 
diff --git a/packages/statemanager/src/rpcStateManager.ts b/packages/statemanager/src/rpcStateManager.ts
@@ -42,7 +42,7 @@ export class RPCStateManager implements StateManagerInterface {
     this.DEBUG =
       typeof window === 'undefined' ? (process?.env?.DEBUG?.includes('ethjs') ?? false) : false
 
-    this._debug = debugDefault('statemanager:rpcStateManager')
+    this._debug = debugDefault('statemanager:rpc')
     if (typeof opts.provider === 'string' && opts.provider.startsWith('http')) {
       this._provider = opts.provider
     } else {
diff --git a/packages/statemanager/src/statefulVerkleStateManager.ts b/packages/statemanager/src/statefulVerkleStateManager.ts
@@ -84,7 +84,7 @@ export class StatefulVerkleStateManager implements StateManagerInterface {
     this._trie =
       opts.trie ??
       new VerkleTree({ verkleCrypto: opts.verkleCrypto, db: new MapDB<Uint8Array, Uint8Array>() })
-    this._debug = debugDefault('statemanager:statefulVerkleStatemanager')
+    this._debug = debugDefault('statemanager:verkle:stateful')
     this.originalStorageCache = new OriginalStorageCache(this.getStorage.bind(this))
     this._caches = opts.caches
     this.keccakFunction = opts.common?.customCrypto.keccak256 ?? keccak256
diff --git a/packages/statemanager/src/statelessVerkleStateManager.ts b/packages/statemanager/src/statelessVerkleStateManager.ts
@@ -42,7 +42,7 @@ import type {
   VerkleProof,
 } from '@ethereumjs/util'
 
-const debug = debugDefault('statemanager:verkle')
+const debug = debugDefault('statemanager:verkle:stateless')
 
 const PUSH_OFFSET = 95
 // eslint-disable-next-line @typescript-eslint/no-unused-vars
diff --git a/packages/trie/README.md b/packages/trie/README.md
@@ -335,12 +335,9 @@ npm run profiling
 
 ## Debugging
 
-The `Trie` class features optional debug logging.. Individual debug selections can be activated on the CL with `DEBUG=ethjs,[Logger Selection]`.
+This library uses the [debug](https://github.com/visionmedia/debug) debugging utility package.
 
-`ethjs` **must** be included in the `DEBUG` environment variables to enable **any** logs.
-Additional log selections can be added with a comma separated list (no spaces). Logs with extensions can be enabled with a colon `:`, and `*` can be used to include all extensions.
-
-`DEBUG=ethjs,thislog,thatlog,otherlog,otherlog:sublog,anotherLog:* node myscript.js`
+The `Trie` class features optional debug logging. Individual debug selections can be activated on the CL with `DEBUG=ethjs,[Logger Selection]`.
 
 The following options are available:
 
@@ -389,6 +386,11 @@ Run with max logging:
 DEBUG=ethjs,trie:* npx vitest test/util/log.spec.ts
 ```
 
+`ethjs` **must** be included in the `DEBUG` environment variables to enable **any** logs.
+Additional log selections can be added with a comma separated list (no spaces). Logs with extensions can be enabled with a colon `:`, and `*` can be used to include all extensions.
+
+`DEBUG=ethjs,tie:PUT,trie:FIND_PATH:* npx vitest test/proof.spec.ts`
+
 ## References
 
 - Wiki
diff --git a/packages/verkle/README.md b/packages/verkle/README.md
@@ -80,6 +80,52 @@ const { EthereumJSClass } = require('@ethereumjs/[PACKAGE_NAME]')
 
 Using ESM will give you additional advantages over CJS beyond browser usage like static code analysis / Tree Shaking, which CJS cannot provide.
 
+## Debugging
+
+This library uses the [debug](https://github.com/visionmedia/debug) debugging utility package.
+
+The `Verkle` class features optional debug logging. Individual debug selections can be activated on the CL with `DEBUG=ethjs,[Logger Selection]`.
+
+The following options are available:
+
+| Logger              | Description                                      |
+| ------------------- | ------------------------------------------------ |
+| `verkle`            | minimal info logging for all verkle methods      |
+| `verkle:<METHOD>`   | debug logging for specific verkle method         |
+| `verkle:<METHOD>:*` | verbose debug logging for specific verkle method |
+| `verkle:*`          | verbose debug logging for all verkle methods     |
+
+To observe the logging in action at different levels:
+
+Run with minimal logging:
+
+```shell
+DEBUG=ethjs,verkle npx vitest test/verkle.spec.ts
+```
+
+Run with **put** method logging:
+
+```shell
+DEBUG=ethjs,verkle:PUT npx vitest test/verkle.spec.ts
+```
+
+Run with **verkle** + **put**/**get**/**del** logging:
+
+```shell
+DEBUG=ethjs,verkle,verkle:PUT,verkle:GET,verkle:DEL npx vitest test/verkle.spec.ts
+```
+
+Run with max logging:
+
+```shell
+DEBUG=ethjs,verkle:* npx vitest test/verkle.spec.ts
+```
+
+`ethjs` **must** be included in the `DEBUG` environment variables to enable **any** logs.
+Additional log selections can be added with a comma separated list (no spaces). Logs with extensions can be enabled with a colon `:`, and `*` can be used to include all extensions.
+
+`DEBUG=ethjs,tie:PUT,trie:FIND_PATH:* npx vitest test/proof.spec.ts`
+
 ## References
 
 - Wiki
