Skip to content

Comments

Enhance Connection Strategy, ENR Validations and add Database Documentation#76

Merged
MatheusFreixo merged 4 commits intomasterfrom
feat/dynamic-backoff
Aug 25, 2025
Merged

Enhance Connection Strategy, ENR Validations and add Database Documentation#76
MatheusFreixo merged 4 commits intomasterfrom
feat/dynamic-backoff

Conversation

@MatheusFreixo
Copy link
Contributor

@MatheusFreixo MatheusFreixo commented Jul 27, 2025

Overview

This PR bundles three related sets of changes:

  1. Dynamic backoff & fork support
  2. Stricter ENR fork-digest filtering & validation
  3. Comprehensive crawler database documentation

Feature: Dynamic backoff for connection attempts

  • dynamicBackoff: Implements an exponential retry delay, adapting base delays by error type.
  • Deprecation window extended from 3 hours to 1 month to reduce premature peer pruning.
  • Fork support: Adds “Electra” fork digest to the ForkDigests map.
  • ISP cleanup: Introduces CleanISP to normalize ISP strings before database insertion.

Feature: Improve ENR fork digest filtering & validation

  • DV5 service (dv5_service.go)

    • Only invoke GetForkDigest() when FilterDigest != "all".
    • Skip nodes lacking valid Eth2 data instead of panicking.
    • Enhance trace logs on digest mismatches or missing data.
  • Ethereum local node (ethereum/enode.go)

  • Add constants for expected hex lengths (ETH2_DATA_HEX_LENGTH, etc.).

  • Strict normalization & validation of fork-digest and attnets hex inputs.

  • Return errors on incomplete/malformed entries; warn on non-standard lengths.

  • ENR parsing (ethereum/enr.go & entries.go)

    • Introduce a structured ParseError to capture ENR parse failures.
    • Track presence and parse errors for both Eth2Data and Attnets on EnrNode.
    • Provide helpers HasValidEth2Data() and GetForkDigest().
    • Enforce a fixed 16-byte size for eth2 data, with clear error reporting.
    • Generalize CountBits() to safely handle variable-length bitfields.

Feature: Add crawler database documentation

  • New file database.md fully describes all tables:
    • peer_info, eth_nodes, eth_status, eth_blocks, eth_attestations, conn_events, active_peers, ips
  • Details columns, types, defaults and indexes.
  • Clarifies differences between ENR discovery vs. successful connections (eth_nodes vs. eth_status).

- implements dynamicBackoff function that computes an exponentially increasing retry delay, using different base delays per error type.
- increases the deprecation time from three hours to a month
- adds Electra fork digest in the ForkDigests map
- implements CleanISP function to store a clean isp information in the database
@MatheusFreixo MatheusFreixo self-assigned this Jul 27, 2025
@MatheusFreixo MatheusFreixo changed the title Feat/dynamic backoff Enhance Connection Strategy, ENR Validations and add Database Documentation Jul 28, 2025
@MatheusFreixo MatheusFreixo merged commit 2dc9099 into master Aug 25, 2025
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants