Skip to content

Commit 61fdcea

Browse files
stakepeterclaude
andcommitted
Migrate go-eth2-client to stakefish fork; fix Caplin missed-slot crash
Two related changes in one session: 1. Switch github.com/attestantio/go-eth2-client to the stakefish fork (feat/erigon-caplin-support, commit 781f0c7f) via a replace directive. The fork's UnmarshalJSON methods on phase0.Slot/Epoch/Gwei/ValidatorIndex and electra.DepositRequest natively tolerate Caplin's bare-number JSON, so the regex-based caplin_compat rewriter (transport + integration test) is now redundant and removed. caplin_parse_test.go stays as a regression guard for the fork's native Caplin parsing. service.go's HTTP client is built via newInstrumentedHTTPClient (same Dialer tuning + metrics wrap, minus the body rewriter). 2. Fix the staging panic at epoch 94898 (slot 3036736 missed → Caplin 404 "block not found N" on the validators-resolve call → Must() panic). GetValidatorIndexes now walks forward through the epoch's slots when Caplin returns 404 on the state_id lookup. Validator set is stable within an epoch, so any canonical slot answers the same query. TestGetValidatorIndexes_ProbeForwardOnMissedFirstSlot locks in the probe-forward behaviour against synthetic 404→200 routes. Plus a CLAUDE.md sweep that removes stale references to the deleted shim and rewrites the "Caplin returns 404 on slot-ID state queries" gotcha to describe the fix; package-comment cleanup in monitoring/doc.go (Package pkg → Package monitoring); subtest rename in service_e2e_test.go. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
1 parent b6101eb commit 61fdcea

11 files changed

Lines changed: 161 additions & 366 deletions

File tree

CLAUDE.md

Lines changed: 8 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -8,7 +8,7 @@ Ethereum 2.0 validator performance monitor built by stakefish. Tracks attestatio
88

99
- **Language:** Go 1.25 (go.mod: 1.25.10; `.tool-versions`: 1.25.10; CI: `'1.25'` in `golangci-lint.yml`, `1.25.x` in `main.yml`)
1010
- **CLI Framework:** Cobra (`github.com/spf13/cobra`)
11-
- **Beacon Chain Client:** `github.com/attestantio/go-eth2-client` v0.28.1 (HTTP transport)
11+
- **Beacon Chain Client:** `github.com/attestantio/go-eth2-client` v0.28.1 — redirected via `replace` directive in `go.mod` to `github.com/stakefish/go-eth2-client@feat/erigon-caplin-support` (commit `781f0c7f`) for native Caplin JSON tolerance. Imports stay as `github.com/attestantio/go-eth2-client/...`; the fork keeps the upstream module path. Revert by dropping the replace directive when upstream absorbs the fix.
1212
- **Logging:** zerolog (`github.com/rs/zerolog`)
1313
- **Metrics:** Prometheus (`github.com/prometheus/client_golang`)
1414
- **Error Wrapping:** `github.com/pkg/errors`
@@ -29,14 +29,12 @@ internal/
2929
opts/
3030
opts.go -- Global CLI flag variables (package-level vars)
3131
beaconchain/
32-
service.go -- BeaconChain wrapper around go-eth2-client (HTTP)
33-
caplin_compat.go -- HTTP transport that rewrites unquoted amount/index JSON fields in Caplin block responses
34-
metrics.go -- Beacon API request CounterVec/HistogramVec instrumentation
35-
caplin_compat_integration_test.go -- Rewriter regex tests + real-block fixture pass-through via the production transport
36-
caplin_parse_test.go -- json.Unmarshal block_canonical.json into electra.SignedBeaconBlock (schema-drift detector)
32+
service.go -- BeaconChain wrapper around go-eth2-client (HTTP); builds the metrics-instrumented http.Client via newInstrumentedHTTPClient
33+
metrics.go -- Beacon API request CounterVec/HistogramVec instrumentation; defines instrumentingTransport
34+
caplin_parse_test.go -- json.Unmarshal block_canonical.json into electra.SignedBeaconBlock (regression guard that the forked go-eth2-client still tolerates Caplin's bare-number JSON natively)
3735
service_fixture_test.go -- Offline GetBlock canonical+missed via fixtureServer (no live endpoint)
3836
metrics_e2e_test.go -- Live-fire metric-detection coverage for the 7 monitor endpoints (build tag: `e2e`)
39-
service_e2e_test.go -- Live-fire tests for all six BeaconChain methods (build tag: `e2e`)
37+
service_e2e_test.go -- Live-fire tests covering each BeaconChain wrapper method (GetValidatorIndexes, GetBlock, GetProposerDuties, GetAttesterDuties, GetCommitteeLengths) against the staging endpoint (build tag: `e2e`)
4038
testdata_test.go -- embed.FS + loadFixture(scenario,name)/loadSharedFixture/loadMeta(scenario)/fixtureServer(scenario,routes) helpers; `defaultChain = "hoodi"`
4139
testdata/
4240
beacon/
@@ -53,7 +51,8 @@ internal/
5351
consts.go -- SLOTS_PER_EPOCH=32, SECONDS_PER_SLOT=12
5452
routines.go -- Epoch/Slot conversion helpers
5553
monitoring/
56-
monitoring.go -- Orchestrator loop + SubscribeToEpochs + LoadKeys/LoadMEVRelays (package monitoring; was pkg/ before restructure)
54+
doc.go -- Package-level overview (entry points + flow)
55+
monitoring.go -- Orchestrator loop + SubscribeToEpochs + LoadKeys/LoadMEVRelays
5756
epoch_context.go -- Per-epoch state fetch: EpochContext + BuildEpochContext + ResolveValidatorKeys + ListProposerDuties / ListEpochBlocks + SlotsWithBlocks
5857
attestations.go -- Attestation-issue detection: processAttestations + BuildCommitteeLookup + PruneSeenAttestations + FinalizeMissedAttestations + CommitteeInfo
5958
proposals.go -- Proposal-issue detection: isBlockEmpty + CheckProposal + FinalizeMissedProposals
@@ -256,8 +255,7 @@ Most monitoring/beaconchain tests are now fixture-backed integration tests (see
256255

257256
- **GetBlock fails on pre-Fusaka slots** -- returns error `"unsupported block version"` for any slot before the Fulu fork
258257
- **Validator cache has a 30-minute TTL** -- `internal/monitoring/cache.go` persists the `Validators` map plus `LastEpoch` to disk JSON (`$TMPDIR/stakefish-eth2-monitor-cache.json`). `CachedIndex.At` is consulted by `ResolveValidatorKeys` to refresh entries older than 30 minutes; `VALIDATOR_INDEX_INVALID` sentinel entries are also TTL-bounded so a newly-active validator becomes visible within the window. On restart `LastEpoch` gates skip-ahead so cumulative counters don't double-count re-processed epochs. Writes use atomic tmpfile + fsync + rename + dir-fsync for crash durability. Delete the file to force a clean run.
259-
- **Caplin `amount`/`index` JSON quoting** -- `internal/beaconchain/caplin_compat.go` installs an HTTP transport that rewrites *only* the `"amount":N` and `"index":N` fields (regex `unquotedNumericField`) on `/eth/v2/beacon/blocks/` JSON responses. Other Caplin endpoints, other unquoted uint64 fields (e.g. anything under `solid/`), and SSZ responses are untouched -- those still need a fix upstream in go-eth2-client.
260-
- **Caplin returns 404 on slot-ID state queries for missed slots** -- `GetValidatorIndexes` uses `fmt.Sprintf("%d", spec.EpochLowestSlot(epoch))` as the state ID. Caplin resolves slot-id states by first finding the block at that slot, so if the first slot of the requested epoch was missed it returns `404 block not found`. Production code has no probe-back logic, so this is a latent flake at epoch-boundary missed slots; the `service_e2e_test.go` `get_validator_indexes_roundtrip` subtest works around it by walking back to an epoch whose first slot has a canonical block.
258+
- **Caplin returns 404 on slot-ID state queries for missed slots** -- `/eth/v1/beacon/states/{slot}/validators` 404s with `block not found N` when slot N was missed; Caplin resolves a slot state_id by walking to the block AT that slot. `GetValidatorIndexes` walks forward through the epoch's slots (validator set is stable within an epoch) until one resolves, capping at the epoch's last slot. The e2e test (`get_validator_indexes_roundtrip`) and the fixture-backed regression test (`TestGetValidatorIndexes_ProbeForwardOnMissedFirstSlot`) both lock this in. Found via staging crash 2026-05-13: epoch 94898's first slot (3036736) was missed and the monitor panicked at `Must(BuildEpochContext)` before the fix.
261259
- **Slashed validators silently excluded from monitoring** -- `GetValidatorIndexes` filters via `IsAttesting()`, which is false for `active_slashed` *and* for any post-exit state. Once a key is slashed it never reappears in duties or reports (slashed and exited are both filtered) -- surprising during incident response when "where is validator X?" has no log line.
262260
- **Attestation dedup requires consecutive epoch processing** -- `processAttestations` keys `seenAttestations` on `(validator, slot)` and the cross-epoch lookahead window assumes E and E+1 are processed in order. Skipping an epoch (SSE jump, replay-epoch gap) produces false missed-attestation reports.
263261
- **`vendor/` is not in git** -- `.gitignore` has `/vendor/` and the directory is genuinely untracked (`git ls-files vendor/` is empty). After a fresh clone vendor/ is absent; `go build` falls back to the module cache. Run `go mod vendor` only if you want a vendored local build. Older docs/comments that imply vendor/ is checked in are stale.

go.mod

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -54,3 +54,5 @@ require (
5454

5555
// See https://github.com/prysmaticlabs/prysm/blob/d035be29cd549ca38b257e67bb6d9e6e76e9fba7/go.mod#L270
5656
replace github.com/grpc-ecosystem/grpc-gateway/v2 => github.com/prysmaticlabs/grpc-gateway/v2 v2.3.1-0.20230315201114-09284ba20446
57+
58+
replace github.com/attestantio/go-eth2-client => github.com/stakefish/go-eth2-client v0.0.0-20260513052533-781f0c7f902c

go.sum

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,5 @@
11
github.com/OffchainLabs/go-bitfield v0.0.0-20251031151322-f427d04d8506 h1:d/SJkN8/9Ca+1YmuDiUJxAiV4w/a9S8NcsG7GMQSrVI=
22
github.com/OffchainLabs/go-bitfield v0.0.0-20251031151322-f427d04d8506/go.mod h1:6TZI4FU6zT8x6ZfWa1J8YQ2NgW0wLV/W3fHRca8ISBo=
3-
github.com/attestantio/go-eth2-client v0.28.1 h1:sDntLnHtlWZ6KK4zybPXMLE20hy0WgM1eDfYzlCciG4=
4-
github.com/attestantio/go-eth2-client v0.28.1/go.mod h1:1MBvz1DVe6jZduHJDJuJ/1S4LzYmDmvx+r5alo4pyrM=
53
github.com/beorn7/perks v1.0.1 h1:VlbKKnNfV8bJzeqoa4cOKqO6bYr3WgKZxO8Z16+hsOM=
64
github.com/beorn7/perks v1.0.1/go.mod h1:G2ZrVWU2WbWT9wwq4/hrbKbnv/1ERSJQ0ibhJ6rlkpw=
75
github.com/cespare/xxhash/v2 v2.2.0 h1:DC2CZ1Ep5Y4k3ZQ899DldepgrayRUGE6BBZ/cd9Cj44=
@@ -108,6 +106,8 @@ github.com/spf13/cobra v1.5.0 h1:X+jTBEBqF0bHN+9cSMgmfuvv2VHJ9ezmFNf9Y/XstYU=
108106
github.com/spf13/cobra v1.5.0/go.mod h1:dWXEIy2H428czQCjInthrTRUg7yKbok+2Qi/yBIJoUM=
109107
github.com/spf13/pflag v1.0.5 h1:iy+VFUOCP1a+8yFto/drg2CJ5u0yRoB7fZw3DKv/JXA=
110108
github.com/spf13/pflag v1.0.5/go.mod h1:McXfInJRrz4CZXVZOBLb0bTZqETkiAhM9Iw0y3An2Bg=
109+
github.com/stakefish/go-eth2-client v0.0.0-20260513052533-781f0c7f902c h1:ThyoyIu/Um7qX+9Nl8YhmFYiT4CY3tU1x49MjfWPBoc=
110+
github.com/stakefish/go-eth2-client v0.0.0-20260513052533-781f0c7f902c/go.mod h1:1MBvz1DVe6jZduHJDJuJ/1S4LzYmDmvx+r5alo4pyrM=
111111
github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
112112
github.com/stretchr/testify v1.4.0/go.mod h1:j7eGeouHqKxXV5pUuKE4zz7dFj8WfuZ+81PSLYec5m4=
113113
github.com/stretchr/testify v1.7.0/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=

internal/beaconchain/caplin_compat.go

Lines changed: 0 additions & 79 deletions
This file was deleted.

0 commit comments

Comments
 (0)