Skip to content

Conversation

@qdm12
Copy link

@qdm12 qdm12 commented Mar 6, 2025

Why this should be merged

The alternative would be to change all occurrences of metrics.New* by metrics.GetOrRegister* in both the consumer code and in the libevm code (because it's unpredictable which one would register first), but that produces a lot more diffs.

Just for historical purposes, I did try to have a custom registry on a consumer (coreth) for all metrics but:

  • if the same metric is used in both the consumer and libevm (I think there are a few, given CI fails quite a bit), we would lose metrics from the libevm side because it uses another registry (and not sure we can merge both metrics from 2 different registries)
  • it introduces a few hundred lines of diffs which is undesirable on the consumer side (coreth) at this time

How this works

  • Change all metrics.New* function implementations to use metrics.GetOrRegister* functions.
  • Add GetOrRegisterOrOverrideCounter because counter metrics in consumers are defined as timer metrics in libevm, so consumers need to use this function when this happens.

How this was tested

  • CI passing
  • GetOrRegisterOrOverrideCounter needs test

qdm12 added 2 commits March 6, 2025 08:22
qdm12 added a commit to ava-labs/coreth that referenced this pull request Mar 6, 2025
@ARR4N
Copy link
Collaborator

ARR4N commented Mar 6, 2025

Is this a correct understanding of the status quo?

  1. Either libevm or a consumer calls, for example, NewRegisteredCounter("foo", nil).
  2. The other party then calls the exact same function but r.Register() returns an error that isn't checked.
  3. The second caller is now holding an unregistered Counter so all of its recordings are dropped.

Is there a strict need for consumers to use the same metrics? What problems would be caused by name-spacing the metric name to guarantee no conflict?

EDIT: I hadn't considered the yet-to-transition packages that exist in coreth and will eventually be deleted in lieu of their libevm counterparts. I think this pattern is a better approach as it leaves libevm unchanged and is simply a global sed on coreth along with imports of libevm packages we'll be adding anyway.

qdm12 added a commit to ava-labs/coreth that referenced this pull request Mar 6, 2025
@ARR4N ARR4N changed the title fix(metrics): New* constructors use GetOrRegister* functions refactor(metrics): New* constructors use GetOrRegister* functions Mar 6, 2025
qdm12 added a commit to ava-labs/coreth that referenced this pull request Mar 6, 2025
@ARR4N
Copy link
Collaborator

ARR4N commented Mar 6, 2025

The alternative would be to change all occurrences of metrics.New* by metrics.GetOrRegister* in both the consumer code and in the libevm code (because it's unpredictable which one would register first)

The order can be forced with a blank import for side effects. Here's a pattern that I think has the desired effect without the need to change libevm at all. It assumes that we know the correspondence between libevm and coreth metrics but (a) they should always be in the mirrored packages, and (b) our refactoring approach always involves the coreth package importing its libevm mirror just as in the pattern so we're eventually going to force the ordering anyway. Once the ordering is guaranteed, it's just a matter of changing New*() to GetOrRegister*() in the consumer only.

@qdm12
Copy link
Author

qdm12 commented Mar 6, 2025

Packages which are:

  • defining global metrics using metrics.New* functions
  • defined in coreth
  • imported by coreth from libevm

are the following:

  • core
  • core/state/snapshot
  • rpc
  • triedb/hashdb
  • triedb/pathdb

Now, the good thing is all their metrics are unexported, so that reduces the problem scope.

However, say for the core package, it's rather tough to find out if a coreth package other than core transitively imports libevm's core without importing coreth's core package. This could already be the case, and even if it is not, it could well be in the future. If, say coreth/xyz, imports libevm/xyz which itself imports libevm/core, it's not a direct import but this may execute before coreth/core (depending on compilation and other things outside our control I would say)

EDIT: using underscore imports in those packages above + GetOrRegister in consumers solves this

@ARR4N
Copy link
Collaborator

ARR4N commented Mar 6, 2025

EDIT: using underscore imports in those packages above + GetOrRegister in consumers solves this

Closing in favour of ava-labs/coreth#860.

@ARR4N ARR4N closed this Mar 6, 2025
ARR4N added a commit to ava-labs/coreth that referenced this pull request Mar 7, 2025
`geth` (and hence `libevm`) global metrics tend to be created with `NewRegistered*()`, which assumes that there are no name conflicts on the same `Registry`. If there are then the second call fails silently to register the new metric.

Except for in `warp` (where there won't be any conflicts) metric registration is changed globally to use `GetOrRegister*()` instead of `NewRegistered*()`. With GNU tools (not the Mac budget replacements):

```shell
find -iname '*.go' -not -iwholename '*/warp/*' | xargs sed -i 's|metrics.NewRegistered|metrics.GetOrRegister|g'
```

All modified files that don't already import their `libevm` counterparts now `_` import them to force the initialisation order. The `libevm` construction with `NewRegistered*()` therefore always comes before the respective `GetOrRegister*()` so both call sites have the same metric instance.

Metrics that we expect to be counters instead of timers are unregistered and newly created with the correct type, borrowing from ava-labs/libevm#159.
qdm12 pushed a commit to ava-labs/coreth that referenced this pull request Mar 7, 2025
`geth` (and hence `libevm`) global metrics tend to be created with `NewRegistered*()`, which assumes that there are no name conflicts on the same `Registry`. If there are then the second call fails silently to register the new metric.

Except for in `warp` (where there won't be any conflicts) metric registration is changed globally to use `GetOrRegister*()` instead of `NewRegistered*()`. With GNU tools (not the Mac budget replacements):

```shell
find -iname '*.go' -not -iwholename '*/warp/*' | xargs sed -i 's|metrics.NewRegistered|metrics.GetOrRegister|g'
```

All modified files that don't already import their `libevm` counterparts now `_` import them to force the initialisation order. The `libevm` construction with `NewRegistered*()` therefore always comes before the respective `GetOrRegister*()` so both call sites have the same metric instance.

Metrics that we expect to be counters instead of timers are unregistered and newly created with the correct type, borrowing from ava-labs/libevm#159.
qdm12 pushed a commit to ava-labs/coreth that referenced this pull request Mar 7, 2025
`geth` (and hence `libevm`) global metrics tend to be created with `NewRegistered*()`, which assumes that there are no name conflicts on the same `Registry`. If there are then the second call fails silently to register the new metric.

Except for in `warp` (where there won't be any conflicts) metric registration is changed globally to use `GetOrRegister*()` instead of `NewRegistered*()`. With GNU tools (not the Mac budget replacements):

```shell
find -iname '*.go' -not -iwholename '*/warp/*' | xargs sed -i 's|metrics.NewRegistered|metrics.GetOrRegister|g'
```

All modified files that don't already import their `libevm` counterparts now `_` import them to force the initialisation order. The `libevm` construction with `NewRegistered*()` therefore always comes before the respective `GetOrRegister*()` so both call sites have the same metric instance.

Metrics that we expect to be counters instead of timers are unregistered and newly created with the correct type, borrowing from ava-labs/libevm#159.
ARR4N added a commit to ava-labs/coreth that referenced this pull request Mar 10, 2025
`geth` (and hence `libevm`) global metrics tend to be created with `NewRegistered*()`, which assumes that there are no name conflicts on the same `Registry`. If there are then the second call fails silently to register the new metric.

Except for in `warp` (where there won't be any conflicts) metric registration is changed globally to use `GetOrRegister*()` instead of `NewRegistered*()`. With GNU tools (not the Mac budget replacements):

```shell
find -iname '*.go' -not -iwholename '*/warp/*' | xargs sed -i 's|metrics.NewRegistered|metrics.GetOrRegister|g'
```

All modified files that don't already import their `libevm` counterparts now `_` import them to force the initialisation order. The `libevm` construction with `NewRegistered*()` therefore always comes before the respective `GetOrRegister*()` so both call sites have the same metric instance.

Metrics that we expect to be counters instead of timers are unregistered and newly created with the correct type, borrowing from ava-labs/libevm#159.
qdm12 pushed a commit to ava-labs/subnet-evm that referenced this pull request Mar 19, 2025
`geth` (and hence `libevm`) global metrics tend to be created with `NewRegistered*()`, which assumes that there are no name conflicts on the same `Registry`. If there are then the second call fails silently to register the new metric.

Except for in `warp` (where there won't be any conflicts) metric registration is changed globally to use `GetOrRegister*()` instead of `NewRegistered*()`. With GNU tools (not the Mac budget replacements):

```shell
find -iname '*.go' -not -iwholename '*/warp/*' | xargs sed -i 's|metrics.NewRegistered|metrics.GetOrRegister|g'
```

All modified files that don't already import their `libevm` counterparts now `_` import them to force the initialisation order. The `libevm` construction with `NewRegistered*()` therefore always comes before the respective `GetOrRegister*()` so both call sites have the same metric instance.

Metrics that we expect to be counters instead of timers are unregistered and newly created with the correct type, borrowing from ava-labs/libevm#159.

See original PR ava-labs/coreth#860
qdm12 pushed a commit to ava-labs/subnet-evm that referenced this pull request Mar 21, 2025
`geth` (and hence `libevm`) global metrics tend to be created with `NewRegistered*()`, which assumes that there are no name conflicts on the same `Registry`. If there are then the second call fails silently to register the new metric.

Except for in `warp` (where there won't be any conflicts) metric registration is changed globally to use `GetOrRegister*()` instead of `NewRegistered*()`. With GNU tools (not the Mac budget replacements):

```shell
find -iname '*.go' -not -iwholename '*/warp/*' | xargs sed -i 's|metrics.NewRegistered|metrics.GetOrRegister|g'
```

All modified files that don't already import their `libevm` counterparts now `_` import them to force the initialisation order. The `libevm` construction with `NewRegistered*()` therefore always comes before the respective `GetOrRegister*()` so both call sites have the same metric instance.

Metrics that we expect to be counters instead of timers are unregistered and newly created with the correct type, borrowing from ava-labs/libevm#159.

See original PR ava-labs/coreth#860
qdm12 pushed a commit to ava-labs/subnet-evm that referenced this pull request Mar 21, 2025
`geth` (and hence `libevm`) global metrics tend to be created with `NewRegistered*()`, which assumes that there are no name conflicts on the same `Registry`. If there are then the second call fails silently to register the new metric.

Except for in `warp` (where there won't be any conflicts) metric registration is changed globally to use `GetOrRegister*()` instead of `NewRegistered*()`. With GNU tools (not the Mac budget replacements):

```shell
find -iname '*.go' -not -iwholename '*/warp/*' | xargs sed -i 's|metrics.NewRegistered|metrics.GetOrRegister|g'
```

All modified files that don't already import their `libevm` counterparts now `_` import them to force the initialisation order. The `libevm` construction with `NewRegistered*()` therefore always comes before the respective `GetOrRegister*()` so both call sites have the same metric instance.

Metrics that we expect to be counters instead of timers are unregistered and newly created with the correct type, borrowing from ava-labs/libevm#159.

See original PR ava-labs/coreth#860
qdm12 pushed a commit to ava-labs/subnet-evm that referenced this pull request Mar 24, 2025
`geth` (and hence `libevm`) global metrics tend to be created with `NewRegistered*()`, which assumes that there are no name conflicts on the same `Registry`. If there are then the second call fails silently to register the new metric.

Except for in `warp` (where there won't be any conflicts) metric registration is changed globally to use `GetOrRegister*()` instead of `NewRegistered*()`. With GNU tools (not the Mac budget replacements):

```shell
find -iname '*.go' -not -iwholename '*/warp/*' | xargs sed -i 's|metrics.NewRegistered|metrics.GetOrRegister|g'
```

All modified files that don't already import their `libevm` counterparts now `_` import them to force the initialisation order. The `libevm` construction with `NewRegistered*()` therefore always comes before the respective `GetOrRegister*()` so both call sites have the same metric instance.

Metrics that we expect to be counters instead of timers are unregistered and newly created with the correct type, borrowing from ava-labs/libevm#159.

See original PR ava-labs/coreth#860
qdm12 pushed a commit to ava-labs/subnet-evm that referenced this pull request Mar 24, 2025
`geth` (and hence `libevm`) global metrics tend to be created with `NewRegistered*()`, which assumes that there are no name conflicts on the same `Registry`. If there are then the second call fails silently to register the new metric.

Except for in `warp` (where there won't be any conflicts) metric registration is changed globally to use `GetOrRegister*()` instead of `NewRegistered*()`. With GNU tools (not the Mac budget replacements):

```shell
find -iname '*.go' -not -iwholename '*/warp/*' | xargs sed -i 's|metrics.NewRegistered|metrics.GetOrRegister|g'
```

All modified files that don't already import their `libevm` counterparts now `_` import them to force the initialisation order. The `libevm` construction with `NewRegistered*()` therefore always comes before the respective `GetOrRegister*()` so both call sites have the same metric instance.

Metrics that we expect to be counters instead of timers are unregistered and newly created with the correct type, borrowing from ava-labs/libevm#159.

See original PR ava-labs/coreth#860
qdm12 pushed a commit to ava-labs/subnet-evm that referenced this pull request Mar 24, 2025
`geth` (and hence `libevm`) global metrics tend to be created with `NewRegistered*()`, which assumes that there are no name conflicts on the same `Registry`. If there are then the second call fails silently to register the new metric.

Except for in `warp` (where there won't be any conflicts) metric registration is changed globally to use `GetOrRegister*()` instead of `NewRegistered*()`. With GNU tools (not the Mac budget replacements):

```shell
find -iname '*.go' -not -iwholename '*/warp/*' | xargs sed -i 's|metrics.NewRegistered|metrics.GetOrRegister|g'
```

All modified files that don't already import their `libevm` counterparts now `_` import them to force the initialisation order. The `libevm` construction with `NewRegistered*()` therefore always comes before the respective `GetOrRegister*()` so both call sites have the same metric instance.

Metrics that we expect to be counters instead of timers are unregistered and newly created with the correct type, borrowing from ava-labs/libevm#159.

See original PR ava-labs/coreth#860
qdm12 pushed a commit to ava-labs/subnet-evm that referenced this pull request Mar 28, 2025
`geth` (and hence `libevm`) global metrics tend to be created with `NewRegistered*()`, which assumes that there are no name conflicts on the same `Registry`. If there are then the second call fails silently to register the new metric.

Except for in `warp` (where there won't be any conflicts) metric registration is changed globally to use `GetOrRegister*()` instead of `NewRegistered*()`. With GNU tools (not the Mac budget replacements):

```shell
find -iname '*.go' -not -iwholename '*/warp/*' | xargs sed -i 's|metrics.NewRegistered|metrics.GetOrRegister|g'
```

All modified files that don't already import their `libevm` counterparts now `_` import them to force the initialisation order. The `libevm` construction with `NewRegistered*()` therefore always comes before the respective `GetOrRegister*()` so both call sites have the same metric instance.

Metrics that we expect to be counters instead of timers are unregistered and newly created with the correct type, borrowing from ava-labs/libevm#159.

See original PR ava-labs/coreth#860
qdm12 added a commit to ava-labs/subnet-evm that referenced this pull request Mar 28, 2025
`geth` (and hence `libevm`) global metrics tend to be created with `NewRegistered*()`, which assumes that there are no name conflicts on the same `Registry`. If there are then the second call fails silently to register the new metric.

Except for in `warp` (where there won't be any conflicts) metric registration is changed globally to use `GetOrRegister*()` instead of `NewRegistered*()`. With GNU tools (not the Mac budget replacements):

```shell
find -iname '*.go' -not -iwholename '*/warp/*' | xargs sed -i 's|metrics.NewRegistered|metrics.GetOrRegister|g'
```

All modified files that don't already import their `libevm` counterparts now `_` import them to force the initialisation order. The `libevm` construction with `NewRegistered*()` therefore always comes before the respective `GetOrRegister*()` so both call sites have the same metric instance.

Metrics that we expect to be counters instead of timers are unregistered and newly created with the correct type, borrowing from ava-labs/libevm#159.

See original PR ava-labs/coreth#860
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants