Skip to content

Conversation

@tbg
Copy link
Member

@tbg tbg commented Jan 7, 2026

status: expose additional Go runtime metrics

This PR adds three new metrics from Go's runtime/metrics package:

  • sys.go.heap.objects: live + unswept object count on heap
  • sys.go.heap.livebytes: bytes of live objects marked by previous GC
  • sys.gc.total.ns: total CPU time spent on GC tasks

These provide better visibility into GC behavior and object allocation
patterns, useful for diagnosing memory pressure and GC overhead.

Inspired by poking at #160195

Epic: none

tbg added 2 commits January 7, 2026 11:09
Add two new metrics sourced from Go's runtime/metrics package:

- sys.go.heap.objects: number of objects (live + unswept) on the heap,
  from /gc/heap/objects:objects
- sys.go.heap.livebytes: bytes of live heap objects marked by the
  previous GC, from /gc/heap/live:bytes

These complement the existing heap byte metrics and provide visibility
into object allocation patterns, which can be useful for diagnosing GC
pressure from high object churn.

Release note: None
Add sys.gc.total.ns metric sourced from Go's
/cpu/classes/gc/total:cpu-seconds runtime metric. This provides
visibility into total CPU time spent on GC tasks, complementing the
existing sys.gc.assist.ns which only captures mutator assist time.

Release note: None
@blathers-crl
Copy link

blathers-crl bot commented Jan 7, 2026

It looks like your PR touches production code but doesn't add or edit any test code. Did you consider adding tests to your PR?

🦉 Hoot! I am a Blathers, a bot for CockroachDB. My owner is dev-inf.

@cockroach-teamcity
Copy link
Member

This change is Reviewable

@tbg

This comment was marked as resolved.

@tbg tbg requested review from a team and angeladietz January 8, 2026 16:31
@tbg tbg marked this pull request as ready for review January 8, 2026 16:31
@tbg tbg requested a review from a team as a code owner January 8, 2026 16:31
@tbg tbg requested review from dhartunian and removed request for a team January 8, 2026 16:31
@tbg tbg requested a review from a team as a code owner January 8, 2026 16:35
@tbg tbg requested review from golgeek and shailendra-patel and removed request for a team January 8, 2026 16:35
@blathers-crl
Copy link

blathers-crl bot commented Jan 8, 2026

Metrics change detected

This PR adds or updates one or more CRDB metrics. If you want these metrics to be exported by CRDB Cloud clusters to Internal CRL Datadog and/or included in the customer metric export integration (Essential metrics for standard deployment, and Essential metrics for advanced deployment), refer to this Installation and Usage guide of a CLI tool that syncs the metric mappings in managed-service. Run this CLI tool after your CRDB PR is merged.

  • The CLI opens a PR in managed-service with the required config changes.
  • Please track that PR and ensure it merges so your metrics become available to CRDB Cloud clusters.

Note: Your metric will appear in Internal CRL Datadog only after the managed-service PR merges and the new OTel configuration rolls out to at least one cluster running a CRDB build that includes this metric.

Docs: cockroach-metric-sync

Questions: reach out to @obs-india-prs

Copy link
Contributor

@angeladietz angeladietz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:lgtm:

@angeladietz made 1 comment.
Reviewable status: :shipit: complete! 1 of 0 LGTMs obtained (waiting on @dhartunian, @golgeek, and @shailendra-patel).

@tbg
Copy link
Member Author

tbg commented Jan 9, 2026

Test failure is unrelated infra flake

  [mixed-version-test/16_run-maybe-run-backup] 2026/01/08 16:49:22 runner.go:350: mixed-version test failure while running step 16 (run "maybe run backup"): pq: failed to create blob client: connecting to node 1: gRPC: failed to connect to n1 at ubuntu-big-2004-20260108-163513-0563523826-001:29100: grpc: node unavailable; try another peer [code 2/Unknown]

TFTR!

bors r+

@craig
Copy link
Contributor

craig bot commented Jan 9, 2026

@craig craig bot merged commit 03e29d7 into cockroachdb:master Jan 9, 2026
27 of 28 checks passed
@tbg tbg deleted the heap-count-metrics branch January 9, 2026 07:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants