stdoutmetric: self-observability: exporter metrics #7492

mahendrabishnoi2 · 2025-10-11T17:39:51Z

This PR adds support for below self observability metrics for stdoutmetric exporter

otel.sdk.exporter.metric_data_point.inflight
otel.sdk.exporter.metric_data_point.exported
otel.sdk.exporter.operation.duration

These metrics are experimental and hence behind a feature flag OTEL_GO_X_OBSERVABILITY.
Definition of above metrics is available at https://github.com/open-telemetry/semantic-conventions/blob/v1.36.0/docs/otel/sdk-metrics.md

Observability Implementation Checklist

Based on the project Observability guidelines, ensure the following are completed:

Environment Variable Activation

Observability features are disabled by default
Features are activated through the OTEL_GO_X_OBSERVABILITY environment variable
Use consistent pattern with x.Observability.Enabled() check ¹
Follow established experimental feature pattern ²³

Encapsulation

Instrumentation is encapsulated within a dedicated struct (e.g., Instrumentation)
Instrumentation is not mixed into the instrumented component
Instrumentation code is in its own file or package if complex/reused
Instrumentation setup doesn't bloat the main component code

Initialization

Initialization is only done when observability is enabled
Setup is explicit and side-effect free
Return errors from initialization when appropriate
Use the global Meter provider (e.g., otel.GetMeterProvider())
Include proper meter configuration with:
- Instrumentation package name is used for the Meter
- Instrumentation version (e.g. Version)
- Schema URL (e.g. SchemaURL)

Performance

Little to no overhead when observability is disabled
Expensive operations are only executed when observability is enabled
When enabled, instrumentation code paths are optimized to reduce allocation/computation overhead

Attribute and Option Allocation Management

Use sync.Pool for attribute slices and options with dynamic attributes
Pool objects are properly reset before returning to pool
Pools are scoped for maximum efficiency while ensuring correctness

Caching

Static attribute sets known at compile time are pre-computed and cached
Common attribute combinations use lookup tables/maps

Benchmarking

Benchmarks provided for all instrumentation code
Benchmark scenarios include both enabled and disabled observability
Benchmark results show impact on allocs/op, B/op, and ns/op (use b.ReportAllocs() in benchmarks)

Error Handling and Robustness

Errors are reported back to caller when possible
Partial failures are handled gracefully
Use partially initialized components when available
Return errors to caller instead of only using otel.Handle()
Use otel.Handle() only when component cannot report error to user

Context Propagation

Observability measurements receive the context from the function being measured (don't break context propagation by using context.Background())

Semantic Conventions Compliance

All metrics follow OpenTelemetry Semantic Conventions
Use the otelconv convenience package for metric semantic conventions
Component names follow semantic conventions
Use package path scope type as stable identifier for non-standard components
Component names are stable unique identifiers
Use global counter for uniqueness if necessary
Component ID counter is resettable for deterministic testing

Testing

Use deterministic testing with isolated state
Restore previous state after tests (t.Cleanup())
Isolate meter provider for testing
Use t.Setenv() for environment variable testing
Reset component ID counter for deterministic component names
Test order doesn't affect results

…GO_X_SELF_OBSERVABILITY

…etrics 1. otel.sdk.exporter.metric_data_point.inflight 2. otel.sdk.exporter.metric_data_point.exported 3. otel.sdk.exporter.operation.duration

…exporter

… the user

- use pool to amortize slice allocation - pass actual context - use t.Cleanup instead of defer in tests - improve readability by returning without using err var

- use metricdatatest for comparision in testcase

…ng in tests

codecov · 2025-10-11T18:09:59Z

Codecov Report

❌ Patch coverage is 89.65517% with 15 lines in your changes missing coverage. Please review.
✅ Project coverage is 86.3%. Comparing base (b5b6989) to head (c540ab0).

Files with missing lines	Patch %	Lines
...rs/stdout/stdoutmetric/internal/observ/exporter.go	83.0%	9 Missing and 3 partials ⚠️
exporters/stdout/stdoutmetric/exporter.go	92.3%	2 Missing and 1 partial ⚠️

Additional details and impacted files

@@          Coverage Diff           @@
##            main   #7492    +/-   ##
======================================
  Coverage   86.3%   86.3%            
======================================
  Files        294     298     +4     
  Lines      25987   26129   +142     
======================================
+ Hits       22431   22557   +126     
- Misses      3183    3195    +12     
- Partials     373     377     +4

Files with missing lines	Coverage Δ
...rs/stdout/stdoutmetric/internal/counter/counter.go	`100.0% <100.0%> (ø)`
...porters/stdout/stdoutmetric/internal/x/features.go	`100.0% <100.0%> (ø)`
exporters/stdout/stdoutmetric/internal/x/x.go	`100.0% <100.0%> (ø)`
exporters/stdout/stdoutmetric/exporter.go	`94.1% <92.3%> (-0.9%)`	⬇️
...rs/stdout/stdoutmetric/internal/observ/exporter.go	`83.0% <83.0%> (ø)`

... and 1 file with indirect coverage changes

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

… `OTEL_GO_X_SELF_OBSERVABILITY` with `OTEL_GO_X_OBSERVABILITY`

mahendrabishnoi2 · 2025-10-12T07:18:43Z

Benchmarks

exporter_1.txt (commit=c106988)

goos: darwin
goarch: arm64
pkg: go.opentelemetry.io/otel/exporters/stdout/stdoutmetric/internal/selfobservability
cpu: Apple M1 Pro
BenchmarkTrackExport/Success-8         	 2128890	       657.0 ns/op	    1185 B/op	      13 allocs/op
BenchmarkTrackExport/Success-8         	 2040018	       593.2 ns/op	    1185 B/op	      13 allocs/op
BenchmarkTrackExport/Success-8         	 2096229	       583.6 ns/op	    1185 B/op	      13 allocs/op
BenchmarkTrackExport/Success-8         	 2097553	       591.9 ns/op	    1185 B/op	      13 allocs/op
BenchmarkTrackExport/Success-8         	 2071342	       598.0 ns/op	    1185 B/op	      13 allocs/op
BenchmarkTrackExport/Success-8         	 1933758	       595.3 ns/op	    1185 B/op	      13 allocs/op
BenchmarkTrackExport/Success-8         	 2066118	       584.8 ns/op	    1185 B/op	      13 allocs/op
BenchmarkTrackExport/Success-8         	 2086310	       652.5 ns/op	    1185 B/op	      13 allocs/op
BenchmarkTrackExport/Success-8         	 2051514	       576.4 ns/op	    1185 B/op	      13 allocs/op
BenchmarkTrackExport/Success-8         	 2041674	       582.2 ns/op	    1185 B/op	      13 allocs/op
BenchmarkTrackExport/WithError-8       	 1635116	       734.6 ns/op	    1443 B/op	      13 allocs/op
BenchmarkTrackExport/WithError-8       	 1599358	       825.2 ns/op	    1442 B/op	      13 allocs/op
BenchmarkTrackExport/WithError-8       	 1554500	       738.6 ns/op	    1443 B/op	      13 allocs/op
BenchmarkTrackExport/WithError-8       	 1631379	       744.9 ns/op	    1443 B/op	      13 allocs/op
BenchmarkTrackExport/WithError-8       	 1628688	       743.6 ns/op	    1443 B/op	      13 allocs/op
BenchmarkTrackExport/WithError-8       	 1623816	       731.9 ns/op	    1443 B/op	      13 allocs/op
BenchmarkTrackExport/WithError-8       	 1631787	       740.9 ns/op	    1443 B/op	      13 allocs/op
BenchmarkTrackExport/WithError-8       	 1652600	       739.9 ns/op	    1443 B/op	      13 allocs/op
BenchmarkTrackExport/WithError-8       	 1646541	       733.1 ns/op	    1443 B/op	      13 allocs/op
BenchmarkTrackExport/WithError-8       	 1642182	       736.2 ns/op	    1443 B/op	      13 allocs/op
PASS
ok  	go.opentelemetry.io/otel/exporters/stdout/stdoutmetric/internal/selfobservability	38.504s

exporter_2.txt (commit=ff814ca)

goos: darwin
goarch: arm64
pkg: go.opentelemetry.io/otel/exporters/stdout/stdoutmetric/internal/observ
cpu: Apple M1 Pro
BenchmarkTrackExport/Success-8         	31497596	        37.67 ns/op	      64 B/op	       1 allocs/op
BenchmarkTrackExport/Success-8         	33536414	        39.01 ns/op	      64 B/op	       1 allocs/op
BenchmarkTrackExport/Success-8         	30539595	        37.28 ns/op	      64 B/op	       1 allocs/op
BenchmarkTrackExport/Success-8         	29877006	        36.79 ns/op	      64 B/op	       1 allocs/op
BenchmarkTrackExport/Success-8         	32583902	        38.99 ns/op	      64 B/op	       1 allocs/op
BenchmarkTrackExport/Success-8         	27887736	        52.16 ns/op	      64 B/op	       1 allocs/op
BenchmarkTrackExport/Success-8         	28600880	        42.28 ns/op	      64 B/op	       1 allocs/op
BenchmarkTrackExport/Success-8         	29103373	        41.58 ns/op	      64 B/op	       1 allocs/op
BenchmarkTrackExport/Success-8         	29177941	        45.74 ns/op	      64 B/op	       1 allocs/op
BenchmarkTrackExport/Success-8         	33234475	        40.57 ns/op	      64 B/op	       1 allocs/op
BenchmarkTrackExport/WithError-8       	 5372739	       222.7 ns/op	     304 B/op	       4 allocs/op
BenchmarkTrackExport/WithError-8       	 5602773	       220.8 ns/op	     304 B/op	       4 allocs/op
BenchmarkTrackExport/WithError-8       	 5389269	       224.0 ns/op	     304 B/op	       4 allocs/op
BenchmarkTrackExport/WithError-8       	 5368492	       217.7 ns/op	     304 B/op	       4 allocs/op
BenchmarkTrackExport/WithError-8       	 5425812	       249.0 ns/op	     304 B/op	       4 allocs/op
BenchmarkTrackExport/WithError-8       	 5417944	       221.5 ns/op	     304 B/op	       4 allocs/op
BenchmarkTrackExport/WithError-8       	 5525214	       223.8 ns/op	     304 B/op	       4 allocs/op
BenchmarkTrackExport/WithError-8       	 5295582	       229.8 ns/op	     304 B/op	       4 allocs/op
BenchmarkTrackExport/WithError-8       	 5546864	       217.8 ns/op	     304 B/op	       4 allocs/op
BenchmarkTrackExport/WithError-8       	 5580722	       214.9 ns/op	     304 B/op	       4 allocs/op
PASS
ok  	go.opentelemetry.io/otel/exporters/stdout/stdoutmetric/internal/observ	29.447s

goos: darwin
goarch: arm64
pkg: go.opentelemetry.io/otel/exporters/stdout/stdoutmetric/internal/observ
cpu: Apple M1 Pro
                        │ exporter_1.txt │            exporter_2.txt            │
                        │     sec/op     │    sec/op     vs base                │
TrackExport/Success-8      592.55n ± 10%   39.79n ± 15%  -93.28% (p=0.000 n=10)
TrackExport/WithError-8     739.2n ±  1%   222.1n ±  3%  -69.96% (p=0.000 n=10)
geomean                     661.8n         94.01n        -85.80%

                        │ exporter_1.txt │           exporter_2.txt           │
                        │      B/op      │    B/op     vs base                │
TrackExport/Success-8       1185.00 ± 0%   64.00 ± 0%  -94.60% (p=0.000 n=10)
TrackExport/WithError-8      1443.0 ± 0%   304.0 ± 0%  -78.93% (p=0.000 n=10)
geomean                     1.277Ki        139.5       -89.33%

                        │ exporter_1.txt │           exporter_2.txt           │
                        │   allocs/op    │ allocs/op   vs base                │
TrackExport/Success-8        13.000 ± 0%   1.000 ± 0%  -92.31% (p=0.000 n=10)
TrackExport/WithError-8      13.000 ± 0%   4.000 ± 0%  -69.23% (p=0.000 n=10)
geomean                       13.00        2.000       -84.62%

flc1125

To be continued

flc1125 · 2025-10-13T01:17:57Z

exporters/stdout/stdoutmetric/internal/x/features.go

+// to the case-insensitive string value of "true" (i.e. "True" and "TRUE"
+// will also enable this).
+var Observability = newFeature(
+	[]string{"OBSERVABILITY", "SELF_OBSERVABILITY"},


Suggested change

[]string{"OBSERVABILITY", "SELF_OBSERVABILITY"},

[]string{"OBSERVABILITY"},

flc1125 · 2025-10-13T01:18:45Z

exporters/stdout/stdoutmetric/internal/x/features_test.go

+
+	const altKey = "OTEL_GO_X_SELF_OBSERVABILITY"
+	require.Contains(t, Observability.Keys(), altKey)


Suggested change

const altKey = "OTEL_GO_X_SELF_OBSERVABILITY"

require.Contains(t, Observability.Keys(), altKey)

Since it is a new component, we do not need to support SELF_OBSERVABILITY.

flc1125 · 2025-10-13T01:24:53Z

exporters/stdout/stdoutmetric/internal/observ/exporter.go

+	mp := otel.GetMeterProvider()
+	m := mp.Meter(
+		scope,
+		metric.WithInstrumentationVersion(sdk.Version()),


We need a Version that should follow the version of its parent module (go.opentelemetry.io/otel/exporters/stdout/stdoutmetric).

Some reference approaches:

opentelemetry-go/exporters/stdout/stdouttrace/internal/version.go

Line 8 in 73cbc69

const Version = "1.38.0"

opentelemetry-go/exporters/stdout/stdouttrace/internal/observ/instrumentation.go

Line 43 in 73cbc69

Version = internal.Version

opentelemetry-go/versions.yaml

Lines 46 to 48 in 73cbc69

go.opentelemetry.io/otel/exporters/stdout/stdouttrace:

version-refs:

- ./exporters/stdout/stdouttrace/internal/version.go

flc1125 · 2025-10-13T01:27:28Z

exporters/stdout/stdoutmetric/internal/observ/exporter.go

+	return em, err
+}
+
+func (em *Instrumentation) TrackExport(ctx context.Context, count int64) func(err error) {


We can refer to this PR and return a struct to improve performance:

Optimize the return type of ExportSpans #7405

opentelemetry-go/exporters/stdout/stdouttrace/internal/observ/instrumentation.go

Lines 167 to 172 in 73cbc69

type ExportOp struct {

ctx context.Context

start time.Time

nSpans int64

inst *Instrumentation

}

flc1125 · 2025-10-13T01:28:25Z

exporters/stdout/stdoutmetric/internal/observ/exporter.go

I think naming this file instrumentation.go might be better, and the same applies to the unit tests.

flc1125 · 2025-10-13T01:33:02Z

exporters/stdout/stdoutmetric/exporter.go

-func (e *exporter) Export(ctx context.Context, data *metricdata.ResourceMetrics) error {
-	if err := ctx.Err(); err != nil {
+func (e *exporter) Export(ctx context.Context, data *metricdata.ResourceMetrics) (err error) {
+	trackExportFunc := e.trackExport(ctx, countDataPoints(data))


I'm wondering if we can directly use the following approach without defining a function (even if not enabled, it still incurs overhead because it returns an empty anonymous function).

if e.inst != nil { // ... }

flc1125 · 2025-10-13T01:35:19Z

exporters/stdout/stdoutmetric/internal/observ/exporter.go

+		em.inflight.Add(ctx, -count, em.addOpts...)
+		if err == nil { // short circuit in case of success to avoid allocations
+			em.exported.Int64Counter.Add(ctx, count, em.addOpts...)
+			em.duration.Float64Histogram.Record(ctx, durationSeconds, em.recordOpts...)


Suggested change

em.duration.Float64Histogram.Record(ctx, durationSeconds, em.recordOpts...)

em.duration.Inst().Record(ctx, durationSeconds, em.recordOpts...)

flc1125 · 2025-10-13T01:35:27Z

exporters/stdout/stdoutmetric/internal/observ/exporter.go

+		durationSeconds := time.Since(begin).Seconds()
+		em.inflight.Add(ctx, -count, em.addOpts...)
+		if err == nil { // short circuit in case of success to avoid allocations
+			em.exported.Int64Counter.Add(ctx, count, em.addOpts...)


Suggested change

em.exported.Int64Counter.Add(ctx, count, em.addOpts...)

em.exported.Inst().Add(ctx, count, em.addOpts...)

mahendrabishnoi2 and others added 30 commits August 4, 2025 22:18

copied code with minor modifications for docs and feature flag: OTEL_…

48b1568

…GO_X_SELF_OBSERVABILITY

Merge branch 'main' into stdoutmetric-auto-instrumentation

c03b9f0

Merge branch 'main' into stdoutmetric-auto-instrumentation

4ddd0b3

added self-observability support to stdoutmetric exporter for below m…

987b77e

…etrics 1. otel.sdk.exporter.metric_data_point.inflight 2. otel.sdk.exporter.metric_data_point.exported 3. otel.sdk.exporter.operation.duration

fixed broken link

b4d2756

added changelog entry for self-observability support in stdoutmetric …

b7181a4

…exporter

run make precommit

c6f91ac

fix a bug where attributes defined in ExporterMetrics are mutated

a9ad0e2

test cases for ExporterMetrics

7b1fb2d

test cases for stdoutmetric exporter

ef4a629

Merge branch 'main' into stdoutmetric-auto-instrumentation

51f2f2b

remove unused receiver to make linter (unused-receiver) happy

38eb316

fix version

b67b593

fix version

ecbb337

make stdoutMetricExporterComponentType as constant

d6d0dd8

Use defer to call trackExportFunc, Thanks to @flc1125

ee64105

duration -> durationSeconds

ddfb3c3

Merge branch 'main' into stdoutmetric-auto-instrumentation

db64dc0

suppress linter as err is used in defer statement

089252e

instead of suppressing error, split if and err check on 2 lines

884ade7

Merge branch 'main' into stdoutmetric-auto-instrumentation

32b6f61

addressed review comment: use named return to make code more readable

460f303

Merge branch 'main' into stdoutmetric-auto-instrumentation

9fea018

name component similar to open-telemetry#7195

a6f0637

flatten the self-observability initialization and return the error to…

938cbb0

… the user

Merge branch 'main' into stdoutmetric-auto-instrumentation

b247192

Merge branch 'main' into stdoutmetric-auto-instrumentation

ee0328e

address review comments

4c357a9

- use pool to amortize slice allocation - pass actual context - use t.Cleanup instead of defer in tests - improve readability by returning without using err var

address review comments

2edfed9

- use metricdatatest for comparision in testcase

generate internal counter package so that it can be tested by resetti…

18e0530

…ng in tests

run make precommit

1f1eb84

mahendrabishnoi2 added 16 commits October 11, 2025 23:41

fix CHANGELOG.md

51e48eb

refactoring to accommodate auto generation of x.go, replace usages of…

7f64c38

… `OTEL_GO_X_SELF_OBSERVABILITY` with `OTEL_GO_X_OBSERVABILITY`

benchmark for TrackExport

c106988

use precomputed set for happy path, still not optimal 5 allocs/op

49c5c9a

happy path allocs reduced to 1/op, will fix the ugly code later

1643d6d

minor refactor

3903f85

move feature flag part to observ package.

7ede990

fix failing test cases

d03013d

make precommit

5742661

rename to Instrumentation

a723bfe

fix CHANGELOG.md

3ab2586

rename self observability to observability

e34443f

make constants unexported

69c1612

helper for component name

6233067

update docs

909bc47

update docs

ff814ca

linter happy

325bda9

mahendrabishnoi2 marked this pull request as ready for review October 12, 2025 07:21

mahendrabishnoi2 requested review from MrAlias, XSAM, dashpole, dmathieu, flc1125 and pellared as code owners October 12, 2025 07:21

flc1125 reviewed Oct 13, 2025

View reviewed changes

MrAlias added this to the v1.39.0 milestone Oct 16, 2025

Merge branch 'main' into stdoutmetric-auto-instrumentation

c540ab0

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

stdoutmetric: self-observability: exporter metrics #7492

stdoutmetric: self-observability: exporter metrics #7492

Uh oh!

mahendrabishnoi2 commented Oct 11, 2025 •

edited

Loading

Uh oh!

codecov bot commented Oct 11, 2025 •

edited

Loading

Uh oh!

mahendrabishnoi2 commented Oct 12, 2025

Uh oh!

flc1125 left a comment

Uh oh!

flc1125 Oct 13, 2025

Uh oh!

flc1125 Oct 13, 2025

Uh oh!

flc1125 Oct 13, 2025

Uh oh!

flc1125 Oct 13, 2025

Uh oh!

flc1125 Oct 13, 2025

Uh oh!

flc1125 Oct 13, 2025

Uh oh!

flc1125 Oct 13, 2025

Uh oh!

flc1125 Oct 13, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

	[]string{"OBSERVABILITY", "SELF_OBSERVABILITY"},
	[]string{"OBSERVABILITY"},


		const altKey = "OTEL_GO_X_SELF_OBSERVABILITY"
		require.Contains(t, Observability.Keys(), altKey)

	go.opentelemetry.io/otel/exporters/stdout/stdouttrace:
	version-refs:
	- ./exporters/stdout/stdouttrace/internal/version.go

	type ExportOp struct {
	ctx context.Context
	start time.Time
	nSpans int64
	inst *Instrumentation
	}

	em.duration.Float64Histogram.Record(ctx, durationSeconds, em.recordOpts...)
	em.duration.Inst().Record(ctx, durationSeconds, em.recordOpts...)

	em.exported.Int64Counter.Add(ctx, count, em.addOpts...)
	em.exported.Inst().Add(ctx, count, em.addOpts...)

stdoutmetric: self-observability: exporter metrics #7492

Are you sure you want to change the base?

stdoutmetric: self-observability: exporter metrics #7492

Uh oh!

Conversation

mahendrabishnoi2 commented Oct 11, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Observability Implementation Checklist

Environment Variable Activation

Encapsulation

Initialization

Performance

Attribute and Option Allocation Management

Caching

Benchmarking

Error Handling and Robustness

Context Propagation

Semantic Conventions Compliance

Testing

Footnotes

Uh oh!

codecov bot commented Oct 11, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

mahendrabishnoi2 commented Oct 12, 2025

Uh oh!

flc1125 left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

mahendrabishnoi2 commented Oct 11, 2025 •

edited

Loading

codecov bot commented Oct 11, 2025 •

edited

Loading