Skip to content

Conversation

mahendrabishnoi2
Copy link
Member

@mahendrabishnoi2 mahendrabishnoi2 commented Oct 11, 2025

Fixes #7014

This PR adds support for below self observability metrics for stdoutmetric exporter

  • otel.sdk.exporter.metric_data_point.inflight
  • otel.sdk.exporter.metric_data_point.exported
  • otel.sdk.exporter.operation.duration

These metrics are experimental and hence behind a feature flag OTEL_GO_X_OBSERVABILITY.
Definition of above metrics is available at https://github.com/open-telemetry/semantic-conventions/blob/v1.36.0/docs/otel/sdk-metrics.md

Observability Implementation Checklist

Observability Implementation Checklist

Based on the project Observability guidelines, ensure the following are completed:

Environment Variable Activation

  • Observability features are disabled by default
  • Features are activated through the OTEL_GO_X_OBSERVABILITY environment variable
  • Use consistent pattern with x.Observability.Enabled() check 1
  • Follow established experimental feature pattern 23

Encapsulation

  • Instrumentation is encapsulated within a dedicated struct (e.g., Instrumentation)
  • Instrumentation is not mixed into the instrumented component
  • Instrumentation code is in its own file or package if complex/reused
  • Instrumentation setup doesn't bloat the main component code

Initialization

  • Initialization is only done when observability is enabled
  • Setup is explicit and side-effect free
  • Return errors from initialization when appropriate
  • Use the global Meter provider (e.g., otel.GetMeterProvider())
  • Include proper meter configuration with:
    • Instrumentation package name is used for the Meter
    • Instrumentation version (e.g. Version)
    • Schema URL (e.g. SchemaURL)

Performance

  • Little to no overhead when observability is disabled
  • Expensive operations are only executed when observability is enabled
  • When enabled, instrumentation code paths are optimized to reduce allocation/computation overhead

Attribute and Option Allocation Management

  • Use sync.Pool for attribute slices and options with dynamic attributes
  • Pool objects are properly reset before returning to pool
  • Pools are scoped for maximum efficiency while ensuring correctness

Caching

  • Static attribute sets known at compile time are pre-computed and cached
  • Common attribute combinations use lookup tables/maps

Benchmarking

  • Benchmarks provided for all instrumentation code
  • Benchmark scenarios include both enabled and disabled observability
  • Benchmark results show impact on allocs/op, B/op, and ns/op (use b.ReportAllocs() in benchmarks)

Error Handling and Robustness

  • Errors are reported back to caller when possible
  • Partial failures are handled gracefully
  • Use partially initialized components when available
  • Return errors to caller instead of only using otel.Handle()
  • Use otel.Handle() only when component cannot report error to user

Context Propagation

  • Observability measurements receive the context from the function being measured (don't break context propagation by using context.Background())

Semantic Conventions Compliance

  • All metrics follow OpenTelemetry Semantic Conventions
  • Use the otelconv convenience package for metric semantic conventions
  • Component names follow semantic conventions
  • Use package path scope type as stable identifier for non-standard components
  • Component names are stable unique identifiers
  • Use global counter for uniqueness if necessary
  • Component ID counter is resettable for deterministic testing

Testing

  • Use deterministic testing with isolated state
  • Restore previous state after tests (t.Cleanup())
  • Isolate meter provider for testing
  • Use t.Setenv() for environment variable testing
  • Reset component ID counter for deterministic component names
  • Test order doesn't affect results

Footnotes

  1. https://github.com/open-telemetry/opentelemetry-go/blob/e4ab3141123d0811125a69823dbbe4d9ec5a9b8f/exporters/stdout/stdouttrace/internal/observ/instrumentation.go#L101-L103

  2. https://github.com/open-telemetry/opentelemetry-go/blob/e4ab3141123d0811125a69823dbbe4d9ec5a9b8f/exporters/stdout/stdouttrace/internal/x/x.go

  3. https://github.com/open-telemetry/opentelemetry-go/blob/e4ab3141123d0811125a69823dbbe4d9ec5a9b8f/sdk/internal/x/x.go

mahendrabishnoi2 and others added 30 commits August 4, 2025 22:18
…etrics

1. otel.sdk.exporter.metric_data_point.inflight
2. otel.sdk.exporter.metric_data_point.exported
3. otel.sdk.exporter.operation.duration
- use pool to amortize slice allocation
- pass actual context
- use t.Cleanup instead of defer in tests
- improve readability by returning without using err var
- use metricdatatest for comparision in testcase
@mahendrabishnoi2
Copy link
Member Author

Benchmarks

exporter_1.txt (commit=c106988)
goos: darwin
goarch: arm64
pkg: go.opentelemetry.io/otel/exporters/stdout/stdoutmetric/internal/selfobservability
cpu: Apple M1 Pro
BenchmarkTrackExport/Success-8         	 2128890	       657.0 ns/op	    1185 B/op	      13 allocs/op
BenchmarkTrackExport/Success-8         	 2040018	       593.2 ns/op	    1185 B/op	      13 allocs/op
BenchmarkTrackExport/Success-8         	 2096229	       583.6 ns/op	    1185 B/op	      13 allocs/op
BenchmarkTrackExport/Success-8         	 2097553	       591.9 ns/op	    1185 B/op	      13 allocs/op
BenchmarkTrackExport/Success-8         	 2071342	       598.0 ns/op	    1185 B/op	      13 allocs/op
BenchmarkTrackExport/Success-8         	 1933758	       595.3 ns/op	    1185 B/op	      13 allocs/op
BenchmarkTrackExport/Success-8         	 2066118	       584.8 ns/op	    1185 B/op	      13 allocs/op
BenchmarkTrackExport/Success-8         	 2086310	       652.5 ns/op	    1185 B/op	      13 allocs/op
BenchmarkTrackExport/Success-8         	 2051514	       576.4 ns/op	    1185 B/op	      13 allocs/op
BenchmarkTrackExport/Success-8         	 2041674	       582.2 ns/op	    1185 B/op	      13 allocs/op
BenchmarkTrackExport/WithError-8       	 1635116	       734.6 ns/op	    1443 B/op	      13 allocs/op
BenchmarkTrackExport/WithError-8       	 1599358	       825.2 ns/op	    1442 B/op	      13 allocs/op
BenchmarkTrackExport/WithError-8       	 1554500	       738.6 ns/op	    1443 B/op	      13 allocs/op
BenchmarkTrackExport/WithError-8       	 1631379	       744.9 ns/op	    1443 B/op	      13 allocs/op
BenchmarkTrackExport/WithError-8       	 1628688	       743.6 ns/op	    1443 B/op	      13 allocs/op
BenchmarkTrackExport/WithError-8       	 1623816	       731.9 ns/op	    1443 B/op	      13 allocs/op
BenchmarkTrackExport/WithError-8       	 1631787	       740.9 ns/op	    1443 B/op	      13 allocs/op
BenchmarkTrackExport/WithError-8       	 1652600	       739.9 ns/op	    1443 B/op	      13 allocs/op
BenchmarkTrackExport/WithError-8       	 1646541	       733.1 ns/op	    1443 B/op	      13 allocs/op
BenchmarkTrackExport/WithError-8       	 1642182	       736.2 ns/op	    1443 B/op	      13 allocs/op
PASS
ok  	go.opentelemetry.io/otel/exporters/stdout/stdoutmetric/internal/selfobservability	38.504s
exporter_2.txt (commit=ff814ca)
goos: darwin
goarch: arm64
pkg: go.opentelemetry.io/otel/exporters/stdout/stdoutmetric/internal/observ
cpu: Apple M1 Pro
BenchmarkTrackExport/Success-8         	31497596	        37.67 ns/op	      64 B/op	       1 allocs/op
BenchmarkTrackExport/Success-8         	33536414	        39.01 ns/op	      64 B/op	       1 allocs/op
BenchmarkTrackExport/Success-8         	30539595	        37.28 ns/op	      64 B/op	       1 allocs/op
BenchmarkTrackExport/Success-8         	29877006	        36.79 ns/op	      64 B/op	       1 allocs/op
BenchmarkTrackExport/Success-8         	32583902	        38.99 ns/op	      64 B/op	       1 allocs/op
BenchmarkTrackExport/Success-8         	27887736	        52.16 ns/op	      64 B/op	       1 allocs/op
BenchmarkTrackExport/Success-8         	28600880	        42.28 ns/op	      64 B/op	       1 allocs/op
BenchmarkTrackExport/Success-8         	29103373	        41.58 ns/op	      64 B/op	       1 allocs/op
BenchmarkTrackExport/Success-8         	29177941	        45.74 ns/op	      64 B/op	       1 allocs/op
BenchmarkTrackExport/Success-8         	33234475	        40.57 ns/op	      64 B/op	       1 allocs/op
BenchmarkTrackExport/WithError-8       	 5372739	       222.7 ns/op	     304 B/op	       4 allocs/op
BenchmarkTrackExport/WithError-8       	 5602773	       220.8 ns/op	     304 B/op	       4 allocs/op
BenchmarkTrackExport/WithError-8       	 5389269	       224.0 ns/op	     304 B/op	       4 allocs/op
BenchmarkTrackExport/WithError-8       	 5368492	       217.7 ns/op	     304 B/op	       4 allocs/op
BenchmarkTrackExport/WithError-8       	 5425812	       249.0 ns/op	     304 B/op	       4 allocs/op
BenchmarkTrackExport/WithError-8       	 5417944	       221.5 ns/op	     304 B/op	       4 allocs/op
BenchmarkTrackExport/WithError-8       	 5525214	       223.8 ns/op	     304 B/op	       4 allocs/op
BenchmarkTrackExport/WithError-8       	 5295582	       229.8 ns/op	     304 B/op	       4 allocs/op
BenchmarkTrackExport/WithError-8       	 5546864	       217.8 ns/op	     304 B/op	       4 allocs/op
BenchmarkTrackExport/WithError-8       	 5580722	       214.9 ns/op	     304 B/op	       4 allocs/op
PASS
ok  	go.opentelemetry.io/otel/exporters/stdout/stdoutmetric/internal/observ	29.447s

goos: darwin
goarch: arm64
pkg: go.opentelemetry.io/otel/exporters/stdout/stdoutmetric/internal/observ
cpu: Apple M1 Pro
                        │ exporter_1.txt │            exporter_2.txt            │
                        │     sec/op     │    sec/op     vs base                │
TrackExport/Success-8      592.55n ± 10%   39.79n ± 15%  -93.28% (p=0.000 n=10)
TrackExport/WithError-8     739.2n ±  1%   222.1n ±  3%  -69.96% (p=0.000 n=10)
geomean                     661.8n         94.01n        -85.80%

                        │ exporter_1.txt │           exporter_2.txt           │
                        │      B/op      │    B/op     vs base                │
TrackExport/Success-8       1185.00 ± 0%   64.00 ± 0%  -94.60% (p=0.000 n=10)
TrackExport/WithError-8      1443.0 ± 0%   304.0 ± 0%  -78.93% (p=0.000 n=10)
geomean                     1.277Ki        139.5       -89.33%

                        │ exporter_1.txt │           exporter_2.txt           │
                        │   allocs/op    │ allocs/op   vs base                │
TrackExport/Success-8        13.000 ± 0%   1.000 ± 0%  -92.31% (p=0.000 n=10)
TrackExport/WithError-8      13.000 ± 0%   4.000 ± 0%  -69.23% (p=0.000 n=10)
geomean                       13.00        2.000       -84.62%

@mahendrabishnoi2 mahendrabishnoi2 marked this pull request as ready for review October 12, 2025 07:21
Copy link
Member

@flc1125 flc1125 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To be continued

mp := otel.GetMeterProvider()
m := mp.Meter(
scope,
metric.WithInstrumentationVersion(sdk.Version()),
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We need a Version that should follow the version of its parent module (go.opentelemetry.io/otel/exporters/stdout/stdoutmetric).

Some reference approaches:

return em, err
}

func (em *Instrumentation) TrackExport(ctx context.Context, count int64) func(err error) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can refer to this PR and return a struct to improve performance:

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good suggestion. Improvement can be seen in updated benchmarks.

@MrAlias MrAlias added this to the v1.39.0 milestone Oct 16, 2025
@mahendrabishnoi2
Copy link
Member Author

Updated Benchmarks

exporter_1.txt (commit=c106988)
goos: darwin
goarch: arm64
pkg: go.opentelemetry.io/otel/exporters/stdout/stdoutmetric/internal/selfobservability
cpu: Apple M1 Pro
BenchmarkTrackExport/Success-8         	 2128890	       657.0 ns/op	    1185 B/op	      13 allocs/op
BenchmarkTrackExport/Success-8         	 2040018	       593.2 ns/op	    1185 B/op	      13 allocs/op
BenchmarkTrackExport/Success-8         	 2096229	       583.6 ns/op	    1185 B/op	      13 allocs/op
BenchmarkTrackExport/Success-8         	 2097553	       591.9 ns/op	    1185 B/op	      13 allocs/op
BenchmarkTrackExport/Success-8         	 2071342	       598.0 ns/op	    1185 B/op	      13 allocs/op
BenchmarkTrackExport/Success-8         	 1933758	       595.3 ns/op	    1185 B/op	      13 allocs/op
BenchmarkTrackExport/Success-8         	 2066118	       584.8 ns/op	    1185 B/op	      13 allocs/op
BenchmarkTrackExport/Success-8         	 2086310	       652.5 ns/op	    1185 B/op	      13 allocs/op
BenchmarkTrackExport/Success-8         	 2051514	       576.4 ns/op	    1185 B/op	      13 allocs/op
BenchmarkTrackExport/Success-8         	 2041674	       582.2 ns/op	    1185 B/op	      13 allocs/op
BenchmarkTrackExport/WithError-8       	 1635116	       734.6 ns/op	    1443 B/op	      13 allocs/op
BenchmarkTrackExport/WithError-8       	 1599358	       825.2 ns/op	    1442 B/op	      13 allocs/op
BenchmarkTrackExport/WithError-8       	 1554500	       738.6 ns/op	    1443 B/op	      13 allocs/op
BenchmarkTrackExport/WithError-8       	 1631379	       744.9 ns/op	    1443 B/op	      13 allocs/op
BenchmarkTrackExport/WithError-8       	 1628688	       743.6 ns/op	    1443 B/op	      13 allocs/op
BenchmarkTrackExport/WithError-8       	 1623816	       731.9 ns/op	    1443 B/op	      13 allocs/op
BenchmarkTrackExport/WithError-8       	 1631787	       740.9 ns/op	    1443 B/op	      13 allocs/op
BenchmarkTrackExport/WithError-8       	 1652600	       739.9 ns/op	    1443 B/op	      13 allocs/op
BenchmarkTrackExport/WithError-8       	 1646541	       733.1 ns/op	    1443 B/op	      13 allocs/op
BenchmarkTrackExport/WithError-8       	 1642182	       736.2 ns/op	    1443 B/op	      13 allocs/op
PASS
ok  	go.opentelemetry.io/otel/exporters/stdout/stdoutmetric/internal/selfobservability	38.504s
exporter_3.txt (commit=545ab7e)
goos: darwin
goarch: arm64
pkg: go.opentelemetry.io/otel/exporters/stdout/stdoutmetric/internal/observ
cpu: Apple M1 Pro
BenchmarkTrackExport/Success-8         	39108700	        29.91 ns/op	       0 B/op	       0 allocs/op
BenchmarkTrackExport/Success-8         	59554951	        40.01 ns/op	       0 B/op	       0 allocs/op
BenchmarkTrackExport/Success-8         	31668709	        42.51 ns/op	       0 B/op	       0 allocs/op
BenchmarkTrackExport/Success-8         	31591450	        45.11 ns/op	       0 B/op	       0 allocs/op
BenchmarkTrackExport/Success-8         	30063759	        43.24 ns/op	       0 B/op	       0 allocs/op
BenchmarkTrackExport/Success-8         	30614202	        40.06 ns/op	       0 B/op	       0 allocs/op
BenchmarkTrackExport/Success-8         	32252391	        46.24 ns/op	       0 B/op	       0 allocs/op
BenchmarkTrackExport/Success-8         	46108329	        36.79 ns/op	       0 B/op	       0 allocs/op
BenchmarkTrackExport/Success-8         	48092174	        43.25 ns/op	       0 B/op	       0 allocs/op
BenchmarkTrackExport/Success-8         	34851410	        47.61 ns/op	       0 B/op	       0 allocs/op
BenchmarkTrackExport/WithError-8       	 6605668	       182.8 ns/op	     240 B/op	       3 allocs/op
BenchmarkTrackExport/WithError-8       	 6558997	       186.4 ns/op	     240 B/op	       3 allocs/op
BenchmarkTrackExport/WithError-8       	 6475551	       190.7 ns/op	     240 B/op	       3 allocs/op
BenchmarkTrackExport/WithError-8       	 6609228	       186.8 ns/op	     240 B/op	       3 allocs/op
BenchmarkTrackExport/WithError-8       	 6534050	       188.7 ns/op	     240 B/op	       3 allocs/op
BenchmarkTrackExport/WithError-8       	 6294747	       190.5 ns/op	     240 B/op	       3 allocs/op
BenchmarkTrackExport/WithError-8       	 6340726	       194.8 ns/op	     240 B/op	       3 allocs/op
BenchmarkTrackExport/WithError-8       	 6104074	       198.2 ns/op	     240 B/op	       3 allocs/op
BenchmarkTrackExport/WithError-8       	 6362425	       197.6 ns/op	     240 B/op	       3 allocs/op
BenchmarkTrackExport/WithError-8       	 6397458	       200.4 ns/op	     240 B/op	       3 allocs/op
PASS
ok  	go.opentelemetry.io/otel/exporters/stdout/stdoutmetric/internal/observ	33.281s
goos: darwin
goarch: arm64
pkg: go.opentelemetry.io/otel/exporters/stdout/stdoutmetric/internal/observ
cpu: Apple M1 Pro
                        │ exporter_1.txt │            exporter_3.txt            │
                        │     sec/op     │    sec/op     vs base                │
TrackExport/Success-8      592.55n ± 10%   42.88n ± 14%  -92.76% (p=0.000 n=10)
TrackExport/WithError-8     739.2n ±  1%   190.6n ±  4%  -74.22% (p=0.000 n=10)
geomean                     661.8n         90.40n        -86.34%

                        │ exporter_1.txt │              exporter_3.txt               │
                        │      B/op      │     B/op      vs base                     │
TrackExport/Success-8       1.157Ki ± 0%   0.000Ki ± 0%  -100.00% (p=0.000 n=10)
TrackExport/WithError-8      1443.0 ± 0%     240.0 ± 0%   -83.37% (p=0.000 n=10)
geomean                     1.277Ki                      ?                       ¹ ²
¹ summaries must be >0 to compute geomean
² ratios must be >0 to compute geomean

                        │ exporter_1.txt │             exporter_3.txt              │
                        │   allocs/op    │ allocs/op   vs base                     │
TrackExport/Success-8         13.00 ± 0%    0.00 ± 0%  -100.00% (p=0.000 n=10)
TrackExport/WithError-8      13.000 ± 0%   3.000 ± 0%   -76.92% (p=0.000 n=10)
geomean                       13.00                    ?                       ¹ ²
¹ summaries must be >0 to compute geomean
² ratios must be >0 to compute geomean

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Metrics SDK observability - stdoutmetric exporter metrics

4 participants