sdk/log: Add FilterProcessor and EnabledParameters #6317

pellared · 2025-02-13T21:13:05Z

We agreed that we can move FilterProcessor directly to sdk/log as Logs SDK does not look to be stabilized soon.

Add the possibility to filter based on the resource and scope which is available for the SDK. The scope information is the most important as it gives the possibility to e.g. filter out logs emitted for a given logger. Thus e.g. Add min_severity to LoggerConfig opentelemetry-specification#4364 is not necessary. See OTEP: Logger.Enabled opentelemetry-specification#4290 (comment) for more context.
It is going be an example for Add Enabled to LogRecordProcessor opentelemetry-specification#4363

There is a little overhead (IMO totally acceptable) because of data transformation. Most importantly, there is no new heap allocation.

goos: linux
goarch: amd64
pkg: go.opentelemetry.io/otel/sdk/log
cpu: 13th Gen Intel(R) Core(TM) i7-13800H
                 │   old.txt   │                 new.txt                  │
                 │   sec/op    │     sec/op      vs base                  │
LoggerEnabled-20   4.589n ± 1%   319.750n ± 16%  +6867.75% (p=0.000 n=10)

                 │   old.txt    │             new.txt             │
                 │     B/op     │     B/op       vs base          │
LoggerEnabled-20   0.000Ki ± 0%   1.093Ki ± 13%  ? (p=0.000 n=10)

                 │  old.txt   │            new.txt             │
                 │ allocs/op  │ allocs/op   vs base            │
LoggerEnabled-20   0.000 ± 0%   0.000 ± 0%  ~ (p=1.000 n=10) ¹
¹ all samples are equal

Logger.Enabled is still more efficient than Logger.Emit (benchmarks from #6315).

goos: linux
goarch: amd64
pkg: go.opentelemetry.io/otel/sdk/log
cpu: 13th Gen Intel(R) Core(TM) i7-13800H
BenchmarkLoggerEmit/5_attributes-20               559934              2391 ns/op           39088 B/op          1 allocs/op
BenchmarkLoggerEmit/10_attributes-20             1000000              5910 ns/op           49483 B/op          5 allocs/op
BenchmarkLoggerEnabled-20                        1605697               968.7 ns/op          1272 B/op          0 allocs/op
PASS
ok      go.opentelemetry.io/otel/sdk/log        10.789s

Prior art:

I also created for tracking purposes:

Stabilize LogRecordProcessor.Enabled opentelemetry-specification#4458

Co-authored-by: Sam Xie <[email protected]>

codecov · 2025-02-13T21:16:00Z

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 81.8%. Comparing base (b80639c) to head (34bae5a).
Report is 1 commits behind head on main.

Additional details and impacted files

@@          Coverage Diff          @@
##            main   open-telemetry/opentelemetry-go#6317   +/-   ##
=====================================
  Coverage   81.8%   81.8%           
=====================================
  Files        283     283           
  Lines      24892   24900    +8     
=====================================
+ Hits       20378   20387    +9     
+ Misses      4110    4109    -1     
  Partials     404     404

see 4 files with indirect coverage changes

sdk/log/filter_processor.go

## Overview This release is the last to support [Go 1.22]. The next release will require at least [Go 1.23]. ### Added - Add `ValueFromAttribute` and `KeyValueFromAttribute` in `go.opentelemetry.io/otel/log`. (#6180) - Add `EventName` and `SetEventName` to `Record` in `go.opentelemetry.io/otel/log`. (#6187) - Add `EventName` to `RecordFactory` in `go.opentelemetry.io/otel/log/logtest`. (#6187) - `AssertRecordEqual` in `go.opentelemetry.io/otel/log/logtest` checks `Record.EventName`. (#6187) - Add `EventName` and `SetEventName` to `Record` in `go.opentelemetry.io/otel/sdk/log`. (#6193) - Add `EventName` to `RecordFactory` in `go.opentelemetry.io/otel/sdk/log/logtest`. (#6193) - Emit `Record.EventName` field in `go.opentelemetry.io/otel/exporters/otlp/otlplog/otlploggrpc`. (#6211) - Emit `Record.EventName` field in `go.opentelemetry.io/otel/exporters/otlp/otlplog/otlploghttp`. (#6211) - Emit `Record.EventName` field in `go.opentelemetry.io/otel/exporters/stdout/stdoutlog` (#6210) - The `go.opentelemetry.io/otel/semconv/v1.28.0` package. The package contains semantic conventions from the `v1.28.0` version of the OpenTelemetry Semantic Conventions. See the [migration documentation](./semconv/v1.28.0/MIGRATION.md) for information on how to upgrade from `go.opentelemetry.io/otel/semconv/v1.27.0`(#6236) - The `go.opentelemetry.io/otel/semconv/v1.30.0` package. The package contains semantic conventions from the `v1.30.0` version of the OpenTelemetry Semantic Conventions. See the [migration documentation](./semconv/v1.30.0/MIGRATION.md) for information on how to upgrade from `go.opentelemetry.io/otel/semconv/v1.28.0`(#6240) - Document the pitfalls of using `Resource` as a comparable type. `Resource.Equal` and `Resource.Equivalent` should be used instead. (#6272) - Support [Go 1.24]. (#6304) - Add `FilterProcessor` and `EnabledParameters` in `go.opentelemetry.io/otel/sdk/log`. It replaces `go.opentelemetry.io/otel/sdk/log/internal/x.FilterProcessor`. Compared to previous version it additionally gives the possibility to filter by resource and instrumentation scope. (#6317) ### Changed - Update `github.com/prometheus/common` to v0.62.0., which changes the `NameValidationScheme` to `NoEscaping`. This allows metrics names to keep original delimiters (e.g. `.`), rather than replacing with underscores. This is controlled by the `Content-Type` header, or can be reverted by setting `NameValidationScheme` to `LegacyValidation` in `github.com/prometheus/common/model`. (#6198) ### Fixes - Eliminate goroutine leak for the processor returned by `NewSimpleSpanProcessor` when `Shutdown` is called and the passed `ctx` is canceled and `SpanExporter.Shutdown` has not returned. (#6368) - Eliminate goroutine leak for the processor returned by `NewBatchSpanProcessor` when `ForceFlush` is called and the passed `ctx` is canceled and `SpanExporter.Export` has not returned. (#6369) [Go 1.23]: https://go.dev/doc/go1.23 [Go 1.22]: https://go.dev/doc/go1.22 --------- Co-authored-by: Robert Pająk <[email protected]> Co-authored-by: Tyler Yahn <[email protected]>

Fixes #4363 Towards #4208 (uses Severity Level passed via Logger.Enabled) Towards stabilization of OpenTelemetry Go Logs API and SDK. ## Use cases Below are some use cases where the new functionality can be used: 1. Bridge features like `LogLevelEnabled` in log bridge/appender implementations. This is needed for **all** (but one) currently supported log bridges in OTel Go Contrib. 2. Configure a minimum log severity level for a certain log processor. 3. Filter out log and event records when they are inside a span that has been sampled out (span is valid and has sampled flag of `false`). 4. **Efficiently** support high-performance logging destination like [Linux user_events](https://docs.kernel.org/trace/user_events.html) and [ETW (Event Tracing for Windows)](https://learn.microsoft.com/windows/win32/etw/about-event-tracing). 5. Bridge Logs API to a language-specific logging library (the other way than usual). ## Changes Add `Enabled` opt-in operation to the `LogRecordProcessor`. I created an OTEP first which was a great for having a lot of discussions and evaluations of different proposals: - #4290 Most importantly from #4290 (comment): > Among Go SIG we were evaluating a few times an alternative to provide some new "filter" abstraction which is decoupled from the "processor". However, we faced more issues than benefits going this route (some if this is described here, but there were more issues: open-telemetry/opentelemetry-go#5825 (comment) . With the current opt-in `Processor.Enabled` we faced less issues so far. > We also do not want to replicate all features from the logging libraries. If someone prefer the log4j (or other) filter design then someone can always use a bridge and use log4j for filtering. `Enabled` callback hook is the simplest design (yet very flexible) which makes it easy to implement in the SDKs. This design is inspired from the design of the two most popular Go structured logging libraries: https://pkg.go.dev/log/slog (standard library) and https://pkg.go.dev/go.uber.org/zap. > > It is worth to adding that Rust design is similar and it also has an `Enabled` hook. See #4363 (comment). Basically we want to add something like https://docs.rs/log/latest/log/trait.Log.html#tymethod.enabled to the `LogRecordProcessor` and allow users to implement `Enabled` in the way that it will meet their requirements. > > I also want to call out form https://github.com/open-telemetry/opentelemetry-specification/blob/main/oteps/0265-event-vision.md#open-questions: > > > How to support routing logs from the Logs API to a language-specific logging library > > To support this we would need a log record processor which bridges the Logs API calls to given logging library. For such case we would need an `Enabled` hook in `Processor` to efficiently bridge `Logger.Enabled` calls. A filterer design would not satisfy such use case. I decided to name the new operation `Enabled` as: 1. this name is already used in logging libraries in many languages: #4439 (comment) 2. it matches the name of the API call (for all trace, metrics and logs APIs). I was also considering `OnEnabled` to have the same pattern as for `Emit` and `OnEmit`. However, we already have `ForceFlush` and `Shutdown` which does not follow this pattern so I preferred to keep the simple `Enabled` name. For `OnEmit` I could also imagine `OnEmitted` (or `OnEmitting`) which does something after (or just before like we have `OnEnding` in `SpanProcessor`) `OnEmit` on all registered processors were called. Yet, I do not imagine something similar for `Enabled` as calling `Enabled` should not have any side-effects. Therefore, I decided to name it `Enabled`. I want to highlight that a processor cannot assume `Enabled` was called before `OnEmit`, because of the following reasons: 1. **Backward compatibility** – Existing processors may already perform filtering without relying on `Enabled`. For example: [Add Advanced Processing to Logs Supplementary Guidelines #4407](#4407). 2. **Self-sufficiency of `OnEmit`** – Since `Enabled` is optional, `OnEmit` should be able to handle filtering independently. A processor filtering events should do so in `OnEmit`, not just in `Enabled`. 3. **Greater flexibility** – Some processors, such as the ETW processor, don’t benefit from redundant filtering. ETW already filters out events internally, making an additional check unnecessary. 4. **Performance considerations** – Calling `Enabled` from `OnEmit` introduces overhead, as it requires converting `OnEmit` parameters to match `Enabled`'s expected input. 5. **Avoiding fragile assumptions** – Enforcing constraints that the compiler cannot validate increases the risk of introducing bugs. This feature is already implemented in OpenTelemetry Go: - open-telemetry/opentelemetry-go#6317 We have one processor in Contrib which takes advantage of this functionality: - https://pkg.go.dev/go.opentelemetry.io/contrib/processors/minsev This feautre (however with some differences) is also avaiable in OTel Rust; #4363 (comment): > OTel Rust also has this capability. Here's an example where it is leveraged to improve performance by dropping unwanted log early. https://github.com/open-telemetry/opentelemetry-rust/blob/88cae2cf7d0ff54a042d281a0df20f096d18bf82/opentelemetry-appender-tracing/benches/logs.rs#L78-L85 --------- Co-authored-by: Tyler Yahn <[email protected]> Co-authored-by: Sam Xie <[email protected]>

pellared and others added 26 commits February 5, 2025 20:30

mv sdk/log/internal/x sdk/xlog

96a79ee

Introduce xlog module with FilterProcessor and EnabledParameters

6282e71

Update changelog

69ff66d

Add trace context fields

bf61d44

Test FilterProcessor functionality

5b3c8ca

Update changelog

f24fc64

go mod tidy

fffc077

Merge branch 'main' into sdk/xlogs

55adcc4

Merge branch 'main' into sdk/xlogs

008cf4d

Remove span context fields from EnabledParameters

4df6669

Simplify doc comments

d31f329

go mod tidy

fe5ae63

Move sdk/xlog to sdk/log/xlog

4e54827

Fix hyperlink

abe7833

Update README.md

d5817c6

Merge branch 'main' into sdk/xlogs

c23714b

Merge branch 'main' into sdk/xlogs

cb91d27

Update sdk/log/example_test.go

71376d1

Co-authored-by: Sam Xie <[email protected]>

Update sdk/log/logger.go

e6b34ae

Co-authored-by: Sam Xie <[email protected]>

Format comment

4ff41e1

Add missing period

628f916

Do not release xlog

98d6811

Improve Go doc comments

5cf1eaf

Merge branch 'main' into sdk/xlogs

6378ea7

Update CHANGELOG.md

46ba89f

Promote FilterProcessor and EnabledParameters to sdk/log

c8deb2f

pellared added pkg:SDK Related to an SDK package area:logs Part of OpenTelemetry logs labels Feb 13, 2025

pellared self-assigned this Feb 13, 2025

pellared added 2 commits February 13, 2025 22:20

Update PR number

c1a5ad7

Merge branch 'main' into filterproc

a7604e1

pellared marked this pull request as ready for review February 13, 2025 21:28

pellared requested review from MrAlias, XSAM, dashpole and dmathieu as code owners February 13, 2025 21:28

pellared added 2 commits February 13, 2025 22:29

Update example_test.go

daf7441

Update example_test.go

2ec602c

pellared mentioned this pull request Jan 23, 2025

Add Enabled to LogRecordProcessor open-telemetry/opentelemetry-specification#4363

Closed

Merge branch 'main' into filterproc

1cdb279

dashpole approved these changes Feb 14, 2025

View reviewed changes

XSAM approved these changes Feb 17, 2025

View reviewed changes

sdk/log/filter_processor.go Show resolved Hide resolved

pellared mentioned this pull request Feb 17, 2025

Stabilize LogRecordProcessor.Enabled open-telemetry/opentelemetry-specification#4458

Open

pellared added this to the v1.35.0 milestone Feb 17, 2025

pellared added 2 commits February 18, 2025 11:55

Merge branch 'main' into filterproc

af3bc56

Merge branch 'main' into filterproc

40478df

MrAlias approved these changes Feb 18, 2025

View reviewed changes

Merge branch 'main' into filterproc

34bae5a

pellared merged commit 1ee7c79 into open-telemetry:main Feb 18, 2025
38 checks passed

pellared deleted the filterproc branch February 18, 2025 21:35

This was referenced Feb 21, 2025

minsev: Update code to the latest FilterProcessor open-telemetry/opentelemetry-go-contrib#6822

Closed

minsev: Update to use sdk/log.FilterProcessor open-telemetry/opentelemetry-go-contrib#6833

Merged

pellared mentioned this pull request Mar 4, 2025

Add opt-in LogRecordProcessor.Enabled open-telemetry/opentelemetry-specification#4439

Merged

XSAM mentioned this pull request Mar 4, 2025

Release v1.35.0/v0.57.0/v0.11.0 #6407

Merged

renovate bot mentioned this pull request Mar 26, 2025

fix(deps): update all non-major go dependencies - autoclosed cloudnative-pg/daggerverse#39

Closed

1 task

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

sdk/log: Add FilterProcessor and EnabledParameters #6317

sdk/log: Add FilterProcessor and EnabledParameters #6317

Uh oh!

pellared commented Feb 13, 2025 •

edited

Loading

Uh oh!

codecov bot commented Feb 13, 2025 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

sdk/log: Add FilterProcessor and EnabledParameters #6317

sdk/log: Add FilterProcessor and EnabledParameters #6317

Uh oh!

Conversation

pellared commented Feb 13, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

codecov bot commented Feb 13, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

pellared commented Feb 13, 2025 •

edited

Loading

codecov bot commented Feb 13, 2025 •

edited

Loading