Add opt-in LogRecordProcessor.Enabled (#4439)

Fixes
#4363

Towards
#4208
(uses Severity Level passed via Logger.Enabled)

Towards stabilization of OpenTelemetry Go Logs API and SDK.

## Use cases

Below are some use cases where the new functionality can be used:

1. Bridge features like `LogLevelEnabled` in log bridge/appender
implementations. This is needed for **all** (but one) currently
supported log bridges in OTel Go Contrib.
2. Configure a minimum log severity level for a certain log processor.
3. Filter out log and event records when they are inside a span that has
been sampled out (span is valid and has sampled flag of `false`).
4. **Efficiently** support high-performance logging destination like
[Linux user_events](https://docs.kernel.org/trace/user_events.html) and
[ETW (Event Tracing for
Windows)](https://learn.microsoft.com/windows/win32/etw/about-event-tracing).
5. Bridge Logs API to a language-specific logging library (the other way
than usual).

## Changes

Add `Enabled` opt-in operation to the `LogRecordProcessor`.

I created an OTEP first which was a great for having a lot of
discussions and evaluations of different proposals:
-
#4290

Most importantly from
#4290 (comment):

> Among Go SIG we were evaluating a few times an alternative to provide
some new "filter" abstraction which is decoupled from the "processor".
However, we faced more issues than benefits going this route (some if
this is described here, but there were more issues:
open-telemetry/opentelemetry-go#5825 (comment)
. With the current opt-in `Processor.Enabled` we faced less issues so
far.
> We also do not want to replicate all features from the logging
libraries. If someone prefer the log4j (or other) filter design then
someone can always use a bridge and use log4j for filtering. `Enabled`
callback hook is the simplest design (yet very flexible) which makes it
easy to implement in the SDKs. This design is inspired from the design
of the two most popular Go structured logging libraries:
https://pkg.go.dev/log/slog (standard library) and
https://pkg.go.dev/go.uber.org/zap.
> 
> It is worth to adding that Rust design is similar and it also has an
`Enabled` hook. See
#4363 (comment).
Basically we want to add something like
https://docs.rs/log/latest/log/trait.Log.html#tymethod.enabled to the
`LogRecordProcessor` and allow users to implement `Enabled` in the way
that it will meet their requirements.
> 
> I also want to call out form
https://github.com/open-telemetry/opentelemetry-specification/blob/main/oteps/0265-event-vision.md#open-questions:
> 
> > How to support routing logs from the Logs API to a language-specific
logging library
> 
> To support this we would need a log record processor which bridges the
Logs API calls to given logging library. For such case we would need an
`Enabled` hook in `Processor` to efficiently bridge `Logger.Enabled`
calls. A filterer design would not satisfy such use case.

I decided to name the new operation `Enabled` as:
1. this name is already used in logging libraries in many languages:
#4439 (comment)
2. it matches the name of the API call (for all trace, metrics and logs
APIs).

I was also considering `OnEnabled` to have the same pattern as for
`Emit` and `OnEmit`. However, we already have `ForceFlush` and
`Shutdown` which does not follow this pattern so I preferred to keep the
simple `Enabled` name. For `OnEmit` I could also imagine `OnEmitted` (or
`OnEmitting`) which does something after (or just before like we have
`OnEnding` in `SpanProcessor`) `OnEmit` on all registered processors
were called. Yet, I do not imagine something similar for `Enabled` as
calling `Enabled` should not have any side-effects. Therefore, I decided
to name it `Enabled`.

I want to highlight that a processor cannot assume `Enabled` was called
before `OnEmit`, because of the following reasons:

1. **Backward compatibility** – Existing processors may already perform
filtering without relying on `Enabled`. For example: [Add Advanced
Processing to Logs Supplementary Guidelines
#4407](#4407).
2. **Self-sufficiency of `OnEmit`** – Since `Enabled` is optional,
`OnEmit` should be able to handle filtering independently. A processor
filtering events should do so in `OnEmit`, not just in `Enabled`.
3. **Greater flexibility** – Some processors, such as the ETW processor,
don’t benefit from redundant filtering. ETW already filters out events
internally, making an additional check unnecessary.
4. **Performance considerations** – Calling `Enabled` from `OnEmit`
introduces overhead, as it requires converting `OnEmit` parameters to
match `Enabled`'s expected input.
5. **Avoiding fragile assumptions** – Enforcing constraints that the
compiler cannot validate increases the risk of introducing bugs.


This feature is already implemented in OpenTelemetry Go:
- open-telemetry/opentelemetry-go#6317
We have one processor in Contrib which takes advantage of this
functionality:
- https://pkg.go.dev/go.opentelemetry.io/contrib/processors/minsev

This feautre (however with some differences) is also avaiable in OTel
Rust;
#4363 (comment):

> OTel Rust also has this capability. Here's an example where it is
leveraged to improve performance by dropping unwanted log early.
https://github.com/open-telemetry/opentelemetry-rust/blob/88cae2cf7d0ff54a042d281a0df20f096d18bf82/opentelemetry-appender-tracing/benches/logs.rs#L78-L85

---------

Co-authored-by: Tyler Yahn <[email protected]>
Co-authored-by: Sam Xie <[email protected]>

Author: 3 people

CHANGELOG.md

@@ -15,6 +15,9 @@ release.
 
 ### Logs
 
+- Add `Enabled` opt-in operation to the `LogRecordProcessor`.
+  ([#4439](https://github.com/open-telemetry/opentelemetry-specification/pull/4439))
+
 ### Baggage
 
 ### Resource

spec-compliance-matrix.md

@@ -203,6 +203,7 @@ Disclaimer: this list of features is still a work in progress, please refer to t
 | SimpleLogRecordProcessor                     |          |     | +    |     | +      |      |        | +   |      | +   |      |       |
 | BatchLogRecordProcessor                      |          |     | +    |     | +      |      |        | +   |      | +   |      |       |
 | Can plug custom LogRecordProcessor           |          |     | +    |     | +      |      |        | +   |      | +   |      |       |
+| LogRecordProcessor.Enabled                   | X        | +   |      |     |        |      |        |     |      |     |      |       |
 | OTLP/gRPC exporter                           |          |     | +    |     | +      |      |        | +   |      | +   | +    |       |
 | OTLP/HTTP exporter                           |          |     | +    |     | +      |      |        | +   |      | +   | +    |       |
 | OTLP File exporter                           |          |     | -    |     | -      |      |        |     |      | +   | -    |       |

specification/logs/sdk.md

@@ -24,6 +24,7 @@
 - [LogRecordProcessor](#logrecordprocessor)
   * [LogRecordProcessor operations](#logrecordprocessor-operations)
     + [OnEmit](#onemit)
+    + [Enabled](#enabled-1)
     + [ShutDown](#shutdown)
     + [ForceFlush](#forceflush-1)
   * [Built-in processors](#built-in-processors)
@@ -198,7 +199,9 @@ It consists of the following parameters:
 `Enabled` MUST return `false` when:
 
 - there are no registered [`LogRecordProcessors`](#logrecordprocessor),
-- `Logger` is disabled ([`LoggerConfig.disabled`](#loggerconfig) is `true`).
+- `Logger` is disabled ([`LoggerConfig.disabled`](#loggerconfig) is `true`),
+- all registered `LogRecordProcessors` implement [`Enabled`](#enabled-1),
+  and a call to `Enabled` on each of them returns `false`.
 
 Otherwise, it SHOULD return `true`.
 It MAY return `false` to support additional optimizations and features.
@@ -345,6 +348,46 @@ A `LogRecordProcessor` may freely modify `logRecord` for the duration of
 the `OnEmit` call. If `logRecord` is needed after `OnEmit` returns (i.e. for
 asynchronous processing) only reads are permitted.
 
+#### Enabled
+
+**Status**: [Development](../document-status.md)
+
+`Enabled` is an operation that a `LogRecordProcessor` MAY implement
+in order to support filtering via [`Logger.Enabled`](api.md#enabled).
+
+**Parameters:**
+
+* [Context](../context/README.md) explicitly passed by the caller or the current
+  Context
+* [Instrumentation Scope](./data-model.md#field-instrumentationscope) associated
+  with the `Logger`
+* [Severity Number](./data-model.md#field-severitynumber) passed by the caller
+
+**Returns:** `Boolean`
+
+An implementation should return `false` if a `LogRecord` (if ever created)
+is supposed to be filtered out for the given parameters.
+It should default to returning `true` for any indeterminate state, for example,
+when awaiting configuration.
+
+Any modifications to parameters inside `Enabled` MUST NOT be propagated to the
+caller. Parameters are immutable or passed by value.
+
+This operation is usually called synchronously, therefore it should not block
+or throw exceptions.
+
+`LogRecordProcessor` implementations responsible for filtering and supporting
+the `Enable` operation should ensure that [`OnEmit`](#onemit) handles filtering
+independently. API users cannot be expected to call [`Enabled`](api.md#enabled)
+before invoking [`Emit a LogRecord`](api.md#emit-a-logrecord).
+Moreover, the filtering logic in `OnEmit` and `Enabled` may differ.
+
+`LogRecordProcessor` implementations that wrap other `LogRecordProcessor`
+(which may perform filtering) can implement `Enabled` and delegate to
+the wrapped processor’s `Enabled`, if available. However, the `OnEmit`
+implementation of such processors should never call the wrapped processor’s
+`Enabled`, as `OnEmit` is responsible for handling filtering independently.
+
 #### ShutDown
 
 Shuts down the processor. Called when the SDK is shut down. This is an

specification/logs/supplementary-guidelines.md

@@ -203,6 +203,19 @@ func (p *SeverityProcessor) OnEmit(ctx context.Context, record *sdklog.Record) e
 	}
 	return p.Processor.OnEmit(ctx, record)
 }
+
+// Enabled returns false if the severity is lower than p.Min.
+func (p *SeverityProcessor) Enabled(ctx context.Context, param sdklog.EnabledParameters) bool {
+	sev := param.Severity
+	if sev != log.SeverityUndefined && sev < p.Min {
+		return false
+	}
+	if fp, ok := p.Processor.(sdklog.FilterProcessor); ok {
+		// The wrapped processor is also a filtering processor.
+		return p.Processor.Enabled(ctx, param)
+	}
+	return true
+}
 ```
 
 > [!NOTE]
@@ -242,6 +255,29 @@ func (p *IsolatedProcessor) OnEmit(ctx context.Context, record *log.Record) erro
 	}
 	return rErr
 }
+
+// Enabled honors Enabled of the wrapped processors.
+func (p *IsolatedProcessor) Enabled(ctx context.Context, param sdklog.EnabledParameters) bool  {
+	fltrProcessors := make([]sdklog.FilterProcessor, len(p.Processors))
+	for i, proc := range p.Processors {
+		fp, ok := proc.(sdklog.FilterProcessor)
+		if !ok {
+			// Processor not implementing Enabled.
+			// We assume it will be processed.
+			return true
+		}
+		fltrProcessors[i] = fp
+	}
+
+	for _, proc := range fltrProcessors {
+		if proc.Enabled(ctx, param) {
+			// At least one Processor will process the Record.
+			return true
+		}
+	}
+	// No processor will process the record.
+	return false
+}
 ```
 
 #### Routing
@@ -271,6 +307,29 @@ func (p *LogEventRouteProcessor) OnEmit(ctx context.Context, record *log.Record)
 	}
 	return p.LogProcessor.OnEmit(ctx, record)
 }
+
+// Enabled honors Enabled of the wrapped processors.
+func (p *LogEventRouteProcessor) Enabled(ctx context.Context, param sdklog.EnabledParameters) bool  {
+	fp1, ok := p.EventProcessor.(sdklog.FilterProcessor)
+	if !ok {
+		// Processor not implementing Enabled.
+		return true
+	}
+	fp2, ok := p.LogProcessor.(sdklog.FilterProcessor)
+	if !ok {
+		// Processor not implementing Enabled.
+		return true
+	}
+
+	if fp1.Enabled(ctx, param) {
+		return true
+	}
+	if fp2.Enabled(ctx, param) {
+		return true
+	}
+	// No processor will process the record.
+	return false
+}
 ```
 
 #### Setup