-
Notifications
You must be signed in to change notification settings - Fork 3.2k
Description
Component(s)
processor/redaction
What happened?
Description
As far as I can tell the redaction processor is being overly aggressive processing database statements and span names.
All sanitization rules appear to be applied to database statements in order as opposed to the relevant ones. E.g. SQL rules should only be applied to mysql, postgresql etc... and should not have subsequent rules like Redis or Monogo sanitization applied.
It also appears that the URL sanitizer is being applied to span names, resulting in unexpected span names.
Steps to Reproduce
Submit database client spans that
{"resourceSpans":[{"resource":{"attributes":[{"key":"service.name","value":{"stringValue":"example-service"}},{"key":"service.version","value":{"stringValue":"1.0.0"}}]},"scopeSpans":[{"scope":{"name":"example-instrumentation","version":"1.0.0"},"spans":[{"traceId":"5b8aa5a2d2c872e8321cf37308d69df2","spanId":"7c8f4d2e9a1b3f45","name":"SELECT","kind":3,"startTimeUnixNano":"1699876543150000000","endTimeUnixNano":"1699876543250000000","attributes":[{"key":"db.system","value":{"stringValue":"mysql"}},{"key":"db.name","value":{"stringValue":"users_db"}},{"key":"db.statement","value":{"stringValue":"SELECT id, username, email, password_hash FROM users WHERE email = '[email protected]' AND status = 'active'"}},{"key":"db.operation","value":{"stringValue":"SELECT"}},{"key":"db.sql.table","value":{"stringValue":"users"}},{"key":"net.peer.name","value":{"stringValue":"mysql.example.com"}},{"key":"net.peer.port","value":{"intValue":"3306"}}],"status":{"code":0}},{"traceId":"5b8aa5a2d2c872e8321cf37308d69df2","spanId":"8d9e5f3a0c2d4e56","name":"GET","kind":3,"startTimeUnixNano":"1699876543260000000","endTimeUnixNano":"1699876543320000000","attributes":[{"key":"db.system","value":{"stringValue":"redis"}},{"key":"db.statement","value":{"stringValue":"GET user:session:abc123def456 user:profile:[email protected]"}},{"key":"db.operation","value":{"stringValue":"GET"}},{"key":"net.peer.name","value":{"stringValue":"redis.example.com"}},{"key":"net.peer.port","value":{"intValue":"6379"}}],"status":{"code":0}}]}]}]}Expected Result
The span name should remain unchanged.
The specified attributes should, only apply the appropriate rule based on DB systems
otel-collector-1 | 2025-11-13T07:15:19.549Z info ResourceSpans #0
otel-collector-1 | Resource SchemaURL:
otel-collector-1 | Resource attributes:
otel-collector-1 | -> service.name: Str(example-service)
otel-collector-1 | -> service.version: Str(1.0.0)
otel-collector-1 | ScopeSpans #0
otel-collector-1 | ScopeSpans SchemaURL:
otel-collector-1 | InstrumentationScope example-instrumentation 1.0.0
otel-collector-1 | Span #0
otel-collector-1 | Trace ID : 5b8aa5a2d2c872e8321cf37308d69df2
otel-collector-1 | Parent ID :
otel-collector-1 | ID : 7c8f4d2e9a1b3f45
otel-collector-1 | Name : SELECT
otel-collector-1 | Kind : Client
otel-collector-1 | Start time : 2023-11-13 11:55:43.15 +0000 UTC
otel-collector-1 | End time : 2023-11-13 11:55:43.25 +0000 UTC
otel-collector-1 | Status code : Unset
otel-collector-1 | Status message :
otel-collector-1 | Attributes:
otel-collector-1 | -> db.system: Str(mysql)
otel-collector-1 | -> db.name: Str(users_db)
otel-collector-1 | -> db.statement: Str(SELECT id, username, email, password_hash FROM users WHERE email = ? AND status = ?)
otel-collector-1 | -> db.operation: Str(SELECT)
otel-collector-1 | -> db.sql.table: Str(users)
otel-collector-1 | -> net.peer.name: Str(mysql.example.com)
otel-collector-1 | -> net.peer.port: Int(3306)
otel-collector-1 | -> log.file.name: Str(input.json)
otel-collector-1 | -> redaction.masked.keys: Str(db.statement)
otel-collector-1 | -> redaction.masked.count: Int(1)
otel-collector-1 | Span #1
otel-collector-1 | Trace ID : 5b8aa5a2d2c872e8321cf37308d69df2
otel-collector-1 | Parent ID :
otel-collector-1 | ID : 8d9e5f3a0c2d4e56
otel-collector-1 | Name : GET
otel-collector-1 | Kind : Client
otel-collector-1 | Start time : 2023-11-13 11:55:43.26 +0000 UTC
otel-collector-1 | End time : 2023-11-13 11:55:43.32 +0000 UTC
otel-collector-1 | Status code : Unset
otel-collector-1 | Status message :
otel-collector-1 | Attributes:
otel-collector-1 | -> db.system: Str(redis)
otel-collector-1 | -> db.statement: Str(GET user:session:? user:profile:?)
otel-collector-1 | -> db.operation: Str(GET)
otel-collector-1 | -> net.peer.name: Str(redis.example.com)
otel-collector-1 | -> net.peer.port: Int(6379)
otel-collector-1 | -> log.file.name: Str(input.json)
otel-collector-1 | -> redaction.masked.keys: Str(db.statement)
otel-collector-1 | -> redaction.masked.count: Int(1)
otel-collector-1 | {"resource": {"service.instance.id": "2362d918-eef4-414e-88d6-acb200660add", "service.name": "otelcol-contrib", "service.version": "0.139.0"}, "otelcol.component.id": "debug", Actual Result
Output with all rules applied:
otel-collector-1 | 2025-11-13T07:15:19.549Z info ResourceSpans #0
otel-collector-1 | Resource SchemaURL:
otel-collector-1 | Resource attributes:
otel-collector-1 | -> service.name: Str(example-service)
otel-collector-1 | -> service.version: Str(1.0.0)
otel-collector-1 | ScopeSpans #0
otel-collector-1 | ScopeSpans SchemaURL:
otel-collector-1 | InstrumentationScope example-instrumentation 1.0.0
otel-collector-1 | Span #0
otel-collector-1 | Trace ID : 5b8aa5a2d2c872e8321cf37308d69df2
otel-collector-1 | Parent ID :
otel-collector-1 | ID : 7c8f4d2e9a1b3f45
otel-collector-1 | Name : ...
otel-collector-1 | Kind : Client
otel-collector-1 | Start time : 2023-11-13 11:55:43.15 +0000 UTC
otel-collector-1 | End time : 2023-11-13 11:55:43.25 +0000 UTC
otel-collector-1 | Status code : Unset
otel-collector-1 | Status message :
otel-collector-1 | Attributes:
otel-collector-1 | -> db.system: Str(mysql)
otel-collector-1 | -> db.name: Str(users_db)
otel-collector-1 | -> db.statement: Str(...)
otel-collector-1 | -> db.operation: Str(SELECT)
otel-collector-1 | -> db.sql.table: Str(users)
otel-collector-1 | -> net.peer.name: Str(mysql.example.com)
otel-collector-1 | -> net.peer.port: Int(3306)
otel-collector-1 | -> log.file.name: Str(input.json)
otel-collector-1 | -> redaction.masked.keys: Str(db.statement)
otel-collector-1 | -> redaction.masked.count: Int(1)
otel-collector-1 | Span #1
otel-collector-1 | Trace ID : 5b8aa5a2d2c872e8321cf37308d69df2
otel-collector-1 | Parent ID :
otel-collector-1 | ID : 8d9e5f3a0c2d4e56
otel-collector-1 | Name : ...
otel-collector-1 | Kind : Client
otel-collector-1 | Start time : 2023-11-13 11:55:43.26 +0000 UTC
otel-collector-1 | End time : 2023-11-13 11:55:43.32 +0000 UTC
otel-collector-1 | Status code : Unset
otel-collector-1 | Status message :
otel-collector-1 | Attributes:
otel-collector-1 | -> db.system: Str(redis)
otel-collector-1 | -> db.statement: Str(...)
otel-collector-1 | -> db.operation: Str(GET)
otel-collector-1 | -> net.peer.name: Str(redis.example.com)
otel-collector-1 | -> net.peer.port: Int(6379)
otel-collector-1 | -> log.file.name: Str(input.json)
otel-collector-1 | -> redaction.masked.keys: Str(db.statement)
otel-collector-1 | -> redaction.masked.count: Int(1)
otel-collector-1 | {"resource": {"service.instance.id": "2362d918-eef4-414e-88d6-acb200660add", "service.name": "otelcol-contrib", "service.version": "0.139.0"}, "otelcol.component.id": "debug", "otelcol.component.kind": "exporter", "otelcol.signal": "traces"}Output with only sql sanitization applied:
otel-collector-1 | 2025-11-13T07:29:48.164Z info ResourceSpans #0
otel-collector-1 | Resource SchemaURL:
otel-collector-1 | Resource attributes:
otel-collector-1 | -> service.name: Str(example-service)
otel-collector-1 | -> service.version: Str(1.0.0)
otel-collector-1 | ScopeSpans #0
otel-collector-1 | ScopeSpans SchemaURL:
otel-collector-1 | InstrumentationScope example-instrumentation 1.0.0
otel-collector-1 | Span #0
otel-collector-1 | Trace ID : 5b8aa5a2d2c872e8321cf37308d69df2
otel-collector-1 | Parent ID :
otel-collector-1 | ID : 7c8f4d2e9a1b3f45
otel-collector-1 | Name : SELECT
otel-collector-1 | Kind : Client
otel-collector-1 | Start time : 2023-11-13 11:55:43.15 +0000 UTC
otel-collector-1 | End time : 2023-11-13 11:55:43.25 +0000 UTC
otel-collector-1 | Status code : Unset
otel-collector-1 | Status message :
otel-collector-1 | Attributes:
otel-collector-1 | -> db.system: Str(mysql)
otel-collector-1 | -> db.name: Str(users_db)
otel-collector-1 | -> db.statement: Str(SELECT id, username, email, password_hash FROM users WHERE email = ? AND status = ?)
otel-collector-1 | -> db.operation: Str(SELECT)
otel-collector-1 | -> db.sql.table: Str(users)
otel-collector-1 | -> net.peer.name: Str(mysql.example.com)
otel-collector-1 | -> net.peer.port: Int(3306)
otel-collector-1 | -> log.file.name: Str(input.json)
otel-collector-1 | -> redaction.masked.keys: Str(db.statement)
otel-collector-1 | -> redaction.masked.count: Int(1)
otel-collector-1 | Span #1
otel-collector-1 | Trace ID : 5b8aa5a2d2c872e8321cf37308d69df2
otel-collector-1 | Parent ID :
otel-collector-1 | ID : 8d9e5f3a0c2d4e56
otel-collector-1 | Name : GET
otel-collector-1 | Kind : Client
otel-collector-1 | Start time : 2023-11-13 11:55:43.26 +0000 UTC
otel-collector-1 | End time : 2023-11-13 11:55:43.32 +0000 UTC
otel-collector-1 | Status code : Unset
otel-collector-1 | Status message :
otel-collector-1 | Attributes:
otel-collector-1 | -> db.system: Str(redis)
otel-collector-1 | -> db.statement: Str(GET user:session:abc?def? user:profile:[email protected])
otel-collector-1 | -> db.operation: Str(GET)
otel-collector-1 | -> net.peer.name: Str(redis.example.com)
otel-collector-1 | -> net.peer.port: Int(6379)
otel-collector-1 | -> log.file.name: Str(input.json)
otel-collector-1 | -> redaction.masked.keys: Str(db.statement)
otel-collector-1 | -> redaction.masked.count: Int(1)
otel-collector-1 | {"resource": {"service.instance.id": "4039722f-0bc5-4632-801c-caa2bd206a0e", "service.name": "otelcol-contrib", "service.version": "0.139.0"}, "otelcol.component.id": "debug", "otelcol.component.kind": "exporter", "otelcol.signal": "traces"}
Collector version
v0.138.0+
Environment information
Environment
OS: (e.g., "Ubuntu 20.04")
Compiler(if manually compiled): (e.g., "go 14.2")
OpenTelemetry Collector configuration
receivers:
otlpjsonfile:
include:
- /input/input.json
start_at: beginning
processors:
redaction:
allow_all_keys: true
db_sanitizer:
sql:
enabled: true
attributes: ["db.statement"]
redis:
enabled: false
attributes: ["db.statement"]
memcached:
enabled: false
attributes: ["db.statement"]
mongo:
enabled: false
attributes: ["db.statement"]
opensearch:
enabled: false
attributes: ["db.statement"]
es:
enabled: false
attributes: ["db.statement"]
summary: debug
exporters:
debug:
verbosity: detailed
service:
pipelines:
traces:
receivers: [otlpjsonfile]
processors: [redaction]
exporters: [debug]
receivers:
otlpjsonfile:
include:
- /input/input.json
start_at: beginning
processors:
redaction:
allow_all_keys: true
db_sanitizer:
sql:
enabled: true
attributes: ["db.statement"]
redis:
enabled: true
attributes: ["db.statement"]
memcached:
enabled: true
attributes: ["db.statement"]
mongo:
enabled: true
attributes: ["db.statement"]
opensearch:
enabled: true
attributes: ["db.statement"]
es:
enabled: true
attributes: ["db.statement"]
summary: debug
exporters:
debug:
verbosity: detailed
service:
pipelines:
traces:
receivers: [otlpjsonfile]
processors: [redaction]
exporters: [debug]Log output
Additional context
No response
Tip
React with 👍 to help prioritize this issue. Please use comments to provide useful context, avoiding +1 or me too, to help us triage it. Learn more here.