Skip to content

Database Sanitization is Overly Aggressive #44229

@arielvalentin

Description

@arielvalentin

Component(s)

processor/redaction

What happened?

Description

As far as I can tell the redaction processor is being overly aggressive processing database statements and span names.

All sanitization rules appear to be applied to database statements in order as opposed to the relevant ones. E.g. SQL rules should only be applied to mysql, postgresql etc... and should not have subsequent rules like Redis or Monogo sanitization applied.

It also appears that the URL sanitizer is being applied to span names, resulting in unexpected span names.

Steps to Reproduce

Submit database client spans that

{"resourceSpans":[{"resource":{"attributes":[{"key":"service.name","value":{"stringValue":"example-service"}},{"key":"service.version","value":{"stringValue":"1.0.0"}}]},"scopeSpans":[{"scope":{"name":"example-instrumentation","version":"1.0.0"},"spans":[{"traceId":"5b8aa5a2d2c872e8321cf37308d69df2","spanId":"7c8f4d2e9a1b3f45","name":"SELECT","kind":3,"startTimeUnixNano":"1699876543150000000","endTimeUnixNano":"1699876543250000000","attributes":[{"key":"db.system","value":{"stringValue":"mysql"}},{"key":"db.name","value":{"stringValue":"users_db"}},{"key":"db.statement","value":{"stringValue":"SELECT id, username, email, password_hash FROM users WHERE email = '[email protected]' AND status = 'active'"}},{"key":"db.operation","value":{"stringValue":"SELECT"}},{"key":"db.sql.table","value":{"stringValue":"users"}},{"key":"net.peer.name","value":{"stringValue":"mysql.example.com"}},{"key":"net.peer.port","value":{"intValue":"3306"}}],"status":{"code":0}},{"traceId":"5b8aa5a2d2c872e8321cf37308d69df2","spanId":"8d9e5f3a0c2d4e56","name":"GET","kind":3,"startTimeUnixNano":"1699876543260000000","endTimeUnixNano":"1699876543320000000","attributes":[{"key":"db.system","value":{"stringValue":"redis"}},{"key":"db.statement","value":{"stringValue":"GET user:session:abc123def456 user:profile:[email protected]"}},{"key":"db.operation","value":{"stringValue":"GET"}},{"key":"net.peer.name","value":{"stringValue":"redis.example.com"}},{"key":"net.peer.port","value":{"intValue":"6379"}}],"status":{"code":0}}]}]}]}

Expected Result

The span name should remain unchanged.

The specified attributes should, only apply the appropriate rule based on DB systems

otel-collector-1  | 2025-11-13T07:15:19.549Z    info    ResourceSpans #0
otel-collector-1  | Resource SchemaURL: 
otel-collector-1  | Resource attributes:
otel-collector-1  |      -> service.name: Str(example-service)
otel-collector-1  |      -> service.version: Str(1.0.0)
otel-collector-1  | ScopeSpans #0
otel-collector-1  | ScopeSpans SchemaURL: 
otel-collector-1  | InstrumentationScope example-instrumentation 1.0.0
otel-collector-1  | Span #0
otel-collector-1  |     Trace ID       : 5b8aa5a2d2c872e8321cf37308d69df2
otel-collector-1  |     Parent ID      : 
otel-collector-1  |     ID             : 7c8f4d2e9a1b3f45
otel-collector-1  |     Name           : SELECT
otel-collector-1  |     Kind           : Client
otel-collector-1  |     Start time     : 2023-11-13 11:55:43.15 +0000 UTC
otel-collector-1  |     End time       : 2023-11-13 11:55:43.25 +0000 UTC
otel-collector-1  |     Status code    : Unset
otel-collector-1  |     Status message : 
otel-collector-1  | Attributes:
otel-collector-1  |      -> db.system: Str(mysql)
otel-collector-1  |      -> db.name: Str(users_db)
otel-collector-1  |      -> db.statement: Str(SELECT id, username, email, password_hash FROM users WHERE email = ? AND status = ?)
otel-collector-1  |      -> db.operation: Str(SELECT)
otel-collector-1  |      -> db.sql.table: Str(users)
otel-collector-1  |      -> net.peer.name: Str(mysql.example.com)
otel-collector-1  |      -> net.peer.port: Int(3306)
otel-collector-1  |      -> log.file.name: Str(input.json)
otel-collector-1  |      -> redaction.masked.keys: Str(db.statement)
otel-collector-1  |      -> redaction.masked.count: Int(1)
otel-collector-1  | Span #1
otel-collector-1  |     Trace ID       : 5b8aa5a2d2c872e8321cf37308d69df2
otel-collector-1  |     Parent ID      : 
otel-collector-1  |     ID             : 8d9e5f3a0c2d4e56
otel-collector-1  |     Name           : GET
otel-collector-1  |     Kind           : Client
otel-collector-1  |     Start time     : 2023-11-13 11:55:43.26 +0000 UTC
otel-collector-1  |     End time       : 2023-11-13 11:55:43.32 +0000 UTC
otel-collector-1  |     Status code    : Unset
otel-collector-1  |     Status message : 
otel-collector-1  | Attributes:
otel-collector-1  |      -> db.system: Str(redis)
otel-collector-1  |      -> db.statement: Str(GET user:session:? user:profile:?)
otel-collector-1  |      -> db.operation: Str(GET)
otel-collector-1  |      -> net.peer.name: Str(redis.example.com)
otel-collector-1  |      -> net.peer.port: Int(6379)
otel-collector-1  |      -> log.file.name: Str(input.json)
otel-collector-1  |      -> redaction.masked.keys: Str(db.statement)
otel-collector-1  |      -> redaction.masked.count: Int(1)
otel-collector-1  |     {"resource": {"service.instance.id": "2362d918-eef4-414e-88d6-acb200660add", "service.name": "otelcol-contrib", "service.version": "0.139.0"}, "otelcol.component.id": "debug", 

Actual Result

Output with all rules applied:

otel-collector-1  | 2025-11-13T07:15:19.549Z    info    ResourceSpans #0
otel-collector-1  | Resource SchemaURL: 
otel-collector-1  | Resource attributes:
otel-collector-1  |      -> service.name: Str(example-service)
otel-collector-1  |      -> service.version: Str(1.0.0)
otel-collector-1  | ScopeSpans #0
otel-collector-1  | ScopeSpans SchemaURL: 
otel-collector-1  | InstrumentationScope example-instrumentation 1.0.0
otel-collector-1  | Span #0
otel-collector-1  |     Trace ID       : 5b8aa5a2d2c872e8321cf37308d69df2
otel-collector-1  |     Parent ID      : 
otel-collector-1  |     ID             : 7c8f4d2e9a1b3f45
otel-collector-1  |     Name           : ...
otel-collector-1  |     Kind           : Client
otel-collector-1  |     Start time     : 2023-11-13 11:55:43.15 +0000 UTC
otel-collector-1  |     End time       : 2023-11-13 11:55:43.25 +0000 UTC
otel-collector-1  |     Status code    : Unset
otel-collector-1  |     Status message : 
otel-collector-1  | Attributes:
otel-collector-1  |      -> db.system: Str(mysql)
otel-collector-1  |      -> db.name: Str(users_db)
otel-collector-1  |      -> db.statement: Str(...)
otel-collector-1  |      -> db.operation: Str(SELECT)
otel-collector-1  |      -> db.sql.table: Str(users)
otel-collector-1  |      -> net.peer.name: Str(mysql.example.com)
otel-collector-1  |      -> net.peer.port: Int(3306)
otel-collector-1  |      -> log.file.name: Str(input.json)
otel-collector-1  |      -> redaction.masked.keys: Str(db.statement)
otel-collector-1  |      -> redaction.masked.count: Int(1)
otel-collector-1  | Span #1
otel-collector-1  |     Trace ID       : 5b8aa5a2d2c872e8321cf37308d69df2
otel-collector-1  |     Parent ID      : 
otel-collector-1  |     ID             : 8d9e5f3a0c2d4e56
otel-collector-1  |     Name           : ...
otel-collector-1  |     Kind           : Client
otel-collector-1  |     Start time     : 2023-11-13 11:55:43.26 +0000 UTC
otel-collector-1  |     End time       : 2023-11-13 11:55:43.32 +0000 UTC
otel-collector-1  |     Status code    : Unset
otel-collector-1  |     Status message : 
otel-collector-1  | Attributes:
otel-collector-1  |      -> db.system: Str(redis)
otel-collector-1  |      -> db.statement: Str(...)
otel-collector-1  |      -> db.operation: Str(GET)
otel-collector-1  |      -> net.peer.name: Str(redis.example.com)
otel-collector-1  |      -> net.peer.port: Int(6379)
otel-collector-1  |      -> log.file.name: Str(input.json)
otel-collector-1  |      -> redaction.masked.keys: Str(db.statement)
otel-collector-1  |      -> redaction.masked.count: Int(1)
otel-collector-1  |     {"resource": {"service.instance.id": "2362d918-eef4-414e-88d6-acb200660add", "service.name": "otelcol-contrib", "service.version": "0.139.0"}, "otelcol.component.id": "debug", "otelcol.component.kind": "exporter", "otelcol.signal": "traces"}

Output with only sql sanitization applied:

otel-collector-1  | 2025-11-13T07:29:48.164Z    info    ResourceSpans #0
otel-collector-1  | Resource SchemaURL: 
otel-collector-1  | Resource attributes:
otel-collector-1  |      -> service.name: Str(example-service)
otel-collector-1  |      -> service.version: Str(1.0.0)
otel-collector-1  | ScopeSpans #0
otel-collector-1  | ScopeSpans SchemaURL: 
otel-collector-1  | InstrumentationScope example-instrumentation 1.0.0
otel-collector-1  | Span #0
otel-collector-1  |     Trace ID       : 5b8aa5a2d2c872e8321cf37308d69df2
otel-collector-1  |     Parent ID      : 
otel-collector-1  |     ID             : 7c8f4d2e9a1b3f45
otel-collector-1  |     Name           : SELECT
otel-collector-1  |     Kind           : Client
otel-collector-1  |     Start time     : 2023-11-13 11:55:43.15 +0000 UTC
otel-collector-1  |     End time       : 2023-11-13 11:55:43.25 +0000 UTC
otel-collector-1  |     Status code    : Unset
otel-collector-1  |     Status message : 
otel-collector-1  | Attributes:
otel-collector-1  |      -> db.system: Str(mysql)
otel-collector-1  |      -> db.name: Str(users_db)
otel-collector-1  |      -> db.statement: Str(SELECT id, username, email, password_hash FROM users WHERE email = ? AND status = ?)
otel-collector-1  |      -> db.operation: Str(SELECT)
otel-collector-1  |      -> db.sql.table: Str(users)
otel-collector-1  |      -> net.peer.name: Str(mysql.example.com)
otel-collector-1  |      -> net.peer.port: Int(3306)
otel-collector-1  |      -> log.file.name: Str(input.json)
otel-collector-1  |      -> redaction.masked.keys: Str(db.statement)
otel-collector-1  |      -> redaction.masked.count: Int(1)
otel-collector-1  | Span #1
otel-collector-1  |     Trace ID       : 5b8aa5a2d2c872e8321cf37308d69df2
otel-collector-1  |     Parent ID      : 
otel-collector-1  |     ID             : 8d9e5f3a0c2d4e56
otel-collector-1  |     Name           : GET
otel-collector-1  |     Kind           : Client
otel-collector-1  |     Start time     : 2023-11-13 11:55:43.26 +0000 UTC
otel-collector-1  |     End time       : 2023-11-13 11:55:43.32 +0000 UTC
otel-collector-1  |     Status code    : Unset
otel-collector-1  |     Status message : 
otel-collector-1  | Attributes:
otel-collector-1  |      -> db.system: Str(redis)
otel-collector-1  |      -> db.statement: Str(GET user:session:abc?def? user:profile:[email protected])
otel-collector-1  |      -> db.operation: Str(GET)
otel-collector-1  |      -> net.peer.name: Str(redis.example.com)
otel-collector-1  |      -> net.peer.port: Int(6379)
otel-collector-1  |      -> log.file.name: Str(input.json)
otel-collector-1  |      -> redaction.masked.keys: Str(db.statement)
otel-collector-1  |      -> redaction.masked.count: Int(1)
otel-collector-1  |     {"resource": {"service.instance.id": "4039722f-0bc5-4632-801c-caa2bd206a0e", "service.name": "otelcol-contrib", "service.version": "0.139.0"}, "otelcol.component.id": "debug", "otelcol.component.kind": "exporter", "otelcol.signal": "traces"}

Collector version

v0.138.0+

Environment information

Environment

OS: (e.g., "Ubuntu 20.04")
Compiler(if manually compiled): (e.g., "go 14.2")

OpenTelemetry Collector configuration

receivers:
  otlpjsonfile:
    include:
      - /input/input.json
    start_at: beginning

processors:
  redaction:
    allow_all_keys: true
    db_sanitizer:
      sql:
        enabled: true
        attributes: ["db.statement"]
      redis:
        enabled: false
        attributes: ["db.statement"]
      memcached:
        enabled: false
        attributes: ["db.statement"]
      mongo:
        enabled: false
        attributes: ["db.statement"]
      opensearch:
        enabled: false
        attributes: ["db.statement"]
      es:
        enabled: false
        attributes: ["db.statement"]
    summary: debug

exporters:
  debug:
    verbosity: detailed

service:
  pipelines:
    traces:
      receivers: [otlpjsonfile]
      processors: [redaction]
      exporters: [debug]




receivers:
  otlpjsonfile:
    include:
      - /input/input.json
    start_at: beginning

processors:
  redaction:
    allow_all_keys: true
    db_sanitizer:
      sql:
        enabled: true
        attributes: ["db.statement"]
      redis:
        enabled: true
        attributes: ["db.statement"]
      memcached:
        enabled: true
        attributes: ["db.statement"]
      mongo:
        enabled: true
        attributes: ["db.statement"]
      opensearch:
        enabled: true
        attributes: ["db.statement"]
      es:
        enabled: true
        attributes: ["db.statement"]
    summary: debug

exporters:
  debug:
    verbosity: detailed

service:
  pipelines:
    traces:
      receivers: [otlpjsonfile]
      processors: [redaction]
      exporters: [debug]

Log output

Additional context

No response

Tip

React with 👍 to help prioritize this issue. Please use comments to provide useful context, avoiding +1 or me too, to help us triage it. Learn more here.

Metadata

Metadata

Assignees

Labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions