Sentry logs not respecting Sidekiq/ActiveJob logs configuration?

[marcelolx]

Issue Description

We have a Rails app that uses Sidekiq for background job processing in combination with ActiveJob, and we have the following configuration for the production environment 👇 with the goal of not logging any ActiveJob logs & only sidekiq warn level logs

Sidekiq.configure_server do |config|
  config.logger.formatter = Sidekiq::Logger::Formatters::JSON.new
  config.logger.level = Logger::WARN
end 

ActiveJob::Base.logger = Logger.new(IO::NULL)

But somehow, ActiveJob seem the picked up by Sentry even though we have set the AJ logger to Logger.new(IO::NULL)?

Reproduction Steps

1. Create a rails project from scratch
2. Add sentry-ruby, sentry-rails, sentry-sidekiq, sidekiq gems
3. Configure the sidekiq server log level to warn and set the ActiveJob logger to Logger.new(IO::NULL)
4. Configure Sentry to send logs (relevant sentry.rb configs)

config.enable_logs = true
config.enabled_patches = [:logger]  
config.rails.structured_logging.enabled = false

Expected Behavior

ActiveJob logs don't get reported to Sentry

Actual Behavior

ActiveJob logs are being reported to Sentry

Ruby Version

3.4.5

SDK Version

6.2.0

Integration and Its Version

Rails/Sidekiq 6.2.0

Sentry Config

# frozen_string_literal: true

#
# See: https://docs.sentry.io/platforms/ruby/guides/rails/
#
Sentry.init do |config|
  config.dsn = ENV["SENTRY_DSN"]

  config.enabled_environments = ["production"]

  config.enable_logs = true
  config.enabled_patches = [:logger]
  config.rails.structured_logging.enabled = false
  
  config.breadcrumbs_logger = [:active_support_logger, :http_logger]

  config.traces_sampler = lambda do |sampling_context|
    # if this is the continuation of a trace, just use that decision (rate controlled by the caller)
    unless sampling_context[:parent_sampled].nil?
      next sampling_context[:parent_sampled]
    end

    # transaction_context is the transaction object in hash form
    # keep in mind that sampling happens right after the transaction is initialized
    # for example, at the beginning of the request
    transaction_context = sampling_context[:transaction_context]

    # transaction_context helps you sample transactions with more sophistication
    # for example, you can provide different sample rates based on the operation or name
    op = transaction_context[:op]
    transaction_name = transaction_context[:name]

    case op
    when /http/
      # for Rails applications, transaction_name would be the request's path (env["PATH_INFO"]) instead of "Controller#action"
      case transaction_name
      when /hubspot\/cards/
        0.001
      when /webhook/
        0.001
      when /viewings/
        0.0001
      else
        0.01
      end
    when /sidekiq/
      case transaction_name
      when /WebhookJob/
        0.0001
      when /ComputedFields/
        0.0001
      when /Inbox/
        0.0001
      when /Postmark/
        0.0001
      when /Mailer/
        0.0001
      when /Fetch/
        0.0001
      when /Poll/
        0.0001
      when /SyncPortal/
        0.0001
      when /SyncInstance/
        0.0001
      else
        0.001
      end
    else
      0.0 # ignore all other transactions
    end
  end

  # The profiles_sample_rate setting is relative to the traces_sample_rate setting.
  config.profiles_sample_rate = 1.0
end

[linear]

RUBY-135

[sl0thentr0py]

@solnic not sure what to do here, should we return early in our patch if it's a IO::NULL logger? Is that a safe assumption to make?

[solnic]

@sl0thentr0py I don't think it is. The fact that log device is IO::NULL does not mean that there is no logger, it's just that it's side-effect of logging somewhere visible is suppressed by the usage of IO::NULL.

@marcelolx could you tell me the reason why you decided to go with the patch instead of rails instrumentation?

[marcelolx]

@solnic by rails instrumentation you mean Rails structured logs? If that's the case, the reason was simply that we were getting too many logs with structured logs.

And for this patch 👇 it was mostly because we were lazy and didn't want to replace all of our manual Rails.logger.info calls with Sentry.logger (mostly because we were trying out logs) — I can try removing that patch to see how it works out (if that would prevent the activejob logs)

config.enabled_patches = [:logger]

[solnic]

@marcelolx OK so your goal is to:

1. Send all Rails logs to Sentry
2. Skip ActiveJob logs coming from its logger

Yes?

You could use before_send_log to filter out ActiveJob log entries, see https://docs.sentry.io/platforms/ruby/configuration/options/#before_send_log