Error logging telemetry causes Puma to exit

[gregbeech]

If a target has an error logging the telemetry then the error is logged using log_writer.error:

log_writer.error "plugin=telemetry err=#{e.class} msg=#{e.message.inspect}"

Unfortunately this method is implemented as:

def error(str)
  @error_logger.info(text: @formatter.call("ERROR: #{str}"))
  exit 1
end

So logging this error causes the master process to exit. As this is just a telemetry logging error which we'd reasonably expect to fail on occasion, particularly if logging performs IO, then this is really undesirable. It'd be better, IMO, if the server logged the error and then continued to serve. Using the unknown_error method would achieve this, e.g.

log_writer.unknown_error(nil, nil, "plugin=telemetry err=#{e.class} msg=#{e.message.inspect}")

Perhaps a more advance approach would be to have a threshold number of failures before exiting, which could be set to 0 to keep compatibility with the existing behaviour if that's important.

[stevenharman]

I wonder if this might be a better way to log the error:

log_writer.unknown_error(e.class.name, nil, "plugin=telemetry msg=#{e.message.inspect}")

Since unknown_error already accepts an "error" argument as the first parameter?