Add the ability to automatically remove PII from logs

[kylemilloy]

It would be great if we could have some way to remove PII from log context automatically by specifying keys that we don't want to have shown. Being able to register in a config or service provider specific keys that should be omitted and then turn the redacting feature on or off for each channel would be killer. The idea would be if I have it turned on for my channel when I do:

// in a service provider or config maybe?
Log::redact('name', 'email');

// later
Log::debug('Unable to log in user', $user->toArray());

the output in the log would be like:

Unable to log in user
id: 1
name: [REDACTED]
email: [REDACTED]
created_at: 1990-10-05T18:03:52Z
updated_at: 2024-06-15T12:07:21Z

More than happy to open a PR for this but thought I'd see if it's welcome first or not.

[ghost]

Hi.

protected $hidden = [];

from the model does not suit this? Like:

 $user->seHidden(['name', 'email'])->toArray()

[ghost]

Not if

Log::debug('Unable to log in user', $user->seHidden(['name', 'email'])->toArray());

[kylemilloy]

Also, I want this for anything passed to logs. So if I have data from Spatie data or am using a resource or something, anytime I log it I want it redacted.

[kylemilloy]

Yeah again though....in your example I have to do that every time and all the other developers I work with have to remember to do this and I have to remember to catch this in code review....

It's just prone to failure imo.

Edit, giving a quick look I have a small app I'm working on now and we have ~900 log statements. I don't wanna have to do that 10 times, let alone almost 1000.

[ghost]

spatie/laravel-activitylog#217

pr
spatie/laravel-activitylog#442

[kylemilloy]

I appreciate the help but I think we're talking about two different problems. To be clear, I'm not talking about user-defined activity stored in my database, we're talking about log data that might be in sentry, better stack, slack, aws...etc. I apologize if my example pointed you towards that, but the scope of this has to be more general. Some of these providers offer ways to scrub data but not all of them which is why I'm suggesting this enhancement.

To be more clear: I'm talking specifically about logging any arbitrary data here as well, not just user information or a user's action, that can be sent as context to a log. This includes, but is not limited to, user data. This could be responses from other APIs, this could be computed data, this could be a lot of things. When it comes to PII, depending on the context of the data, a lot of stuff could be required to be redacted.

We have found ways around this by augmenting our existing code but I am suggesting making this a native feature of Laravel here so that we don't need to replace the LogManager provider at runtime and be able to handle this behaviour out of the box.

[mbabker]

Log::redact('name', 'email');

The thing that concerns me about a one-size-fits-all "redact any key with this name" approach is it sanitizes too much data.

Log::debug('Fetched events from https://my-events.api: {events}', ['events' => [['id' => 1, 'name' => 'Meetup']]]);

In context, the name key isn't sensitive information and I wouldn't expect it to be redacted here.

That said, since Laravel is using Monolog for its log handling, there's really not much of a need to add this to the framework. There's a package at https://github.com/leocavalcante/redact-sensitive which provides a Monolog processor that looks to be able to do what you want in the way you're trying to accomplish it.

[kylemilloy]

Yeah I totally agree, it's not a magic bullet, but when we have to hide this information typically we prefer to err on the side of hiding too much by accident rather than showing too much. I have no illusion that I'll still have to look at what people are logging to confirm no PII is being leaked but it's at least a safe guard to help if I miss something that there's a hope that this will be caught here instead.

The nice thing here is that because I (presumably) have access to this "my events" api in your example, because I still have the ID in the log I can track down which event is having the error to do digging without having the name...so while it might be a little frustrating, it's not the end of the world.

I've used the redact-sensitive via the tap command before and I found that while it does an okay job, it's not great. Lots of customization has to be done to handle Laravel paradigms for Models, Resources, etc. When augmenting this I have to customize and look for Laravel-specific interfaces (such as, Arrayable and Jsonable) which is why I would wanna bring this functionality into the fold rather than drag it around from project to project.