Parsing audit.log*

[mbneimann-at-work]

Hi,

I want to parse /var/log/audit/audit.log* files but cannot find any predefined parser or examples on how to do this.

I have created the following transformer, but would like to know if there is a better way to do it?

# Parse .message into tokens. Regex accepts values with or without quotations
tokens = parse_regex_all!(.message, r'(?P<key>\w*)=(?P<value>((\'[^\']*\')|\"[^\"]*\"|(\S*)))')

# Parse tokens and set each token as a key/value in .message
map_values(tokens) -> |token| {
    if token.key == "msg" {
        if starts_with(to_string(token.value), "audit(") {
            audit = parse_regex!(token.value, r'audit\((?P<ts>[^:]+):(?P<id>\d+)\):')
            .timestamp = to_timestamp(to_float!(audit.ts)) ?? .timestamp
            .msg = set!(.msg, ["timestamp"], audit.ts)
            .msg = set!(.msg, ["id"], audit.id)
        } else {
            # Parse value of "msg" field into tokens
            msg_tokens = parse_regex_all!(token.value, r'(?P<key>\w*)=(?P<value>((\'[^\']*\')|\"[^\"]*\"|(\S*)))')
            map_values(msg_tokens) -> |msg_token| {
                .msg = set!(.msg, [msg_token.key], msg_token.value)
            }
        }
    } else {
        .= set!(., [token.key], token.value)
    }
}

[spencergilbert]

👋 I don't think I've seen a parser in the wild before, and we don't have a function for that. I'd recommend opening a feature request to add a specific parsing function to VRL.

Your existing implementation looks reasonable at a high level 👍

[mbneimann-at-work]

@spencergilbert Thank you for your reply!

Somebody in vectordotdev/vrl#66 (comment) linked to this https://gist.github.com/kierdavis/b13776847ad2e2fd55740fa5789a44cb

The idea seems to be the same as mine...