Ruby exception occurred: Invalid FieldReference: `auditLogTrailer[messages]

[mtudisco49]

auditLogTrailer.messages

{ "tag": "OWASP_CRS/WEB_ATTACK/XSS", "id": "973338", "msg": "XSS Filter - Category 3: Javascript URI Vector", "file": "/etc/httpd/modsecurity.d/activated_rules/modsecurity_crs_41_xss_attacks.conf", "line": "28", "data": "Matched Data: astyle_ found within ARGS:themeId: deandastyle_WAR_deandastyletheme", "severity": "CRITICAL", "info": "Warning. Pattern match "(?i)((?:=|U\sR\sL\s*\()\s*[^>]\sS\sC\sR\sI\sP\sT\s:|:|[\s\S]allowscriptaccess[\s\S]|[\s\S]src[\s\S]|[\s\S]data:text\/html[\s\S]|[\s\S]xlink:href[\s\S]|[\s\S]base64[\s\S]|[\s\S]xmlns[\s\S]|[\s\S]xht ..." at ARGS:themeId." }, { "id": "981204", "msg": "Inbound Anomaly Score Exceeded (Total Inbound Score: 5, SQLi=0, XSS=5): XSS Filter - Category 3: Javascript URI Vector", "file": "/etc/httpd/modsecurity.d/activated_rules/modsecurity_crs_60_correlation.conf", "line": "37", "info": "Warning. Operator GE matched 5 at TX:inbound_anomaly_score." }

[mtudisco49]

Hi,

first of all, thank you very much for providing this repo! This helped me a lot.

I'm currently having an issue with some log data in sectionPartH not generating the fild modsecSeverity

in the logstash log i get:
[ERROR][logstash.filters.ruby ] Ruby exception occurred: Invalid FieldReference: auditLogTrailer[messages]

however when i see the data severity is there

auditLogTrailer.messages

{ "tag": "OWASP_CRS/WEB_ATTACK/XSS", "id": "973338", "msg": "XSS Filter - Category 3: Javascript URI Vector", "file": "/etc/httpd/modsecurity.d/activated_rules/modsecurity_crs_41_xss_attacks.conf", "line": "28", "data": "Matched Data: astyle_ found within ARGS:themeId: deandastyle_WAR_deandastyletheme", "severity": "CRITICAL", "info": "Warning. Pattern match "(?i)((?:=|U\\sR\\sL\\s*\\()\\s*[^>]\\sS\\sC\\sR\\sI\\sP\\sT\\s:|:|[\\s\\S]allowscriptaccess[\\s\\S]|[\\s\\S]src[\\s\\S]|[\\s\\S]data:text\\/html[\\s\\S]|[\\s\\S]xlink:href[\\s\\S]|[\\s\\S]base64[\\s\\S]|[\\s\\S]xmlns[\\s\\S]|[\\s\\S]xht ..." at ARGS:themeId." }, { "id": "981204", "msg": "Inbound Anomaly Score Exceeded (Total Inbound Score: 5, SQLi=0, XSS=5): XSS Filter - Category 3: Javascript URI Vector", "file": "/etc/httpd/modsecurity.d/activated_rules/modsecurity_crs_60_correlation.conf", "line": "37", "info": "Warning. Operator GE matched 5 at TX:inbound_anomaly_score." }

Any idea out there?

Best regards

[mtudisco49]

Hi, i manage to find a workarround.
i changed file "2089_filter_section_h_example_severities.conf"
from:

filter {

  #~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  # Proceed to extract all distinct "severities"
  # and store them in a top-level "modsecSeverities" array
  #~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

  if [rawSectionH] =~ /.+/ {

    # extract distinct severities from the messages built in 2080_filter_section_h_parse_messages_to_auditLogTrailerMessages.conf
    ruby {
      code => "
          modsecSeverities = Set.new
          trailerMsgs = event.get('auditLogTrailer[messages]')
          trailerMsgs.each {|m|
            if m.key?('severity')
              modsecSeverities.add(m['severity'])
            end
          }
          event.set('modsecSeverities', modsecSeverities.to_a)
        "
    }
  }

}

to:

filter {
#~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  # Proceed to extract all distinct "severities"
  # and store them in a top-level "modsecSeverities" array
 #~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  if [rawSectionH] =~ /.+/ {
    # extract distinct severities from the messages built in ```
2080_filter_section_h_parse_messages_to_auditLogTrailerMessages.conf
    ruby {
      code => "
          modsecSeverities = Set.new
#          trailerMsgs = event.get('auditLogTrailer[messages]')
          pepe = event.get('auditLogTrailer')
          trailerMsgs = pepe['messages']
          trailerMsgs.each {|m|
            if m.key?('severity')
              modsecSeverities.add(m['severity'])
            end
          }
          event.set('modsecSeverities', modsecSeverities.to_a)
        "
    }
  }
}

[bitsofinfo]

Please submit a PR if this fix goes beyond your particular use-case and would benefit others. Also please describe your setup (logstash version, modsec version etc)

[cukal]

I ran into the same issue using:

CentOS7 3.10.0-1062.4.3.el7.x86_64
mod_security 2.9.2
Logstash 7.5.1