Skip to content

Lowercase mutate filter followed by gsub not working. #152

@adison88

Description

@adison88

Please post all product and debugging questions on our forum. Your questions will reach our wider community members there, and if we confirm that there is a bug, then we can open a new issue here.

For all general issues, please provide the following details for fast resolution:

  • Version:
Logstash 6.8.5
Elasticsearch & Kibana 6.3
Filebeat 6.3
Winlogbeat 6.0
  • Operating System:
  • Config File (if you have sensitive info, please remove it):

`
mutate {
add_field => { "Static_Url" => "%{Url}" }
}

    mutate {
        gsub => [
            "Static_Url", "\\/[({]?[a-fA-F0-9]{8}[-]?([a-fA-F0-9]{4}[-]?){3}[a-fA-F0-9]{12}[})]?+", "/{uuid}"
        ]
    }

    mutate {
        gsub => [
            "Static_Url", "\\/[0-9]+( \\/|\\/|$)", "/{id}/"
        ]
    }

    mutate {
        lowercase => [ "Static_Url" ]
    }

`

  • Sample Data:

`
{
"_type": "doc",
"_version": 1,
"_score": null,
"_source": {
"log_timestamp": "2021-03-29T02:31:28.000Z",
"type": "lumberjack-json_line",
"Url": "/DeviceServices/Dm.svc/token/jW6Q8/333CB41462686848A9BA73ECA4794B05",
"path": "e:\AirWatch\Logs\IIS\W3SVC1\u_ex210329.log",
"LogSize": 1432,
"sc_bytes": "1097",
"@Version": "1",
"beat": {
"name": "",
"version": "6.8.5",
"hostname": ""
},
"Static_Url": "/DeviceServices/Dm.svc/token/jW6Q8/{uuid}",
"tags": [
"filebeat",
"ws1",
"iis-log",
"us04_ls",
"beats_input_codec_plain_applied",
"lumberjack-json_line-5006"
],
"delay": {
"downstream_logstash_ack": -12.848,
"filebeat_read": "2021-03-29T02:31:31.587Z"
},
"@timestamp": "2021-03-29T02:31:31.587Z",
"WinStatus": "0",
"_logzio_pattern": 3516387,
"Username": "-",
"SubStatusCode": "0",
"fields": {
"filebeat_config_version": "20.12",
"env_name": "CN1258",
"ws1_release": "20.11.0.7"
}
},
"fields": {
"log_timestamp": [
"2021-03-29T02:31:28.000Z"
],
"@timestamp": [
"2021-03-29T02:31:31.587Z"
]
},
"highlight": {
"tags": [
"@kibana-highlighted-field@iis@/kibana-highlighted-field@-@kibana-highlighted-field@log@/kibana-highlighted-field@"
]
},
"sort": [
1616985091587
]
}

`

  • Steps to Reproduce:

I've 2 gsub filters followed by lowercase filter on newly added field. gsub filters work properly however lowercase filter does not work.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions