The AVPair field of CISE_Passed_Authentications logs are parsed wrong, or more precisely not at all.

How to reproduce:

POST _ingest/pipeline/logs-cisco_ise.log-1.0.0/_simulate { "docs": [ { "_index": "logs-cisco_ise.log-debug", "_id": "0001", "_source": { "message": """<181>Sep 1 22:13:44 isepsn-v101 CISE_Passed_Authentications 0042127282 4 0 2022-09-01 22:13:44.934 +02:00 1170106382 5203 NOTICE Device-Administration: Session Authorization succeeded, ConfigVersionId=1053, Device IP Address=1.2.3.4, DestinationIPAddress=5.6.7.8, DestinationPort=49, UserName=blabla, CmdSet=[ CmdAV= ], Protocol=Tacacs, RequestLatency=7, NetworkDeviceName=foo-bar, Type=Authorization, Privilege-Level=1, Authen-Type=PAP, Service=Login, User=alice, Port= REST, Remote-Address=1.2.3.4, Authen-Method=TacacsPlus, AVPair=cisco-av-pair*, AVPair=shell:roles*, Service-Argument=shell, NetworkDeviceProfileId=b0699505-3150-4215-a80e-6753d45bf56c, AcsSessionID=foo/22212221, AuthenticationIdentityStore=foo.bar.net, AuthenticationMethod=Lookup, SelectedAccessService=Default Device Admin, SelectedShellPro file=ACI_Access, IsMachineAuthentication=false, Step=13005, Step=15049, Step=15008, Step=15048, Step=15048, Step=15041, Step=15013, Step=24432,\r""" } } ] }

Output:

{ "docs": [ { "doc": { "_index": "logs-cisco_ise.log-debug", "_id": "0001", "_version": "-3", "_source": { "cisco_ise": { "log": { "acs": { "session": { "id": "foo/22212221" } }, "request": { "latency": 7 }, "log_details": "ConfigVersionId=1053, Device IP Address=1.2.3.4, DestinationIPAddress=5.6.7.8, DestinationPort=49, UserName=blabla, CmdSet=[ CmdAV= ], Protocol=Tacacs, RequestLatency=7, NetworkDeviceName=foo-bar, Type=Authorization, Privilege-Level=1, Authen-Type=PAP, Service=Login, User=alice, Port= REST, Remote-Address=1.2.3.4, Authen-Method=TacacsPlus, AVPair=cisco-av-pair*, AVPair=shell:roles*, Service-Argument=shell, NetworkDeviceProfileId=b0699505-3150-4215-a80e-6753d45bf56c, AcsSessionID=foo/22212221, AuthenticationIdentityStore=foo.bar.net, AuthenticationMethod=Lookup, SelectedAccessService=Default Device Admin, SelectedShellPro file=ACI_Access, IsMachineAuthentication=false, Step=13005, Step=15049, Step=15008, Step=15048, Step=15048, Step=15041, Step=15013, Step=24432", "segment": { "number": 0, "total": 4 }, "step": [ "13005", "15049", "15008", "15048", "15048", "15041", "15013", "24432" ], "category": { "name": "CISE_Passed_Authentications" }, "message": { "code": "5203", "description": "Device-Administration: Session Authorization succeeded", "id": "0042127282" }, "config_version": { "id": 1053 }, "selected": { "access": { "service": "Default Device Admin" } }, "authentication": { "method": "Lookup", "identity_store": "foo.bar.net" }, "network": { "device": { "profile_id": "b0699505-3150-4215-a80e-6753d45bf56c", "name": "foo-bar" } } } }, "User": "alice", "log": { "level": "notice", "syslog": { "severity": { "name": "notice" }, "priority": 181 } }, "Port": " REST", "destination": { "port": 49, "ip": "5.6.7.8" }, "Authen-Method": "TacacsPlus", "network": { "protocol": "tacacs" }, "ecs": { "version": "8.4.0" }, "related": { "hosts": [ "isepsn-v101" ], "ip": [ "5.6.7.8", "1.2.3.4" ], "user": [ "blabla" ] }, "host": { "hostname": "isepsn-v101" }, "Remote-Address": "1.2.3.4", "client": { "ip": "1.2.3.4" }, "event": { "sequence": 1170106382, "kind": "event", "timezone": "+02:00", "action": "device-administration" }, "AVPair": [ "cisco-av-pair*", "shell:roles*" ], "Service": "Login", "CmdSet": "[ CmdAV= ]", "message": """2022-09-01 22:13:44.934 +02:00 1170106382 5203 NOTICE Device-Administration: Session Authorization succeeded, ConfigVersionId=1053, Device IP Address=1.2.3.4, DestinationIPAddress=5.6.7.8, DestinationPort=49, UserName=blabla, CmdSet=[ CmdAV= ], Protocol=Tacacs, RequestLatency=7, NetworkDeviceName=foo-bar, Type=Authorization, Privilege-Level=1, Authen-Type=PAP, Service=Login, User=alice, Port= REST, Remote-Address=1.2.3.4, Authen-Method=TacacsPlus, AVPair=cisco-av-pair*, AVPair=shell:roles*, Service-Argument=shell, NetworkDeviceProfileId=b0699505-3150-4215-a80e-6753d45bf56c, AcsSessionID=foo/22212221, AuthenticationIdentityStore=foo.bar.net, AuthenticationMethod=Lookup, SelectedAccessService=Default Device Admin, SelectedShellPro file=ACI_Access, IsMachineAuthentication=false, Step=13005, Step=15049, Step=15008, Step=15048, Step=15048, Step=15041, Step=15013, Step=24432,\r""", "Type": "Authorization", "@timestamp": "2022-09-01T22:13:44.934+02:00", "Service-Argument": "shell", "IsMachineAuthentication": "false", "Privilege-Level": "1", "Authen-Type": "PAP", "user": { "name": [ "blabla" ] }, "SelectedShellPro file": "ACI_Access" }, "_ingest": { "timestamp": "2022-09-08T07:44:12.244866994Z" } } } ] }

As you can see, the output contains a AVPair field at root level. As far as I understand the code of the pipeline (logs-cisco_ise.log-1.0.0-pipeline_passed_authentications), there are processors available for cisco-av-pair. fields, but they make no sense to me, since they seem not to exist. IMO there is a piece of logic missing.

Back to my problem. Due to another cisco_ise parsing error I have reported already, our indices have already a mapping for the AVPair field and thus theese logs are rejected. I have temporarily fixed this by renaming the AVPair field to tmp.AVPair, make sure the AVPair is removed at the end and add a error processor.

Please have a look at this and provide a solution to either remove the field or parse it correctly

Regards
Bernhard