Custom syslog ingest parser

[SackOfHacks]

Hi, I am using Security Onion 2.3.21.
Is anyone familiar with adding a custom Syslog ingest parser to Security Onion. I placed my parser (my.syslog) file in /opt/so/saltstack/local/salt/elasticsearch/files/ingest/ and restarted elasticsearch but the parser is not being implemented. Any help would be great.

[weslambert]

You can check to see if your pipeline was loaded with so-elasticsearch-pipelines-list

You can also check for invocations of certain processors in your pipeline with so-elasticsearch-pipeline-stats $pipelinename once you've confirmed the pipeline is loaded. This way, you can see how far an event made it through the pipeline before failing to be modified or exit (or if it even entered the pipeline at all).

Thanks,
Wes

[SackOfHacks]

Thanks, you have helped me a long way into the right direction. Learned a ton about Elasticsearch parsing but I can't seem to get the right parser in place to not break something. I keep ending up messing up the Elasticsearch data types. Maybe you can send me in the right direction for the task at hand... how do I add a CEF parser to the Security Onion setup? My syslog is sending CEF formatted messages, and I want to extract IP, Port and message in a way that doesnt break things.

[weslambert]

If you are able to provide a scrubbed example of the message field (where it is unparsed), we may be able to offer some additional assistance.

[SackOfHacks]

Thanks for the offer. Here is a scrubbed example message:

May 15 21:07:15 11:22:33:44:55:66 CEF:1|My personal Firewall|Acme Firewall|01.2.03|304005|Acme Firewall: ACL Violation|6|msg=ACL violated due to incorrect network address, protocol, port, rate limit and/or state FirewallMode=OPERATIONAL smac=22:33:44:55:66:77 src=192.168.1.1 spt=14607 dmac=77:66:55:44:33:22 dst=10.0.0.2 dpt=12547 proto=IPv4/UDP EthType=800 TTL=128 PhysIn=eth0

[Skullack]

Did you manage to get this working I would like to get the same

[InfosecGoon]

Did you manage to get this working I would like to get the same

I would suggest starting a fresh thread with the details of what you're trying to do -- this post is two years old now.