CEF integration not ingesting

[kspringer-maf]

Version

2.4.120

Installation Method

Security Onion ISO image

Description

other (please provide detail below)

Installation Type

Distributed

Location

on-prem with Internet access

Hardware Specs

Exceeds minimum requirements

CPU

lots

RAM

lots

Storage for /

lots

Storage for /nsm

lots

Network Traffic Collection

other (please provide detail below)

Network Traffic Speeds

1Gbps to 10Gbps

Status

Yes, all services on all nodes are running OK

Salt Status

No, there are no failures

Logs

No, there are no additional clues

Detail

I've added the CEF integration per these instructions.
https://docs.securityonion.net/en/latest/third-party-integrations.html#adding-an-integration

In the CEF module I defined the udp 9003 and tcp 9004 ports, and changed localhost to 0.0.0.0 per the instructions. I've got it tied to the FleetServer policy per the instructions. All shows running and Healthy.

I've configured the Firewall customportgroup0 to allow udp 9003 and tcp 9004, and customhostgroup0 to allow 0.0.0.0/0

I've got my source system sending it's CEF logs to Security Onion. I can easily switch between udp 9003 or tcp 9004.

The SO system doesn't seem to be ingesting the data. I can see traffic happening via the zeek.conn logs, and I can run netstat from the terminal and the ports are listening properly. Running tcpdump shows the traffic is hitting the interface, but I've been banging away at this all day and no matter what I do or which ports/protocols I use, I'm not getting traffic ingested into the CEF pipeline. If I send from the source to SO over Syslog port 514 the data gets ingested via the syslog pipeline as expected.

The reason I'm attempting to get CEF working is because the data ingested via syslog is not being parsed correctly and I'm hoping to get better results with CEF. Also, this is my first integration attempt and if it works I'll have a baseline of knowledge to setup some others. I feel like things are very close to working, but there's a disconnect somewhere that I'm not understanding. Any advice would be appreciated.

Guidelines

• I have read the discussion guidelines at Read before posting! #1720 and assert that I have followed the guidelines.

[InfosecGoon]

"I've configured the Firewall customportgroup0 to allow udp 9003 and tcp 9004, and customhostgroup0 to allow 0.0.0.0/0"

Did you also set customhostgroup0 to use customportgroup0 in the proper system role?

Docs: https://docs.securityonion.net/en/2.4/firewall.html#creating-a-custom-host-group-with-a-custom-port-group

[kspringer-maf]

Thanks for pointing me there. I hadn't done that.
https://docs.securityonion.net/en/2.4/firewall.html#creating-a-custom-host-group-with-a-custom-port-group

I followed the instructions and navigated to firewall> role> idh> chain> DOCKER-USER> hostgroups> customhostgroup0> portgroups> chose my Manager> entered customportgroup0
Saved and Synced Grid

Tested and Hunted for the traffic and am still only seeing zeek.conn traffic. Nothing being ingested by a CEF pipeline.

I thought maybe the DOCKER-USER menu category might not be correct, so I navigated to firewall> role> idh> chain> INPUT> hostgroups> customhostgroup0> portgroups> chose my Manager> entered customportgroup0
Saved and Synced Grid

Same results. I see only zeek.conn traffic. Nothing being ingested by a CEF pipeline. I've tried both udp 9003 and tcp 9004 without success. The zeek logs see the incoming tcp requests and say there is no reply.

[kspringer-maf]

I never got this to work. I've abandoned the task.

[InfosecGoon]

Sorry - you need to add it to the Grid Nodes policy, not the Fleet Server policy. The Elastic Agent for the Fleet Server runs inside of a container, while the one for the node itself runs on the host operating system.

[kspringer-maf]

I wanted to close the loop on this discussion and say that I got it working. Here's the instructions.

These instructions will allow the 'Receiver' node to accept incoming CEF formatted logs. If you want a different grid node to be defined, adjust accordingly.

1. Open 'Elastic Fleet' from SO sidebar.
   Open 'Integrations' from Elastic sidebar.
   Find the 'CEF' integration.
   Click 'Add Common Event Format (CEF)' blue button from top right.

   Set the following settings:
   input: udp Syslog Host 0.0.0.0
   Syslog Port 9003
   input: tcp Syslog Host 0.0.0.0
   Syslog Port 9004

   Existing hosts Agent policies so-grid-nodes_general

   'Save and continue'
   'Save and deploy changes'

2. Open 'Administration> Configuration> Advanced settings'

   Define ports
   firewall> portgroups> customportgroup0> udp> soreceiver> 9003
   firewall> portgroups> customportgroup0> tcp> soreceiver> 9004

   Define remote allowed hosts
   firewall> hostgroups> customhostgroup0> soreceiver> 0.0.0.0/0

   Tie customportgroup0 to customhostgroup0 on the receiver node
   firewall> role> receiver> chain> INPUT> hostgroups> customhostgroup0> portgroups> customportgroup0

3. Sync settings. Click the 'Synchronize Grid' button at the top of the Configuration page.

4. Test using nmap from a machine on the same network as your intended Sender. If SSH doesn't connect, neither will the new ports. Be patient when waiting for the grid to Sync, it often takes up to 10 min to finish.
   $ nmap -Pn -p 22 10.11.12.13
   $ nmap -Pn -p 9004 10.11.12.13

5. Configure ADAudit to send logs in CEF format.
   Admin> SIEM Integration> ArcSight (CEF)
   CEF Server 10.11.12.13
   Port 9004
   Protocol TCP
   Folder size 5 GB

6. Hunt for cef formatted logs in SO
   event.dataset:"cef.log"