PFsense log digestion in Eval node

[aredey]

Hi Everyone, I'm investigating the capabilities of the the current version of Security Onion (SO), trying to ingest PFSense logs as described in this great video: https://www.youtube.com/watch?v=aoH8qZwAxek&t=450s (also through https://docs.securityonion.net/en/2.4/firewall.html#host-firewall > https://docs.securityonion.net/en/2.4/pfsense.html#pfsense). I tried to follow the process to the letter but SO is not recognising the syslog traffic sent by the firewall. I can see the traffic hitting the SO physical host, by running tcpdump, but neither Hunt nor the Dashboard are showing any records related to the firewall.

I followed these steps as described, except the step at 7:14, as I installed the Evaluation version (due to lack of RAM on my zima board) instead of the Standalone one - actually I have added "customportgroup0 to both the "Eval" and the "Standalone" roles, just in case. Is adding PfSense log monitoring to Evaluation level installations work (theoretically) at all or is it a lost cause?

my Node Status is as follows:
Grid ID:(local)
ID:so
Role:Evaluation
Address:192.168.123.123
Version:2.4.141
...
and

[root@so ~]# free -h
               total        used        free      shared  buff/cache   available
Mem:          **3.2Gi**      2.8Gi       103Mi       8.0Mi       561Mi       411Mi
Swap:          8.0Gi       4.4Gi       3.6Gi

this is why I installed eval as standard install bails out on me due to insufficient RAM capacity. (It is in testing so not much traffic is expected to be handled. Is forced Standard install with 3GB RAM possible?)
Here is the packet trace I observe on SO:

[root@so ~]# tcpdump -nnqi enp3s0 port 9001
dropped privs to tcpdump
tcpdump: verbose output suppressed, use -v[v]... for full protocol decode
listening on enp3s0, link-type EN10MB (Ethernet), snapshot length 262144 bytes
18:19:29.857162 IP 192.168.123.1.514 > 192.168.123.123.9001: UDP, length 272
18:19:30.032324 IP 192.168.123.1.514 > 192.168.123.123.9001: UDP, length 271
18:19:30.850339 IP 192.168.123.1.514 > 192.168.123.123.9001: UDP, length 218

.. but I can't find the process/container that listens on udp:9001, both of the following yield to no hits:

[root@so ~]# netstat -unlp | grep 9001
[root@so ~]# ss -unlp | grep 9001

Many thanks, for any pointers/advice!

[Zer0-cyber-web]

Hi! I did this few month ago, and als faced some problems. It would be much more effective if you could notice also all the other information that is required for your post, but here are my findings:

1. Be sure that you installed pfSense integration.
2. Check if this integration is assigned to elastic-fleet agent policy that is collecting all your data from other devices. In my case it was so-grid-nodes_general
3. Check that pfSense integration has exclusive port control (no other integrations use the same port that this integration is using. This is important)
4. Check for pfsense indexes in Elastic Search. All the DashBoards and Hunts (even outside of Kebana) are using those indexes. So they have to exist, otherwise - no other functionality will be able to see that information. This is some other way to check if your SO instance is receiving logs from pfSense.

I followed these steps as described, except the step at 7:14, as I installed the Evaluation version (due to lack of RAM on my zima board) instead of the Standalone one - actually I have added "customportgroup0 to both the "Eval" and the "Standalone" roles, just in case.

5. Also - I would suggest to see if your firewall rules are working properly - use this:
   sudo iptables -nvL|grep 9001
   and you will see if there is a connection to port 9001 from your pfSense installation.
6. And of course - not to say that you have to doublecheck pfSense settings. :)

.. but I can't find the process/container that listens on udp:9001, both of the following yield to no hits:

You will not find a process that is listening to that port because SO uses DOCKER containers, which contain that procedd. And personally me using lsof command to find that process:
lsof -i :9001

Hope this will help.

[aredey]

Hi Zer0, many thanks for the response.
I have installed SO from the current ISO, into an Intel Celeron single board computer (SBC). The CPU and Storage requirements have been met, the memory requirement hasn't, as mentioned, this was the reason I used the so_eval type install, as standalone/standard installation failed due to lack of memory:

[root@so ~]# neofetch 
OS: Oracle Linux Server 9.5 x86_64
Kernel: 5.15.0-308.179.6.3.el9uek.x86_64
Shell: bash 5.1.8
CPU: Intel Celeron N3450 (4) @ 2.200GHz
GPU: Intel HD Graphics 500
Memory: 2625MiB / 3303MiB

[root@so ~]# lsblk
NAME            MAJ:MIN RM  SIZE RO TYPE MOUNTPOINTS
...
└─nvme0n1p3     259:3    0  3.6T  0 part 
  ├─system-root 252:0    0  293G  0 lvm  /
  ├─system-swap 252:1    0    8G  0 lvm  [SWAP]
  ├─system-nsm  252:2    0  3.3T  0 lvm  /nsm
  └─system-tmp  252:3    0    2G  0 lvm  /tmp

As I originally intended to install the single node into a firewalled (non directly accessible Internet, using a tagged VLAN) that failed (I don't think the install script supports this method, from what I found) I ended up reconfiguring the SO node using the /root/SecurityOnion/so-desktop-install script (I attempted the reinstall with the .../so-setup-network script as well in my tries), over the original Oracle Linux Server install.

1. I have used the pfSense integration as per both the linked video as well as the Install Document (Option B) description.
2. My own notes, based on the linked video is as follows:

Elastic Fleet
    Agent policies > so-grid-nodes_general > Add Integration Search for "pfsense", 
    Add pfsense-1
      Syslog Host: localhost > 0.0.0.0
      syslog Port: 9001
      Save and continue
Security Onion
  Administration > Configuration 
    Options > Enable (drop down)  > Show all conf...
  Administration > Configuration > firewall > hostgroups > syslog: 192.168.123.1/32
    firewall > 
      hostgroups > cutomhostgroup0: 192.168.123.1, check
      portgroups > cutomportgroup0 > udp: 9001, check
      role > **eval**/(also standalone) > chain > INPUT > hostgroups > customhostgroup0 > portgroups: cutomportgroup0, check
    Options > SYNCHRONIZE GRID

3. I have a single integration installed: pfsense (as per the video)
4. No indexes are showing up anywhere in Elastic Search (in the Dashboard or Hunt) anywhere. I am very new to SO so not quite sure how to find these indexes, this was my first try to install SO so that I can learn/experiment with these.
5. The firewall rules seems to be correct, I can see the packets arriving in tcpdump without error;

[root@so ~]# sudo iptables -nvL|grep 9001
 8058 1741K ACCEPT     udp  --  *      *       192.168.123.1          0.0.0.0/0            udp dpt:9001
[root@so ~]# 

I also flashed the entire iptables rule set, so that I can verify the operating system processes and all the docker processes without the network filtering interference - which of course broke the entire installation, This is why I was wondering what process is serving the remote syslog traffic, but found none:
6. The pfsense config seems to be ok, as I log to a couple of other systems (like an external LibreNMS docker container) and the SO tcpdump is showing traffic generated by the firewall (there are snort alerts identified by my firewall, for example, in syslog messages, generated by constant probing for SSH on my Interent interface).

[root@so ~]# lsof -i :9001
[root@so ~]#

Also draws blank, hence my puzzlement for no process/container seemingly running on udp:9001
I have installed portainer on the SO docker host (yet am unable to access the web interface, so far, hence the flushing of iptables earlier. Yet the list of containers also shows the lack of a process on udp:9001:

[root@so ~]# docker ps -a | grep 9001
[root@so ~]# 
[root@so ~]# docker ps -a | grep 9300   # testing for another known installed process on tcp:9300 for testing
5d2e604f4c1f   so:5000/security-onion-solutions/so-elasticsearch:8.17.3                     "/bin/tini -- /usr/l…"   6 days ago    Up 27 hours             0.0.0.0:9200->9200/tcp, 0.0.0.0:9300->9300/tcp                                                       so-elasticsearch
[root@so ~]# 

[aredey]

Ok, I have created a brand new SO instance as a VM, using the same parameters as as before and as described in the video in the first message, except the SO node is installed in so_eval mode (same as before for the physical device install). I can see log items coming as before but now the SO VM does have a process listening on (udp) port 9001:

[root@so-test ~]# ss -nulip | grep 9001
UNCONN 0      0                  *:9001            *:*    users:(("agentbeat",pid=1834,fd=13))

Yet, no log messages show up under the Dashboard or the Hunt page, under "Group Metrics" as at 9:35 in the video: No data available.
Is it because none of the integrations are available in Eval mode installs at all? Under Hunt > Events I do see log records coming in (and showing errors):

error.message	[
  "Provided Grok expressions do not match field value: [<78>May 30 14:55:00 /usr/sbin/cron[52009]: (root) CMD (/usr/bin/nice -n20 /sbin/pfctl -q -t snort2c -T expire 3600)]"
]
event.kind	pipeline_error
soc_source	so-test:.ds-logs-pfsense.log-default-2025.05.30-000001

If the (pfsense) integrations are not available in Evaluation level SO installs, is there a way to convert the VM to Standard/Standalone service? Is there a description for what service are (and aren't) available in the various installation levels - and how/where they are checked and enforced?

[Zer0-cyber-web]

If the (pfsense) integrations are not available in Evaluation level SO installs, is there a way to convert the VM to Standard/Standalone service? Is there a description for what service are (and aren't) available in the various installation levels - and how/where they are checked and enforced?

Yes, it can be converted to Standalone with running so-setup. I did it many times while determining optimal architecture for my setup.

[aredey]

Found it: https://docs.securityonion.net/en/2.4/architecture.html#evaluation Probably time to close the thread: ...it does not support adding Elastic agents...

[Zer0-cyber-web]

It does not support adding Elastic Fleet agents outside of server (as well as other SO nodes). But it contains one agent for receive logs from outter entites. As states it's architecture scheme:
https://docs.securityonion.net/en/2.4/_images/eval.png