Filebeat & Distributed Setup

[kennyrogersjr]

Version

2.4.60

Installation Method

Other (please provide detail below)

Description

other (please provide detail below)

Installation Type

Standalone

Location

cloud

Hardware Specs

Exceeds minimum requirements

CPU

8

RAM

32

Storage for /

275 GB

Storage for /nsm

275 GB

Network Traffic Collection

span port

Network Traffic Speeds

1Gbps to 10Gbps

Status

Yes, all services on all nodes are running OK

Salt Status

No, there are no failures

Logs

No, there are no additional clues

Detail

Hoping to pick the collective brain here. We deployed SO to AWS using the latest SO AMI as a standalone manager. We installed SO as a sensor/forwarder on prem. Everything is working as intended, however, we're trying to ensure all logs (think addl zeek logs for ex) are also sent to the manager. We want this because we set up a splunk forwarder in the manager for our SIEM. Having the logs at the manager enables forwarding to the SIEM where our sec team can store and access them which is optimal and preferred.

Looking through docker on both manager and sensor, we do not see too many mentions of filebeat nor do we see any images or procs for it. Is there another function that will forward the logs from the sensor or should filebeat be installed on the sensor? How have you all accomplished this?

Guidelines

• I have read the discussion guidelines at Read before posting! #1720 and assert that I have followed the guidelines.

[TotieBash]

I would install Splunk forwarder on the SO Sensor on-prem itself and have it ingest /nsm/zeek and /nsm/suricata logs on its native format. This will be the best option since Splunk has native parsers for Zeek and Suricata.

You can also play with SO Logstash to forward your logs to Splunk from SO standalone manager in AWS. But I would go this route as last resort.. I can see parsing issues or logs not getting in time or logs not arriving at all issues.

[kennyrogersjr]

Having splunk forwarder on the sensor is our last option, but it is an option. We wanted to ensure all flows and files were sent straight to the manager vs individual set ups and rules for each splunk forwarder sensor. The goal is to future-proof sensor flows for network and business changes, that way we can just turn on a setting and log files and flows all ship to the cloud manager and it does the work directly with the SIEM as the sole "provider", per se. Hopefully that makes sense. To correct my original post, looks like filebeat was incorrect and it should be logstash. We're just not finding anything that'll capture and send the unparsed log files to the master. Sent from my Verizon, Samsung Galaxy smartphone Get Outlook for Android<https://aka.ms/AAb9ysg>

…

________________________________ From: TotieBash ***@***.***> Sent: Friday, May 3, 2024 1:19:55 PM To: Security-Onion-Solutions/securityonion ***@***.***> Cc: kennyrogersjr ***@***.***>; Author ***@***.***> Subject: Re: [Security-Onion-Solutions/securityonion] Filebeat & Distributed Setup (Discussion #12924) I would install Splunk forwarder on the SO Sensor on-prem itself and have it ingest /nsm/zeek and /nsm/suricata logs on its native format. This will be the best option since Splunk has native parsers for Zeek and Suricata. You can also play with SO Logstash to forward your logs to Splunk from SO standalone manager in AWS. But I would go this route as last resort.. I can see parsing issues or logs not getting in time or logs not arriving at all issues. — Reply to this email directly, view it on GitHub<#12924 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AKQMKW2CJQTV2HPC7IX2UZLZAPIMXAVCNFSM6AAAAABHFXQZTCVHI2DSMVQWIX3LMV43SRDJONRXK43TNFXW4Q3PNVWWK3TUHM4TGMBYGU3DA>. You are receiving this because you authored the thread.Message ID: ***@***.***>

[TotieBash]

Gotcha... I assume you have seen this logstash forwarding method:
https://docs.securityonion.net/en/2.4/logstash.html#original-event-forwarding

[kennyrogersjr]

I have, but last week when I wasn't on that step. I got derailed today when I was errantly looking for filebeat forwarding and wondering why I don't have any filebeat settings, procs or images on my nodes... I'll re-attack Mon when I'll have access. I'll update the main thread if I have issues. Thank you so much!

[InfosecGoon]

One of the changes in 2.4 from 2.3 is that log forwarding is now done via the Elastic Agent rather than Filebeat, which is why you're not seeing any Filebeat configuration or documentation.

If you want to forward those logs from the Manager into Splunk using something like syslog, documentation is here: https://docs.securityonion.net/en/2.4/logstash.html#original-event-forwarding

[kennyrogersjr]

Correct. However, sending raw log files to the manager was the goal. We ended up just implementing the splunk forwarder on the sensor. The alternative would have involved a few scripts using rsync, a script to handle the log managment, a script to manage cron jobs, and then the crons jobs themselves. Too much work and not enough dividends, so we gave in and implemented the splunk forwarder.