Rotate stale logs from /nsm to a separate drive

[bigacci]

Version

2.4.170

Installation Method

Security Onion ISO image

Description

configuration

Installation Type

Standalone

Location

on-prem with Internet access

Hardware Specs

Exceeds minimum requirements

CPU

12

RAM

100GB

Storage for /

210GB

Storage for /nsm

423GB

Network Traffic Collection

tap

Network Traffic Speeds

1Gbps to 10Gbps

Status

Yes, all services on all nodes are running OK

Salt Status

No, there are no failures

Logs

No, there are no additional clues

Detail

Goal: Rotate stale /nsm logs to another drive once they're 30 days old.

I'm a SOC Analyst who is working on our SO2 lab installation. I'm the only person in during the overnight shift, so convening with our SOC engineer is rather difficult. I have a tap setup to capture LAN to LAN traffic for devices connected to our lab VLAN.

I've been tasked with rotating stale logs our of /nsm and into another drive once they're 30 days old. Currently, /nsm is sitting at 94.6% and rising. I'm planning to archive them to this separate drive in case they're needed further in the future. They'll be rotated out of the second drive separately.

If I'm going about this the wrong way, I'd love to know. I'm learning Linux and SO2, so any pointers would be greatly appreciated.

Guidelines

• I have read the discussion guidelines at Read before posting! #1720 and assert that I have followed the guidelines.

[reyesj2]

For elasticsearch data itself you could just create snapshots and store those snapshots else where until you need it. Then you can restore it and search for relevant logs. https://www.elastic.co/guide/en/elasticsearch/reference/8.18/snapshot-restore.html

For raw event logs like suricata /zeek they are found in /nsm/(zeek/suricata)/ and will have archive folders already available you can move off. (these logs are ingested into elasticsearch, so you probably don't need this unless you need the original raw log for any reason)

If you need to store as much elasticsearch data as possible with the given 423GB for /nsm you could disable PCAP (if you don't need it) that will save you some space when not writing the network traffic to disk.

Overall, if your goal is retention you should consider redeploying your grid as a distributed grid. https://docs.securityonion.net/en/2.4/architecture.html#distributed. Distributed grids give you more flexibility. Example being if you needed longer retention for elasticsearch data you could add a searchnode to your grid