What happens if my NSM directory dies?

[GeorgePatches]

Version

2.4.111

Installation Method

Security Onion ISO image

Description

other (please provide detail below)

Installation Type

Standalone

Location

on-prem with Internet access

Hardware Specs

Exceeds minimum requirements

CPU

4

RAM

32

Storage for /

500

Storage for /nsm

1000

Network Traffic Collection

span port

Network Traffic Speeds

Less than 1Gbps

Status

Yes, all services on all nodes are running OK

Salt Status

No, there are no failures

Logs

No, there are no additional clues

Detail

Just to start, I have no issues currently, this is a theoretical DR question.

I've been playing with my SO PCAPs lately and I really like the ability to go back in time and grab data from a broad window if I desire. I want to store more data, but I don't want to "waste" a bunch of storage/IO on my primary array storing all that. I'd like to move the NSM directory to it's own dedicated storage, but I don't want to back it up to the same level as my current combined disk. If the worst happens and the drives storing the NSM dies, what happens? I'm not particular concerned about losing all the history, but I've made a fair number of custom tuning of suricata alerts that I don't want to lose. Can a SO deployment be restored with a blank NSM directory?

Guidelines

• I have read the discussion guidelines at Read before posting! #1720 and assert that I have followed the guidelines.

[zfcarsonb]

It depends on what you're trying to protect:

If you want to increase redundancy/fault tolerance, consider placing the /nsm directory on a RAID array. That way, if a drive dies, you can swap it out with a new drive and the array will rebuild any lost data.

If you want to keep backups that you can restore in the case of a grid failure:

1. The configuration for Security Onion itself is regularly backed up to /nsm/backup. These backups can be used to restore your grid in the event of a failure. Docs Reference
2. Suricata alerts (and all other Elasticsearch data) can be backed up by configuring a snapshot repository in Kibana. To set up snapshots, go to Kibana -> Stack Management -> Snapshot and Restore. Docs Reference
3. Raw PCAPs are stored directly on disk. Depending on whether you use Suricata or Stenographer, you would need to choose a different backup strategy: Suricata stores PCAPs as plain .PCAP files that can be copied off the system, while Stenographer uses an index and custom format. Docs Reference

I would recommend both the RAID array plus taking regular backups if you are able to do both with your deployment. However, either strategy reduces risk of data loss to a degree.

[InfosecGoon]

I would suggest setting up a crontab to copy off the contents of /nsm/backup every so often. All the custom configuration for your deployment is stored in tarballs in that directory.

[GeorgePatches]

Thanks all, the "save your /nsm/backup directory" advice was exactly what I needed to know. I'll still be backing up my root partition, but I wanted to know if I would be shooting myself in the foot not backing up the nsm with it. All of this is virtual with my setup, and I will be storing the nsm on a mirrored ZFS pool, but for performance reasons the nsm zvol is running with sync disabled. Without sync, there's a nonzero chance that a badly timed hard reset could corrupt the metadata of the whole pool, effectively wiping it out. Since only the nsm directory is stored here, I'm not super worried. I'll setup a job to copy the /nsm/backup over to the root partition that is backed up and I think that will be good enough for me purposes.