Deployment advice needed for inherited hardware

[bmarshal]

I've inherited two new server appliances, and I'm seeking advice on the best way to deploy security onion in production. Unfortunately, there is no additional money to purchase more hardware so I have to roll with what I have available. Each server has the following specs.

2x Intel Xeon 3rd gen CPU's, 32 core each
256GB memory
12x 7.68TB NVMe drives, Intel VROC raid and JBOD support
4x 100GbE NIC's

I intend to ingest logs from multiple cloud API's, firewall syslogs, Active Directory, 100 production servers (various OS's), and would like to monitor traffic from 500 production VLAN's.

KVM with VM's or direct install?
If KVM, should I pool the drives together using vSAN? How many VM's, hardware specs etc...?
If distributed direct install, how should I break the services up between the two servers?

[dougburks]

Have you reviewed the documentation?

https://docs.securityonion.net/en/2.4/best-practices.html
https://docs.securityonion.net/en/2.4/architecture.html
https://docs.securityonion.net/en/2.4/hardware.html
https://docs.securityonion.net/en/2.4/performance.html

[bmarshal]

Yes, I reviewed the documentation, but the recommendations don't seem to cover a scenario where I'm limited to just two nodes. My current understanding is that I should set up one Manager/Search node and one Forward node, but I'm not sure if a Heavy node is preferred over just a Forward node with my limitations of only having two physical servers. I just want to ensure this is the best approach to take.

[jwmasekre]

If I were designing a setup using this hardware, I'd be running a hypervisor on each device and develop the architecture via VMs, instead of running baremetal, but that's what I'm comfortable with.

[bmarshal]

Thanks @jwmasekre, in a two-node hypervisor setup, would you lean towards hardware raid, software raid like ZFS, or pooled storage option between the two nodes? How many TBs of storage would you dedicate to each Search and Forward node VM?

[jwmasekre]

Resource requirements should be governed by ingest rate. per the documentation, if you can find how much data your forwards are going to be ingesting, you can work out the math on how much space, memory, and CPU cores you need. see if whatever system you plan on using to forward traffic to the security onion will track data rates so you can scope out your resource requirements accurately

[dougburks]

My current understanding is that I should set up one Manager/Search node and one Forward node, but I'm not sure if a Heavy node is preferred over just a Forward node with my limitations of only having two physical servers.

Please make sure that you read the warning for heavy nodes as we don't recommend them for most use cases:
https://docs.securityonion.net/en/2.4/architecture.html#heavy-node

Regarding virtualization and pooled storage, that can be made to work but please make sure you consider the cost. Extra complexity usually requires more maintenance and more tuning. It can also make troubleshooting more difficult when problems arise. For most use cases, we recommend keeping things simple so that you can focus your time on catching bad guys. For more information, please see:
https://docs.securityonion.net/en/2.4/best-practices.html