Distributed setup proposal for ~100Gbit/s *without* full PCAP

[vincentvlk]

Hi community,

Security Onion version: 2.4.50
Design: Distributed
Deployment: On-Prem
Install type: Security Onion ISO

We are planning distributed Security Onion setup for aggregated ~100Gbit/s traffic analysis with ET Pro rule-set for Suricata, but without full PCAP. We plan to enable Steno/PCAP only in on-demand fashion filtered with BPF.

Starting with HW proposal for Forward Nodes:

Nodes count: 4
CPUs per node: 2 x AMD EPYC 9354 3.25GHz, 32C/64T, 256M Cache (280W) DDR5-4800
RAM per node: 512GB DDR5-4800 ECC
Eth. NICs per node: 2x100Gbps, 2x25Gbps, 2x10Gbps
Local storage: 4 x 1.92TB SSD SAS, RI, up to 24Gbps 512e 2.5in Hot-Plug, AG Drive
Analysis Interfaces (bond0): 2x100Gbps
Cluster Comms interfaces: 2x25Gbps
Management interfaces: 2x10Gbps

Log/Alert Retentions: ~14 days. We are aiming more at automatized event alerts/playbooks than on manual investigation.

Our plan is to distribute the aggregated analysis load to 4 x 25Gbit/s. We are also considering to add another 2 Forward nodes to distribute the load in ~6 x 17 Gbit/s fashion. Logs from analysis will be forwarded to 1x Manager VM/Node (Considering 8 x CPUs, 32GB DDR5-4800-ECC, 4TB-SAS-SSD).

HW proposal for Search Nodes:

Nodes count: 2
HW setup is similar to Forward Nodes.

Please, is this setup viable? I'm open to clarify more details.

Thanks in advance for any direction and have a nice day.

[KimkimCyber]

I'd be interested to know if the 4 forward nodes with 2x9354 each, will be able to handle 100Gbps without packet drops.
If you load balance, that would be like 25Gbps per node. Are you going to use a smart nic ?

To me it seems ok, but only from an educational guess point of view as I'm still waiting myself for my hardware to start my setup:
2xForward node, 2x9554 AMD for 10Gbps (5Gbps per forward node) with Zeek, Suricata, Strelka and full pcap capture.