Advice needed for architecture design

[ionetworkadmin]

I've been using SO for years and love it. My past networks have consisted of a Manager/Search server and 3-4 Forward nodes. That worked great. But now I'm looking to implement a much larger Grid of 25 Forward nodes, and I'm wondering about the best way to design the Manager. I've looked at the documentation and I think that having a Manager with separate Search nodes is the way to go, but I'm wondering how exactly those work? Should I make it a Manager/Search node with additional Search nodes as needed? Do I still point the Forward nodes at the Manager, and it load balances with the Search nodes automatically? Due to the quantity of Forward nodes I intend to feed into it, if I run into storage issues on the Search node do I just add another Search node to help balance the load? Is it automated? I'd just like some advice before I build the Manager.

[rosswakelin]

Hi There

We have a grid with 35 sensors/forward nodes scattered across the country. We have one manager, one Receiver and three search nodes. It seems to be running ok. The three data nodes are running 47%, 52% and 22% full on the /nsm partition so it looks like the load balancing is not quite perfect, but seems to work ok.

[ionetworkadmin]

Thanks for the info. I hadn't considered needing a Receiver node also, but am now liking the idea. Do the Forwarding Sensors really need to be 200 Gigs of disk if PCAP is turned off? My current Sensors are only using 50G or less. It seems like 100G would be more than adequate for the Sensors.

[rosswakelin]

We have the receiver node so that when we reboot the manager for upgrades etc. the sensors still have somewhere to send their data. Our sensors have either 500Gb or 1Tb of space allocated to their /nsm partitions so we can keep PCAP history for investigations. We have "small" sensors that are monitoring 1-200Mb/s of data, and "large" sensors where we have multiple active 10Gb links, and the monitoring traffic can sit up to 1Gb/s