I do NOT have duplicate Zeek logs and I am not sure why

[subs1138]

This is really just for my own understanding. I know that I do not actually want duplicate Zeek logs.

Current setup SecOnion 2.3, running on single server. Network data is being sent from SPANs on a couple switches that I know should be seeing a few of the same packets in transit. I am not seeing double Zeek logs like I expected. Is this because the data broker, Redis, is doing dedup? Part of Zeek autocorrecting? or is there something else in SecOnion that I need to get smart on?

Once I rebuild to a proper distributed architecture and these points start feeding different forwarders, should I expect to see duplicate zeek logs, and if so what should I do about it?

[dougburks]

Current setup SecOnion 2.3, running on single server. Network data is being sent from SPANs on a couple switches that I know should be seeing a few of the same packets in transit. I am not seeing double Zeek logs like I expected. Is this because the data broker, Redis, is doing dedup? Part of Zeek autocorrecting? or is there something else in SecOnion that I need to get smart on?

I don't think there is anything in 2.3 that would be de-duplicating the logs. To answer why you're not seeing duplication would require much more knowledge of your network, IP ranges, NAT, switch configuration, and Security Onion configuration. If the Security Onion box is under-powered, then it's possible that it's dropping packets and that could be the explanation.

Once I rebuild to a proper distributed architecture and these points start feeding different forwarders, should I expect to see duplicate zeek logs,

Perhaps but again it would require more knowledge about your network.

and if so what should I do about it?

It depends. Some folks are OK with some duplication. If it's a problem for you, you might consider changing your SPAN configuration or using BPFs:
https://docs.securityonion.net/en/2.4/bpf.html