No log sources coming in after adding a forwarder and 2 Indexers from a standalone system to distributed

[idamansudo]

Version

2.4.111

Installation Method

Security Onion ISO image

Description

configuration

Installation Type

Distributed

Location

on-prem with Internet access

Hardware Specs

Meets minimum requirements

CPU

8

RAM

16

Storage for /

500

Storage for /nsm

200

Network Traffic Collection

other (please provide detail below)

Network Traffic Speeds

1Gbps to 10Gbps

Status

Yes, all services on all nodes are running OK

Salt Status

Yes, there are salt failures (please provide detail below)

Logs

Yes, there are additional clues in /opt/so/log/ (please provide detail below)

Detail

local:
Data failed to compile:

The function "state.highstate" is running as PID 93494 and was started at 2025, Feb 03 16:48:17.095152 with jid 2025020316481709515      

Guidelines

• I have read the discussion guidelines at Read before posting! #1720 and assert that I have followed the guidelines.

[idamansudo]

Ok, we re-installed all 4 VM nodes successfully and are back online. We suspect that we ran into the issue by installing a 3rd party certificate on the Manager for the web console and that did not get over to the 2 Indexers and 1 forwarder. We did see an x509 error in the logstash logs and that gave us our clue to rebuild. Thanks everyone!

[cm-ops]

The error above is just letting you know a highstate is running.

When you copied the certificate and key into SOC did you follow https://docs.securityonion.net/en/2.4/nginx.html#replacing-default-cert?

For the logstash error, do you still have a copy of the errors?

[idamansudo]

All the logs are from this thread, started by my manager - #14178

[cm-ops]

Okay, there was a search node with a cluster UUID trying to join a manager with a different UUID. Thanks.