logstash bad_certificate error

[tsmullins]

Version

2.4.10

Installation Method

Security Onion ISO image

Description

installation

Installation Type

Distributed

Location

on-prem with Internet access

Hardware Specs

Exceeds minimum requirements

CPU

32

RAM

128G

Storage for /

300G

Storage for /nsm

7TB

Network Traffic Collection

tap

Network Traffic Speeds

Less than 1Gbps

Status

Yes, all services on all nodes are running OK

Salt Status

Yes, there are salt failures (please provide detail below)

Logs

Yes, there are additional clues in /opt/so/log/ (please provide detail below)

Detail

We have a fresh four node install of 2.4.10. No data is being sent to Elastic/Kibana. Everything looks fine except a bad_certificate error in /opt/so/log/logstash/logstash.log. From the manager, I have tried a:

salt-call state.apply ssl
so-elasticsearch-restart

Any input would be greatly appreciated.

Here are the errors on the salt-call state.highstate and the error from the logstash log (on the manager).

#salt-call state.highstate failures
ID: so-elastic-fleet-auto-configure-logstash-outputs
Function: cmd.run
Name: /usr/sbin/so-elastic-fleet-outputs-update
Result: False
Comment: Attempt 1: Returned a result of "False", with the following comment: "Command "/usr/sbin/so-elastic-fleet-outputs-update" run"
Command "/usr/sbin/so-elastic-fleet-outputs-update" run
Started: 03:38:38.361036
Duration: 30209.618 ms
Changes:
----------
pid:
209849
retcode:
1
stderr:
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed

                0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
              100    98  100    98    0     0   2969      0 --:--:-- --:--:-- --:--:--  2882
              100    98  100    98    0     0   2969      0 --:--:-- --:--:-- --:--:--  2882
          stdout:
              Failed to query for current Logstash Outputs...

#logstash cert error
[2023-09-18T03:13:22,913][INFO ][org.logstash.beats.BeatsHandler] [local: 172.17.1.29:5055, remote: 10.16.228.13:40392] Handling exception: io.netty.handler.codec.DecoderException: javax.net.ssl.SSLHandshakeException: Received fatal alert: bad_certificate (caused by: javax.net.ssl.SSLHandshakeException: Received fatal alert: bad_certificate)

Guidelines

• I have read the above statement and can confirm my post is relevant to Security Onion 2.4.

[linuxlurak]

Hi there. Happened to me. Check the remote IPs that are listed in your logs and uninstall + install elastic agent (or whatever you use to send to beats) on that client. If this makes the exceptions disappear the problem is solved. I tried to reset certs/pki etc. but this happened directly after setup of a fresh SO 2.4.30 standalone box. --> Problem were elastic agents with outdated certs.