No alerts appearing in SOC Alerts Interface

[Yehakin]

Version

2.4.200

Installation Method

Security Onion ISO image

Description

other (please provide detail below)

Installation Type

Distributed

Location

on-prem with Internet access

Hardware Specs

Exceeds minimum requirements

CPU

Manager: 8 CPU

RAM

Manager: 32 GB

Storage for /

82 GB total, 26 GB available

Storage for /nsm

159 GB total, 35 GB available

Network Traffic Collection

tap

Network Traffic Speeds

1Gbps to 10Gbps

Status

Yes, all services on all nodes are running OK

Salt Status

No, there are no failures

Logs

Yes, there are additional clues in /opt/so/log/ (please provide detail below)

Detail

About a month ago, alerts stopped appearing. Upgraded from SO ver 2.4.180 to ver 2.4.201, still no alerts appearing. Rebooted nodes, still no alerts appearing.

All grid (manager, search, and sensor) node's services are showing green "Running".

Sensor node grid status shows "Suricata Rules: 48018 loaded"

Detections tab:

"Total Found: 69,903"
ElastAlert: OK
Strelka: OK
Suricata: OK

Clicking Options, Suricata "Full Update" completes successful.

Alerts tab:

"Total Found: 0"

I see Suricata logs on the sensor:

[root@allium-sensor suricata]# pwd
/nsm/suricata
[root@allium-sensor suricata]# ls -lt | head
total 221140
-rw-r--r--. 1 suricata suricata 15777499 Feb 27 16:15 eve-2026-02-27-16:01.json
-rw-r--r--. 1 suricata suricata 82733795 Feb 27 16:00 eve-2026-02-27-15:01.json
-rw-r--r--. 1 suricata suricata  6307128 Feb 27 15:00 eve-2026-02-27-14:01.json.gz
-rw-r--r--. 1 suricata suricata  4096001 Feb 27 14:00 eve-2026-02-27-13:01.json.gz
-rw-r--r--. 1 suricata suricata  3651243 Feb 27 13:00 eve-2026-02-27-12:01.json.gz
-rw-r--r--. 1 suricata suricata  3576506 Feb 27 12:00 eve-2026-02-27-11:01.json.gz
-rw-r--r--. 1 suricata suricata  3573115 Feb 27 11:00 eve-2026-02-27-10:01.json.gz
-rw-r--r--. 1 suricata suricata  3583284 Feb 27 10:00 eve-2026-02-27-09:01.json.gz
-rw-r--r--. 1 suricata suricata  3386183 Feb 27 09:00 eve-2026-02-27-08:01.json.gz

Guidelines

• I have read the discussion guidelines at Read before posting! #1720 and assert that I have followed the guidelines.

[cm-ops]

On your sensor node, what does the Elastic agent status show? sudo elastic-agent status

If that shows ok, go to your manager node and check the Logstash log, /opt/so/log/logstash/logstash.log for any errors or clues.

You can also check you Elasticsearch for any issues with sudo so-elasticsearch-troubleshoot

[Yehakin]

Thanks Chris for the assistance.

"On your sensor node, what does the Elastic agent status show? sudo elastic-agent status"

[root@allium-sensor ]# elastic-agent status
┌─ fleet
│  └─ status: (HEALTHY) Connected
└─ elastic-agent
   └─ status: (HEALTHY) Running

"If that shows ok, go to your manager node and check the Logstash log, /opt/so/log/logstash/logstash.log for any errors or clues."

[2026-03-03T00:00:01,731][INFO ][org.logstash.beats.BeatsHandler] [local: 172.17.1.29:5055, remote: 139.70.2.106:38014] Handling exception: io.netty.handler.codec.DecoderException: javax.net.ssl.SSLHandshakeException: PKIX path validation failed: java.security.cert.CertPathValidatorException: validity check failed (caused by: java.security.cert.CertificateExpiredException: NotAfter: Wed Jan 14 21:01:54 GMT 2026)
[2026-03-03T00:00:01,732][WARN ][io.netty.channel.DefaultChannelPipeline] An exceptionCaught() event was fired, and it reached at the tail of the pipeline. It usually means the last handler in the pipeline did not handle the exception.
io.netty.handler.codec.DecoderException: javax.net.ssl.SSLHandshakeException: PKIX path validation failed: java.security.cert.CertPathValidatorException: validity check failed
---snip---

"You can also check you Elasticsearch for any issues with sudo so-elasticsearch-troubleshoot"

[root@allium logstash]# so-elasticsearch-troubleshoot

================ Elasticsearch Status ================

 All health indicators are green

================ Disk Usage Check ================

LOW:80% HIGH:85% FLOOD:90%

 allium disk usage: 78.41%
 allium-search disk usage: 0.71%

================ Unassigned Shards Check ================

 All shards are assigned

I checked the logstash cert on the manager.

[root@allium pki]# openssl x509 -in elasticfleet-logstash.crt -noout -startdate
notBefore=Jan 14 21:06:59 2026 GMT
[root@allium pki]# openssl x509 -in elasticfleet-logstash.crt -noout -enddate
notAfter=Apr 13 21:06:59 2028 GMT

[cm-ops]

If you go into Kibana > Fleet > Settings > Outputs > grid-logstash (click the pencil to edit) and check the Client SSL certificate for expiration. If it is expired, replace the Client SSL certificate and key with /etc/pki/elasticfleet-logstash.crt and .key

[Yehakin]

Thanks Chris for the solution.

The cert expired Jan 14, 2026. For some reason the renewed cert was never pushed out?

I replaced the expired .crt and .key, restarted logstash so-logstash-restart, then alerts started to populate in the Alerts Interface.