No alerts after updating

[BradMic]

Version

2.4.110

Installation Method

Security Onion ISO image

Description

other (please provide detail below)

Installation Type

Distributed

Location

on-prem with Internet access

Hardware Specs

Meets minimum requirements

CPU

4

RAM

16

Storage for /

130GB

Storage for /nsm

12TB

Network Traffic Collection

tap

Network Traffic Speeds

Less than 1Gbps

Status

Yes, all services on all nodes are running OK

Salt Status

No, there are no failures

Logs

No, there are no additional clues

Detail

I have not been getting any alerts since updating to v2.4.110.

When I go to http://testmynids.org/uid/index.html, no alert is generated.
I do see the traffic with tcpdump on the sensor's monitoring interface.
I can see the pcap files being created on the sensor in /nsm/pcap.
Queries to /detections?q=2100498 shows that the rule is enabled.
Metadata is set to ZEEK.
HOME_NET is set correctly.
All the nodes in the grid are green.

sudo so-status on run on the sensor show suricate is running.
sudo so-checkin runs on the manager with no failures.
sudo so-elasticsearch-indices-list does NOT show any .ds-logs-suricata-so-* indices.

I don't know where else to look. Thank you for any pointers.

Guidelines

• I have read the discussion guidelines at Read before posting! #1720 and assert that I have followed the guidelines.

[rh-ops]

Try checking your endpoint policies and make sure they aren't applied to the wrong places. If they are correct, then run the following command and provide the output.

Salt-call pillar.get logstash

[BradMic]

I'm not sure what you mean by endpoint policies but there is no output from the command sudo salt-call pillar.get logstash when run on the manager, except for the word local:

The salt minion service is running on all the nodes.

[BradMic]

Looks like there was already a highstate running so that's why it showed nothing before. When I ran it again, it seems to have completed successfully.

I ran sudo so-test but still no laerts were shown in the SOC Alerts page.

Runing the command sudo docker logs so-filebeat on every node returns Error response from daemon: No such container: so-filebeat

Running the command sudo docker images | grep filebeat returns nothing

[BradMic]

I just discovered that the elastic agent won't start on the forward node. It's running fine on all the other nodes.

[BradMic]

So I ended up having to uninstall/reinstall the elastic agent on the sensor, which fixed the issue. The command /usr/bin/elastic-agent uninstall errored out, so I had to manually delete the /opt/Elastic folder and the files /etc/systemd/system/elastic-agent.service, usr/bin/elastic-agent. Then the command sudo salt-call state.apply elasticfleet.install_agent_grid would work to reinstall the elastic agent back on the sensor.

[rh-ops]

So now all your issues have been resolved? You're receiving alerts again?

[BradMic]

yes

[NeoLife-iT]

my new deployment show all node are green but no data received. Where do you think I should Look?

[rh-ops]

my new deployment show all node are green but no data received. Where do you think I should Look?

Please open a new discussion for this matter.