How to Troubleshoot no Data in Dashboard

[cocjos]

Version

2.4.60

Installation Method

Security Onion ISO image

Description

configuration

Installation Type

Distributed

Location

on-prem with Internet access

Hardware Specs

Exceeds minimum requirements

CPU

4

RAM

16 gb

Storage for /

1TB

Storage for /nsm

1TB

Network Traffic Collection

other (please provide detail below)

Network Traffic Speeds

Less than 1Gbps

Status

Yes, all services on all nodes are running OK

Salt Status

No, there are no failures

Logs

No, there are no additional clues

Detail

I do not see any log in information in the dashboard. I am trying to connect windows elastic agents to the SO manager. I see the agents in fleet manager. The agents are connected to SO manager when I look at the agents' netstat.

I see this error Failed to send backlog of events to Redis in logstash.log. But I am not sure what I am looking at yet.

Guidelines

• I have read the discussion guidelines at Read before posting! #1720 and assert that I have followed the guidelines.

[cm-ops]

Looks like an issue with the Logstash pipeline. Would you restart Logstash, then re-check the /opt/so/log/logstash/logstash.log for any clues.

Also, in Fleet, do you see all the agents as healthy?

[cocjos]

Chris, I do see all the agents in fleet. The agents are all good. Joseph Lamb

…

________________________________ From: Chris Morgret ***@***.***> Sent: Tuesday, May 28, 2024 5:49:24 AM To: Security-Onion-Solutions/securityonion ***@***.***> Cc: Joseph Lamb ***@***.***>; Author ***@***.***> Subject: Re: [Security-Onion-Solutions/securityonion] How to Troubleshoot no Data in Dashboard (Discussion #13077) You don't often get email from ***@***.*** Learn why this is important<https://aka.ms/LearnAboutSenderIdentification> Looks like an issue with the Logstash pipeline. Would you restart Logstash, then re-check the /opt/so/log/logstash/logstash.log for any clues. Also, in Fleet, do you see all the agents as healthy? — Reply to this email directly, view it on GitHub<#13077 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/BIPZBMLV6E2GO5WFNFIFY6TZER4NJAVCNFSM6AAAAABIHWE6DOVHI2DSMVQWIX3LMV43SRDJONRXK43TNFXW4Q3PNVWWK3TUHM4TKOBQG42DC>. You are receiving this because you authored the thread.Message ID: ***@***.***>

[cm-ops]

Restart Logstash, tail the /opt/so/log/logstash/logstash.log, look for any issues with the pipeline(s) starting.

[cocjos]

Chris, The pipeline is not starting workers. This is the start: [2024-05-29T13:58:57,068][WARN ][logstash.outputs.redis ] Failed to send backlog of events to Redis "/usr/share/logstash/logstash-core/lib/logstash/outputs/base.rb:104:in `multi_receive'", "org/logstash/config/ir/c ompiler/AbstractOutputDelegatorExt.java:121:in `multi_receive'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:304:in `block in st art_workers'"]}

…

________________________________ From: Chris Morgret ***@***.***> Sent: Wednesday, May 29, 2024 5:58 AM To: Security-Onion-Solutions/securityonion ***@***.***> Cc: Joseph Lamb ***@***.***>; Author ***@***.***> Subject: Re: [Security-Onion-Solutions/securityonion] How to Troubleshoot no Data in Dashboard (Discussion #13077) You don't often get email from ***@***.*** Learn why this is important<https://aka.ms/LearnAboutSenderIdentification> Restart Logstash, tail the /opt/so/log/logstash/logstash.log, look for any issues with the pipeline(s) starting. — Reply to this email directly, view it on GitHub<#13077 (reply in thread)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/BIPZBMM7TFFNATB27DNQWK3ZEXGIJAVCNFSM6AAAAABIHWE6DOVHI2DSMVQWIX3LMV43SRDJONRXK43TNFXW4Q3PNVWWK3TUHM4TKOJTGU2TG>. You are receiving this because you authored the thread.Message ID: ***@***.***>

[cm-ops]

See if you see a pipeline terminated in the log, also check your /opt/so/log/elasticsearch/securityonion.log for any issues.

How is the memory utilization? Another issue could be a Logstash heap issue shutting down the pipeline(s).

[dougburks]

@DrOnionMaster Please review the discussion guidelines at #1720:

Start a new discussion instead of replying to somebody else's discussion
Please search to see if you can find similar discussions that may help you. However, in order to avoid confusion, please do NOT reply to somebody else's discussion with your own issue. Instead, please start a new discussion and in that new discussion you can provide a hyperlink to the related discussion.

[DrOnionMaster]

My apologies, comments deleted.