"The search query encountered a failure within the elaticsearch cluster"

[idamansudo]

Version

2.4.120

Installation Method

Security Onion ISO image

Description

configuration

Installation Type

Distributed

Location

on-prem with Internet access

Hardware Specs

Meets minimum requirements

CPU

8

RAM

16GB

Storage for /

500GB

Storage for /nsm

500GB

Network Traffic Collection

span port

Network Traffic Speeds

1Gbps to 10Gbps

Status

Yes, all services on all nodes are running OK

Salt Status

No, there are no failures

Logs

No, there are no additional clues

Detail

Getting intermittent log flow into the system. A restart resolves it for a couple hours. Request assistance looking into logs to determine cause

Guidelines

• I have read the discussion guidelines at Read before posting! #1720 and assert that I have followed the guidelines.

[dougburks]

How many total nodes do you have in your distributed deployment?

How many of those nodes are search nodes?

What are the specs of your manager and each of your search nodes? (CPU, RAM, type of storage - NVMe, SSD, or rotational)

Have you checked the logs in /opt/so/log/elasticsearch/ for additional clues?

How much network traffic are you monitoring in Gbps?

Are you consuming other logs and, if so, what is the EPS?

[idamansudo]

Hi Doug, How many total nodes do you have in your distributed deployment? 4 How many of those nodes are search nodes? 2 What are the specs of your manager and each of your search nodes? (CPU, RAM, type of storage - NVMe, SSD, or rotational) Recommended Have you checked the logs in /opt/so/log/elasticsearch/ for additional clues? Yes How much network traffic are you monitoring in Gbps? 40mbs Are you consuming other logs and, if so, what is the EPS? 50000

…

________________________________ From: Doug Burks ***@***.***> Sent: Wednesday, March 5, 2025 8:58 AM To: Security-Onion-Solutions/securityonion ***@***.***> Cc: idamansudo ***@***.***>; Author ***@***.***> Subject: Re: [Security-Onion-Solutions/securityonion] "The search query encountered a failure within the elaticsearch cluster" (Discussion #14334) How many total nodes do you have in your distributed deployment? How many of those nodes are search nodes? What are the specs of your manager and each of your search nodes? (CPU, RAM, type of storage - NVMe, SSD, or rotational) Have you checked the logs in /opt/so/log/elasticsearch/ for additional clues? How much network traffic are you monitoring in Gbps? Are you consuming other logs and, if so, what is the EPS? — Reply to this email directly, view it on GitHub<#14334 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/BO55O65G3TY7GHR6QASHCHL2S4UMBAVCNFSM6AAAAABYKKTPW2VHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTENBQGQZDGMA>. You are receiving this because you authored the thread.

[dougburks]

What are the specs of your manager and each of your search nodes? (CPU, RAM, type of storage - NVMe, SSD, or rotational)
Recommended

Please provide the DETAILED specs requested so that we can help you.

Have you checked the logs in /opt/so/log/elasticsearch/ for additional clues?
Yes

Please provide DETAILED information from those logs so that we can help you.

[idamansudo]

This issue was resolved by doubling the memory allocations as described here - #14322