Elasticsearch Status: Fault | after reboot all grind

[cherryCokeBack]

Version

2.4.70

Installation Method

Security Onion ISO image

Description

other (please provide detail below)

Installation Type

Distributed

Location

on-prem with Internet access

Hardware Specs

Exceeds minimum requirements

CPU

12

RAM

20

Storage for /

2to

Storage for /nsm

20to

Network Traffic Collection

span port

Network Traffic Speeds

1Gbps to 10Gbps

Status

No, one or more services are failed (please provide detail below)

Salt Status

No, there are no failures

Logs

Yes, there are additional clues in /opt/so/log/ (please provide detail below)

Detail

Hello,

After reboot all gring node ( reboot request ).
I have a problem with the manager on elasticsearch

I've run the following commands and have no problems with the returns

*so-docker-refresh
*salt-call state.highstate

I looked in the logs in the following directory :
opt/so/log/elasticsearch
here's part of the result:

Caused by: org.elasticsearch.action.search.SearchPhaseExecutionException: Search rejected due to missing shards [[.ds-logs-elastic_agent-default-2024.06.03-000005][0], [.ds-logs-elastic_agent-default-2024.06.03-000006][0], [.ds-logs-elastic_agent.filebeat-default-2024.05.09-000004][0], [.ds-logs-elastic_agent.filebeat-default-2024.06.03-000006][0], [.ds-logs-elastic_agent.fleet_server-default-2024.04.09-000002][0], [.ds-logs-m365_defender.incident-default-2024.02.27-000008][0], [.ds-logs-m365_defender.incident-default-2024.05.27-000011][0], [.ds-logs-panw.panos-default-2024.06.16-000012][0], [.ds-logs-playbook.alerts-so-2024.05.21-000004][0], [.ds-logs-suricata-so-2024.05.07-000035][0], [.ds-logs-suricata-so-2024.05.11-000039][0], [.ds-logs-suricata-so-2024.05.20-000048][0], [.ds-logs-suricata-so-2024.05.21-000049][0], [.ds-logs-suricata-so-2024.06.01-000060][0], [.ds-logs-suricata-so-2024.06.02-000061][0], [.ds-logs-suricata-so-2024.06.05-000068][0], [.ds-logs-suricata-so-2024.06.06-000069][0], [.ds-logs-suricata-so-2024.06.07-000070][0], [.ds-logs-suricata-so-2024.06.12-000075][0], [.ds-logs-suricata-so-2024.06.15-000079][0], [.ds-logs-suricata-so-2024.06.18-000082][0], [.ds-logs-suricata-so-2024.06.20-000084][0], [.ds-logs-suricata-so-2024.06.24-000088][0], [.ds-logs-zeek-so-2024.03.12-000006][1], [.ds-logs-zeek-so-2024.05.11-000008][1], [.ds-logs-zeek-so-2024.06.10-000014][1]]. Consider using allow_partial_search_results setting to bypass this error.
at org.elasticsearch.action.search.CanMatchPreFilterSearchPhase.checkNoMissingShards(CanMatchPreFilterSearchPhase.java:213) ~[elasticsearch-8.10.4.jar:?]
at org.elasticsearch.action.search.CanMatchPreFilterSearchPhase.run(CanMatchPreFilterSearchPhase.java:132) ~[elasticsearch-8.10.4.jar:?]
at org.elasticsearch.action.search.CanMatchPreFilterSearchPhase$1.doRun(CanMatchPreFilterSearchPhase.java:429) ~[elasticsearch-8.10.4.jar:?]
at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:26) ~[elasticsearch-8.10.4.jar:?]
... 6 more
[2024-06-26T10:34:05,321][WARN ][rest.suppressed ] path: /.ds-logs-/_eql/search, params: {ignore_unavailable=true, index=.ds-logs-}
org.elasticsearch.action.search.SearchPhaseExecutionException: start
at org.elasticsearch.action.search.CanMatchPreFilterSearchPhase.onPhaseFailure(CanMatchPreFilterSearchPhase.java:435) ~[elasticsearch-8.10.4.jar:?]
at org.elasticsearch.action.search.CanMatchPreFilterSearchPhase$1.onFailure(CanMatchPreFilterSearchPhase.java:424) ~[elasticsearch-8.10.4.jar:?]
at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:28) ~[elasticsearch-8.10.4.jar:?]
at org.elasticsearch.common.util.concurrent.TimedRunnable.doRun(TimedRunnable.java:33) ~[elasticsearch-8.10.4.jar:?]
at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:983) ~[elasticsearch-8.10.4.jar:?]
at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:26) ~[elasticsearch-8.10.4.jar:?]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144) ~[?:?]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642) ~[?:?]
at java.lang.Thread.run(Thread.java:1583) ~[?:?]
Caused by: org.elasticsearch.action.search.SearchPhaseExecutionException: Search rejected due to missing shards [[.ds-logs-elastic_agent-default-2024.06.03-000005][0], [.ds-logs-elastic_agent-default-2024.06.03-000006][0], [.ds-logs-elastic_agent.filebeat-default-2024.05.09-000004][0], [.ds-logs-elastic_agent.filebeat-default-2024.06.03-000006][0], [.ds-logs-elastic_agent.fleet_server-default-2024.04.09-000002][0], [.ds-logs-m365_defender.incident-default-2024.02.27-000008][0], [.ds-logs-m365_defender.incident-default-2024.05.27-000011][0], [.ds-logs-panw.panos-default-2024.06.16-000012][0], [.ds-logs-playbook.alerts-so-2024.05.21-000004][0], [.ds-logs-suricata-so-2024.05.07-000035][0], [.ds-logs-suricata-so-2024.05.11-000039][0], [.ds-logs-suricata-so-2024.05.20-000048][0], [.ds-logs-suricata-so-2024.05.21-000049][0], [.ds-logs-suricata-so-2024.06.01-000060][0], [.ds-logs-suricata-so-2024.06.02-000061][0], [.ds-logs-suricata-so-2024.06.05-000068][0], [.ds-logs-suricata-so-2024.06.06-000069][0], [.ds-logs-suricata-so-2024.06.07-000070][0], [.ds-logs-suricata-so-2024.06.12-000075][0], [.ds-logs-suricata-so-2024.06.15-000079][0], [.ds-logs-suricata-so-2024.06.18-000082][0], [.ds-logs-suricata-so-2024.06.20-000084][0], [.ds-logs-suricata-so-2024.06.24-000088][0], [.ds-logs-zeek-so-2024.03.12-000006][1], [.ds-logs-zeek-so-2024.05.11-000008][1], [.ds-logs-zeek-so-2024.06.10-000014][1]]. Consider using allow_partial_search_results setting to bypass this error.
at org.elasticsearch.action.search.CanMatchPreFilterSearchPhase.checkNoMissingShards(CanMatchPreFilterSearchPhase.java:213) ~[elasticsearch-8.10.4.jar:?]
at org.elasticsearch.action.search.CanMatchPreFilterSearchPhase.run(CanMatchPreFilterSearchPhase.java:132) ~[elasticsearch-8.10.4.jar:?]
at org.elasticsearch.action.search.CanMatchPreFilterSearchPhase$1.doRun(CanMatchPreFilterSearchPhase.java:429) ~[elasticsearch-8.10.4.jar:?]
at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:26) ~[elasticsearch-8.10.4.jar:?]
... 6 more
[2024-06-26T10:34:05,326][WARN ][rest.suppressed ] path: /.ds-logs-/_eql/search, params: {ignore_unavailable=true, index=.ds-logs-}
org.elasticsearch.action.search.SearchPhaseExecutionException: start
at org.elasticsearch.action.search.CanMatchPreFilterSearchPhase.onPhaseFailure(CanMatchPreFilterSearchPhase.java:435) ~[elasticsearch-8.10.4.jar:?]
at org.elasticsearch.action.search.CanMatchPreFilterSearchPhase$1.onFailure(CanMatchPreFilterSearchPhase.java:424) ~[elasticsearch-8.10.4.jar:?]
at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:28) ~[elasticsearch-8.10.4.jar:?]
at org.elasticsearch.common.util.concurrent.TimedRunnable.doRun(TimedRunnable.java:33) ~[elasticsearch-8.10.4.jar:?]
at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:983) ~[elasticsearch-8.10.4.jar:?]
at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:26) ~[elasticsearch-8.10.4.jar:?]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144) ~[?:?]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642) ~[?:?]
at java.lang.Thread.run(Thread.java:1583) ~[?:?]
Caused by: org.elasticsearch.action.search.SearchPhaseExecutionException: Search rejected due to missing shards [[.ds-logs-elastic_agent-default-2024.06.03-000005][0], [.ds-logs-elastic_agent-default-2024.06.03-000006][0], [.ds-logs-elastic_agent.filebeat-default-2024.05.09-000004][0], [.ds-logs-elastic_agent.filebeat-default-2024.06.03-000006][0], [.ds-logs-elastic_agent.fleet_server-default-2024.04.09-000002][0], [.ds-logs-m365_defender.incident-default-2024.02.27-000008][0], [.ds-logs-m365_defender.incident-default-2024.05.27-000011][0], [.ds-logs-panw.panos-default-2024.06.16-000012][0], [.ds-logs-playbook.alerts-so-2024.05.21-000004][0], [.ds-logs-suricata-so-2024.05.07-000035][0], [.ds-logs-suricata-so-2024.05.11-000039][0], [.ds-logs-suricata-so-2024.05.20-000048][0], [.ds-logs-suricata-so-2024.05.21-000049][0], [.ds-logs-suricata-so-2024.06.01-000060][0], [.ds-logs-suricata-so-2024.06.02-000061][0], [.ds-logs-suricata-so-2024.06.05-000068][0], [.ds-logs-suricata-so-2024.06.06-000069][0], [.ds-logs-suricata-so-2024.06.07-000070][0], [.ds-logs-suricata-so-2024.06.12-000075][0], [.ds-logs-suricata-so-2024.06.15-000079][0], [.ds-logs-suricata-so-2024.06.18-000082][0], [.ds-logs-suricata-so-2024.06.20-000084][0], [.ds-logs-suricata-so-2024.06.24-000088][0], [.ds-logs-zeek-so-2024.03.12-000006][1], [.ds-logs-zeek-so-2024.05.11-000008][1], [.ds-logs-zeek-so-2024.06.10-000014][1]]. Consider using allow_partial_search_results setting to bypass this error.
at org.elasticsearch.action.search.CanMatchPreFilterSearchPhase.checkNoMissingShards(CanMatchPreFilterSearchPhase.java:213) ~[elasticsearch-8.10.4.jar:?]
at org.elasticsearch.action.search.CanMatchPreFilterSearchPhase.run(CanMatchPreFilterSearchPhase.java:132) ~[elasticsearch-8.10.4.jar:?]
at org.elasticsearch.action.search.CanMatchPreFilterSearchPhase$1.doRun(CanMatchPreFilterSearchPhase.java:429) ~[elasticsearch-8.10.4.jar:?]
at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:26) ~[elasticsearch-8.10.4.jar:?]
... 6 more
[2024-06-26T10:34:05,339][WARN ][rest.suppressed ] path: /.ds-logs-/_eql/search, params: {ignore_unavailable=true, index=.ds-logs-}
org.elasticsearch.action.search.SearchPhaseExecutionException: start
at org.elasticsearch.action.search.CanMatchPreFilterSearchPhase.onPhaseFailure(CanMatchPreFilterSearchPhase.java:435) ~[elasticsearch-8.10.4.jar:?]
at org.elasticsearch.action.search.CanMatchPreFilterSearchPhase$1.onFailure(CanMatchPreFilterSearchPhase.java:424) ~[elasticsearch-8.10.4.jar:?]
at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:28) ~[elasticsearch-8.10.4.jar:?]
at org.elasticsearch.common.util.concurrent.TimedRunnable.doRun(TimedRunnable.java:33) ~[elasticsearch-8.10.4.jar:?]
at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:983) ~[elasticsearch-8.10.4.jar:?]
at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:26) ~[elasticsearch-8.10.4.jar:?]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144) ~[?:?]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642) ~[?:?]
at java.lang.Thread.run(Thread.java:1583) ~[?:?]
Caused by: org.elasticsearch.action.search.SearchPhaseExecutionException: Search rejected due to missing shards [[.ds-logs-elastic_agent-default-2024.06.03-000005][0], [.ds-logs-elastic_agent-default-2024.06.03-000006][0], [.ds-logs-elastic_agent.filebeat-default-2024.05.09-000004][0], [.ds-logs-elastic_agent.filebeat-default-2024.06.03-000006][0], [.ds-logs-elastic_agent.fleet_server-default-2024.04.09-000002][0], [.ds-logs-m365_defender.incident-default-2024.02.27-000008][0], [.ds-logs-m365_defender.incident-default-2024.05.27-000011][0], [.ds-logs-panw.panos-default-2024.06.16-000012][0], [.ds-logs-playbook.alerts-so-2024.05.21-000004][0], [.ds-logs-suricata-so-2024.05.07-000035][0], [.ds-logs-suricata-so-2024.05.11-000039][0], [.ds-logs-suricata-so-2024.05.20-000048][0], [.ds-logs-suricata-so-2024.05.21-000049][0], [.ds-logs-suricata-so-2024.06.01-000060][0], [.ds-logs-suricata-so-2024.06.02-000061][0], [.ds-logs-suricata-so-2024.06.05-000068][0], [.ds-logs-suricata-so-2024.06.06-000069][0], [.ds-logs-suricata-so-2024.06.07-000070][0], [.ds-logs-suricata-so-2024.06.12-000075][0], [.ds-logs-suricata-so-2024.06.15-000079][0], [.ds-logs-suricata-so-2024.06.18-000082][0], [.ds-logs-suricata-so-2024.06.20-000084][0], [.ds-logs-suricata-so-2024.06.24-000088][0], [.ds-logs-zeek-so-2024.03.12-000006][1], [.ds-logs-zeek-so-2024.05.11-000008][1], [.ds-logs-zeek-so-2024.06.10-000014][1]]. Consider using allow_partial_search_results setting to bypass this error.
at org.elasticsearch.action.search.CanMatchPreFilterSearchPhase.checkNoMissingShards(CanMatchPreFilterSearchPhase.java:213) ~[elasticsearch-8.10.4.jar:?]
at org.elasticsearch.action.search.CanMatchPreFilterSearchPhase.run(CanMatchPreFilterSearchPhase.java:132) ~[elasticsearch-8.10.4.jar:?]
at org.elasticsearch.action.search.CanMatchPreFilterSearchPhase$1.doRun(CanMatchPreFilterSearchPhase.java:429) ~[elasticsearch-8.10.4.jar:?]
at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:26) ~[elasticsearch-8.10.4.jar:?]
... 6 more
[2024-06-26T10:34:05,344][WARN ][rest.suppressed ] path: /.ds-logs-/_eql/search, params: {ignore_unavailable=true, index=.ds-logs-}
org.elasticsearch.action.search.SearchPhaseExecutionException: start
at org.elasticsearch.action.search.CanMatchPreFilterSearchPhase.onPhaseFailure(CanMatchPreFilterSearchPhase.java:435) ~[elasticsearch-8.10.4.jar:?]
at org.elasticsearch.action.search.CanMatchPreFilterSearchPhase$1.onFailure(CanMatchPreFilterSearchPhase.java:424) ~[elasticsearch-8.10.4.jar:?]
at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:28) ~[elasticsearch-8.10.4.jar:?]
at org.elasticsearch.common.util.concurrent.TimedRunnable.doRun(TimedRunnable.java:33) ~[elasticsearch-8.10.4.jar:?]
at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:983) ~[elasticsearch-8.10.4.jar:?]
at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:26) ~[elasticsearch-8.10.4.jar:?]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144) ~[?:?]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642) ~[?:?]
at java.lang.Thread.run(Thread.java:1583) ~[?:?]
Caused by: org.elasticsearch.action.search.SearchPhaseExecutionException: Search rejected due to missing shards [[.ds-logs-elastic_agent-default-2024.06.03-000005][0], [.ds-logs-elastic_agent-default-2024.06.03-000006][0], [.ds-logs-elastic_agent.filebeat-default-2024.05.09-000004][0], [.ds-logs-elastic_agent.filebeat-default-2024.06.03-000006][0], [.ds-logs-elastic_agent.fleet_server-default-2024.04.09-000002][0], [.ds-logs-m365_defender.incident-default-2024.02.27-000008][0], [.ds-logs-m365_defender.incident-default-2024.05.27-000011][0], [.ds-logs-panw.panos-default-2024.06.16-000012][0], [.ds-logs-playbook.alerts-so-2024.05.21-000004][0], [.ds-logs-suricata-so-2024.05.07-000035][0], [.ds-logs-suricata-so-2024.05.11-000039][0], [.ds-logs-suricata-so-2024.05.20-000048][0], [.ds-logs-suricata-so-2024.05.21-000049][0], [.ds-logs-suricata-so-2024.06.01-000060][0], [.ds-logs-suricata-so-2024.06.02-000061][0], [.ds-logs-suricata-so-2024.06.05-000068][0], [.ds-logs-suricata-so-2024.06.06-000069][0], [.ds-logs-suricata-so-2024.06.07-000070][0], [.ds-logs-suricata-so-2024.06.12-000075][0], [.ds-logs-suricata-so-2024.06.15-000079][0], [.ds-logs-suricata-so-2024.06.18-000082][0], [.ds-logs-suricata-so-2024.06.20-000084][0], [.ds-logs-suricata-so-2024.06.24-000088][0], [.ds-logs-zeek-so-2024.03.12-000006][1], [.ds-logs-zeek-so-2024.05.11-000008][1], [.ds-logs-zeek-so-2024.06.10-000014][1]]. Consider using allow_partial_search_results setting to bypass this error.
at org.elasticsearch.action.search.CanMatchPreFilterSearchPhase.checkNoMissingShards(CanMatchPreFilterSearchPhase.java:213) ~[elasticsearch-8.10.4.jar:?]
at org.elasticsearch.action.search.CanMatchPreFilterSearchPhase.run(CanMatchPreFilterSearchPhase.java:132) ~[elasticsearch-8.10.4.jar:?]
at org.elasticsearch.action.search.CanMatchPreFilterSearchPhase$1.doRun(CanMatchPreFilterSearchPhase.java:429) ~[elasticsearch-8.10.4.jar:?]
at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:26) ~[elasticsearch-8.10.4.jar:?]
... 6 more
[2024-06-26T10:34:05,374][WARN ][rest.suppressed ] path: /.ds-logs-/_eql/search, params: {ignore_unavailable=true, index=.ds-logs-}
org.elasticsearch.action.search.SearchPhaseExecutionException: start
at org.elasticsearch.action.search.CanMatchPreFilterSearchPhase.onPhaseFailure(CanMatchPreFilterSearchPhase.java:435) ~[elasticsearch-8.10.4.jar:?]
at org.elasticsearch.action.search.CanMatchPreFilterSearchPhase$1.onFailure(CanMatchPreFilterSearchPhase.java:424) ~[elasticsearch-8.10.4.jar:?]
at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:28) ~[elasticsearch-8.10.4.jar:?]
at org.elasticsearch.common.util.concurrent.TimedRunnable.doRun(TimedRunnable.java:33) ~[elasticsearch-8.10.4.jar:?]
at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:983) ~[elasticsearch-8.10.4.jar:?]
at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:26) ~[elasticsearch-8.10.4.jar:?]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144) ~[?:?]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642) ~[?:?]
at java.lang.Thread.run(Thread.java:1583) ~[?:?]
Caused by: org.elasticsearch.action.search.SearchPhaseExecutionException: Search rejected due to missing shards [[.ds-logs-elastic_agent-default-2024.06.03-000005][0], [.ds-logs-elastic_agent-default-2024.06.03-000006][0], [.ds-logs-elastic_agent.filebeat-default-2024.05.09-000004][0], [.ds-logs-elastic_agent.filebeat-default-2024.06.03-000006][0], [.ds-logs-elastic_agent.fleet_server-default-2024.04.09-000002][0], [.ds-logs-m365_defender.incident-default-2024.02.27-000008][0], [.ds-logs-m365_defender.incident-default-2024.05.27-000011][0], [.ds-logs-panw.panos-default-2024.06.16-000012][0], [.ds-logs-playbook.alerts-so-2024.05.21-000004][0], [.ds-logs-suricata-so-2024.05.07-000035][0], [.ds-logs-suricata-so-2024.05.11-000039][0], [.ds-logs-suricata-so-2024.05.20-000048][0], [.ds-logs-suricata-so-2024.05.21-000049][0], [.ds-logs-suricata-so-2024.06.01-000060][0], [.ds-logs-suricata-so-2024.06.02-000061][0], [.ds-logs-suricata-so-2024.06.05-000068][0], [.ds-logs-suricata-so-2024.06.06-000069][0], [.ds-logs-suricata-so-2024.06.07-000070][0], [.ds-logs-suricata-so-2024.06.12-000075][0], [.ds-logs-suricata-so-2024.06.15-000079][0], [.ds-logs-suricata-so-2024.06.18-000082][0], [.ds-logs-suricata-so-2024.06.20-000084][0], [.ds-logs-suricata-so-2024.06.24-000088][0], [.ds-logs-zeek-so-2024.03.12-000006][1], [.ds-logs-zeek-so-2024.05.11-000008][1], [.ds-logs-zeek-so-2024.06.10-000014][1]]. Consider using allow_partial_search_results setting to bypass this error.
at org.elasticsearch.action.search.CanMatchPreFilterSearchPhase.checkNoMissingShards(CanMatchPreFilterSearchPhase.java:213) ~[elasticsearch-8.10.4.jar:?]
at org.elasticsearch.action.search.CanMatchPreFilterSearchPhase.run(CanMatchPreFilterSearchPhase.java:132) ~[elasticsearch-8.10.4.jar:?]
at org.elasticsearch.action.search.CanMatchPreFilterSearchPhase$1.doRun(CanMatchPreFilterSearchPhase.java:429) ~[elasticsearch-8.10.4.jar:?]
at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:26) ~[elasticsearch-8.10.4.jar:?]

Now I can't use the manager, how can I fix the problem?
thank you in advance for your help

Guidelines

• I have read the discussion guidelines at Read before posting! #1720 and assert that I have followed the guidelines.

[cm-ops]

Search rejected due to missing shards There is a list of shards Elasticsearch is reporting as missing.

What is the output of these commands?

sudo so-elasticsearch-query _cluster/health?pretty
sudo so-elasticsearch-query _cat/shards
sudo so-elasticsearch-query _cat/nodes

[cherryCokeBack]

thanks for the answer

I found the shards in error:
Index
.ds-logs-elastic_agent-default-2024.06.03-000006 -----------
Type
no_shard_available_action_exception

Shard
0
Index
.ds-logs-elastic_agent.filebeat-default-2024.05.09-000004
Type
no_shard_available_action_exception

Shard
0
Index
.ds-logs-elastic_agent.filebeat-default-2024.06.03-000006 ----------
Type
no_shard_available_action_exception

Shard
0
Index
.ds-logs-elastic_agent.fleet_server-default-2024.04.09-000002
Type
no_shard_available_action_exception

Shard
0
Index
.ds-logs-m365_defender.incident-default-2024.02.27-000008
Type
no_shard_available_action_exception

Shard
0
Index
.ds-logs-m365_defender.incident-default-2024.05.27-000011 ------------
Type
no_shard_available_action_exception

Shard
0
Index
.ds-logs-panw.panos-default-2024.06.16-000012 ------------------
Type
no_shard_available_action_exception

Shard
0
Index
.ds-logs-playbook.alerts-so-2024.05.21-000004
Type
no_shard_available_action_exception

Shard
0
Index
.ds-logs-suricata-so-2024.05.07-000035
Type
no_shard_available_action_exception

Shard
0
Index
.ds-logs-suricata-so-2024.05.11-000039
Type
no_shard_available_action_exception

Shard
0

Index
.ds-logs-suricata-so-2024.05.20-000048
Type
no_shard_available_action_exception

Shard
0
Index
.ds-logs-suricata-so-2024.05.21-000049
Type
no_shard_available_action_exception

Shard
0
Index
.ds-logs-suricata-so-2024.06.01-000060
Type
no_shard_available_action_exception

Shard
0
Index
.ds-logs-suricata-so-2024.06.02-000061
Type
no_shard_available_action_exception

Shard
0
Index
.ds-logs-suricata-so-2024.06.05-000068
Type
no_shard_available_action_exception

Shard
0
Index
.ds-logs-suricata-so-2024.06.06-000069
Type
no_shard_available_action_exception

Shard
0
Index
.ds-logs-suricata-so-2024.06.07-000070
Type
no_shard_available_action_exception

Shard
0
Index
.ds-logs-suricata-so-2024.06.12-000075
Type
no_shard_available_action_exception

Shard
0
Index
.ds-logs-suricata-so-2024.06.15-000079
Type
no_shard_available_action_exception

Shard
0
Index
.ds-logs-suricata-so-2024.06.18-000082
Type
no_shard_available_action_exception

Shard
0

Index
.ds-logs-suricata-so-2024.06.20-000084
Type
no_shard_available_action_exception

Shard
0
Index
.ds-logs-suricata-so-2024.06.24-000088
Type
no_shard_available_action_exception

Shard
1
Index
.ds-logs-zeek-so-2024.03.12-000006
Type
no_shard_available_action_exception

Shard
1
Index
.ds-logs-zeek-so-2024.05.11-000008
Type
no_shard_available_action_exception

Shard
1
Index
.ds-logs-zeek-so-2024.06.10-000014
Type
no_shard_available_action_exception

-reste : 
Shard
0

Index
.ds-logs-elastic_agent-default-2024.06.03-000006
Type
no_shard_available_action_exception

Shard
0
Index
.ds-logs-elastic_agent.filebeat-default-2024.06.03-000006
Type
no_shard_available_action_exception

Shard
0
Index
.ds-logs-m365_defender.incident-default-2024.05.27-000011
Type
no_shard_available_action_exception

Shard
0
Index
.ds-logs-panw.panos-default-2024.06.16-000012
Type
no_shard_available_action_exception

Shard
1
Index
.ds-logs-zeek-so-2024.06.10-000014
Type
no_shard_available_action_exception

I integrated a new search server but it crashed completely after one night.

i've tried deleting the default indexes but there are still 5 that i can't delete