Force Forward node to talk to Receiver Node #12777

alveare1 · 2024-04-11T04:11:22Z

alveare1
Apr 11, 2024

Version

2.4.60

Installation Method

Security Onion ISO image

Description

configuration

Installation Type

Distributed

Location

on-prem with Internet access

Hardware Specs

Exceeds minimum requirements

CPU

8 to 16 depending on node

RAM

16 or 32gb

Storage for /

512

Storage for /nsm

512

Network Traffic Collection

other (please provide detail below)

Network Traffic Speeds

Less than 1Gbps

Status

Yes, all services on all nodes are running OK

Salt Status

No, there are no failures

Logs

No, there are no additional clues

Detail

It is in the notes that you have to modify the rules but not sure what ones.

I want my forward only to send to the receiver nodes ignoring the master node.

Guidelines

I have read the discussion guidelines at Read before posting! #1720 and assert that I have followed the guidelines.

Answered by TOoSmOotH

Apr 11, 2024

Actually there is a setting for this.

If you change that to false it will tell all of your elastic agents to use receivers instead of the manager. Make sure you click Advanced to expose this setting.

View full answer

TOoSmOotH · 2024-04-11T13:13:42Z

TOoSmOotH
Apr 11, 2024
Maintainer

Actually there is a setting for this.

If you change that to false it will tell all of your elastic agents to use receivers instead of the manager. Make sure you click Advanced to expose this setting.

6 replies

alveare1 Apr 13, 2024
Author

Docker (unhealthy)
so-influxdb
so-nginx
so-mysql
so-elastic-fleet-packages-registery

so-elastialert is missing

before grinding to an almost dead stop, still trying to get the system stable.

alveare1 Apr 13, 2024
Author

[root@system]# so-elastic-fleet-restart

Restarting elastic-fleet...

This could take a while if another Salt job is running.
Run this command with --force to stop all Salt jobs before proceeding.

Error response from daemon: No such container: so-elastic-fleet

[root@system]# so-elastic-fleet-start

Starting elastic-fleet...

This could take a while if another Salt job is running.
Run this command with --force to stop all Salt jobs before proceeding.

elastic-fleet is already running!

[root@system]# so-status

                   Security Onion Status                        
                     Container │ Status  │              Details

───────────────────────────────────┼─────────┼──────────────────────
so-dockerregistry │ running │ Up 4 hours
so-elastalert │ running │ Up 3 hours
so-elastic-fleet │ missing │
so-elastic-fleet-package-registry │ running │ Up 3 hours (healthy)
so-elasticsearch │ running │ Up 4 hours
so-idstools │ running │ Up 4 hours
so-influxdb │ running │ Up 4 hours (healthy)
so-kibana │ running │ Up 39 minutes
so-kratos │ running │ Up 4 hours
so-logstash │ running │ Up 3 hours
so-mysql │ running │ Up 4 hours (healthy)
so-nginx │ running │ Up 4 hours (healthy)
so-playbook │ running │ Up 3 hours
so-redis │ running │ Up 3 hours
so-sensoroni │ running │ Up 4 hours
so-soc │ running │ Up 4 hours
so-soctopus │ running │ Up 3 hours
so-telegraf │ running │ Up 4 hours

❗ Check container logs and /opt/so/log for more details.

[root@system]# so-elastic-fleet-urls-update
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0
curl: (56) Recv failure: Connection reset by peer
Failed to query for current Fleet Server URLs...

[root@system]# docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
48b83e4f691e hb:5000/security-onion-solutions/so-kibana:2.4.60 "/usr/local/bin/so-k…" 41 minutes ago Up 41 minutes 0.0.0.0:5601->5601/tcp so-kibana
a8aa0fec9a5a hb:5000/security-onion-solutions/so-elastalert:2.4.60 "/opt/elastalert/run…" 16 hours ago Up 3 hours so-elastalert
f3b85d9b7738 hb:5000/security-onion-solutions/so-telegraf:2.4.60 "/entrypoint.sh tele…" 40 hours ago Up 4 hours so-telegraf
6b5ab7aaa7b5 hb:5000/security-onion-solutions/so-influxdb:2.4.60 "/redirect_to_file.s…" 40 hours ago Up 4 hours (healthy) 0.0.0.0:8086->8086/tcp so-influxdb
5b2a82e81fc8 hb:5000/security-onion-solutions/so-logstash:2.4.60 "/usr/local/bin/dock…" 44 hours ago Up 3 hours 0.0.0.0:3765->3765/tcp, 0.0.0.0:5044->5044/tcp, 0.0.0.0:5055-5056->5055-5056/tcp, 0.0.0.0:5644->5644/tcp, 0.0.0.0:6050-6053->6050-6053/tcp, 0.0.0.0:9600->9600/tcp so-logstash
9cdb464c0336 hb:5000/security-onion-solutions/so-elasticsearch:2.4.60 "/bin/tini -- /usr/l…" 44 hours ago Up 4 hours 0.0.0.0:9200->9200/tcp, 0.0.0.0:9300->9300/tcp so-elasticsearch
db2e2fa68786 hb:5000/security-onion-solutions/so-soc:2.4.60 "/opt/sensoroni/sens…" 44 hours ago Up 4 hours 0.0.0.0:9822->9822/tcp so-soc
820d39150b71 hb:5000/security-onion-solutions/so-nginx:2.4.60 "/docker-entrypoint.…" 4 days ago Up 4 hours (healthy) 0.0.0.0:80->80/tcp, 0.0.0.0:443->443/tcp, 0.0.0.0:7788->7788/tcp, 0.0.0.0:8443->8443/tcp so-nginx
c7f3edb2db29 hb:5000/security-onion-solutions/so-redis:2.4.60 "redis-server /usr/l…" 4 days ago Up 3 hours 0.0.0.0:6379->6379/tcp, 0.0.0.0:9696->9696/tcp so-redis
866481389cc0 hb:5000/security-onion-solutions/so-mysql:2.4.60 "/entrypoint.sh mysq…" 4 days ago Up 4 hours (healthy) 0.0.0.0:3306->3306/tcp, 33060/tcp so-mysql
74b16c50e276 hb:5000/security-onion-solutions/so-soctopus:2.4.60 "gunicorn -b 0.0.0.0…" 4 days ago Up 3 hours 0.0.0.0:7000->7000/tcp so-soctopus
aa3abb79279e hb:5000/security-onion-solutions/so-playbook:2.4.60 "/docker-entrypoint.…" 3 weeks ago Up 3 hours 0.0.0.0:3000->3000/tcp so-playbook
a10b505d7174 hb:5000/security-onion-solutions/so-elastic-fleet-package-registry:2.4.60 "./package-registry" 3 weeks ago Up 3 hours (healthy) 0.0.0.0:8080->8080/tcp so-elastic-fleet-package-registry
c9457809049d hb:5000/security-onion-solutions/so-idstools:2.4.60 "./entrypoint.sh" 3 weeks ago Up 4 hours so-idstools
8fd71a0fd4b6 hb:5000/security-onion-solutions/so-soc:2.4.60 "/opt/sensoroni/sens…" 3 weeks ago Up 4 hours so-sensoroni
3b8d3f02a500 hb:5000/security-onion-solutions/so-kratos:2.4.60 "/start-kratos.sh" 3 weeks ago Up 4 hours 0.0.0.0:4433-4434->4433-4434/tcp so-kratos
b6551b48edc9 ghcr.io/security-onion-solutions/registry:2.8.2 "/entrypoint.sh /etc…" 3 weeks ago Up 4 hours 0.0.0.0:5000->5000/tcp so-dockerregistry
[root@system]#

TOoSmOotH Apr 13, 2024
Maintainer

Make sure that the true or false that you have for send events to manager is a global setting and not applied to a specific node. Otherwise the other nodes will not pick up the change.

alveare1 Apr 14, 2024
Author

root@system]# so-elastic-agent-status
┌─ fleet
│ └─ status: (FAILED) fail to checkin to fleet-server: all hosts failed: 2 errors occurred:
│ * requester 0/2 to host https://fleet:8220/ errored: Post "https://fleet:8220/api/fleet/agents/939fe7e4-a9d0-40aa-aed9-3ec93039716e/checkin?": lookup fleet on 100.100.100.100:53: no such host
│ * requester 1/2 to host https://manager:8220/ errored: Post "https://manager:8220/api/fleet/agents/939fe7e4-a9d0-40aa-aed9-3ec93039716e/checkin?": dial tcp [::1]:8220: connect: cannot assign requested address
│
│
└─ elastic-agent
└─ status: (HEALTHY) Running

fleet address is wrong and when I am curling the manager

[root@system]# curl https://manager:8220
curl: (35) error:0A000126:SSL routines::unexpected eof while reading

I am expecting a 404

I am not sure what happened to the fleet docker. I have restarted the system with no change.

Kibana
so-elastic-fleet-restart.txt

alveare1 Apr 15, 2024
Author

I can't seem to repair the fleet container. kibana is waiting on it. iptables is clear and allowing access but fleet is reseting the connection or eof while reading.

m0duspwnens · 2024-04-11T17:37:36Z

m0duspwnens
Apr 11, 2024
Maintainer

The documentation for receiver nodes has been updated to explain this feature. https://docs.securityonion.net/en/2.4/architecture.html#receiver-node

0 replies

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Force Forward node to talk to Receiver Node #12777

Uh oh!

{{title}}

Uh oh!

Replies: 2 comments 6 replies

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{editor}}'s edit

{{editor}}'s edit

Uh oh!

Select a reply

Uh oh!

Force Forward node to talk to Receiver Node #12777

Uh oh!

alveare1 Apr 11, 2024

Version

Installation Method

Description

Installation Type

Location

Hardware Specs

CPU

RAM

Storage for /

Storage for /nsm

Network Traffic Collection

Network Traffic Speeds

Status

Salt Status

Logs

Detail

Guidelines

Replies: 2 comments · 6 replies

Uh oh!

TOoSmOotH Apr 11, 2024 Maintainer

Uh oh!

alveare1 Apr 13, 2024 Author

Uh oh!

alveare1 Apr 13, 2024 Author

[root@system]# so-elastic-fleet-restart

This could take a while if another Salt job is running. Run this command with --force to stop all Salt jobs before proceeding.

[root@system]# so-elastic-fleet-start

This could take a while if another Salt job is running. Run this command with --force to stop all Salt jobs before proceeding.

Uh oh!

TOoSmOotH Apr 13, 2024 Maintainer

Uh oh!

alveare1 Apr 14, 2024 Author

Uh oh!

alveare1 Apr 15, 2024 Author

Uh oh!

Uh oh!

m0duspwnens Apr 11, 2024 Maintainer

alveare1
Apr 11, 2024

Replies: 2 comments 6 replies

TOoSmOotH
Apr 11, 2024
Maintainer

alveare1 Apr 13, 2024
Author

alveare1 Apr 13, 2024
Author

This could take a while if another Salt job is running.
Run this command with --force to stop all Salt jobs before proceeding.

This could take a while if another Salt job is running.
Run this command with --force to stop all Salt jobs before proceeding.

TOoSmOotH Apr 13, 2024
Maintainer

alveare1 Apr 14, 2024
Author

alveare1 Apr 15, 2024
Author

m0duspwnens
Apr 11, 2024
Maintainer