Failed to connect to /usr/local/nagios/var/rw/livestatus: Connection refused at /usr/share/thruk/lib/Monitoring/Livestatus/UNIX.pm line 72

[kaushalshriyan]

I am running nagios version 4.5.1 (https://www.nagios.org/projects/nagios-core/4x/)

# rpm -qa | grep thruk
libthruk-3.00-0.x86_64
thruk-base-3.14.2-12970.1.x86_64
thruk-3.14.2-12970.1.x86_64
thruk-plugin-reporting-3.14.2-12970.1.x86_64
# cat /etc/redhat-release
Red Hat Enterprise Linux release 8.9 (Ootpa)
#

# pwd
/usr/local/nagios/var/rw
# ls -l
total 0
srw-rw---- 1 nagios nagios 0 Apr 16 08:30 livestatus
prw-rw---- 1 nagios nagios 0 Apr 16 08:30 nagios.cmd
srw-rw---- 1 nagios nagios 0 Apr 16 08:30 nagios.qh
# echo "GET status" | /usr/local/bin/unixcat /usr/local/nagios/var/rw/livestatus
accept_passive_host_checks;accept_passive_service_checks;cached_log_messages;check_external_commands;check_host_freshness;check_service_freshness;connections;connections_rate;enable_event_handlers;enable_flap_detection;enable_notifications;execute_host_checks;execute_service_checks;external_command_buffer_max;external_command_buffer_slots;external_command_buffer_usage;external_commands;external_commands_rate;forks;forks_rate;host_checks;host_checks_rate;interval_length;last_command_check;last_log_rotation;livecheck_overflows;livecheck_overflows_rate;livechecks;livechecks_rate;livestatus_active_connections;livestatus_queued_connections;livestatus_threads;livestatus_version;log_messages;log_messages_rate;mk_inventory_last;nagios_pid;neb_callbacks;neb_callbacks_rate;num_hosts;num_services;obsess_over_hosts;obsess_over_services;process_performance_data;program_start;program_version;requests;requests_rate;service_checks;service_checks_rate
1;1;0;1;0;1;4;0.0321792;1;1;1;1;1;0;0;0;0;0;0;0;1;0.0107264;60;0;0;0;0;0;0;1;0;10;1.5.0p23;12;0.128717;0;10926;52;0.914182;1;1;0;0;0;1713236448;4.5.1;4;0.0321792;1;0.084375
#

event_broker_options=-1
broker_module=/opt/local/livestatus/livestatus.o /usr/local/nagios/var/rw/livestatus

# /usr/local/nagios/bin/nagios -v /usr/local/nagios/etc/nagios.cfg

Nagios Core 4.5.1
Copyright (c) 2009-present Nagios Core Development Team and Community Contributors
Copyright (c) 1999-2009 Ethan Galstad
Last Modified: 2024-02-28
License: GPL

Website: https://www.nagios.org
Reading configuration data...
   Read main config file okay...
   Read object config files okay...

Running pre-flight check on configuration data...

Checking objects...
	Checked 8 services.
	Checked 1 hosts.
	Checked 1 host groups.
	Checked 0 service groups.
	Checked 1 contacts.
	Checked 1 contact groups.
	Checked 24 commands.
	Checked 5 time periods.
	Checked 0 host escalations.
	Checked 0 service escalations.
Checking for circular paths...
	Checked 1 hosts
	Checked 0 service dependencies
	Checked 0 host dependencies
	Checked 5 timeperiods
Checking global event handlers...
Checking obsessive compulsive processor commands...
Checking misc settings...

Total Warnings: 0
Total Errors:   0

Things look okay - No serious problems were detected during the pre-flight check
#

# systemctl status nagios.service
● nagios.service - Nagios Core 4.5.1
   Loaded: loaded (/usr/lib/systemd/system/nagios.service; enabled; vendor preset: disabled)
   Active: active (running) since Tue 2024-04-16 08:30:48 IST; 2min 40s ago
     Docs: https://www.nagios.org/documentation
  Process: 10791 ExecStopPost=/usr/bin/rm -f /usr/local/nagios/var/rw/nagios.cmd (code=exited, status=0/SUCCESS)
  Process: 10924 ExecStart=/usr/local/nagios/bin/nagios -d /usr/local/nagios/etc/nagios.cfg (code=exited, status=0/SUCCESS)
  Process: 10923 ExecStartPre=/usr/local/nagios/bin/nagios -v /usr/local/nagios/etc/nagios.cfg (code=exited, status=0/SUCCESS)
 Main PID: 10926 (nagios)
    Tasks: 17 (limit: 48838)
   Memory: 7.0M
   CGroup: /system.slice/nagios.service
           ├─10926 /usr/local/nagios/bin/nagios -d /usr/local/nagios/etc/nagios.cfg
           ├─10927 /usr/local/nagios/bin/nagios --worker /usr/local/nagios/var/rw/nagios.qh
           ├─10928 /usr/local/nagios/bin/nagios --worker /usr/local/nagios/var/rw/nagios.qh
           ├─10929 /usr/local/nagios/bin/nagios --worker /usr/local/nagios/var/rw/nagios.qh
           ├─10930 /usr/local/nagios/bin/nagios --worker /usr/local/nagios/var/rw/nagios.qh
           └─10935 /usr/local/nagios/bin/nagios -d /usr/local/nagios/etc/nagios.cfg

Apr 16 08:30:49 niranjan nagios[10926]: livestatus: TIMEPERIOD TRANSITION: 24x7_sans_holidays;-1;1
Apr 16 08:30:49 niranjan nagios[10926]: livestatus: TIMEPERIOD TRANSITION: none;-1;0
Apr 16 08:30:49 niranjan nagios[10926]: livestatus: TIMEPERIOD TRANSITION: us-holidays;-1;0
Apr 16 08:30:49 niranjan nagios[10926]: livestatus: TIMEPERIOD TRANSITION: workhours;-1;0
Apr 16 08:30:49 niranjan nagios[10926]: livestatus: Timeperiod cache not updated, there are no timeperiods (yet)
Apr 16 08:30:49 niranjan nagios[10926]: livestatus: starting main thread and 10 client threads
Apr 16 08:30:49 niranjan nagios[10926]: livestatus: default stack size is 8388608
Apr 16 08:30:49 niranjan nagios[10926]: livestatus: setting thread stack size to 1048576
Apr 16 08:31:58 niranjan nagios[10926]: livestatus: Timeperiod cache not updated, there are no timeperiods (yet)
Apr 16 08:32:58 niranjan nagios[10926]: livestatus: Timeperiod cache not updated, there are no timeperiods (yet)
#

I am encountering the below issue whenever I restart

`491: failed to connect - failed to connect to /usr/local/nagios/var/rw/livestatus: Connection refused at /usr/share/thruk/lib/Monitoring/Livestatus/UNIX.pm line 72.              

	eval {...} called at /usr/share/thruk/lib/Monitoring/Livestatus/UNIX.pm line 64
	Monitoring::Livestatus::UNIX::_open(Monitoring::Livestatus::UNIX=HASH(0x55c2a9d93f70)) called at /usr/share/thruk/lib/Monitoring/Livestatus.pm line 971
	Monitoring::Livestatus::_open(Monitoring::Livestatus=HASH(0x55c2a92c7618)) called at /usr/share/thruk/lib/Monitoring/Livestatus.pm line 1143
	Monitoring::Livestatus::_send_socket_do(Monitoring::Livestatus=HASH(0x55c2a92c7618), "GET status\x{a}Columns: accept_passive_host_checks accept_passive_service_checks check_external_commands check_host_freshness check_service_freshness enable_event_handlers enable_flap_detection enable_notifications execute_host_checks execute_service_checks last_command_check last_log_rotation livestatus_version nagios_pid obsess_over_hosts obsess_over_services process_performance_data program_start program_version interval_length\x{a}OutputFormat: json\x{a}ResponseHeader: fixed16\x{a}") called at /usr/share/thruk/lib/Monitoring/Livestatus.pm line 1111
	eval {...} called at /usr/share/thruk/lib/Monitoring/Livestatus.pm line 1101
	Monitoring::Livestatus::_send_socket(Monitoring::Livestatus=HASH(0x55c2a92c7618), "GET status\x{a}Columns: accept_passive_host_checks accept_passive_service_checks check_external_commands check_host_freshness check_service_freshness enable_event_handlers enable_flap_detection enable_notifications execute_host_checks execute_service_checks last_command_check last_log_rotation livestatus_version nagios_pid obsess_over_hosts obsess_over_services process_performance_data program_start program_version interval_length\x{a}OutputFormat: json\x{a}ResponseHeader: fixed16\x{a}") called at /usr/share/thruk/lib/Monitoring/Livestatus.pm line 823
	Monitoring::Livestatus::_send(Monitoring::Livestatus=HASH(0x55c2a92c7618), "GET status\x{a}Columns: accept_passive_host_checks accept_passive_service_checks check_external_commands check_host_freshness check_service_freshness enable_event_handlers enable_flap_detection enable_notifications execute_host_checks execute_service_checks last_command_check last_log_rotation livestatus_version nagios_pid obsess_over_hosts obsess_over_services process_performance_data program_start program_version interval_length", HASH(0x55c2a9d70fb8)) called at /usr/share/thruk/lib/Monitoring/Livestatus.pm line 302
	Monitoring::Livestatus::selectall_arrayref(Monitoring::Livestatus=HASH(0x55c2a92c7618), "GET status\x{a}Columns: accept_passive_host_checks accept_passive_service_checks check_external_commands check_host_freshness check_service_freshness enable_event_handlers enable_flap_detection enable_notifications execute_host_checks execute_service_checks last_command_check last_log_rotation livestatus_version nagios_pid obsess_over_hosts obsess_over_services process_performance_data program_start program_version interval_length", HASH(0x55c2a8d08788)) called at /usr/share/thruk/lib/Monitoring/Livestatus/Class/Lite.pm line 420
	Monitoring::Livestatus::Class::Lite::_execute(Monitoring::Livestatus::Class::Lite=HASH(0x55c2a8f6ddc0)) called at /usr/share/thruk/lib/Monitoring/Livestatus/Class/Lite.pm line 278
	Monitoring::Livestatus::Class::Lite::hashref_array(Monitoring::Livestatus::Class::Lite=HASH(0x55c2a8f6ddc0)) called at /usr/share/thruk/lib/Monitoring/Livestatus/Class/Lite.pm line 259
	Monitoring::Livestatus::Class::Lite::hashref_pk(Monitoring::Livestatus::Class::Lite=HASH(0x55c2a8f6ddc0), "peer_key") called at /usr/share/thruk/lib/Thruk/Backend/Provider/Livestatus.pm line 304
	Thruk::Backend::Provider::Livestatus::get_processinfo(Thruk::Backend::Provider::Livestatus=HASH(0x55c2a8e717a0)) called at /etc/thruk/plugins//plugins-enabled/conf/lib/Thruk/Controller/conf.pm line 903
	eval {...} called at /etc/thruk/plugins//plugins-enabled/conf/lib/Thruk/Controller/conf.pm line 896`

# cat nagios.log
[1713237093] Nagios 4.5.1 starting... (PID=12405)
[1713237093] Local time is Tue Apr 16 08:41:33 IST 2024
[1713237093] LOG VERSION: 2.0
[1713237093] qh: Socket '/usr/local/nagios/var/rw/nagios.qh' successfully initialized
[1713237093] qh: core query handler registered
[1713237093] qh: echo service query handler registered
[1713237093] qh: help for the query handler registered
[1713237093] wproc: Successfully registered manager as @wproc with query handler
[1713237093] wproc: Registry request: name=Core Worker 12408;pid=12408
[1713237093] wproc: Registry request: name=Core Worker 12409;pid=12409
[1713237093] wproc: Registry request: name=Core Worker 12410;pid=12410
[1713237093] wproc: Registry request: name=Core Worker 12411;pid=12411
[1713237093] livestatus: fl_socket_path=[/usr/local/nagios/var/rw/livestatus], fl_mkeventd_socket_path=[/usr/local/nagios/var/rw/mkeventd/status]
[1713237093] livestatus: Livestatus by Mathias Kettner started with PID 12405
[1713237093] livestatus: version 1.5.0p23 compiled Tue, 09 Apr 2024 21:24:02 +0530 on niranjan
[1713237093] livestatus: built with g++ (GCC) 8.5.0 20210514 (Red Hat 8.5.0-20), using C++11 regex engine
[1713237093] livestatus: Using socket at '/usr/local/nagios/var/rw/livestatus'
[1713237093] livestatus: Please visit us at http://mathias-kettner.de/
[1713237093] livestatus: Hint: Please try out OMD - the Open Monitoring Distribution
[1713237093] livestatus: Please visit OMD at http://omdistro.org
[1713237093] livestatus: removed old socket file /usr/local/nagios/var/rw/livestatus
[1713237093] livestatus: opened UNIX socket at /usr/local/nagios/var/rw/livestatus
[1713237093] livestatus: your event_broker_options are sufficient for livestatus..
[1713237093] livestatus: finished initialization, further log messages go to /usr/local/nagios/var/livestatus.log
[1713237093] Event broker module '/opt/local/livestatus/livestatus.o' initialized successfully.
[1713237094] Successfully launched command file worker with pid 12417
[1713237094] livestatus: TIMEPERIOD TRANSITION: 24x7;-1;1
[1713237094] livestatus: TIMEPERIOD TRANSITION: 24x7_sans_holidays;-1;1
[1713237094] livestatus: TIMEPERIOD TRANSITION: none;-1;0
[1713237094] livestatus: TIMEPERIOD TRANSITION: us-holidays;-1;0
[1713237094] livestatus: TIMEPERIOD TRANSITION: workhours;-1;0
[1713237094] livestatus: Timeperiod cache not updated, there are no timeperiods (yet)
[1713237094] livestatus: starting main thread and 10 client threads
[1713237094] livestatus: default stack size is 8388608
[1713237094] livestatus: setting thread stack size to 1048576
[1713237097] Caught SIGSEGV, shutting down...
[1713237097] Caught SIGTERM, shutting down...
#

Please guide me. Thanks in advance.

[sni]

the issue is Caught SIGSEGV, shutting down which means livestatus and nagios crashed. But again, that's more an issue for nagios or the livestatus module.

[kaushalshriyan]

@sni I have earlier used livestatus along with Nagios version3 using thruk web interface. It worked without any glitch. I am finding it difficult to make it work with Nagios4. Any pointers would be really really helpful. Thanks in advance.

[sni]

certainly, here are some pointers:

• https://github.com/NagiosEnterprises/nagioscore/issues the nagios core issue tracker
• the livestatus for nagios is here: https://github.com/Checkmk/checkmk although they don't have an issue tracker