Hostname of the manager unexplicably changed during update to 2.4.100 #13634

Spinae-SBE · 2024-09-09T13:18:17Z

Spinae-SBE
Sep 9, 2024

Version

2.4.100

Installation Method

Security Onion ISO image

Description

upgrading

Installation Type

Distributed

Location

on-prem with Internet access

Hardware Specs

Exceeds minimum requirements

CPU

4

RAM

32

Storage for /

100G

Storage for /nsm

200G

Network Traffic Collection

span port

Network Traffic Speeds

1Gbps to 10Gbps

Status

No, one or more services are failed (please provide detail below)

Salt Status

Yes, there are salt failures (please provide detail below)

Logs

No, there are no additional clues

Detail

[root@so-manager ~]# cat /etc/soversion
2.4.100

5 days ago, soup was run to update to the 2.4.100. This did not complete succesfully. It seems that in some places, the hostname it is looking for, has changed from so-manager to so-manager-tou. Question is: why?

After some troubleshooting and trial and error, I got most of the containers running, except for the following 2:

so-idstools
so-soc

[root@so-manager ~]# so-status

                          Security Onion Status
                         Container │ Status  │                    Details
───────────────────────────────────┼─────────┼────────────────────────────
                 so-dockerregistry │ running │                 Up 2 hours
                     so-elastalert │ running │           Up About an hour
                  so-elastic-fleet │ running │           Up About an hour
 so-elastic-fleet-package-registry │ running │ Up About an hour (healthy)
                  so-elasticsearch │ running │           Up About an hour
                       so-idstools │ missing │
                       so-influxdb │ running │ Up About an hour (healthy)
                         so-kibana │ running │           Up About an hour
                         so-kratos │ running │                 Up 2 hours
                       so-logstash │ running │           Up About an hour
                          so-nginx │ running │ Up About an hour (healthy)
                          so-redis │ running │           Up About an hour
                      so-sensoroni │ running │           Up About an hour
                            so-soc │ missing │
                       so-telegraf │ running │           Up About an hour

idstools

When trying to start the idstools, I get the following:

[root@so-manager ~]# so-restart idstools
=========================================================================
Restarting idstools...

This could take a while if another Salt job is running.
Run this command with --force to stop all Salt jobs before proceeding.
=========================================================================
Error response from daemon: No such container: so-idstools
Error response from daemon: No such container: so-idstools
[INFO    ] Loading fresh modules for state activity
[INFO    ] Running state [/opt/so/conf/idstools/etc] at time 12:39:31.607893
[INFO    ] Executing state file.directory for [/opt/so/conf/idstools/etc]
[INFO    ] The directory /opt/so/conf/idstools/etc is in the correct state
[INFO    ] Completed state [/opt/so/conf/idstools/etc] at time 12:39:31.609931 (duration_in_ms=2.038)
[INFO    ] Running state [/opt/so/conf/idstools/etc] at time 12:39:31.610205
[INFO    ] Executing state file.recurse for [/opt/so/conf/idstools/etc]
[ERROR   ] Rendering exception occurred
Traceback (most recent call last):
  File "/opt/saltstack/salt/lib/python3.10/site-packages/salt/utils/templates.py", line 468, in render_jinja_tmpl
    output = template.render(**decoded_context)
  File "/opt/saltstack/salt/lib/python3.10/site-packages/jinja2/environment.py", line 1304, in render
    self.environment.handle_exception()
  File "/opt/saltstack/salt/lib/python3.10/site-packages/jinja2/environment.py", line 939, in handle_exception
    raise rewrite_traceback_stack(source=source)
  File "<template>", line 2, in top-level template code
  File "/opt/saltstack/salt/lib/python3.10/site-packages/jinja2/environment.py", line 1408, in make_module
    return TemplateModule(self, ctx)
  File "/opt/saltstack/salt/lib/python3.10/site-packages/jinja2/environment.py", line 1540, in __init__
    body_stream = list(template.root_render_func(context))
  File "/var/cache/salt/minion/files/base/soc/merged.map.jinja", line 11, in top-level template code
    {% do DOCKER_EXTRA_HOSTS.append({GLOBALS.influxdb_host:pillar.node_data[GLOBALS.influxdb_host].ip}) %}
  File "/opt/saltstack/salt/lib/python3.10/site-packages/jinja2/sandbox.py", line 327, in getattr
    value = getattr(obj, attribute)
jinja2.exceptions.UndefinedError: 'dict object' has no attribute 'so-manager-tou'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/opt/saltstack/salt/lib/python3.10/site-packages/salt/utils/templates.py", line 211, in render_tmpl
    output = render_str(tmplstr, context, tmplpath)
  File "/opt/saltstack/salt/lib/python3.10/site-packages/salt/utils/templates.py", line 474, in render_jinja_tmpl
    raise SaltRenderError(f"Jinja variable {exc}{out}", line, tmplstr)
salt.exceptions.SaltRenderError: Jinja variable 'dict object' has no attribute 'so-manager-tou'
/var/cache/salt/minion/files/base/soc/merged.map.jinja(11):
---
[...]
{% from 'vars/globals.map.jinja' import GLOBALS %}
{% from 'soc/defaults.map.jinja' import SOCDEFAULTS with context %}
{% from 'elasticsearch/config.map.jinja' import ELASTICSEARCH_NODES %}
{% from 'manager/map.jinja' import MANAGERMERGED %}
{% set DOCKER_EXTRA_HOSTS = ELASTICSEARCH_NODES %}
{% do DOCKER_EXTRA_HOSTS.append({GLOBALS.influxdb_host:pillar.node_data[GLOBALS.influxdb_host].ip}) %}    <======================

{% set SOCMERGED = salt['pillar.get']('soc', SOCDEFAULTS, merge=true) %}

{% do SOCMERGED.config.server.update({'proxy': MANAGERMERGED.proxy}) %}
{% do SOCMERGED.config.server.update({'additionalCA': MANAGERMERGED.additionalCA}) %}
[...]
---
[ERROR   ] #### /opt/so/conf/idstools/etc/rulecat.conf ####
Unable to manage file: Jinja variable 'dict object' has no attribute 'so-manager-tou'
/var/cache/salt/minion/files/base/soc/merged.map.jinja(11):
---
[...]
{% from 'vars/globals.map.jinja' import GLOBALS %}
{% from 'soc/defaults.map.jinja' import SOCDEFAULTS with context %}
{% from 'elasticsearch/config.map.jinja' import ELASTICSEARCH_NODES %}
{% from 'manager/map.jinja' import MANAGERMERGED %}
{% set DOCKER_EXTRA_HOSTS = ELASTICSEARCH_NODES %}
{% do DOCKER_EXTRA_HOSTS.append({GLOBALS.influxdb_host:pillar.node_data[GLOBALS.influxdb_host].ip}) %}    <======================

{% set SOCMERGED = salt['pillar.get']('soc', SOCDEFAULTS, merge=true) %}

{% do SOCMERGED.config.server.update({'proxy': MANAGERMERGED.proxy}) %}
{% do SOCMERGED.config.server.update({'additionalCA': MANAGERMERGED.additionalCA}) %}
[...]
---
[INFO    ] Completed state [/opt/so/conf/idstools/etc] at time 12:39:34.296722 (duration_in_ms=2686.517)
[INFO    ] Running state [/opt/so/rules/nids/suri] at time 12:39:34.297082
[INFO    ] Executing state file.directory for [/opt/so/rules/nids/suri]
[INFO    ] The directory /opt/so/rules/nids/suri is in the correct state
[INFO    ] Completed state [/opt/so/rules/nids/suri] at time 12:39:34.299105 (duration_in_ms=2.024)
[INFO    ] Running state [/opt/so/rules/nids/suri/] at time 12:39:34.299430
[INFO    ] Executing state file.recurse for [/opt/so/rules/nids/suri/]
[INFO    ] The directory /opt/so/rules/nids/suri/ is in the correct state
[INFO    ] Completed state [/opt/so/rules/nids/suri/] at time 12:39:34.386610 (duration_in_ms=87.179)
[INFO    ] Running state [/opt/so/log/idstools] at time 12:39:34.386961
[INFO    ] Executing state file.directory for [/opt/so/log/idstools]
[INFO    ] The directory /opt/so/log/idstools is in the correct state
[INFO    ] Completed state [/opt/so/log/idstools] at time 12:39:34.388749 (duration_in_ms=1.789)
[INFO    ] Running state [/usr/sbin] at time 12:39:34.389026
[INFO    ] Executing state file.recurse for [/usr/sbin]
[INFO    ] The directory /usr/sbin is in the correct state
[INFO    ] Completed state [/usr/sbin] at time 12:39:34.476427 (duration_in_ms=87.401)
[INFO    ] Running state [/usr/sbin] at time 12:39:34.476864
[INFO    ] Executing state file.recurse for [/usr/sbin]
[INFO    ] The directory /usr/sbin is in the correct state
[INFO    ] Completed state [/usr/sbin] at time 12:39:34.699006 (duration_in_ms=222.14)
[INFO    ] Running state [/nsm/rules/detect-suricata/custom_file] at time 12:39:34.699346
[INFO    ] Executing state file.directory for [/nsm/rules/detect-suricata/custom_file]
[INFO    ] The directory /nsm/rules/detect-suricata/custom_file is in the correct state
[INFO    ] Completed state [/nsm/rules/detect-suricata/custom_file] at time 12:39:34.701251 (duration_in_ms=1.906)
[INFO    ] Running state [/nsm/rules/detect-suricata/custom_temp] at time 12:39:34.701541
[INFO    ] Executing state file.directory for [/nsm/rules/detect-suricata/custom_temp]
[INFO    ] The directory /nsm/rules/detect-suricata/custom_temp is in the correct state
[INFO    ] Completed state [/nsm/rules/detect-suricata/custom_temp] at time 12:39:34.703166 (duration_in_ms=1.626)
[INFO    ] Running state [/opt/so/conf/so-status/so-status.conf] at time 12:39:34.703479
[INFO    ] Executing state file.append for [/opt/so/conf/so-status/so-status.conf]
/opt/saltstack/salt/lib/python3.10/site-packages/salt/utils/psutil_compat.py:16: DeprecationWarning: Please stop importing 'salt.utils.psutil_compat' and instead import 'psutil' directly as there's no longer a need for a compatability layer. The 'salt.utils.psutil_compat' will go away on Salt 3008.0 (Argon).
  salt.utils.versions.warn_until(
[INFO    ] Executing command git in directory '/root'
[INFO    ] Executing command 'grep' in directory '/root'
[INFO    ] ['unless condition is true']
[INFO    ] Completed state [/opt/so/conf/so-status/so-status.conf] at time 12:39:36.164594 (duration_in_ms=1461.115)
[INFO    ] Running state [/opt/so/conf/so-status/so-status.conf] at time 12:39:36.165322
[INFO    ] Executing state file.uncomment for [/opt/so/conf/so-status/so-status.conf]
[INFO    ] Pattern already uncommented
[INFO    ] Completed state [/opt/so/conf/so-status/so-status.conf] at time 12:39:36.168793 (duration_in_ms=3.47)
[INFO    ] Running state [/usr/sbin/so-rule-update > /opt/so/log/idstools/download_cron.log 2>&1] at time 12:39:36.169068
[INFO    ] Executing state cron.present for [/usr/sbin/so-rule-update > /opt/so/log/idstools/download_cron.log 2>&1]
[INFO    ] Executing command 'crontab' in directory '/root'
[INFO    ] Cron /usr/sbin/so-rule-update > /opt/so/log/idstools/download_cron.log 2>&1 already present
[INFO    ] Completed state [/usr/sbin/so-rule-update > /opt/so/log/idstools/download_cron.log 2>&1] at time 12:39:36.174900 (duration_in_ms=5.831)
local:
----------
          ID: idstoolsdir
    Function: file.directory
        Name: /opt/so/conf/idstools/etc
      Result: True
     Comment: The directory /opt/so/conf/idstools/etc is in the correct state
     Started: 12:39:31.607893
    Duration: 2.038 ms
     Changes:
----------
          ID: idstoolsetcsync
    Function: file.recurse
        Name: /opt/so/conf/idstools/etc
      Result: False
     Comment: #### /opt/so/conf/idstools/etc/rulecat.conf ####
              Unable to manage file: Jinja variable 'dict object' has no attribute 'so-manager-tou'
              /var/cache/salt/minion/files/base/soc/merged.map.jinja(11):
              ---
              [...]
              {% from 'vars/globals.map.jinja' import GLOBALS %}
              {% from 'soc/defaults.map.jinja' import SOCDEFAULTS with context %}
              {% from 'elasticsearch/config.map.jinja' import ELASTICSEARCH_NODES %}
              {% from 'manager/map.jinja' import MANAGERMERGED %}
              {% set DOCKER_EXTRA_HOSTS = ELASTICSEARCH_NODES %}
              {% do DOCKER_EXTRA_HOSTS.append({GLOBALS.influxdb_host:pillar.node_data[GLOBALS.influxdb_host].ip}) %}    <======================

              {% set SOCMERGED = salt['pillar.get']('soc', SOCDEFAULTS, merge=true) %}

              {% do SOCMERGED.config.server.update({'proxy': MANAGERMERGED.proxy}) %}
              {% do SOCMERGED.config.server.update({'additionalCA': MANAGERMERGED.additionalCA}) %}
              [...]
              ---
     Started: 12:39:31.610205
    Duration: 2686.517 ms
     Changes:
----------
          ID: rulesdir
    Function: file.directory
        Name: /opt/so/rules/nids/suri
      Result: True
     Comment: The directory /opt/so/rules/nids/suri is in the correct state
     Started: 12:39:34.297081
    Duration: 2.024 ms
     Changes:
----------
          ID: synclocalnidsrules
    Function: file.recurse
        Name: /opt/so/rules/nids/suri/
      Result: True
     Comment: The directory /opt/so/rules/nids/suri/ is in the correct state
     Started: 12:39:34.299431
    Duration: 87.179 ms
     Changes:
----------
          ID: idstoolslogdir
    Function: file.directory
        Name: /opt/so/log/idstools
      Result: True
     Comment: The directory /opt/so/log/idstools is in the correct state
     Started: 12:39:34.386960
    Duration: 1.789 ms
     Changes:
----------
          ID: idstools_sbin
    Function: file.recurse
        Name: /usr/sbin
      Result: True
     Comment: The directory /usr/sbin is in the correct state
     Started: 12:39:34.389026
    Duration: 87.401 ms
     Changes:
----------
          ID: idstools_sbin_jinja
    Function: file.recurse
        Name: /usr/sbin
      Result: True
     Comment: The directory /usr/sbin is in the correct state
     Started: 12:39:34.476866
    Duration: 222.14 ms
     Changes:
----------
          ID: suricatacustomdirsfile
    Function: file.directory
        Name: /nsm/rules/detect-suricata/custom_file
      Result: True
     Comment: The directory /nsm/rules/detect-suricata/custom_file is in the correct state
     Started: 12:39:34.699345
    Duration: 1.906 ms
     Changes:
----------
          ID: suricatacustomdirsurl
    Function: file.directory
        Name: /nsm/rules/detect-suricata/custom_temp
      Result: True
     Comment: The directory /nsm/rules/detect-suricata/custom_temp is in the correct state
     Started: 12:39:34.701540
    Duration: 1.626 ms
     Changes:
----------
          ID: append_so-idstools_so-status.conf
    Function: file.append
        Name: /opt/so/conf/so-status/so-status.conf
      Result: True
     Comment: unless condition is true
     Started: 12:39:34.703479
    Duration: 1461.115 ms
     Changes:
----------
          ID: so-idstools
    Function: docker_container.running
      Result: False
     Comment: One or more requisite failed: idstools.sync_files.idstoolsetcsync
     Started: 12:39:36.165219
    Duration: 0.004 ms
     Changes:
----------
          ID: delete_so-idstools_so-status.disabled
    Function: file.uncomment
        Name: /opt/so/conf/so-status/so-status.conf
      Result: True
     Comment: Pattern already uncommented
     Started: 12:39:36.165323
    Duration: 3.47 ms
     Changes:
----------
          ID: so-rule-update
    Function: cron.present
        Name: /usr/sbin/so-rule-update > /opt/so/log/idstools/download_cron.log 2>&1
      Result: True
     Comment: Cron /usr/sbin/so-rule-update > /opt/so/log/idstools/download_cron.log 2>&1 already present
     Started: 12:39:36.169069
    Duration: 5.831 ms
     Changes:
----------
          ID: run_so-rule-update
    Function: cmd.run
        Name: /usr/sbin/so-rule-update > /opt/so/log/idstools/download_idstools_state.log 2>&1
      Result: False
     Comment: One or more requisite failed: idstools.enabled.so-idstools, idstools.sync_files.idstoolsetcsync
     Started: 12:39:36.175638
    Duration: 0.004 ms
     Changes:

Summary for local
-------------
Succeeded: 11
Failed:     3
-------------
Total states run:     14
Total run time:    4.563 s

This mentions the so-manager-tou

 Comment: #### /opt/so/conf/idstools/etc/rulecat.conf ####
              Unable to manage file: Jinja variable 'dict object' has no attribute 'so-manager-tou'
              /var/cache/salt/minion/files/base/soc/merged.map.jinja(11):
              ---

However, the hostname is so-manager

[root@so-manager ~]# cat /etc/hosts
127.0.0.1   so-manager so-manager.localdomain localhost localhost.localdomain localhost4 localhost4.localdomain
::1   so-manager so-manager.localdomain localhost localhost.localdomain localhost6 localhost6.localdomain6

[root@so-manager ~]# hostname
so-manager

I have searched (grep -r) all over the filesystem for files where this so-manager-tou is mentioned, but to no avail except for the log files. Not a single configuration file or anything...

soc

When trying to start the soc, I get the following:

[root@so-manager ~]# so-restart soc
=========================================================================
Restarting soc...

This could take a while if another Salt job is running.
Run this command with --force to stop all Salt jobs before proceeding.
=========================================================================
Error response from daemon: No such container: so-soc
Error response from daemon: No such container: so-soc
[INFO    ] Loading fresh modules for state activity
[ERROR   ] Rendering exception occurred
Traceback (most recent call last):
  File "/opt/saltstack/salt/lib/python3.10/site-packages/salt/utils/templates.py", line 468, in render_jinja_tmpl
    output = template.render(**decoded_context)
  File "/opt/saltstack/salt/lib/python3.10/site-packages/jinja2/environment.py", line 1304, in render
    self.environment.handle_exception()
  File "/opt/saltstack/salt/lib/python3.10/site-packages/jinja2/environment.py", line 939, in handle_exception
    raise rewrite_traceback_stack(source=source)
  File "<template>", line 6, in top-level template code
  File "/opt/saltstack/salt/lib/python3.10/site-packages/jinja2/environment.py", line 1408, in make_module
    return TemplateModule(self, ctx)
  File "/opt/saltstack/salt/lib/python3.10/site-packages/jinja2/environment.py", line 1540, in __init__
    body_stream = list(template.root_render_func(context))
  File "/var/cache/salt/minion/files/base/soc/merged.map.jinja", line 11, in top-level template code
    {% do DOCKER_EXTRA_HOSTS.append({GLOBALS.influxdb_host:pillar.node_data[GLOBALS.influxdb_host].ip}) %}
  File "/opt/saltstack/salt/lib/python3.10/site-packages/jinja2/sandbox.py", line 327, in getattr
    value = getattr(obj, attribute)
jinja2.exceptions.UndefinedError: 'dict object' has no attribute 'so-manager-tou'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/opt/saltstack/salt/lib/python3.10/site-packages/salt/utils/templates.py", line 211, in render_tmpl
    output = render_str(tmplstr, context, tmplpath)
  File "/opt/saltstack/salt/lib/python3.10/site-packages/salt/utils/templates.py", line 474, in render_jinja_tmpl
    raise SaltRenderError(f"Jinja variable {exc}{out}", line, tmplstr)
salt.exceptions.SaltRenderError: Jinja variable 'dict object' has no attribute 'so-manager-tou'
/var/cache/salt/minion/files/base/soc/merged.map.jinja(11):
---
[...]
{% from 'vars/globals.map.jinja' import GLOBALS %}
{% from 'soc/defaults.map.jinja' import SOCDEFAULTS with context %}
{% from 'elasticsearch/config.map.jinja' import ELASTICSEARCH_NODES %}
{% from 'manager/map.jinja' import MANAGERMERGED %}
{% set DOCKER_EXTRA_HOSTS = ELASTICSEARCH_NODES %}
{% do DOCKER_EXTRA_HOSTS.append({GLOBALS.influxdb_host:pillar.node_data[GLOBALS.influxdb_host].ip}) %}    <======================

{% set SOCMERGED = salt['pillar.get']('soc', SOCDEFAULTS, merge=true) %}

{% do SOCMERGED.config.server.update({'proxy': MANAGERMERGED.proxy}) %}
{% do SOCMERGED.config.server.update({'additionalCA': MANAGERMERGED.additionalCA}) %}
[...]
---
[CRITICAL] Rendering SLS 'base:soc' failed: Jinja variable 'dict object' has no attribute 'so-manager-tou'
/var/cache/salt/minion/files/base/soc/merged.map.jinja(11):
---
[...]
{% from 'vars/globals.map.jinja' import GLOBALS %}
{% from 'soc/defaults.map.jinja' import SOCDEFAULTS with context %}
{% from 'elasticsearch/config.map.jinja' import ELASTICSEARCH_NODES %}
{% from 'manager/map.jinja' import MANAGERMERGED %}
{% set DOCKER_EXTRA_HOSTS = ELASTICSEARCH_NODES %}
{% do DOCKER_EXTRA_HOSTS.append({GLOBALS.influxdb_host:pillar.node_data[GLOBALS.influxdb_host].ip}) %}    <======================

{% set SOCMERGED = salt['pillar.get']('soc', SOCDEFAULTS, merge=true) %}

{% do SOCMERGED.config.server.update({'proxy': MANAGERMERGED.proxy}) %}
{% do SOCMERGED.config.server.update({'additionalCA': MANAGERMERGED.additionalCA}) %}
[...]
---
local:
    Data failed to compile:
----------
    Rendering SLS 'base:soc' failed: Jinja variable 'dict object' has no attribute 'so-manager-tou'
/var/cache/salt/minion/files/base/soc/merged.map.jinja(11):
---
[...]
{% from 'vars/globals.map.jinja' import GLOBALS %}
{% from 'soc/defaults.map.jinja' import SOCDEFAULTS with context %}
{% from 'elasticsearch/config.map.jinja' import ELASTICSEARCH_NODES %}
{% from 'manager/map.jinja' import MANAGERMERGED %}
{% set DOCKER_EXTRA_HOSTS = ELASTICSEARCH_NODES %}
{% do DOCKER_EXTRA_HOSTS.append({GLOBALS.influxdb_host:pillar.node_data[GLOBALS.influxdb_host].ip}) %}    <======================

{% set SOCMERGED = salt['pillar.get']('soc', SOCDEFAULTS, merge=true) %}

{% do SOCMERGED.config.server.update({'proxy': MANAGERMERGED.proxy}) %}
{% do SOCMERGED.config.server.update({'additionalCA': MANAGERMERGED.additionalCA}) %}
[...]
---

Also here this so-manager-tou is popping up.

[CRITICAL] Rendering SLS 'base:soc' failed: Jinja variable 'dict object' has no attribute 'so-manager-tou'
/var/cache/salt/minion/files/base/soc/merged.map.jinja(11):

When doing docker ps before I started troubleshooting, it showed (exerpt I copied at the start of my troubleshooting)

002c32f12e4e   so-manager:5000/security-onion-solutions/so-elastic-fleet-package-registry:2.4.90   "./package-registry"     5 weeks ago      Exited (0) 5 days ago                                                                                                                                                                          so-elastic-fleet-package-registry

As you can see, it tried to find so-manager:5000
After the update using soup to try and get to 2.4.100, the docker ps output now shows so-manager-tou. I still have no clue why.

This is after the upgrade to 2.4.100, after some troubleshooting en trial and error where I got all but 2 containers working:

[root@so-manager ~]# docker ps
CONTAINER ID   IMAGE                                                                                    COMMAND                  CREATED             STATUS                 PORTS                                                                                                                                                                NAMES
b0474939ae95   so-manager-tou:5000/security-onion-solutions/so-elastalert:2.4.100                       "/opt/elastalert/run…"   About an hour ago   Up About an hour

Below also an error I found, pertaining to so-manager-tou:

Comment: Failed to pull so-manager-tou:5000/security-onion-solutions/so-strelka-filestream:2.4.100: Error 500: Get "https://so-manager-tou:5000/v2/": tls: failed to verify certificate: x509: certificate is valid for so-manager, not so-manager-tou

Here it talks about x509 certificate which (I assume during initial setup) has been created for so-manager hostname, and not for so-manager-tou.

x509 certificate

Going down the x509 rabbit hole (and leaving some bits out for readability and security reasons):

[root@so-manager ~]# salt-call grains.get id
local:
    so-manager_manager
[root@so-manager ~]# salt-call mine.get so-manager_manager x509.get_pem_entries
local:
    ----------
    so-manager_manager:
        ----------
        /etc/pki/ca.crt:
            -----BEGIN CERTIFICATE-----
            MIIFhTCCA22gAwIBAgIUQI511x47IJ2p+P+JijuIXS/QUkwwDQYJKoZIhvcNAQEL
            BQAwSjELMAkGA1UEBhMCVVMxDTALBgNVBAgMBFV0YWgxFzAVBgNVBAcMDlNhbHQg
            TGFrZSBDaXR5MRMwEQYDVQQDDApzby1tYW5hZ2VyMB4XDTI0MDUxNTEwMjcxMVoX
            DTM0MDUxMzEwMjcxMVowSjELMAkGA1UEBhMCVVMxDTALBgNVBAgMBFV0YWgxFzAV
            BgNVBAcMDlNhbHQgTGFrZSBDaXR5MRMwEQYDVQQDDApzby1tYW5hZ2VyMIICIjAN
            BgkqhkiG9w0BAQEFAAOCAg8AMIICCgKCAgEAxV/2Dd/gS0xAX5mB5Zmaw9rlCAHE
            zIfncrZ2nfHohII5EXMO/zmb+dirRnPt8WRfN1NOnvP+TSf+7sDIpv4BXujdS9Qj
            K9fuCtwrpQw+C5o2tk1E9tFKI1v1cn4Rtlai9057thxXcwkYRl1w25Fwztn530nw
            tml1l6GxZWvGiJw6mSpX80pP/NAqZ+NjWdrHsC+BbZ4pEAI+ho5T+QzLrViXMZzv
            DfxBzNzKPyJdfnTD+czf3G+vSJo4JwIlu4Yg6cenF9WYubqQmMzjWsLw4cKUcftr
            EMigcwq9Idfh6G8LYj0LI8sg93RdQHBJF7LI8Xpng107TLADWqlcpJNIo39EtYxS
            +3P00smXppIIxd3o/ijWW1OG3cEEF7NG1RBvfLVXvVJ4kI0SX2YN96Fpy1Z1Y7Fs
            gxy2y6fg3WMTU5rQRayUNPd7eOhMfd2ofdK8Duxk2b6yxTeuBShe69bi5x/kdvBg
            h1KOouqxoZiSs0FXr2dYIBRSptQzPAwlGaDK0+oK3JyeeM6ooXYHIfDdR7PjjhPg
            Vm360k7SPO9CtS2+RzA9oS83EaRLokMkSjvSX0jOhSh3XGF0ocT5SR0GFgMtcxu7
            LkZvMMXg5pgVTMKvSdebrruJn0dQ/XgtiU2tyozAsTlxo4FcyfZaeUsFNysVCXK7
            ZC2Gq5Jyea57Kn8CAwEAAaNjMGEwDwYDVR0TAQH/BAUwAwEB/zAOBgNVHQ8BAf8E
            BAMCAQYwHQYDVR0OBBYEFCGQMiPcV27SVJn2y/o/DIjCrcHvMB8GA1UdIwQYMBaA
            FCGQMiPcV27SVJn2y/o/DIjCrcHvMA0GCSqGSIb3DQEBCwUAA4ICAQCF3f10LLnt
            1P4mCkeTzlzpL7/FHxvxzcv1KUjtu/nag/c6G9H72Jf05hA2KzJPNAGP1/o3+IwD
            //EDrx9XxFImuc1Au8HMktTM/PvWGiAu4bRp1KSq0XJ/CchrN2Joq/wK78048Tx4
            DueG+kVjjNY3MIOpBpExoCvB7qEo8FHjRtDuOGsb2t8ndBhCmuCkzYbXlK/1aEMy
            MCVQ7jwOmTLR3eQ58AO/OR+kiL1W493iTtybnpvZQFVfIJSx3SPgbKvqzKsHp0yW
            HI6efPl4grsClGamgZlYdF80oA/nKVMntSYd+kiS+xkFIb5hXbclVdNvNC1j5TJK
            ycMrqGJLvQiRQQ5sbJQqvqyODt5BMtxPARJaGZLjbTwxlOFFv/51JFtzHdCl/dIL
            uELXrL9sbpnxZnVNFvhgeHqYibBW2oT58jlAEGc4QN3wd4u9Rrx8p2AiVWaW/D3A
            E9OMUFbC7RRCqqPcgoITNaGV5DybmT/2jXveljnMTIn/yZxxnib+aJ+LvM0fMjuL
            v59TI+KdrJ63LxUZMubji1e0DeYMeKJYxXo8rMvReROTwv3Qs08gRkGqZDjsHXAb
            xk+npkRbhF1akrHzUvWWsoq1yijh/rzNaeBwHcyYSgyMDTXHsbxmRQuA9MMz8te8
            hDPaXemJzi3JHrdnzoL9cbOOusc/ZWVcWw==
            -----END CERTIFICATE-----

[root@so-manager ~]# openssl x509 -in /etc/pki/ca.crt -text -noout
Certificate:
    Data:
        Version: 3 (0x2)
        Serial Number:
            40:8e:75:d7
        Signature Algorithm: sha256WithRSAEncryption
        Issuer: C = US, ST = Utah, L = Salt Lake City, CN = so-manager
        Validity
            Not Before: May 15 10:27:11 2024 GMT
            Not After : May 13 10:27:11 2034 GMT
        Subject: C = US, ST = Utah, L = Salt Lake City, CN = so-manager
        Subject Public Key Info:
            Public Key Algorithm: rsaEncryption
                Public-Key: (4096 bit)
                Modulus:
                    00:c5:5f:f6:0d:df:e0:4b:4c:40:5f:99:81:
                    7b:2a:7f
                Exponent: 65537 (0x10001)
        X509v3 extensions:
            X509v3 Basic Constraints: critical
                CA:TRUE
            X509v3 Key Usage: critical
                Certificate Sign, CRL Sign
            X509v3 Subject Key Identifier:
                21:90:32:23:DC:57:6E:D2:54:99:F6:CB:FA:3F:0C:88:C2:AD:C1:EF
            X509v3 Authority Key Identifier:
                21:90:32:23:DC:57:6E:D2:54:99:F6:CB:FA:3F:0C:88:C2:AD:C1:EF
    Signature Algorithm: sha256WithRSAEncryption
    Signature Value:
        85:dd:fd:74:2c:b9:ed:d4:fe:26:0a:47:93:ce:5c:b3:
        8e:ba:c7:3f:65:65:5c:5b

Here it mentions the so-manager name, not the so-manager-tou

The one thing I can think of right now in the direction of so-manager-tou is the following: I have found out that, in the local DNS server in the network, there is an A-record for so-manager-tou, but there is no A-record for so-manager. Has there been some reverse DNS lookup, resulting in the new name (with the -tou appended)? I have no idea.

I have verified: the manager has been installed with so-manager as hostname. There has not been a change of hostname (taking into account https://docs.securityonion.net/en/2.4/hostname.html)

Any help would be highly appreciated, because I am lost here...

Guidelines

I have read the discussion guidelines at Read before posting! #1720 and assert that I have followed the guidelines.

Answered by Spinae-SBE

Sep 9, 2024

And also the last one which refused to start earlier, now is able to start:

[root@so-manager etc]# so-status

                       Security Onion Status
                         Container │ Status  │              Details
───────────────────────────────────┼─────────┼──────────────────────
                 so-dockerregistry │ running │           Up 2 hours
                     so-elastalert │ running │           Up 2 hours
                  so-elastic-fleet │ running │           Up 2 hours
 so-elastic-fleet-package-registry │ running │ Up 2 hours (healthy)
                  so-elasticsearch │ running │           Up 2 hours
                       so-idstools │ running │         Up 5 minut…

View full answer

Spinae-SBE · 2024-09-09T13:25:29Z

Spinae-SBE
Sep 9, 2024
Author

And, as always, you are searching for hours on end without finding anything. You decide to start a discussion on github, gather as much usefull info as possible, you file the case and POOF : progress!

Seems I overlooked the following file where this so-manager-tou IS mentioned.

[root@so-manager etc]# cat /opt/so/saltstack/local/pillar/global/soc_global.sls
global:
    airgap: false
    endgamehost: ""
    fleet_grid_enrollment_token_general: ekpYamU0OEJ0=
    fleet_grid_enrollment_token_heavy: enBYamU0O=
    ids: Suricata
    imagerepo: security-onion-solutions
    influxdb_host: so-manager-tou
    managerip: 10.X.Y.240
    mdengine: ZEEK
    registry_host: so-manager-tou
    repo_host: so-manager-tou
    soversion: 2.4.100
    url_base: 10.X.Y..240

[root@so-manager etc]# so-status

                       Security Onion Status
                         Container │ Status  │              Details
───────────────────────────────────┼─────────┼──────────────────────
                 so-dockerregistry │ running │           Up 2 hours
                     so-elastalert │ running │           Up 2 hours
                  so-elastic-fleet │ running │           Up 2 hours
 so-elastic-fleet-package-registry │ running │ Up 2 hours (healthy)
                  so-elasticsearch │ running │           Up 2 hours
                       so-idstools │ running │         Up 3 minutes
                       so-influxdb │ running │ Up 2 hours (healthy)
                         so-kibana │ running │           Up 2 hours
                         so-kratos │ running │           Up 2 hours
                       so-logstash │ running │           Up 2 hours
                          so-nginx │ running │ Up 2 hours (healthy)
                          so-redis │ running │           Up 2 hours
                      so-sensoroni │ running │           Up 2 hours
                            so-soc │ missing │
                       so-telegraf │ running │           Up 2 hours

1 down, 1 to go...

0 replies

Spinae-SBE · 2024-09-09T13:29:58Z

Spinae-SBE
Sep 9, 2024
Author

And also the last one which refused to start earlier, now is able to start:

[root@so-manager etc]# so-status

                       Security Onion Status
                         Container │ Status  │              Details
───────────────────────────────────┼─────────┼──────────────────────
                 so-dockerregistry │ running │           Up 2 hours
                     so-elastalert │ running │           Up 2 hours
                  so-elastic-fleet │ running │           Up 2 hours
 so-elastic-fleet-package-registry │ running │ Up 2 hours (healthy)
                  so-elasticsearch │ running │           Up 2 hours
                       so-idstools │ running │         Up 5 minutes
                       so-influxdb │ running │ Up 2 hours (healthy)
                         so-kibana │ running │           Up 2 hours
                         so-kratos │ running │           Up 2 hours
                       so-logstash │ running │           Up 2 hours
                          so-nginx │ running │ Up 2 hours (healthy)
                          so-redis │ running │           Up 2 hours
                      so-sensoroni │ running │           Up 2 hours
                            so-soc │ running │        Up 34 seconds
                       so-telegraf │ running │           Up 2 hours

 ✔ This onion is ready to make your adversaries cry!

But my question remains: why did it suddenly wanted to use so-manager-tou instead of so-manager ?

Anyways, my setup seems to work again. I will not look any further.

Maybe someone has some use in the info I posted here. Have a nice one!

0 replies

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Hostname of the manager unexplicably changed during update to 2.4.100 #13634

Uh oh!

{{title}}

Uh oh!

Replies: 2 comments

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{editor}}'s edit

{{editor}}'s edit

Uh oh!

Uh oh!

{{title}}

Uh oh!

Select a reply

Uh oh!

Hostname of the manager unexplicably changed during update to 2.4.100 #13634

Uh oh!

Spinae-SBE Sep 9, 2024

Version

Installation Method

Description

Installation Type

Location

Hardware Specs

CPU

RAM

Storage for /

Storage for /nsm

Network Traffic Collection

Network Traffic Speeds

Status

Salt Status

Logs

Detail

idstools

soc

x509 certificate

Guidelines

Replies: 2 comments

Uh oh!

Uh oh!

Spinae-SBE Sep 9, 2024 Author

Uh oh!

Spinae-SBE Sep 9, 2024 Author

Spinae-SBE
Sep 9, 2024

Spinae-SBE
Sep 9, 2024
Author

Spinae-SBE
Sep 9, 2024
Author