SOUP 2.4.50 Power Loss

[8inaryMata1eao]

Well after years of SO I finally think I broke one. My distributed system is running 3 search nodes 3 senors and one manager spanning a few states in the US. I was running Soup and in the middle my work station(house) lost power. When I got back online and checked the grid everything seamed fine. I ran soup and it showed good to go. In the web dashboard all systems showed version 2.4.40 but if I typed "cat /etc/soversion" in the manager it reported 2.4.50. I rebooted the entire grid and to my surprise it never came back up. I went in and reset the socversion and /opt/so/saltstack/local/pillar/global/soc_global.sls to 2.4.40, and ran soup. It looked like it was working, downloading new images and containers. At the end errors. Then i ran so-docker-refresh only to see all failures all like " dial tcp 127.0.0.1:5000: connect: connection refused" . When executing docker ps it only shows so-elastichsearch running.

Any help would be greatly appreciated.

so-docker-refresh

Downloading so-strelka-filestream
Executing command with retry support: docker pull ghcr.io/security-onion-solutions/so-strelka-filestream:2.4.40
Results: 2.4.40: Pulling from security-onion-solutions/so-strelka-filestream
Digest: sha256:502b93434ba5842837c5a1ca45c7979b5b14c0772404905fed2da36a6f0203a1
Status: Image is up to date for ghcr.io/security-onion-solutions/so-strelka-filestream:2.4.40
ghcr.io/security-onion-solutions/so-strelka-filestream:2.4.40 (0)
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   566  100   566    0     0   3605      0 --:--:-- --:--:-- --:--:--  3605
The push refers to repository [so-mgr-01:5000/security-onion-solutions/so-strelka-filestream]
Get "http://so-mgr-01:5000/v2/": dial tcp 127.0.0.1:5000: connect: connection refused
Downloading so-strelka-frontend
Executing command with retry support: docker pull ghcr.io/security-onion-solutions/so-strelka-frontend:2.4.40
Results: 2.4.40: Pulling from security-onion-solutions/so-strelka-frontend
Digest: sha256:171302b4ce0797b61bfad668b92246daf6e599afb62a92fc17940afb8365e120
Status: Image is up to date for ghcr.io/security-onion-solutions/so-strelka-frontend:2.4.40
ghcr.io/security-onion-solutions/so-strelka-frontend:2.4.40 (0)
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   566  100   566    0     0   3699      0 --:--:-- --:--:-- --:--:--  3699
The push refers to repository [so-mgr-01:5000/security-onion-solutions/so-strelka-frontend]
Get "http://so-mgr-01:5000/v2/": dial tcp 127.0.0.1:5000: connect: connection refused
Downloading so-strelka-manager
Executing command with retry support: docker pull ghcr.io/security-onion-solutions/so-strelka-manager:2.4.40
Results: 2.4.40: Pulling from security-onion-solutions/so-strelka-manager
Digest: sha256:f19eedae8e81b9ed4598af02dc7b6780a9da9ee7bc2530078350e5a8f3bb75ea
Status: Image is up to date for ghcr.io/security-onion-solutions/so-strelka-manager:2.4.40
ghcr.io/security-onion-solutions/so-strelka-manager:2.4.40 (0)
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   566  100   566    0     0   3930      0 --:--:-- --:--:-- --:--:--  3930
The push refers to repository [so-mgr-01:5000/security-onion-solutions/so-strelka-manager]
Get "http://so-mgr-01:5000/v2/": dial tcp 127.0.0.1:5000: connect: connection refused
Downloading so-suricata
Executing command with retry support: docker pull ghcr.io/security-onion-solutions/so-suricata:2.4.40
Results: 2.4.40: Pulling from security-onion-solutions/so-suricata
Digest: sha256:329b6cade2c24e73757ad4a235a4e0ea0daa89ef6875502cb430a609a6783bc3
Status: Image is up to date for ghcr.io/security-onion-solutions/so-suricata:2.4.40
ghcr.io/security-onion-solutions/so-suricata:2.4.40 (0)
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   566  100   566    0     0   3252      0 --:--:-- --:--:-- --:--:--  3252
The push refers to repository [so-mgr-01:5000/security-onion-solutions/so-suricata]
Get "http://so-mgr-01:5000/v2/": dial tcp 127.0.0.1:5000: connect: connection refused
Downloading so-telegraf
Executing command with retry support: docker pull ghcr.io/security-onion-solutions/so-telegraf:2.4.40
Results: 2.4.40: Pulling from security-onion-solutions/so-telegraf
Digest: sha256:3555c530245182a99d8763074a3164093220c6f38e5554da18b9aa7da0821191
Status: Image is up to date for ghcr.io/security-onion-solutions/so-telegraf:2.4.40
ghcr.io/security-onion-solutions/so-telegraf:2.4.40 (0)
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   566  100   566    0     0   3699      0 --:--:-- --:--:-- --:--:--  3723
The push refers to repository [so-mgr-01:5000/security-onion-solutions/so-telegraf]
Get "http://so-mgr-01:5000/v2/": dial tcp 127.0.0.1:5000: connect: connection refused
Downloading so-zeek
Executing command with retry support: docker pull ghcr.io/security-onion-solutions/so-zeek:2.4.40
Results: 2.4.40: Pulling from security-onion-solutions/so-zeek
Digest: sha256:4691a99d3d76ad927aa2d80970d2b83be4899ac8b70f5d18c2b60171ea879ac2
Status: Image is up to date for ghcr.io/security-onion-solutions/so-zeek:2.4.40
ghcr.io/security-onion-solutions/so-zeek:2.4.40 (0)
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   566  100   566    0     0   5717      0 --:--:-- --:--:-- --:--:--  5717
The push refers to repository [so-mgr-01:5000/security-onion-solutions/so-zeek]
Get "http://so-mgr-01:5000/v2/": dial tcp 127.0.0.1:5000: connect: connection refused

soup

Performing upgrade from Security Onion 2.4.40 to Security Onion 2.4.50.


Stopping crond service at 19:51:46.470135
Successfully stopped crond.

Updating dockers to 2.4.50.
Error response from daemon: No such container: so-dockerregistry

Starting crond service at 19:51:46.563282
Successfully started crond.


Starting salt-master service at 19:51:46.594419
Successfully started salt-master.


Starting salt-minion service at 19:51:46.615495
Successfully started salt-minion.

Enabling highstate.
[INFO    ] Creating module dir '/var/cache/salt/minion/extmods/grains'
[INFO    ] Syncing grains for environment 'base'
[INFO    ] Loading cache from salt://_grains, for base
[INFO    ] Caching directory '_grains/' for environment 'base'
local:
    ----------
    msg:
        Info: highstate state already enabled.
    res:
        True

Soup failed with error 1: Unhandled error
Unhandled error occured, please check /root/soup.log for details.

docker ps

[root@so-mgr-01 ]# docker ps
CONTAINER ID   IMAGE                                                                 COMMAND                  CREATED          STATUS          PORTS                                            NAMES
62da22224f2e  so-mgr-01:5000/security-onion-solutions/so-elasticsearch:2.4.40   "/bin/tini -- /usr/l…"   40 minutes ago   Up 35 minutes   0.0.0.0:9200->9200/tcp, 0.0.0.0:9300->9300/tcp   so-elasticsearch

---

Rebooted this morning. After running soup I get a message below.

Checking if we can access the salt master and that it is ready at: Feb 22 15:09:15
Error running 'state.show_top': Pillar failed to render. Additional info follows:

- Specified SLS 'data.*' in environment 'base' is not available on the salt master
- Specified SLS 'global' in environment 'base' is not available on the salt master
Can't access salt master or it is not ready at: Feb 22 15:09:19
Gave up trying to access salt-master
ERROR: Could not talk to salt master: Please run 'systemctl status salt-master' to ensure the salt-master service is running and check the log at /opt/so/log/salt/master.
Exiting.

When running "systemctl status salt-master"

● salt-master.service - The Salt Master Server
     Loaded: loaded (/usr/lib/systemd/system/salt-master.service; enabled; preset: disabled)
     Active: active (running) since Thu 2024-02-22 13:56:30 UTC; 1h 13min ago
       Docs: man:salt-master(1)
             file:///usr/share/doc/salt/html/contents.html
             https://docs.saltproject.io/en/latest/contents.html
   Main PID: 844 (/opt/saltstack/)
      Tasks: 32 (limit: 150631)
     Memory: 783.6M
        CPU: 19min 48.785s
     CGroup: /system.slice/salt-master.service
             ├─   844 "/opt/saltstack/salt/bin/python3.10 /usr/bin/salt-master MainProcess"
             ├─  2308 "/opt/saltstack/salt/bin/python3.10 /usr/bin/salt-master PubServerChannel._publish_daemon"
             ├─  2309 "/opt/saltstack/salt/bin/python3.10 /usr/bin/salt-master EventPublisher"
             ├─  2326 "/opt/saltstack/salt/bin/python3.10 /usr/bin/salt-master Engine(salt.loaded.ext.engines.checkmine)"
             ├─  2328 "/opt/saltstack/salt/bin/python3.10 /usr/bin/salt-master ReqServer ReqServer_ProcessManager"
             ├─  2330 "/opt/saltstack/salt/bin/python3.10 /usr/bin/salt-master ReqServer MWorkerQueue"
             ├─  2334 "/opt/saltstack/salt/bin/python3.10 /usr/bin/salt-master ReqServer MWorker-0"
             ├─  2335 "/opt/saltstack/salt/bin/python3.10 /usr/bin/salt-master ReqServer MWorker-1"
             ├─  2342 "/opt/saltstack/salt/bin/python3.10 /usr/bin/salt-master ReqServer MWorker-2"
             ├─  2343 "/opt/saltstack/salt/bin/python3.10 /usr/bin/salt-master ReqServer MWorker-3"
             ├─  2345 "/opt/saltstack/salt/bin/python3.10 /usr/bin/salt-master ReqServer MWorker-4"
             ├─217881 "/opt/saltstack/salt/bin/python3.10 /usr/bin/salt-master Maintenance"
             └─220169 "/opt/saltstack/salt/bin/python3.10 /usr/bin/salt-master FileserverUpdate"

Feb 22 15:09:18 so-mgr-01 salt-master[2345]: **[CRITICAL] Pillar render error: Specified SLS 'global' in environment 'base' is not available on the salt master**
Feb 22 15:09:25 so-mgr-01 salt-master[2326]: [INFO    ] Runner completed: 20240222150925050182
Feb 22 15:09:26 so-mgr-01 salt-master[2326]: [INFO    ] Runner completed: 20240222150925505356
Feb 22 15:09:26 so-mgr-01 salt-master[2326]: [INFO    ] Runner completed: 20240222150926408156
Feb 22 15:09:27 so-mgr-01 salt-master[2326]: [INFO    ] Runner completed: 20240222150927206241
Feb 22 15:09:28 so-mgr-01 salt-master[2326]: [INFO    ] Runner completed: 20240222150928078312
Feb 22 15:09:30 so-mgr-01 salt-master[2326]: [INFO    ] Runner completed: 20240222150929581423
Feb 22 15:09:30 so-mgr-01 salt-master[2326]: [INFO    ] Runner completed: 20240222150930422540
Feb 22 15:09:31 so-mgr-01 salt-master[2326]: [INFO    ] Runner completed: 20240222150931239389
Feb 22 15:09:32 so-mgr-01 salt-master[2326]: [INFO    ] Runner completed: 20240222150932051125
Feb 22 15:15:11 so-mgr-01 salt-master[2345]: [INFO    ] Got return from so-sch-02_searchnode for job 20240222151511905309
Feb 22 15:15:12 so-mgr-01 salt-master[2334]: [INFO    ] Runner completed: 20240222151512538502
Feb 22 15:15:13 so-mgr-01 salt-master[2334]: [INFO    ] Runner completed: 20240222151512971639
Feb 22 15:15:13 so-mgr-01 salt-master[2334]: [ERROR   ] Specified SLS 'zeeklogs' in environment 'base' is not available on the salt master
Feb 22 15:15:13 so-mgr-01 salt-master[2334]: [ERROR   ] Template was specified incorrectly: False
Feb 22 15:15:13 so-mgr-01 salt-master[2334]: [ERROR   ] Specified SLS 'global' in environment 'base' is not available on the salt master
Feb 22 15:15:13 so-mgr-01 salt-master[2334]: [ERROR   ] Template was specified incorrectly: False
Feb 22 15:15:13 so-mgr-01 salt-master[2334]: [CRITICAL] Pillar render error: Specified SLS 'zeeklogs' in environment 'base' is not available on the salt master
Feb 22 15:15:13 so-mgr-01 salt-master[2334]: [CRITICAL] Pillar render error: Specified SLS 'global' in environment 'base' is not available on the salt master
Feb 22 15:15:13 so-mgr-01 salt-master[2343]: [INFO    ] Got return from so-fwd-02_sensor for job 20240222151513734547

[cm-ops]

What does cat /opt/so/saltstack/default/salt/top.sls show?

[8inaryMata1eao]

Thanks for the input, here it is.

{% set ZEEKVER = salt['pillar.get']('global:mdengine', '') %}
{% set WAZUH = salt['pillar.get']('global:wazuh', '0') %}
{% set PLAYBOOK = salt['pillar.get']('manager:playbook', '0') %}
{% set FREQSERVER = salt['pillar.get']('manager:freq', '0') %}
{% set DOMAINSTATS = salt['pillar.get']('manager:domainstats', '0') %}
{% set FLEETMANAGER = salt['pillar.get']('global:fleet_manager', False) %}
{% set FLEETNODE = salt['pillar.get']('global:fleet_node', False) %}
{% set ELASTALERT = salt['pillar.get']('elastalert:enabled', True) %}
{% set ELASTICSEARCH = salt['pillar.get']('elasticsearch:enabled', True) %}
{% set FILEBEAT = salt['pillar.get']('filebeat:enabled', True) %}
{% set KIBANA = salt['pillar.get']('kibana:enabled', True) %}
{% set LOGSTASH = salt['pillar.get']('logstash:enabled', True) %}
{% set REDIS = salt['pillar.get']('redis:enabled', True) %}
{% set STRELKA = salt['pillar.get']('strelka:enabled', '0') %}
{% import_yaml 'salt/minion.defaults.yaml' as saltversion %}
{% set saltversion = saltversion.salt.minion.version %}
{% set INSTALLEDSALTVERSION = grains.saltversion %}

base:

  '*':
    - cron.running
    - repo.client

  'not G@saltversion:{{saltversion}}':
    - match: compound
    - salt.minion-state-apply-test
    - salt.minion

  'G@os:CentOS and G@saltversion:{{saltversion}}':
    - match: compound
    - yum.packages

  '* and G@saltversion:{{saltversion}}':
    - match: compound
    - salt.minion
    - patch.os.schedule
    - motd
    - salt.minion-check
    - salt.lasthighstate

  'not *_workstation and G@saltversion:{{saltversion}}':
    - match: compound
    - common

  '*_helixsensor and G@saltversion:{{saltversion}}':
    - match: compound
    - salt.master
    - ca
    - ssl
    - registry
    - sensoroni
    - telegraf
    - firewall
    - idstools
    - suricata.manager
    - pcap
    - suricata
    - zeek
    - redis
    - elasticsearch
    - logstash
    {%- if FILEBEAT %}
    - filebeat
    {%- endif %}
    - schedule

  '*_sensor and G@saltversion:{{saltversion}}':
    - match: compound
    - ssl
    - sensoroni
    - telegraf
    - firewall
    - nginx
    - pcap
    - suricata
    - healthcheck
    {%- if ZEEKVER != 'SURICATA' %}
    - zeek
    {%- endif %}
    {%- if WAZUH != 0 %}
    - wazuh
    {%- endif %}
    {%- if STRELKA %}
    - strelka
    {%- endif %}
    {%- if FILEBEAT %}
    - filebeat
    {%- endif %}
    {%- if FLEETMANAGER or FLEETNODE %}
    - fleet.install_package
    {%- endif %}
    - schedule
    - docker_clean

  '*_eval and G@saltversion:{{saltversion}}':
    - match: compound
    - salt.master
    - ca
    - ssl
    - registry
    - sensoroni
    - manager
    - nginx
    - telegraf
    - influxdb
    - grafana
    - soc
    - kratos
    - firewall
    - idstools
    - suricata.manager
    - healthcheck
    {%- if (FLEETMANAGER or FLEETNODE) or PLAYBOOK != 0 %}
    - mysql
    {%- endif %}
    {%- if WAZUH != 0 %}
    - wazuh
    {%- endif %}
    {%- if ELASTICSEARCH %}
    - elasticsearch
    {%- endif %}
    {%- if KIBANA %}
    - kibana.so_savedobjects_defaults
    {%- endif %}
    - pcap
    - suricata
    {%- if ZEEKVER != 'SURICATA' %}
    - zeek
    {%- endif %}
    {%- if STRELKA %}
    - strelka
    {%- endif %}
    {%- if FILEBEAT %}
    - filebeat
    {%- endif %}
    - curator
    {%- if ELASTALERT %}
    - elastalert
    {%- endif %}
    {%- if FLEETMANAGER or FLEETNODE %}
    - redis
    - fleet
    - fleet.install_package
    {%- endif %}
    - utility
    - schedule
    - soctopus
    {%- if PLAYBOOK != 0 %}
    - playbook
    - redis
    {%- endif %}
    {%- if FREQSERVER != 0 %}
    - freqserver
    {%- endif %}
    {%- if DOMAINSTATS != 0 %}
    - domainstats
    {%- endif %}
    - docker_clean
    - pipeline.load
    - learn

  '*_manager and G@saltversion:{{saltversion}}':
    - match: compound
    - salt.master
    - ca
    - ssl
    - registry
    - sensoroni
    - nginx
    - telegraf
    - influxdb
    - grafana
    - soc
    - kratos
    - firewall
    - manager
    - idstools
    - suricata.manager
    {%- if (FLEETMANAGER or FLEETNODE) or PLAYBOOK != 0 %}
    - mysql
    {%- endif %}
    {%- if WAZUH != 0 %}
    - wazuh
    {%- endif %}
    {%- if ELASTICSEARCH %}
    - elasticsearch
    {%- endif %}
    {%- if LOGSTASH %}
    - logstash
    {%- endif %}
    {%- if REDIS %}
    - redis
    {%- endif %}
    {%- if KIBANA %}
    - kibana.so_savedobjects_defaults
    {%- endif %}
    - curator
    {%- if ELASTALERT %}
    - elastalert
    {%- endif %}
    {%- if FILEBEAT %}
    - filebeat
    {%- endif %}
    - curator
    - utility
    - schedule
    {%- if FLEETMANAGER or FLEETNODE %}
    - fleet
    - fleet.install_package
    {%- endif %}
    - soctopus
    {%- if PLAYBOOK != 0 %}
    - playbook
    {%- endif %}
    {%- if FREQSERVER != 0 %}
    - freqserver
    {%- endif %}
    {%- if DOMAINSTATS != 0 %}
    - domainstats
    {%- endif %}
    - docker_clean
    - pipeline.load
    - learn

  '*_standalone and G@saltversion:{{saltversion}}':
    - match: compound
    - salt.master
    - ca
    - ssl
    - registry
    - sensoroni
    - manager
    - nginx
    - telegraf
    - influxdb
    - grafana
    - soc
    - kratos
    - firewall
    - idstools
    - suricata.manager
    - healthcheck
    {%- if (FLEETMANAGER or FLEETNODE) or PLAYBOOK != 0 %}
    - mysql
    {%- endif %}
    {%- if WAZUH != 0 %}
    - wazuh
    {%- endif %}
    {%- if ELASTICSEARCH %}
    - elasticsearch
    {%- endif %}
    {%- if LOGSTASH %}
    - logstash
    {%- endif %}
    {%- if REDIS %}
    - redis
    {%- endif %}
    {%- if KIBANA %}
    - kibana.so_savedobjects_defaults
    {%- endif %}
    - pcap
    - suricata
    {%- if ZEEKVER != 'SURICATA' %}
    - zeek
    {%- endif %}
    {%- if STRELKA %}
    - strelka
    {%- endif %}
    {%- if FILEBEAT %}
    - filebeat
    {%- endif %}
    - curator
    {%- if ELASTALERT %}
    - elastalert
    {%- endif %}
    {%- if FLEETMANAGER or FLEETNODE %}
    - fleet
    - fleet.install_package
    {%- endif %}
    - utility
    - schedule
    - soctopus
    {%- if PLAYBOOK != 0 %}
    - playbook
    {%- endif %}
    {%- if FREQSERVER != 0 %}
    - freqserver
    {%- endif %}
    {%- if DOMAINSTATS != 0 %}
    - domainstats
    {%- endif %}
    - docker_clean
    - pipeline.load
    - learn

  '*_searchnode and G@saltversion:{{saltversion}}':
    - match: compound
    - ssl
    - sensoroni
    - nginx
    - telegraf
    - firewall
    {%- if WAZUH != 0 %}
    - wazuh
    {%- endif %}
    {%- if ELASTICSEARCH %}
    - elasticsearch
    {%- endif %}
    {%- if LOGSTASH %}
    - logstash
    {%- endif %}
    - curator
    {%- if FILEBEAT %}
    - filebeat
    {%- endif %}
    {%- if FLEETMANAGER or FLEETNODE %}
    - fleet.install_package
    {%- endif %}
    - schedule
    - docker_clean
    - pipeline.load

  '*_managersearch and G@saltversion:{{saltversion}}':
    - match: compound
    - salt.master
    - ca
    - ssl
    - registry
    - sensoroni
    - nginx
    - telegraf
    - influxdb
    - grafana
    - soc
    - kratos
    - firewall
    - manager
    - idstools
    - suricata.manager
    {%- if (FLEETMANAGER or FLEETNODE) or PLAYBOOK != 0 %}
    - mysql
    {%- endif %}
    {%- if WAZUH != 0 %}
    - wazuh
    {%- endif %}
    {%- if ELASTICSEARCH %}
    - elasticsearch
    {%- endif %}
    {%- if LOGSTASH %}
    - logstash
    {%- endif %}
    {%- if REDIS %}
    - redis
    {%- endif %}
    - curator
    {%- if KIBANA %}
    - kibana.so_savedobjects_defaults
    {%- endif %}
    {%- if ELASTALERT %}
    - elastalert
    {%- endif %}
    {%- if FILEBEAT %}
    - filebeat
    {%- endif %}
    - utility
    - schedule
    {%- if FLEETMANAGER or FLEETNODE %}
    - fleet
    - fleet.install_package
    {%- endif %}
    - soctopus
    {%- if PLAYBOOK != 0 %}
    - playbook
    {%- endif %}
    {%- if FREQSERVER != 0 %}
    - freqserver
    {%- endif %}
    {%- if DOMAINSTATS != 0 %}
    - domainstats
    {%- endif %}
    - docker_clean
    - pipeline.load
    - learn

  '*_heavynode and G@saltversion:{{saltversion}}':
    - match: compound
    - ssl
    - sensoroni
    - nginx
    - telegraf
    - firewall
    {%- if WAZUH != 0 %}
    - wazuh
    {%- endif %}
    {%- if ELASTICSEARCH %}
    - elasticsearch
    {%- endif %}
    {%- if LOGSTASH %}
    - logstash
    {%- endif %}
    {%- if REDIS %}
    - redis
    {%- endif %}
    - curator
    {%- if FILEBEAT %}
    - filebeat
    {%- endif %}
    {%- if STRELKA %}
    - strelka
    {%- endif %}
    {%- if FLEETMANAGER or FLEETNODE %}
    - fleet.install_package
    {%- endif %}
    - pcap
    - suricata
    {%- if ZEEKVER != 'SURICATA' %}
    - zeek
    {%- endif %}
    {%- if FILEBEAT %}
    - filebeat
    {%- endif %}
    - schedule
    - docker_clean
    - pipeline.load

  '*_fleet and G@saltversion:{{saltversion}}':
    - match: compound
    - ssl
    - sensoroni
    - nginx
    - telegraf
    - firewall
    - mysql
    - redis
    - fleet
    - fleet.install_package
    {%- if FILEBEAT %}
    - filebeat
    {%- endif %}
    - schedule
    - docker_clean

  '*_import and G@saltversion:{{saltversion}}':
    - match: compound
    - salt.master
    - ca
    - ssl
    - registry
    - sensoroni
    - manager
    - nginx
    - soc
    - kratos
    - firewall
    - idstools
    - suricata.manager
    - pcap
    {%- if ELASTICSEARCH %}
    - elasticsearch
    {%- endif %}
    {%- if KIBANA %}
    - kibana.so_savedobjects_defaults
    {%- endif %}
    {%- if FILEBEAT %}
    - filebeat
    {%- endif %}
    - utility
    - suricata
    - zeek
    - schedule
    - docker_clean
    - pipeline.load
    - learn

  '*_receiver and G@saltversion:{{saltversion}}':
    - match: compound
    - ssl
    - sensoroni
    - telegraf
    - firewall
    {%- if WAZUH != 0 %}
    - wazuh
    {%- endif %}
    {%- if LOGSTASH %}
    - logstash
    {%- endif %}
    {%- if REDIS %}
    - redis
    {%- endif %}
    {%- if FILEBEAT %}
    - filebeat
    {%- endif %}
    {%- if FLEETMANAGER or FLEETNODE %}
    - fleet.install_package
    {%- endif %}
    - schedule
    - docker_clean

  '*_idh and G@saltversion:{{saltversion}}':
    - match: compound
    - ssl
    - sensoroni
    - telegraf
    - firewall
    {%- if WAZUH != 0 %}
    - wazuh
    {%- endif %}
    {%- if FLEETMANAGER or FLEETNODE %}
    - fleet.install_package
    {%- endif %}
    - schedule
    - docker_clean
    {%- if FILEBEAT %}
    - filebeat
    {%- endif %}
    - idh

  'J@workstation:gui:enabled:^[Tt][Rr][Uu][Ee]$ and ( G@saltversion:{{saltversion}} and G@os:CentOS )':
    - match: compound
    - workstation

  'J@workstation:gui:enabled:^[Ff][Aa][Ll][Ss][Ee]$ and ( G@saltversion:{{saltversion}} and G@os:CentOS )':
    - match: compound
    - workstation.remove_gui

[cm-ops]

That is a 2.3 top.sls file, did you try and soup from 2.3 to 2.4?

[8inaryMata1eao]

No it was a fresh install from the iso for 2.4.30. We actually just went and reinstalled all of our instances last month to prep for the EOL.
Could i copy the top.sls from another instance?

[cm-ops]

Depending on which version of 2.4.30 you have you can see the top.sls here https://github.com/Security-Onion-Solutions/securityonion/blob/2.4.30-20231228/salt/top.sls that is for 2.4.30-20231228

If that was a fresh install, where did the 2.3 top.sls come from?

[8inaryMata1eao]

Thats what makes no sense to us. We did the iso "securityonion-2.4.30-20231228.iso". I will take a look. I pulled the backups from /nsm/backup so i can go through it, I didnt see a documented method of restoring them though. But you can see the only backups are version 2.4.X and no 2.3. This was a fresh install so Im not sure where that change came from. This all steamed from a power outage while i was performing an update.

ISO

Backups

[8inaryMata1eao]

Appears to be a lost cause. Will create new discussion about restore.