Soup failing when trying to upgrade from 2.4.80

[Johnny111125]

Version

2.4.80

Installation Method

Security Onion ISO image

Description

upgrading

Installation Type

Standalone

Location

on-prem with Internet access

Hardware Specs

Exceeds minimum requirements

CPU

4

RAM

32 GB

Storage for /

200 GB

Storage for /nsm

132 GB

Network Traffic Collection

span port

Network Traffic Speeds

1Gbps to 10Gbps

Status

Yes, all services on all nodes are running OK

Salt Status

Yes, there are salt failures (please provide detail below)

Logs

No, there are no additional clues

Detail

When trying to update with soup the script ends with the following:

      Soup failed with error 2: No such file or directory

Not sure what that is referring to. Please let me know if I should provide anything else to help troubleshoot.

Any help on getting this to update is greatly appreciated. Thanks in advance!

Salt Status:
[admin@data ~]$ sudo salt-call state.highstate
[INFO ] Creating module dir '/var/cache/salt/minion/extmods/grains'
[INFO ] Syncing grains for environment 'base'
[INFO ] Loading cache from salt://_grains, for base
[INFO ] Caching directory '_grains/' for environment 'base'
local:
Data failed to compile:

The function "state.highstate" is running as PID 557540 and was started at 2024, Aug 02 14:08:07.642086 with jid 20240802140807642086

Guidelines

• I have read the discussion guidelines at Read before posting! #1720 and assert that I have followed the guidelines.

[cm-ops]

Do you see any errors in the /root/soup.log?

[Johnny111125]

The only errors I see in the log are as follows:

Killing all Salt jobs across the grid.
ERROR: Minions returned with non-zero exit code
cyber_idh:
data_standalone:
Minion did not return. [Not connected]

I will attach the full log file just in case I missed something.
soup.log

[cm-ops]

Is the standalone showing an updated version? Run cat /etc/soversion.

If so, what does sudo so-checkin show? If not, check the minion log at /opt/so/log/salt/minion for any clues.

[Johnny111125]

Not showing an updated version.

[admin@data ~]$ cat /etc/soversion
2.4.80

I checked the minion log, but don't see any errors.

[fcblank]

same issue here:
Starting salt-minion service at 17:27:13.632247
Successfully started salt-minion.

Enabling highstate.
[INFO ] Creating module dir '/var/cache/salt/minion/extmods/grains'
[INFO ] Syncing grains for environment 'base'
[INFO ] Loading cache from salt://_grains, for base
[INFO ] Caching directory '_grains/' for environment 'base'
local:
----------
msg:
Info: highstate state enabled.
res:
True

Soup failed with error 2: No such file or directory

[m0duspwnens]

Can you attach a copy of /root/soup.log?

[Johnny111125]

Sure. Here is the log...
soup.log

[reyesj2]

sorry we did not see that you had previously shared the soup.log file

Check that the file /opt/so/saltstack/local/pillar/kafka/soc_kafka.sls exists. it should look like below

kafka:
  cluster_id: cHcsj5qFbaCiw8gkuKoP5x
  password: f5ksrCk8Sdy8zkhlyNYz

if it looks very different than above or does not exist at all try restoring it by taking the most recent backup in /nsm/backup/ and unpacking it.

mkdir /tmp/newdir && /nsm/backup/so-config-backup-2024_08_03.tar -C /tmp/newdir

using the backup restore the file soc_kafka.sls

mv /tmp/newdir/opt/so/saltstack/local/pillar/kafka/soc_kafka.sls /opt/so/saltstack/local/pillar/kafka/soc_kafka.sls

finally make sure the permissions for the file are correct

chown socore:socore /opt/so/saltstack/local/pillar/kafka/soc_kafka.sls && chmod 644 /opt/so/saltstack/local/pillar/kafka/soc_kafka.sls

Then try rerunning soup

[fcblank]

Hi Restoring the soc_kafka.sls and performing a soup afterwards did the trick for me. I'm on 2.4.90 now. many thanks for your help!

[mathurin68]

Anyone else looking for this... mine failed too, but, I had somehow deleted the backup. My soc_kafka.sls had a few extra lines in it so I commented them out to look like the example above and everything went through and updated!