Error on Soup Upgrade to 2.4.90 - Rendering SLS 'base:logstash' failed

[fitz003]

Version

2.4.90

Installation Method

Security Onion ISO image

Description

upgrading

Installation Type

Standalone

Location

on-prem with Internet access

Hardware Specs

Exceeds minimum requirements

CPU

8

RAM

16

Storage for /

219GB

Storage for /nsm

323GB

Network Traffic Collection

span port

Network Traffic Speeds

Less than 1Gbps

Status

No, one or more services are failed (please provide detail below)

Salt Status

Yes, there are salt failures (please provide detail below)

Logs

No, there are no additional clues

Detail

Upgrade to 2.4.90 worked on one standalone sensor, but failed on another. Upon failure, I tried rebooting, running soup again, and running both sudo salt-call state.highstate and sudo so-checkin as per the guidance here: https://docs.securityonion.net/en/2.4/soup.html#errors

Files:
highstate.log
map.jinja.txt
soup.log

Running salt '*' mine.get '*' network.ip_addrs does return the expected IP address.

securityonion_standalone:
    ----------
    securityonion_standalone:
        - 10.2.x.x

Very similar issue to what was documented here #7046, but consistently running salt-call state.highstate did not fix the issue.

Guidelines

• I have read the discussion guidelines at Read before posting! #1720 and assert that I have followed the guidelines.

[cm-ops]

What does this output? sudo salt-call pillar.get logstash

[fitz003]

This is what I got:

[INFO    ] Creating module dir '/var/cache/salt/minion/extmods/grains'
[INFO    ] Syncing grains for environment 'base'
[INFO    ] Loading cache from salt://_grains, for base
[INFO    ] Caching directory '_grains/' for environment 'base'
local:
    ----------
    config:
        ----------
        pipeline_x_workers:
            8
    enabled:
        False
    nodes:
        None
    settings:
        ----------
        lsheap:
            1000m

[cm-ops]

Looks like Logstash is disabled. Can you go into SOC > Administration >Configuration > logstash > enabled and change it to true.

Then try sudo so-checkin.

[fitz003]

We disabled Logstash because we actually send all of the logs to our own Elastic Cloud instance via the Elastic Agent that we install. I also cannot re-enable Logstash as the web interface is not loading. If there's a command I can run to manually enable it let me know and I can run that.

[petiepooo]

#7655 suggests running sudo salt '*' mine.update and trying another sudo so-checkin. That has worked for me in the past on weird rendering issues.

[fitz003]

Appreciate the suggestion, I saw this issue (#7655) when I first started looking for a fix action and unfortunately these commands produced the errors that I posted about above in the files: https://github.com/user-attachments/files/16603057/highstate.log

[samuel809]

is there is a node thar is not reachable?

if so, remove its keys.

to list salt-key -L
to delete salt-key -d 'minion1'

run again salt-call state.highstate from the node again.

regards,