Elastalert fails to start with validate error on rule

[danielbidwell]

Version

2.4.140

Installation Method

Security Onion ISO image

Description

upgrading

Installation Type

Standalone

Location

on-prem with Internet access

Hardware Specs

Meets minimum requirements

CPU

8

RAM

32gb

Storage for /

250GB

Storage for /nsm

252g

Network Traffic Collection

tap

Network Traffic Speeds

Less than 1Gbps

Status

Yes, all services on all nodes are running OK

Salt Status

Yes, there are salt failures (please provide detail below)

Logs

Yes, there are additional clues in /opt/so/log/ (please provide detail below)

Detail

Everything is running except so-elastalert
/opt/so/log/elastalert/stderr.log has:
Traceback (most recent call last):
File "/usr/local/lib/python3.13/site-packages/elastalert/loaders.py", line 315, in load_options
self.rule_schema.validate(rule)
~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^
File "/usr/local/lib/python3.13/site-packages/jsonschema/validators.py", line 451, in validate
raise error
jsonschema.exceptions.ValidationError: 'type' is a required property

Failed validating 'required' in schema:
{'$schema': 'http://json-schema.org/draft-07/schema#',
'definitions': {'arrayOfStrings': {'type': ['string', 'array'],
'items': {'type': 'string'}},
'arrayOfStringsOrOtherArrays': {'type': ['string',
'array'],
'items': {'type': ['string',
'array']}},
...
On instance:
{'title': 'Vim GTFOBin Abuse - Linux',
'id': '7ab8f73a-fcff-428b-84aa-6a5ff7877dea',

I don't know where this rule comes from or how to get rid of it.

I have the standard sigmaRulePackages of core and emerging_threats_addon.

Guidelines

• I have read the discussion guidelines at Read before posting! #1720 and assert that I have followed the guidelines.

[reyesj2]

Via Security Onion -> Detections

You can search for "GTFOBin"
GTFOBin | groupby so_detection.ruleset so_detection.isEnabled | groupby so_detection.category | groupby so_detection.product

click on 'view'

Far right shows an enable/disable toggle

[danielbidwell]

When I try this, it comes back with an orange error box that says "The search query contains an unsupported segment (pipe) function" and then pointers to the help system.

[dougburks]

Did you copy and paste the query exactly? It seems to work correctly for me:

I'll paste the query again in a different format which should make it a little easier to copy:

GTFOBin | groupby so_detection.ruleset so_detection.isEnabled | groupby so_detection.category | groupby so_detection.product

If you continue to have issues, please post a screenshot of the query that you are running.

[danielbidwell]

That works. It says that it was disabled. I found the rune in the Sigma Community Rulesets Core. I told it not to use any of the Sigma Community Rulesets. It still fails on this same rule. How do I get rid of this rule so I can get elastalert to run?

[reyesj2]

In Detections what does the status on the top right say next to each detection engine? Have you tried running a sync? https://docs.securityonion.net/en/2.4/detections.html#options

[danielbidwell]

My screen shot was too large. It says: ElastAlert: OK, Strelka: OK, Suricata: OK, Total Found 46,731 Automatically apply filters, groupings, and date ranges ElastAlert Under Group Metrics it shows Count Ruleset Severity 26,018 ETOPEN high 10,882 ETOPEN informational 3,884 securitiononion-yara unknown 3,053 ETOPEN critical 2,434 ETOPEN low 183 core high 123 ETOPEN unknown 74 core critical 50 emerging_threats_addon critical 16 securityonion-resources critical 8 emerging_threats_addon high 3 securityonion-resources high 1 emerging_threats_addon medium 1 custom high 1 custom medium so-status shows so-ElastAlert as missing. Running so-elastalert- restart doesn't change it's status. /opt/so/log/elastalert/stderr.log shows that it fails to restart with the following line: elastalert.util.EAException: Invalid Rule file: /opt/elastalert/rules/rules/custom/proc_creation_lnx_vim_shell_executio n.yml 'type' is a required property This is from the sigma rule group. I have removed these from the web administration, but they persist. Elastalert appears to scan through the rules, hit this one, find an error and quit. How can I remove the sigma rules so Elastalert will continue running?

…

On Fri, 2025-04-04 at 15:31 +0000, Jorge Reyes wrote: In Detections what does the status on the top right say next to each detection engine? Have you tried running a sync? https://docs.securityonion.net/en/2.4/detections.html#options — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you authored the thread.Message ID: <Security-Onion-Solutions/securityonion/repo- ***@***.***>

-- Daniel R. Bidwell | ***@***.*** TikvahShalomConsulting,LLC If two always agree, one of them is unnecessary. Karma is getting what you deserve, mercy is not getting what you deserve grace is getting what you do not deserve. In theory, theory and practice are the same. In practice, they are not.

[danielbidwell]

I found that if I moved all of the rules in the /opt/so/rules/elastalert/rules/custom/*.yml to someplace like /opt/so/rules/save/ and ran so-rule-update ; so-elastalert-restart. The elastalert process would parse the remaining rules and run sucessfully.

With this I should close this ticket and open a new one for getting the custom rules to work.