Playbook dont works

[Taraskobilskiy]

Version

2.4.30

Installation Method

Security Onion ISO image

Description

other (please provide detail below)

Installation Type

Standalone

Location

on-prem with Internet access

Hardware Specs

Exceeds minimum requirements

CPU

26

RAM

128

Storage for /

200 Gb

Storage for /nsm

1 tb

Network Traffic Collection

span port

Network Traffic Speeds

Less than 1Gbps

Status

Yes, all services on all nodes are running OK

Salt Status

No, there are no failures

Logs

No, there are no additional clues

Detail

Hi team

When you enable rules in the Playbook (for example, creating a new account in Windows), the event does not appear in Alerts. Although the rule is written correctly,
SIGMA example
title: Local User Creation \TEST
status: experimental
description: Create user
author: Taras
tags:

• attack.persistence

• attack.t1136
  logsource:
  product: windows
  service: security
  detection:
  selection:
  EventID: 4720
  condition: selection
  false positives:

• None
  fields:

• EventCode

• AccountName

• AccountDomain
  level: high

  View ElastAlert Config
  "any where (event.code : "4720" and winlog.channel : "Security")"

Query in ELK-DEVTOOLS-Console:

Click the Variables button, above, to create your own variables.

GET /.ds-logs-*/_eql/search
{
"query": """
any where (event.code : "4720" and winlog.channel : "Security")
"""
}

Returns the necessary information, but the trigger itself does not appear in Alerts and so on for all plays

I understand that this is related to the transition from Lucene to EQL, please could you direct me where to look for the problem or read redundant information on how it works now

Guidelines

• I have read the above statement and can confirm my post is relevant to Security Onion 2.4.

[Mav1814]

Hi,
try to change the status to test and not experimental.
Check then on kibana with the result query (event.code : "4720" and winlog.channel : "Security") if it appears there...
You can also see on the logs of the alerts if the playbook is triggered, by doing

tail -f /opt/so/log/elastalert/elastalert.log | grep "Local User Creation \TEST"

Try on the side of the windows to generate a log and check it on the kibana side and then see if is triggered on elasticalert.

I hope it help.
cheers

[Taraskobilskiy]

Judging by the log, there was a hit when creating an account, but in SOC the Alerts tab is still empty.

tail -f /opt/so/log/elastalert/elastalert.log | grep "Local User Creation TEST"
2024-01-11 10:09:21,967 INFO elastalert Local User Creation TEST - afadef0fb range 600
2024-01-11 10:09:21,967 INFO apscheduler.executors.default Job "Rule: Local User Creation TEST - afadef0fb (trigger: interval[0:03:00], next run at: 2024-01-11 10:12:24 UTC)" executed successfully
2024-01-11 10:12:24,635 INFO apscheduler.executors.default Running job "Rule: Local User Creation TEST - afadef0fb (trigger: interval[0:03:00], next run at: 2024-01-11 10:15:25 UTC)" (scheduled at 2024-01-11 10:12:24.634300+00:00)
2024-01-11 10:12:24,977 INFO elastalert Queried rule Local User Creation TEST - afadef0fb from 2024-01-11 10:02 UTC to 2024-01-11 10:12 UTC: 1 / 1 hits
2024-01-11 10:12:25,038 INFO elastalert Ran Local User Creation TEST - afadef0fb from 2024-01-11 10:02 UTC to 2024-01-11 10:12 UTC: 1 query hits (0 already seen), 1 matches, 1 alerts sent
2024-01-11 10:12:25,038 INFO elastalert Local User Creation

TEST - afadef0fb range 600
2024-01-11 10:12:25,038 INFO apscheduler.executors.default Job "Rule: Local User Creation TEST - afadef0fb (trigger: interval[0:03:00], next run at: 2024-01-11 10:15:25 UTC)" executed successfully

[Taraskobilskiy]

[Taraskobilskiy]

I also noticed strange events on the dashboard 10 - * Missing

with the following events:

[Mav1814]

Did you change the status to test?
Do so-playbook-sync to see if is okay.

[Taraskobilskiy]

Yes, in the play I changed the value to status: test.
result of sudo so-playbook-sync command:

.
.
.

Thank you for your prompt response

[Mav1814]

Try to put the status inactive, do the sync, then put again on active and try it again.

[Mav1814]

Post here the code of the play please.

[Taraskobilskiy]

I did as you said: changed it to inactive status, then did so-playbook-sync, then switched it to Active, created a user on the test host. the notification did not appear in alerts

-= Started: 2024-01-11 11:26:05.115871-=

Active offset: 100

-= Parsed Playbook Plays: 1 -=

ab382b285
All Good - Elastalert Config Exists
Inactive offset: 200
Inactive offset: 300
Inactive offset: 400
Inactive offset: 500
Inactive offset: 600
Inactive offset: 700
Inactive offset: 800
Inactive offset: 900
Inactive offset: 1000
Inactive offset: 1100
Inactive offset: 1200
Inactive offset: 1300
Inactive offset: 1400
Inactive offset: 1500
Inactive offset: 1600
Inactive offset: 1700
Inactive offset: 1800
Inactive offset: 1900
Inactive offset: 2000
Inactive offset: 2100
Inactive offset: 2200
Inactive offset: 2300

Inactive - afadef0fb

Inactive - 78fc4b742
Inactive - 851c232a1
.
.
.
.

Inactive - ed034dc95

-= Maintenance Summary =-

Active Plays: 1

Missing ElastAlert Configs: 0
Inactive Plays: 2221

Out of Sync ElastAlert Configs: 0

-= Completed: 2024-01-11 11:26:27.819690-=

[Taraskobilskiy]

[Mav1814]

I'm asking because the playID are different, the above one from the last result.

[Taraskobilskiy]

I'm sorry, I have 2 identical rules, the log worked for both, Now I’ll delete Local User Creation TARAS TEST11 so as not to get confused

2024-01-11 11:56:53,106 INFO elastalert Queried rule Local User Creation TARAS TEST - afadef0fb from 2024-01-11 11:46 UTC to 2024-01-11 11:56 UTC: 2 / 2 hits

[Mav1814]

Do again "status inactive, do the sync, then put again on active and try it again."
check if is deleted the other rule.

Sometimes there are troubles..

It should work, im trying to understand the problem. I will try to reproduce in my side.

[Taraskobilskiy]

disabled all Playbooks, made so-playbook-sync:

enabled the required rule, synchronized again:

created a user, ran tail -f /opt/so/log/elastalert/elastalert.log | grep "Local User Creation TARAS TEST"


Yes in the log, but empty in SOC Alerts
I really want to find the problem, but I have no idea

[Taraskobilskiy]

I also saw this error in the log related to parsing the match_body.user.id field:
elasticsearch POST https://seconion-qc:9200/elastalert/_doc [status:400 request:0.005s]
2024-01-11 13:43:26,129
ERROR elastalert Error writing alert info to Elasticsearch: RequestError(400, 'document_parsing_exception', "[1:5146] failed to parse field [match_body.user.id] of type [long] in document with id 'GFrD-IwBRWlvJ4-ebLCt'. Preview of field's value: 'S-1-5-21-3080172177-780197540-4289498753-1182'")
Traceback (most recent call last):
File "/usr/local/lib/python3.11/site-packages/elastalert/elastalert.py", line 1461, in writeback
res = self.writeback_es.index(index=index, body=body)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/elasticsearch/client/utils.py", line 152, in _wrapped
return func(*args, params=params, headers=headers, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/elasticsearch/client/init.py", line 398, in index
return self.transport.perform_request(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/elasticsearch/transport.py", line 392, in perform_request
raise e
File "/usr/local/lib/python3.11/site-packages/elasticsearch/transport.py", line 358, in perform_request
status, headers_response, data = connection.perform_request(
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/elasticsearch/connection/http_requests.py", line 199, in perform_request
self._raise_error(response.status_code, raw_data)
File "/usr/local/lib/python3.11/site-packages/elasticsearch/connection/base.py", line 315, in _raise_error
raise HTTP_EXCEPTIONS.get(status_code, TransportError)(
elasticsearch.exceptions.RequestError: RequestError(400, 'document_parsing_exception', "[1:5146] failed to parse field [match_body.user.id] of type [long] in document with id 'GFrD-IwBRWlvJ4-ebLCt'. Preview of field's value: 'S-1-5-21-3080172177-780197540-4289498753-1182'")

Do you think these problems could be related? because I wrote a simple play for Linux that detects sudo cat etc/sudoers and the rule worked, and the alert appeared in soc alerts. But the rules still don’t appear under Windows.

[Taraskobilskiy]

sudo cat /opt/so/rules/elastalert/playbook/afadef0fb.yaml
alert:

• "modules.so.playbook-es.PlaybookESAlerter"

elasticsearch_host: "10.189.96.18:9200"
play_title: "Local User Creation TARAS TEST"
play_id: "afadef0fb"
event.module: "playbook"
event.dataset: "playbook.alert"
event.severity: 3
rule.category: ""
play_url: "https://10.189.96.18/playbook/issues/2484"
kibana_pivot: "https://10.189.96.18/kibana/app/kibana#/discover?_g=()&_a=(columns:!(_source),interval:auto,query:(language:lucene,query:'_id:{[_id]}'),sort:!('@timestamp',desc))"
soc_pivot: "https://10.189.96.18/#/hunt"
sigma_level: "high"

index: '.ds-logs-*'
name: "Local User Creation TARAS TEST - afadef0fb"
priority: 3
realert:
minutes: 0
type: any
filter:

• eql: >
  any where (event.code : "4720" and winlog.channel : "Security")

[Mav1814]

It seems that my own playbook for windows doesn't work now ..

I had a custom one that worked, but i tested know and it doesn't work .. No alert in SOC.

So from now on i can not help you cuz im on the same situation xD

[Taraskobilskiy]

Clearly, if something becomes known, I would be grateful if you let me know. And thank you very much for taking the time)