Parsing winlogbeat data

[martin8615]

Hello Security Onion Community,

we have an issue parsing Winlogbeat data. In Kibana, we have´nt found any parser for Winlogbeat under Ingest Pipeline. Unfortunately, the data we receive from our Windows systems is not parsed correctly.
Therefore we installed Winlogbeat on a Windows Event Collector Server and tried to create the parsers in Security Onion with the command .\winlogbeat.exe setup –pipelines. But unfortunately we get the following error message:

.\winlogbeat.exe : Exiting: couldn't connect to any of the configured Elasticsearch hosts. Errors: [error connecting to Elasticsearch at http://x.x.x.x:5044/: Get
http://x.x.x.x:5044/: read tcp [Source_IP]:49791->[Destination_IP]:5044: wsarecv: An existing connection was forcibly closed by the remote host.]
At line:1 char:1

• .\winlogbeat.exe setup --pipelines -c C:\ProgramData\Elastic\Beats\wi ...

•   + CategoryInfo          : NotSpecified: (Exiting: couldn...e remote host.]:String) [], RemoteException
    + FullyQualifiedErrorId : NativeCommandError
  
  

We send the events from our Windows Event Collector (on which Winlogbeat is installed in the version the SOC Download tab provides) to a Receiver Node that forwards the logs to our Search Nodes. This whole process itself works fine and the data is displayed in Kibana.

In Ingest Pipeliens there are currently more than 50 parsers that appear to be created automatically when Security Onion is installed.
e.g. Common, Common.nids, ecs, filebeat-7.17.3-xxxx, etc.

How does Security Onion regulate when which parser is used and how can we deploy new ones?

Thank you all in advance and best regards.

[dougburks]

Did you follow the Winlogbeat steps at https://docs.securityonion.net/en/2.3/beats.html#winlogbeat?

Have you reviewed the Winlogbeat video at https://youtu.be/Xz-7oDrZdQY?

[martin8615]

Thank you very much for your help.
Yes, we already read the article and watched the video, but unfortunately it didn't solve our problem.

We would like to understand how we can create the parsers for Winlogbeat in Security Onion.

According to this article, for Winlogbeat the ingest pipelines must be carried out with the installation of the Winlogbeats Agent.

https://www.elastic.co/guide/en/beats/winlogbeat/current/load-ingest-pipelines.html

For us, however, this leads to the errors already mentioned. Is there another way to import the parsers for Winlogbeat into Security Onion?
How does Security Onion decide which parser to use?

Our current Security Onion version is 2.3.260.

Thanks in advance.

[dougburks]

Security Onion 2.3.260 already has ingest parsers for Windows event logs coming from Winlogbeat:
https://github.com/Security-Onion-Solutions/securityonion/blob/master/salt/elasticsearch/files/ingest/beats.common
https://github.com/Security-Onion-Solutions/securityonion/blob/master/salt/elasticsearch/files/ingest/sysmon
https://github.com/Security-Onion-Solutions/securityonion/blob/master/salt/elasticsearch/files/ingest/win.eventlogs

This is why there are only a few steps at https://docs.securityonion.net/en/2.3/beats.html#winlogbeat and it shouldn't require the manual steps that you mentioned from https://www.elastic.co/guide/en/beats/winlogbeat/current/load-ingest-pipelines.html.

Can you provide more information about where the process at https://docs.securityonion.net/en/2.3/beats.html#winlogbeat isn't working for you?

Are you using the correct version of the Winlogbeat agent downloaded from SOC Downloads?

Are you making any other changes to winlogbeat.yml other than what's shown at https://docs.securityonion.net/en/2.3/beats.html#winlogbeat?

Finally, before we spend too much time troubleshooting 2.3 and Winlogbeat, please keep in mind that 2.4 is now available with Elastic Agent and we will announce 2.3 End Of Life date soon. So it may be a better use of time to switch to 2.4 and Elastic Agent.

[martin8615]

Thanks for your help Doug. We definitely switch to 2.4 in the future.
In the meantime we are trying to figure out our current problem. Maybe we're looking in the wrong place here.
The problem we have is that when we use the Community Playbooks they don't work for us because some fields that are specified in the Community Playbooks don't exist for us.
For example with Powershell.
The fields “process.command_line.security” are often mentioned in the playbooks here. However, we do not have this field. We thought this was because the parser for Winlogbeat was not present in Security Onion.
If we download the current version of Winlogbeat from the Elastic website, there are several files there. For example, the file winlogbeat\module\powershell\ingest\Powershell.yml. This file specifies the fields (for example: process.command_line.security) that we do not have.

How do we get the PowerShell field into our Security Onion environment?

As an example. The following Elasticsearch query does not work for us because the fields “process.command_line.security” do not exist .
(event.code.security:"1" AND winlog.channel.security:"Microsoft-Windows-Sysmon/Operational" AND process.command_line.security:\HarddiskVolumeShadowCopy AND process.command_line.security:System32\config\sam AND process.command_line.security:(Copy-Item OR cp\ $_. OR cpi\ $_. OR copy\ $_. OR .File]::Copy())

[defensivedepth]

@martin8615 Do the systems that you are shipping the logs from have Sysmon installed on them? If they don't, then you wont have logs with process.command_line.

For the powershell logs - have you configured Winglogbeat to send them? Also, which Powershell logs are you referring to, there are a few different ones.

[martin8615]

Yes, Sysmon and Winlogbeat are installed on the system where the logs are sent to the Security Onion Receiver. You can find our winlogbeat.yml file in the appendix. Do we have a mistake here?

We also send everything with the PowerShell logs and we see the logs in Security Onion, but the “process.command_line” field is always empty and therefore the playbooks do not work either.

winlogbeat.yml.txt
winlogbeat-20230925.bak.ndjson.txt

[defensivedepth]

Ok, do you see any sysmon logs in the Hunt interface? If so, can you post a screenshot of it? (sanitized as needed)

[secuser12]

Hello,

Martin isn't here today, so I'm sending the logs on his behalf. Yes We see Sysmin logs in the hunting interface.

Attached are the screenshots of the Sysmon logs.

I ran the following command using Powershell on a Windows 10 machine.
copy-item -Path C:\test\test.txt -Destination C:\Temp -Force

From what I understand, the “process.command_line.security” field should now be filled in, correct?
Unfortunately, we don't have any data on this in SecOnion either. (See screenshot: process.command_line.Security)

Let us know what more information we can send you. Thank you for your support.