Importing Windows DNS-Logs

[ben-sec]

Hello!

I plan to import and parse Windows DNS logs into Elasticsearch. Now I found the Logstash config /opt/so/saltstack/default/salt/logstash/pipelines/config/so/6301_dns_windows.conf and I'm wondering how I can "enable" this? Just put it in /opt/so/saltstack/local/salt/logstash/pipelines/config/custom/ and edit my manager.sls like

logstash:
   pipelines:
     manager:
       config:
       - so/0009_input_beats.conf
       - so/0010_input_hhbeats.conf
       - custom/dns-import.conf
       - custom/6301_dns_windows.conf
       - so/9999_output_redis.conf.jinja

(custom/dns-import.conf is a configuration to read the logs from a mounted share that is already working fine)

Additionally I have to make sure that my DNS import contains a type that matches "dns".

Anything else or any experiences with this Logstash configuration? Thanks in advance!

Cheers, Ben

[InfosecGoon]

How are you generating the logs and forwarding them into Security Onion?

[ben-sec]

They are written by my Windows DNS server to a network share that my manager node has mounted, I use my custom dns-import.conf to read the plain text files into logstash and currently I write only a copy to a daily plain text file on the manager node, but I would prefer to send the logs to Elasticsearch.

[InfosecGoon]

Have you tried forwarding the contents of the text file into Logstash using Filebeat? They may go through that default dns_windows pipeline automatically if the format is recognized.

[ben-sec]

Well, logstash is already importing my dns logs like

input {
        file {
              path => "/var/log/logstash/dns-share-gdc/dns.log"
              tags => ["dns","gdc"]
              mode => "tail"
              #start_position => beginning
              sincedb_path => "/var/log/logstash/dns-gdc-log"
             }
}

filter { if "gdc" in [tags] {
       if [message] =~ /^\r/ {
       drop {}
       }

       mutate { gsub => [
                        "message", "\s\(\d+\)", " ",
                        "message", "\(\d+\)\r", "\r" ,
                        "message", "\(\d+\)", "."
                        ]
                add_field => {"dns-server" => "gdc"}
                add_field => {"type" => "dns"}
              }
       }

}

Isn't my data now already going through the default pipeline? I have already added "type" => "dns", becaue that gets checked in 6301_dns_windows.conf, but so far I don't see anything comming through to Elasticsearch. But to be honest, I have no clue if any of the logstash output configurations are already sending the dns-logs to Elasticsearch or if I have to do anything else.

[ben-sec]

Any more suggestions on how to proceed with my goal to import Windows DNS logs into Elasticsearch?

[InfosecGoon]

Could you post a sample of the DNS log that you're trying to ingest? I'd like to run it through a test import and see what pipelines it hits.

[ben-sec]

There you go:

{"message":"07.03.2023 10:55:25 1A04 PACKET  00000225671C4610 UDP Snd 1.2.3.4     3e59 R Q [8385 A DR NXDOMAIN] PTR    183.100.16.172.in-addr.arpa\\r","@timestamp":"2023-03-07T09:55:49.493517375Z","tags":["dns","gdc"],"dns-server":"gdc","type":"dns","host":"so-logstash","@version":"1","path":"/var/log/logstash/dns-share-gdc/DNS.log"}
{"message":"07.03.2023 10:55:35 0C1C PACKET  00000225620C08E0 UDP Snd 1.2.3.4     c3ab R Q [8281   DR SERVFAIL] PTR    132.200.0.10.in-addr.arpa\\r","@timestamp":"2023-03-07T09:55:49.494260178Z","tags":["dns","gdc"],"dns-server":"gdc","type":"dns","host":"so-logstash","@version":"1","path":"/var/log/logstash/dns-share-gdc/DNS.log"}
{"message":"07.03.2023 10:55:26 1A04 PACKET  0000022561FF50B0 UDP Rcv 1.2.3.4     379f R Q [8281   DR SERVFAIL] PTR    15.200.0.10.in-addr.arpa\\r","@timestamp":"2023-03-07T09:55:49.493768325Z","tags":["dns","gdc"],"dns-server":"gdc","type":"dns","host":"so-logstash","@version":"1","path":"/var/log/logstash/dns-share-gdc/DNS.log"}
{"message":"07.03.2023 10:54:54 1A04 PACKET  0000022563A71D50 UDP Rcv 1.2.3.4     c86a   Q [0000       NOERROR] SOA    discord.gg\\r","@timestamp":"2023-03-07T09:55:49.491822768Z","tags":["dns","gdc"],"dns-server":"gdc","type":"dns","host":"so-logstash","@version":"1","path":"/var/log/logstash/dns-share-gdc/DNS.log"}
{"message":"07.03.2023 10:55:04 0C1C PACKET  0000022563EBDCC0 UDP Snd 1.2.3.4     2c33   Q [0001   D   NOERROR] PTR    32.200.0.10.in-addr.arpa\\r","@timestamp":"2023-03-07T09:55:49.492504600Z","tags":["dns","gdc"],"dns-server":"gdc","type":"dns","host":"so-logstash","@version":"1","path":"/var/log/logstash/dns-share-gdc/DNS.log"}
{"message":"07.03.2023 10:55:30 1A04 PACKET  00000225620C08E0 UDP Rcv 1.2.3.4     c3ab   Q [0001   D   NOERROR] PTR    132.200.0.10.in-addr.arpa\\r","@timestamp":"2023-03-07T09:55:49.493853794Z","tags":["dns","gdc"],"dns-server":"gdc","type":"dns","host":"so-logstash","@version":"1","path":"/var/log/logstash/dns-share-gdc/DNS.log"}
{"message":"07.03.2023 10:54:56 1A04 PACKET  0000022561C1F4F0 UDP Rcv 1.2.3.4     1969 R Q [8281   DR SERVFAIL] PTR    201.200.0.10.in-addr.arpa\\r","@timestamp":"2023-03-07T09:55:49.492071819Z","tags":["dns","gdc"],"dns-server":"gdc","type":"dns","host":"so-logstash","@version":"1","path":"/var/log/logstash/dns-share-gdc/DNS.log"}
{"message":"07.03.2023 10:55:09 0890 PACKET  000002256E008050 TCP Rcv 1.2.3.4     0000   Q [0000       NOERROR] AXFR   discord.onl\\r","@timestamp":"2023-03-07T09:55:49.492753624Z","tags":["dns","gdc"],"dns-server":"gdc","type":"dns","host":"so-logstash","@version":"1","path":"/var/log/logstash/dns-share-gdc/DNS.log"}
{"message":"07.03.2023 10:55:25 1A04 PACKET  00000225671C4610 UDP Rcv 1.2.3.4     3e59   Q [0001   D   NOERROR] PTR    183.100.16.172.in-addr.arpa\\r","@timestamp":"2023-03-07T09:55:49.493435037Z","tags":["dns","gdc"],"dns-server":"gdc","type":"dns","host":"so-logstash","@version":"1","path":"/var/log/logstash/dns-share-gdc/DNS.log"}
{"message":"07.03.2023 10:55:32 1A04 PACKET  0000022562134080 UDP Snd 1.2.3.4     578f   Q [0001   D   NOERROR] PTR    132.200.0.10.in-addr.arpa\\r","@timestamp":"2023-03-07T09:55:49.494164654Z","tags":["dns","gdc"],"dns-server":"gdc","type":"dns","host":"so-logstash","@version":"1","path":"/var/log/logstash/dns-share-gdc/DNS.log"}

[ben-sec]

Any more advice for me?