2.3.260 Issue to parse fortinet logs with filebeat

[easy13]

Hi,

I have a distributed SO deployment running, and I cannot find what I did wrong so my Fortinet firewall logs are not parsed by the filebeat module.

The architecture

1 manager/search node
2 search node
1 forwarder

Version: 2.3.260 (updated from 2.3.40)

(In this question FW_IP_address mask the real entry)

The firewall (fortigate 6.4.14) is set to sent its logs directly to the udp 9004 manager port
I can see the flow is coming with tcpdump (see below)

What I did:

In /opt/so/saltstack/local/pillar/global.sls

filebeat:
  third_party_filebeat:
    modules:
      fortinet:
        firewall:
          enabled: true
          var.input: udp
          var.syslog_host: 0.0.0.0
          var.syslog_port: 9004
firewall:
  assigned_hostgroups:
    chain:
      DOCKER-USER:
        hostgroups:
          fortigate:
            portgroups:
              - portgroups.fortigate
      INPUT:
        hostgroups:
          fortigate:
            portgroups:
              - portgroups.fortigate

Same in manager.sls, and both search nodes minions sls files without the "INPUT" part.

I created a fortigate hostgroup and portgoup then add the firewall IP address

so-firewall addhostgroup fortigate
so-firewall includehost fortigate FW_IP_address
so-firewall addportgroup fortigate
so-firewall addport fortigate udp  9004

salt-call state.apply firewall | grep FW_IP_address
          ID: insert_DOCKER-USER_syslog_FW_IP_address_514_tcp_91
     Comment: iptables rule for insert_DOCKER-USER_syslog_FW_IP_address_514_tcp_91 already set for ipv4 (/sbin/iptables --wait -t filter -I DOCKER-USER 1 -p tcp --dport 514 --source FW_IP_address --jump ACCEPT)
          ID: insert_DOCKER-USER_syslog_FW_IP_address_514_udp_92
     Comment: iptables rule for insert_DOCKER-USER_syslog_FW_IP_address_514_udp_92 already set for ipv4 (/sbin/iptables --wait -t filter -I DOCKER-USER 1 -p udp --dport 514 --source FW_IP_address --jump ACCEPT)
          ID: insert_DOCKER-USER_fortigate_FW_IP_address_9004_udp_112
     Comment: iptables rule for insert_DOCKER-USER_fortigate_FW_IP_address_9004_udp_112 already set for ipv4 (/sbin/iptables --wait -t filter -I DOCKER-USER 1 -p udp --dport 9004 --source FW_IP_address --jump ACCEPT)
          ID: insert_INPUT_fortigate_FW_IP_address_9004_udp_128
     Comment: iptables rule for insert_INPUT_fortigate_FW_IP_address_9004_udp_128 already set for ipv4 (/sbin/iptables --wait -t filter -I INPUT 1 -p udp --dport 9004 --source FW_IP_address --jump ACCEPT)

That I can see:
The filebeat docker port is open and listening

docker ps | grep 9004
5db33d462520   xxxxxxxxxxx:5000/security-onion-solutions/so-filebeat:2.3.260        "/usr/local/bin/dock…"   4 days ago    Up 4 days             0.0.0.0:514->514/tcp, 0.0.0.0:5066->5066/tcp, 0.0.0.0:514->514/udp, 0.0.0.0:9004->9004/tcp, 0.0.0.0:9004->9004/udp                 so-filebeat

And

netstat -nap | grep 9004
tcp        0      0 0.0.0.0:9004            0.0.0.0:*               LISTEN      26040/docker-proxy
udp        0      0 0.0.0.0:9004            0.0.0.0:*                           26052/docker-proxy

And these logs:

tail -f /opt/so/log/filebeat/filebeat.log-20230919.ndjson
{"log.level":"warn","@timestamp":"2023-10-02T12:49:02.211Z","log.logger":"input.udp","log.origin":{"file.name":"udp/input.go","file.line":249},"message":"failed to get udp stats from /proc: /proc/net/udp entry not found for [0:232c]","service.name":"filebeat","id":"B2D7E7E9079CCBD8","host":"0.0.0.0:9004","ecs.version":"1.6.0"}

tcpdump -i ens192 port 9004
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on ens192, link-type EN10MB (Ethernet), capture size 262144 bytes
12:57:40.516505 IP FW_IP_address.watchme-7272 > manager.9004: UDP, length 606

I try to apply the documentation and aswers from other issues, for example:
#9705

Thanks for any help or clue :)

[cm-ops]

Try putting Filebeat into debug and see if there are any clues in the log? Put the below in your global.sls or minion pillar file where you are sending the logs to.

Filebeat:
  logging:
    level: debug

[easy13]

Thanks for your help cm-ops.

I tried the debug a bit before that, and I can see Filebeat parsing the logs.

[root@mgt01 ~]# tail -f /opt/so/log/filebeat/filebeat.log-20231011-34.ndjson {"log.level":"debug","@timestamp":"2023-10-19T07:37:25.175Z","log.logger":"processors","log.origin":{"file.name":"processing/processors.go","file.line":213},"message":"Publish event: {\n \"@timestamp\": \"2023-10-19T07:37:25.175Z\",\n \"@metadata\": {\n \"beat\": \"filebeat\",\n \"type\": \"_doc\",\n \"version\": \"8.8.2\",\n \"truncated\": false,\n \"pipeline\": \"filebeat-8.8.2-fortinet-firewall-pipeline\"\n },\n \"fileset\": {\n \"name\": \"firewall\"\n },\n \"input\": {\n \"type\": \"udp\"\n },\n \"agent\": {\n \"type\": \"filebeat\",\n \"version\": \"8.8.2\",\n \"ephemeral_id\": \"e90ab7be-c805-46a5-b5b8-caf5dc1a285c\",\n \"id\": \"8489dd79-e3d9-4ae9-8761-2cc3e15ca434\",\n \"name\": \"mgt01\"\n },\n \"event\": {\n \"dataset\": \"fortinet.firewall\",\n \"module\": \"fortinet\",\n \"timezone\": \"+00:00\"\n },\n \"service\": {\n \"type\": \"fortinet\"\n },\n \"ecs\": {\n \"version\": \"1.12.0\"\n },\n \"message\": \"<189>date=2023-10-19 time=09:38:30 devname=\\\"Fortiname\\\" devid=\\\"FGT80xxxxxxxx\\\" eventtime=1697701111256797290 tz=\\\"+0200\\\" logid=\\\"0000000013\\\" type=\\\"traffic\\\" subtype=\\\"forward\\\" level=\\\"notice\\\" vd=\\\"root\\\" srcip=1X.X.X.X srcport=55030 srcintf=\\\"ssl.root\\\" srcintfrole=\\\"undefined\\\" dstip=1X.X.X.Xdstport=80 dstintf=\\\"internal2\\\" dstintfrole=\\\"lan\\\" srccountry=\\\"Reserved\\\" dstcountry=\\\"Reserved\\\" sessionid=71362987 proto=6 action=\\\"deny\\\" policyid=0 policytype=\\\"policy\\\" user=\\\"aaaaaaaa\\\" authserver=\\\bbbbbbbb\\\" service=\\\"HTTP\\\" trandisp=\\\"noop\\\" duration=0 sentbyte=0 rcvdbyte=0 sentpkt=0 rcvdpkt=0 appcat=\\\"unscanned\\\" crscore=30 craction=131072 crlevel=\\\"high\\\"\",\n \"log\": {\n \"source\": {\n \"address\": \"X.X.X.X:47143\"\n }\n },\n \"tags\": [\n \"fortinet-firewall\",\n \"forwarded\"\n ]\n}","service.name":"filebeat","ecs.version":"1.6.0"}

I think the issue is between filebeat and logstash, but I can't see any issue in the logs...

Thanks again.

[chadstout1939]

Hello there!

I have a little different of a setup, but I was able to get Fortinet logging to work with my 2.3 deployment. My setup is two forward nodes, and a managersearch node.

This is what I did to get it to work for me. I don't know if this will specifically help, but maybe we can compare/contrast and troubleshoot from there. I think the main differences are that I'm sending logs directly to the Managersearch, and not to sensor nodes. I also applied the config changes to the specific minion pillar and not to global. Maybe there is an issue between the sensor/forward nodes and the search/manager nodes that gets the filebeat to logstash wrong?

Within /opt/so/saltstack/local/pillar/minions/$managersearch.sls config I added:
filebeat:
third_party_filebeat:
modules:
fortinet:
firewall:
enabled: true
var.input: udp
var.syslog_host: 0.0.0.0
var.syslog_port: 9004

Then I ran: sudo so-allow -> s ->
Run this for each FortiGate you're connecting.

With my network, I have VLANs, so I have to use the gateway IP for how my managersearch node sees each Fortigate.

Within the FortiGate:
Login as an admin, and you can set it to log things to Syslog. But I had to go to the command line interface, and:

config log syslogd setting
set status enabled
set server
set port 9004
set source-ip

Then, back to the Managersearch node. I ran the following commands:
sudo so-firewall addportgroup fortinet
sudo so-firewall addport fortinet udp 9004

Then I edited the managersearch config again (/opt/so/saltstack/local/pillar/minions/$managersearch.sls)
firewall:
assigned_hostgroups:
chain:
DOCKER-USER:
hostgroups:
syslog:
portgroups:
- portgroups.fortinet

Lastly, I ran: sudo salt-call state.apply firewall

And I began getting parsed Fortinet logs. I am running 6.4.14 as well. I don't know if it is parsing all of the logs, but I am getting plenty of them to work with.

[easy13]

Hi Chad,

I also have a forward. It handles zeek/suricata, but I tried to send the fortinet logs thought it before posting.
I also tried with the search node too.

Same result with both:
The logs are accepted and parsed with filebeat, and then transmit to logstash, but logstash just keep them (or whatever it's doing with).
Maybe I forgot something, but I was able to see the flow as expected with tcpdump.
I'll try again your setup.

Thanks for helping.

[easy13]

Hi there,

I tried as you said and still have the same issue.
The debug mode show me that the forward node filebeat is parsing the fortinet logs:

{"log.level":"debug","@timestamp":"2023-10-23T13:19:22.537Z","log.logger":"processors","log.origin":{"file.name":"processing/processors.go","file.line":213},"message":"Publish event: {\n \"@timestamp\": \"2023-10-23T13:19:22.537Z\",\n \"@metadata\": {\n \"beat\": \"filebeat\",\n \"type\": \"_doc\",\n \"version\": \"8.8.2\",\n \"truncated\": false,\n \"pipeline\": \"filebeat-8.8.2-fortinet-firewall-pipeline\"\n },\n \"event\": {\n \"module\": \"fortinet\",\n \"dataset\": \"fortinet.firewall\",\n \"timezone\": \"+00:00\"\n },\n \"fileset\": {\n \"name\": \"firewall\"\n },\n \"message\": \"<189>date=2023-10-23 time=15:19:22 devname=\\\"xxxxxxxx\\\" devid=\\\"FGT80xxxxxxxxx\\\" eventtime=1698067162972064880 tz=\\\"+0200\\\" logid=\\\"0000000013\\\" type=\\\"traffic\\\" subtype=\\\"forward\\\" level=\\\"notice\\\" vd=\\\"root\\\" srcip=xxxxxxxx srcport=52641 srcintf=\\\"ssl.root\\\" srcintfrole=\\\"undefined\\\" dstip=xxxxxx dstport=445 dstintf=\\\"internal2\\\" dstintfrole=\\\"lan\\\" srccountry=\\\"Reserved\\\" dstcountry=\\\"Reserved\\\" sessionid=79434298 proto=6 action=\\\"deny\\\" policyid=0 policytype=\\\"policy\\\" user=\\\"xxxxxxxx\\\" authserver=\\\"xxxxx.ixx\\\" service=\\\"SMB\\\" trandisp=\\\"noop\\\" duration=0 sentbyte=0 rcvdbyte=0 sentpkt=0 rcvdpkt=0 appcat=\\\"unscanned\\\" crscore=30 craction=131072 crlevel=\\\"high\\\"\",\n \"log\": {\n \"source\": {\n \"address\": \"xxxxxxxx:50104\"\n }\n },\n \"tags\": [\n \"fortinet-firewall\",\n \"forwarded\"\n ],\n \"agent\": {\n \"version\": \"8.8.2\",\n \"ephemeral_id\": \"cbf76130-6c86-4db9-b1a7-36f6cda49944\",\n \"id\": \"d9895f95-0892-4985-b694-7fdeb5236420\",\n \"name\": \"socfw01\",\n \"type\": \"filebeat\"\n },\n \"service\": {\n \"type\": \"fortinet\"\n },\n \"input\": {\n \"type\": \"udp\"\n },\n \"ecs\": {\n \"version\": \"1.12.0\"\n }\n}","service.name":"filebeat","ecs.version":"1.6.0"}

But it still do not show up in the dashboard or kibana console.

If I put the logs directly in the syslog port 514 (on the manager), they are visible as "syslog" in both console but not parsed. (I did that just for testing)

An extra step I did, is to add a rules for the forwarder is allowed to send log to the master logstash, after I saw it still do not work. Changed nothing...

Is there a way to check the communication between filebeat and logstash (tcpdump show me they are..) and what logstash is really doing with them?

Thanks!

[cm-ops]

Also check the following:

so-logstash-pipeline-stats $pipeline (manager or search)
so-elasticsearch-pipeline-stats $pipeline (you can get the name with so-elasticsearch-pipelines-list | grep fort)
tail -f /opt/so/log/logstash/logstash.log (on your searchnodes)

See if there are any clues in the above.

[easy13]

Thank you.

I'm using this flow now ,
Fortinet >> forwarder >> manager-search .

On the forwarder there is nothing with theses commands

On the manager:

so-logstash-pipeline-stats

{
  "manager": {
    "events": {
      "in": 630467,
      "queue_push_duration_in_millis": 18295,
      "filtered": 630436,
      "duration_in_millis": 259643,
      "out": 630436
    },
    "flow": {
      "input_throughput": {
        "current": 30.3,
        "last_1_minute": 34.41,
        "last_5_minutes": 40.85,
        "last_15_minutes": 41.98,
        "last_1_hour": 48,
        "lifetime": 64.63
      },
      "queue_backpressure": {
        "current": 0.0005375,
        "last_1_minute": 0.0003852,
        "last_5_minutes": 0.0005213,
        "last_15_minutes": 0.0006009,
        "last_1_hour": 0.0006769,
        "lifetime": 0.001875
      },
      "worker_concurrency": {
        "current": 0.008198,
        "last_1_minute": 0.008567,
        "last_5_minutes": 0.01152,
        "last_15_minutes": 0.0124,
        "last_1_hour": 0.01412,
        "lifetime": 0.02662
      },
      "output_throughput": {
        "current": 28.22,
        "last_1_minute": 34.35,
        "last_5_minutes": 40.75,
        "last_15_minutes": 41.95,
        "last_1_hour": 48,
        "lifetime": 64.63
      },
      "filter_throughput": {
        "current": 28.22,
        "last_1_minute": 34.35,
        "last_5_minutes": 40.75,
        "last_15_minutes": 41.95,
        "last_1_hour": 48,
        "lifetime": 64.63
      }
    },
    "plugins": {
      "inputs": [
        {
          "id": "e300acb9cdf53e26a6b4025d617e8c75a65e054cab5a2659ae13e1ee18b12511",
          "name": "beats",
          "flow": {
            "throughput": {
              "current": 15.99,
              "last_1_minute": 23.87,
              "last_5_minutes": 25.9,
              "last_15_minutes": 28.21,
              "last_1_hour": 32.1,
              "lifetime": 36.99
            }
          },
          "peak_connections": 4,
          "current_connections": 2,
          "events": {
            "queue_push_duration_in_millis": 7468,
            "out": 360797
          }
        },
        {
          "id": "26de1e0d83b75919e0370525f359093ea0bb9bca1ca7c4292cacc0718e9ba16d",
          "name": "beats",
          "flow": {
            "throughput": {
              "current": 14.31,
              "last_1_minute": 10.54,
              "last_5_minutes": 14.95,
              "last_15_minutes": 13.78,
              "last_1_hour": 15.9,
              "lifetime": 27.64
            }
          },
          "peak_connections": 6,
          "current_connections": 4,
          "events": {
            "queue_push_duration_in_millis": 10579,
            "out": 269670
          }
        },
        {
          "id": "endgame_data",
          "name": "http",
          "flow": {
            "throughput": {
              "current": 0,
              "last_1_minute": 0,
              "last_5_minutes": 0,
              "last_15_minutes": 0,
              "last_1_hour": 0,
              "lifetime": 0
            }
          },
          "events": {
            "queue_push_duration_in_millis": 0,
            "out": 0
          }
        }
      ],
      "codecs": [
        {
          "id": "plain_be51d0b4-4b3e-4bdf-a619-681705503932",
          "name": "plain",
          "decode": {
            "duration_in_millis": 0,
            "writes_in": 0,
            "out": 0
          },
          "encode": {
            "duration_in_millis": 0,
            "writes_in": 0
          }
        },
        {
          "id": "es_bulk_e6077e05-ec90-4f0c-a5ce-5a84ea0d5d84",
          "name": "es_bulk",
          "decode": {
            "duration_in_millis": 0,
            "writes_in": 0,
            "out": 0
          },
          "encode": {
            "duration_in_millis": 0,
            "writes_in": 0
          }
        },
        {
          "id": "json_89be47f8-d1fd-4e37-9326-461aadcdcd9e",
          "name": "json",
          "decode": {
            "duration_in_millis": 0,
            "writes_in": 0,
            "out": 0
          },
          "encode": {
            "duration_in_millis": 85139,
            "writes_in": 630436
          }
        },
        {
          "id": "plain_57ebe1d0-5cbd-44c3-91d5-5c3c55e349ae",
          "name": "plain",
          "decode": {
            "duration_in_millis": 0,
            "writes_in": 0,
            "out": 0
          },
          "encode": {
            "duration_in_millis": 0,
            "writes_in": 0
          }
        }
      ],
      "filters": [
        {
          "id": "4a1163d77bf073ae65972efd444041aeff77f57182369b443d6356e661fb86af",
          "name": "mutate",
          "flow": {
            "worker_millis_per_event": {
              "current": "Infinity",
              "last_1_minute": "Infinity",
              "last_5_minutes": "Infinity",
              "last_15_minutes": "Infinity",
              "last_1_hour": "Infinity",
              "lifetime": "Infinity"
            },
            "worker_utilization": {
              "current": 0.00168,
              "last_1_minute": 0.001926,
              "last_5_minutes": 0.001721,
              "last_15_minutes": 0.001766,
              "last_1_hour": 0.001858,
              "lifetime": 0.002009
            }
          },
          "events": {
            "in": 0,
            "duration_in_millis": 784,
            "out": 0
          }
        },
        {
          "id": "5004cf4c668977b9919e1bb9f863457a25353103d8851d6c20368a4e6c70e476",
          "name": "mutate",
          "flow": {
            "worker_millis_per_event": {
              "last_5_minutes": "Infinity",
              "last_15_minutes": "Infinity",
              "last_1_hour": "Infinity",
              "lifetime": "Infinity"
            },
            "worker_utilization": {
              "current": 0,
              "last_1_minute": 0,
              "last_5_minutes": 0.0002459,
              "last_15_minutes": 0.0002717,
              "last_1_hour": 0.000288,
              "lifetime": 0.0002973
            }
          },
          "events": {
            "in": 0,
            "duration_in_millis": 116,
            "out": 0
          }
        },
        {
          "id": "0c58d1a84134c8ae9d136c130b63cf07a76434bca02b3a3551c3b4b085ddf019",
          "name": "mutate",
          "flow": {
            "worker_millis_per_event": {
              "current": "Infinity",
              "last_1_minute": "Infinity",
              "last_5_minutes": "Infinity",
              "last_15_minutes": "Infinity",
              "last_1_hour": "Infinity",
              "lifetime": "Infinity"
            },
            "worker_utilization": {
              "current": 0.00168,
              "last_1_minute": 0.0003852,
              "last_5_minutes": 0.0003279,
              "last_15_minutes": 0.000326,
              "last_1_hour": 0.0003497,
              "lifetime": 0.0004024
            }
          },
          "events": {
            "in": 0,
            "duration_in_millis": 157,
            "out": 0
          }
        },
        {
          "id": "7b18fe6dfc0e2ede486536035d625b7ac03ea75c56f33f0852938172294427dc",
          "name": "json",
          "flow": {
            "worker_millis_per_event": {
              "current": "Infinity",
              "last_1_minute": "Infinity",
              "last_5_minutes": "Infinity",
              "last_15_minutes": "Infinity",
              "last_1_hour": "Infinity",
              "lifetime": "Infinity"
            },
            "worker_utilization": {
              "current": 0.00168,
              "last_1_minute": 0.0003852,
              "last_5_minutes": 0.0002459,
              "last_15_minutes": 0.0002717,
              "last_1_hour": 0.0003565,
              "lifetime": 0.0003332
            }
          },
          "events": {
            "in": 0,
            "duration_in_millis": 130,
            "out": 0
          }
        }
      ],
      "outputs": [
        {
          "id": "20d9f4ec873eab1dc6ce27843e3a140fd0699b2c1dda3c901ad595adfb493d94",
          "name": "redis",
          "flow": {
            "worker_millis_per_event": {
              "current": 0.2738,
              "last_1_minute": 0.2382,
              "last_5_minutes": 0.2735,
              "last_15_minutes": 0.2862,
              "last_1_hour": 0.2854,
              "lifetime": 0.4038
            },
            "worker_utilization": {
              "current": 0.1932,
              "last_1_minute": 0.2045,
              "last_5_minutes": 0.2787,
              "last_15_minutes": 0.3002,
              "last_1_hour": 0.3425,
              "lifetime": 0.6524
            }
          },
          "events": {
            "in": 630436,
            "duration_in_millis": 254578,
            "out": 630436
          }
        }
      ]
    },
    "reloads": {
      "failures": 0,
      "last_error": null,
      "last_success_timestamp": null,
      "last_failure_timestamp": null,
      "successes": 0
    },
    "queue": {
      "type": "memory",
      "events_count": 0,
      "queue_size_in_bytes": 0,
      "max_queue_size_in_bytes": 0
    },
    "hash": "e67181543b6342de495b8816791623537fc956fb00f198b719fda30a787c456b",
    "ephemeral_id": "3a09aacd-708e-4383-b975-e517b2690ad4"
  }
}

so-elasticsearch-pipelines-list | grep fort

"filebeat-8.7.1-fortinet-clientendpoint-pipeline",
 "filebeat-8.7.1-fortinet-firewall-event",
 "filebeat-8.7.1-fortinet-firewall-pipeline",
 "filebeat-8.7.1-fortinet-firewall-traffic",
 "filebeat-8.7.1-fortinet-firewall-utm",
 "filebeat-8.7.1-fortinet-fortimail-pipeline",
 "filebeat-8.8.2-fortinet-clientendpoint-pipeline",
 "filebeat-8.8.2-fortinet-firewall-event",
 "filebeat-8.8.2-fortinet-firewall-pipeline",
 "filebeat-8.8.2-fortinet-firewall-traffic",
 "filebeat-8.8.2-fortinet-firewall-utm",
 "filebeat-8.8.2-fortinet-fortimail-pipeline",

tail -f /opt/so/log/logstash/logstash.log
[2023-10-23T12:55:18,172][INFO ][logstash.javapipeline ] Pipelinemanageris configured withpipeline.ecs_compatibility: disabledsetting. All plugins in this pipeline will default toecs_compatibility => disabledunless explicitly configured otherwise. [2023-10-23T12:55:18,640][INFO ][logstash.javapipeline ] Starting pipeline {:pipeline_id=>"manager", "pipeline.workers"=>4, "pipeline.batch.size"=>125, "pipeline.batch.delay"=>50, "pipeline.max_inflight"=>500, "pipeline.sources"=>["/usr/share/logstash/pipelines/manager/0009_input_beats.conf", "/usr/share/logstash/pipelines/manager/0010_input_hhbeats.conf", "/usr/share/logstash/pipelines/manager/0011_input_endgame.conf", "/usr/share/logstash/pipelines/manager/9999_output_redis.conf"], :thread=>"#<Thread:0x9d6932f@/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:134 run>"} [2023-10-23T12:55:22,207][INFO ][logstash.javapipeline ] Pipeline Java execution initialization time {"seconds"=>3.57} [2023-10-23T12:55:22,518][INFO ][logstash.inputs.beats ] Starting input listener {:address=>"0.0.0.0:5044"} [2023-10-23T12:55:23,669][INFO ][logstash.inputs.beats ] Starting input listener {:address=>"0.0.0.0:5644"} [2023-10-23T12:55:24,638][INFO ][org.logstash.beats.Server] Starting server on port: 5044 [2023-10-23T12:55:24,641][INFO ][logstash.inputs.http ] Starting http input listener {:address=>"0.0.0.0:3765", :ssl=>"true"} [2023-10-23T12:55:24,641][INFO ][logstash.javapipeline ] Pipeline started {"pipeline.id"=>"manager"} [2023-10-23T12:55:24,643][INFO ][org.logstash.beats.Server] Starting server on port: 5644 [2023-10-23T12:55:24,714][INFO ][logstash.agent ] Pipelines running {:count=>1, :running_pipelines=>[:manager], :non_running_pipelines=>[]}

Something is wrong?

Thanks again.

[easy13]

Hi There,

Tried this:
so-elasticsearch-pipeline-stats | grep forti

  "filebeat-8.8.2-fortinet-firewall-event": {
  "filebeat-8.8.2-fortinet-clientendpoint-pipeline": {
  "filebeat-8.7.1-fortinet-firewall-traffic": {
  "filebeat-8.8.2-fortinet-firewall-pipeline": {
        "pipeline:filebeat-8.8.2-fortinet-firewall-event": {
        "pipeline:filebeat-8.8.2-fortinet-firewall-traffic": {
        "pipeline:filebeat-8.8.2-fortinet-firewall-utm": {
  "filebeat-8.8.2-fortinet-firewall-utm": {
  "filebeat-8.7.1-fortinet-firewall-pipeline": {
        "pipeline:filebeat-8.7.1-fortinet-firewall-event": {
        "pipeline:filebeat-8.7.1-fortinet-firewall-traffic": {
        "pipeline:filebeat-8.7.1-fortinet-firewall-utm": {
  "filebeat-8.7.1-fortinet-firewall-utm": {
  "filebeat-8.8.2-fortinet-fortimail-pipeline": {
  "filebeat-8.7.1-fortinet-firewall-event": {
  "filebeat-8.7.1-fortinet-clientendpoint-pipeline": {
  "filebeat-8.7.1-fortinet-fortimail-pipeline": {

Each result is null:

so-elasticsearch-pipeline-stats "filebeat-8.7.1-fortinet-firewall-pipeline"

"count": 0,
 "time_in_millis": 0,
 "current": 0,
 "failed": 0,
 "processors": [
   {
     "set": {
       "type": "set",
       "stats": {
         "count": 0,
         "time_in_millis": 0,
         "current": 0,
         "failed": 0

so-elasticsearch-pipeline-stats "filebeat-8.8.2-fortinet-firewall-traffic"

{
  "count": 0,
  "time_in_millis": 0,
  "current": 0,
  "failed": 0,
  "processors": [
    {
      "set": {
        "type": "set",
        "stats": {
          "count": 0,
          "time_in_millis": 0,
          "current": 0,
          "failed": 0
        }
      }
    },
    {
      "set": {
        "type": "set",
        "stats": {
          "count": 0,
          "time_in_millis": 0,
          "current": 0,
          "failed": 0
        }
      }
    },
    {
      "set": {
        "type": "set",
        "stats": {
          "count": 0,
          "time_in_millis": 0,
          "current": 0,
          "failed": 0
        }
      }
    },
    {
      "append": {
        "type": "append",
        "stats": {
          "count": 0,
          "time_in_millis": 0,
          "current": 0,
          "failed": 0
        }
      }
    },
    {
      "append": {
        "type": "append",
        "stats": {
          "count": 0,
          "time_in_millis": 0,
          "current": 0,
          "failed": 0
        }
      }
    },
    {
      "append": {
        "type": "append",
        "stats": {
          "count": 0,
          "time_in_millis": 0,
          "current": 0,
          "failed": 0
        }
      }
    },
    {
      "append": {
        "type": "append",
        "stats": {
          "count": 0,
          "time_in_millis": 0,
          "current": 0,
          "failed": 0
        }
      }
    },
    {
      "append": {
        "type": "append",
        "stats": {
          "count": 0,
          "time_in_millis": 0,
          "current": 0,
          "failed": 0
        }
      }
    },
    {
      "append": {
        "type": "append",
        "stats": {
          "count": 0,
          "time_in_millis": 0,
          "current": 0,
          "failed": 0
        }
      }
    },
    {
      "append": {
        "type": "append",
        "stats": {
          "count": 0,
          "time_in_millis": 0,
          "current": 0,
          "failed": 0
        }
      }
    },
    {
      "rename": {
        "type": "rename",
        "stats": {
          "count": 0,
          "time_in_millis": 0,
          "current": 0,
          "failed": 0
        }
      }
    },
    {
      "rename": {
        "type": "rename",
        "stats": {
          "count": 0,
          "time_in_millis": 0,
          "current": 0,
          "failed": 0
        }
      }
    },
    {
      "convert": {
        "type": "convert",
        "stats": {
          "count": 0,
          "time_in_millis": 0,
          "current": 0,
          "failed": 0
        }
      }
    },
    {
      "convert": {
        "type": "convert",
        "stats": {
          "count": 0,
          "time_in_millis": 0,
          "current": 0,
          "failed": 0
        }
      }
    },
    {
      "convert": {
        "type": "convert",
        "stats": {
          "count": 0,
          "time_in_millis": 0,
          "current": 0,
          "failed": 0
        }
      }
    },
    {
      "convert": {
        "type": "convert",
        "stats": {
          "count": 0,
          "time_in_millis": 0,
          "current": 0,
          "failed": 0
        }
      }
    },
    {
      "rename": {
        "type": "rename",
        "stats": {
          "count": 0,
          "time_in_millis": 0,
          "current": 0,
          "failed": 0
        }
      }
    },
    {
      "rename": {
        "type": "rename",
        "stats": {
          "count": 0,
          "time_in_millis": 0,
          "current": 0,
          "failed": 0
        }
      }
    },
    {
      "rename": {
        "type": "rename",
        "stats": {
          "count": 0,
          "time_in_millis": 0,
          "current": 0,
          "failed": 0
        }
      }
    },
    {
      "convert": {
        "type": "convert",
        "stats": {
          "count": 0,
          "time_in_millis": 0,
          "current": 0,
          "failed": 0
        }
      }
    },
    {
      "rename": {
        "type": "rename",
        "stats": {
          "count": 0,
          "time_in_millis": 0,
          "current": 0,
          "failed": 0
        }
      }
    },
    {
      "rename": {
        "type": "rename",
        "stats": {
          "count": 0,
          "time_in_millis": 0,
          "current": 0,
          "failed": 0
        }
      }
    },
    {
      "rename": {
        "type": "rename",
        "stats": {
          "count": 0,
          "time_in_millis": 0,
          "current": 0,
          "failed": 0
        }
      }
    },
    {
      "convert": {
        "type": "convert",
        "stats": {
          "count": 0,
          "time_in_millis": 0,
          "current": 0,
          "failed": 0
        }
      }
    },
    {
      "rename": {
        "type": "rename",
        "stats": {
          "count": 0,
          "time_in_millis": 0,
          "current": 0,
          "failed": 0
        }
      }
    },
    {
      "rename": {
        "type": "rename",
        "stats": {
          "count": 0,
          "time_in_millis": 0,
          "current": 0,
          "failed": 0
        }
      }
    },
    {
      "rename": {
        "type": "rename",
        "stats": {
          "count": 0,
          "time_in_millis": 0,
          "current": 0,
          "failed": 0
        }
      }
    },
    {
      "convert": {
        "type": "convert",
        "stats": {
          "count": 0,
          "time_in_millis": 0,
          "current": 0,
          "failed": 0
        }
      }
    },
    {
      "rename": {
        "type": "rename",
        "stats": {
          "count": 0,
          "time_in_millis": 0,
          "current": 0,
          "failed": 0
        }
      }
    },
    {
      "convert": {
        "type": "convert",
        "stats": {
          "count": 0,
          "time_in_millis": 0,
          "current": 0,
          "failed": 0
        }
      }
    },
    {
      "rename": {
        "type": "rename",
        "stats": {
          "count": 0,
          "time_in_millis": 0,
          "current": 0,
          "failed": 0
        }
      }
    },
    {
      "rename": {
        "type": "rename",
        "stats": {
          "count": 0,
          "time_in_millis": 0,
          "current": 0,
          "failed": 0
        }
      }
    },
    {
      "rename": {
        "type": "rename",
        "stats": {
          "count": 0,
          "time_in_millis": 0,
          "current": 0,
          "failed": 0
        }
      }
    },
    {
      "rename": {
        "type": "rename",
        "stats": {
          "count": 0,
          "time_in_millis": 0,
          "current": 0,
          "failed": 0
        }
      }
    },
    {
      "rename": {
        "type": "rename",
        "stats": {
          "count": 0,
          "time_in_millis": 0,
          "current": 0,
          "failed": 0
        }
      }
    },
    {
      "rename": {
        "type": "rename",
        "stats": {
          "count": 0,
          "time_in_millis": 0,
          "current": 0,
          "failed": 0
        }
      }
    },
    {
      "rename": {
        "type": "rename",
        "stats": {
          "count": 0,
          "time_in_millis": 0,
          "current": 0,
          "failed": 0
        }
      }
    },
    {
      "rename": {
        "type": "rename",
        "stats": {
          "count": 0,
          "time_in_millis": 0,
          "current": 0,
          "failed": 0
        }
      }
    },
    {
      "rename": {
        "type": "rename",
        "stats": {
          "count": 0,
          "time_in_millis": 0,
          "current": 0,
          "failed": 0
        }
      }
    },
    {
      "gsub": {
        "type": "gsub",
        "stats": {
          "count": 0,
          "time_in_millis": 0,
          "current": 0,
          "failed": 0
        }
      }
    },
    {
      "remove": {
        "type": "remove",
        "stats": {
          "count": 0,
          "time_in_millis": 0,
          "current": 0,
          "failed": 0
        }
      }
    }
  ]
}

Not sure what to try next...

Thanks

[cm-ops]

Change the below to you sensor's minion pillar file in /opt/so/saltstack/local/pillar/minions/ instead of the managersearch's then restart Filebeat. Also, move the firewall pillar entry to the sensor's. If your flow is Fortinet >> forwarder >> managersearch.

filebeat:
  third_party_filebeat:
    modules:
      fortinet:
        firewall:
          enabled: true
          var.input: udp
          var.syslog_host: 0.0.0.0
          var.syslog_port: 9004

[easy13]

Thanks you,

I'll try that...