Filebeat/Netflow error "Exiting: module netflow is configured but has no enabled filesets"

[kingtriumph]

I'm trying to get netflow (or more specifically sFlow, as I'm using HP/Aruba gear) to ingest in Security Onion version 2.3.180. When trying to enable the ingestion pipeline, I get the dreaded error "Exiting: module netflow is configured but has no enabled filesets"

I have been using the following documentation to configure netflow:
https://www.youtube.com/watch?v=ew5gtVjAs7g
https://docs.securityonion.net/en/2.3/filebeat.html

And I've looked at various forum posts about the error referenced above, including a recent one from this forum:
#8745

So far I am unable to get the pipeline enabled. I am running a distributed server setup. I have tried running the following command on both the manager node and the search node:

docker exec -i so-filebeat filebeat setup modules -pipelines -modules netflow -c /usr/share/filebeat/module-setup.yml

I get the same error on both the manager and search nodes.

Here is the relevant (sanitized) config of the manager pillar .sls file:

filebeat:
  third_party_filebeat:
    modules:
      okta:
        system:
          enabled: true
          var.url: https://oktaurl.here/stuff
          var.api_key: '####'
      fortinet:
        firewall:
          enabled: true
          var.input: udp
          var.syslog_host: 0.0.0.0
          var.syslog_port: 9004
      o365:
        audit:
          enabled: true
          var.application_id: '####'
          var.tenants:
            - id: '####'
              name: 'faketenant.onmicrosoft.com'
          var.certificate: '/usr/share/filebeat/keynamehere.pem'
          var.key: '/usr/share/filebeat/keynamehere2.key'
      netflow:
        log:
          enabled: true
          var.netflow_host: 0.0.0.0
          var.netflow_port: 2055

firewall:
  assigned_hostgroups:
    chain:
      DOCKER-USER:
        hostgroups:
          fortigate:
            portgroups:
              - portgroups.fortigate-syslog
          netflow:
            portgroups:
              - portgroups.netflow
      INPUT:
        hostgroups:
          fortigate:
            portgroups:
              - portgroups.fortigate-syslog
          netflow:
            portgroups:
              - portgroups.netflow

init.sls relevant port binding:

- 0.0.0.0:2055:2055/udp

I can provide other information if necessary. There are devices on the allowed networks that are configured to send sFlow traffic to the Manager node on udp/2055. (Aruba defaults to port udp/6343 for sFlow, but it has been configured to use udp/2055).

Any assistance is very much appreciated. Thanks.

[InfosecGoon]

Good morning!

As noted in that other discussion post that you linked, you should need to manually enable the fileset on a newer version of SO, like the one that you're running.

A few things to check:

• Does "sudo so-firewall includedhosts netflow" show the IPs or subnets that you are expecting traffic from?
• Does "sudo so-firewall listports netflow udp" return 2055?
• Does "sudo docker ps | grep 2055" show the so-filebeat container accepting connections on that port?
• Does "sudo iptables -L | grep 2055" show the firewall hole to let the traffic into the local host and Docker environment?

If that's all clear, then the traffic should be able to come from your devices to the filebeat module.

If you run "sudo so-filebeat-module-setup", does it list the netflow module in the output as its setting up the ingest pipelines?

If all that looks good, try sending traffic to 2055/UDP using a Netflow generator (something like https://github.com/nerdalert/nflow-generator) to confirm that the issue isn't something on the sending/formatting side.

Hope that helps!

--Matt

[kingtriumph]

Hi Matt!

Thanks very much for the response. I get the expected results from the first three commands, but the iptables -L | grep 2055 command returns nothing.

I've rewatched the netflow setup video referenced above and I have confirmed that I've followed the instructions as provided in that video. The command iptables --list -n | grep 2055 returns the expected results. This is basically the same command you suggested above... I'm not sure why the -n option would change the output that much.

I'll see about testing with the nflow generator as well. Thank you.

[kingtriumph]

Hello again. It took a little bit, but I was finally able to get back to troubleshooting this issue.

I have run so-filebeat-module-setup again and it does list netflow as one of the modules. I've also set up an instance of the nflow-generator application and tested netflow ingestion using it. I do receive netflow packets from nflow-generator and can view them as part of the netflow.log dataset in Kibana. I am still not seeing "netflow" packets from my switches though. Technically speaking, it is sFlow, not netflow that I'm sending from my switches. However, I have configured them to use the standard netflow port udp/2055 instead of the default sFlow port of udp/6343. There really isn't much to the switch-side configuration. I've set a sFlow instance pointing at my manager node for the receiver and set the sampling and polling to the standards for a 1 Gbps port.

Any other thoughts on what I might be missing here? All assistance is very much appreciated. Thanks.

[InfosecGoon]

Ah, that may be the problem - the netflow filebeat module supports netflow and IPFIX, but not sflow. Sorry, I missed that in your original post.

From the documentatation at https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-module-netflow.html :

"This is a module for receiving NetFlow and IPFIX flow records over UDP. This input supports NetFlow versions 1, 5, 6, 7, 8 and 9, as well as IPFIX. "

There are some projects for doing sflow parsing as a Logstash codec, but I have no experience with them.

--Matt

[kingtriumph]

Hey Matt, I appreciate the response. Glad to have confirmation that sFlow isn't supported natively in SO or Elastic at least! I might try installing sflowtool on my manager node to see if that will get me where I need to be. Thanks again for your assistance!