Adding ElastiFlow to monitor Network flow

[tientmse62290]

Hi all,
After migrating to SO2.3, I'm very missing the Network flow summary & view from SO16. So I want to make a tutorial/document contributed by the community to this part for a more useful SO. I'm a newbie and lack knowledge about SO so let sharing to make it better!
I will list down all the steps we need to configure ElastiFlow with a normal ELK Stack and you will fill the tutorial step in the SO2.3 docker environment with your experience then I will update to make a complete document. Thanks all!

Please provide any methods that we can keep the config after the salt update

1. Update logstash & install plugins on sensors

sudo /usr/share/logstash/bin/logstash-plugin install logstash-codec-sflow
sudo /usr/share/logstash/bin/logstash-plugin update logstash-codec-netflow
sudo /usr/share/logstash/bin/logstash-plugin update logstash-input-udp
sudo /usr/share/logstash/bin/logstash-plugin update logstash-input-tcp
sudo /usr/share/logstash/bin/logstash-plugin update logstash-filter-dns
sudo /usr/share/logstash/bin/logstash-plugin update logstash-filter-geoip

2. Clone ElastiFlow & configure the pipeline:

• This pipeline config is on sensors.

cd #Home
mkdir flowtemp
cd flowtemp
sudo git clone https://github.com/robcowart/elastiflow.git
sudo cp -arv elastiflow/logstash/elastiflow/. /etc/logstash/elastiflow/
sudo cp -arv elastiflow/logstash.service.d/. /etc/systemd/system/logstash.service.d/
sudo nano /etc/logstash/pipelines.yml
**add pipeline:
---
- pipeline.id: elastiflow
  path.config: "/etc/logstash/elastiflow/conf.d/*.conf"
---

Then reboot sensor!
Verify that Logstash has started…It should be like:

sudo tail /var/log/logstash/logstash-plain.log -f
[2020-02-08T16:13:00,860][INFO ][logstash.inputs.udp      ] Starting UDP listener {:address=>"0.0.0.0:6343"}
[2020-02-08T16:13:00,884][INFO ][logstash.inputs.udp      ] Starting UDP listener {:address=>"0.0.0.0:2055"}
[2020-02-08T16:13:00,968][INFO ][logstash.agent           ] Pipelines running {:count=>1, :running_pipelines=>[:elastiflow], :non_running_pipelines=>[]}
[2020-02-08T16:13:00,987][WARN ][logstash.inputs.udp      ] Unable to set receive_buffer_bytes to desired size. Requested 33554432 but obtained 212992 bytes.
[2020-02-08T16:13:00,988][WARN ][logstash.inputs.udp      ] Unable to set receive_buffer_bytes to desired size. Requested 33554432 but obtained 212992 bytes.
[2020-02-08T16:13:00,989][WARN ][logstash.inputs.udp      ] Unable to set receive_buffer_bytes to desired size. Requested 33554432 but obtained 212992 bytes.
[2020-02-08T16:13:01,002][INFO ][logstash.inputs.udp      ] UDP listener started {:address=>"0.0.0.0:4739", :receive_buffer_bytes=>"212992", :queue_size=>"4096"}
[2020-02-08T16:13:01,002][INFO ][logstash.inputs.udp      ] UDP listener started {:address=>"0.0.0.0:6343", :receive_buffer_bytes=>"212992", :queue_size=>"4096"}
[2020-02-08T16:13:01,002][INFO ][logstash.inputs.udp      ] UDP listener started {:address=>"0.0.0.0:2055", :receive_buffer_bytes=>"212992", :queue_size=>"4096"}
[2020-02-08T16:13:01,220][INFO ][logstash.agent           ] Successfully started Logstash API endpoint {:port=>9600}

3. Adding dashboard
   -Download the dashboard from https://github.com/robcowart/elastiflow/tree/master/kibana
   -Log on the Kibana to Management/Saved Objects. Import the above file.
   -Restart service
   -Logon in again and go to the dashboard. You will be requested to create an index. Just use * and next and follow the prompts.

4. Enjoy!

Again very much appreciate for any contribution!!!

[dougburks]

What do you mean by Network flow summary & view from SO16?