You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
[Network Security Group (NSG) flow logs](nsg-flow-logs-overview.md) provide information that can be used to understand ingress and egress IP traffic on network interfaces. These flow logs show outbound and inbound flows on a per NSG rule basis, the NIC the flow applies to, 5-tuple information about the flow (Source/Destination IP, Source/Destination Port, Protocol), and if the traffic was allowed or denied.
18
18
19
-
You can have many NSGs in your network with flow logging enabled. This amount of logging data makes it cumbersome to parse and gain insights from your logs. This article provides a solution to centrally manage these NSG flow logs using Grafana, an open source graphing tool, ElasticSearch, a distributed search and analytics engine, and Logstash, which is an open source server-side data processing pipeline.
19
+
You can have many NSGs in your network with flow logging enabled. This amount of logging data makes it cumbersome to parse and gain insights from your logs. This article provides a solution to centrally manage these NSG flow logs using Grafana, an open source graphing tool, Elasticsearch, a distributed search and analytics engine, and Logstash, which is an open source server-side data processing pipeline.
20
20
21
21
## Scenario
22
22
23
-
NSG flow logs are enabled using Network Watcher and are stored in Azure blob storage. A Logstash plugin is used to connect and process flow logs from blob storage and send them to ElasticSearch. Once the flow logs are stored in ElasticSearch, they can be analyzed and visualized into customized dashboards in Grafana.
23
+
NSG flow logs are enabled using Network Watcher and are stored in Azure blob storage. A Logstash plugin is used to connect and process flow logs from blob storage and send them to Elasticsearch. Once the flow logs are stored in Elasticsearch, they can be analyzed and visualized into customized dashboards in Grafana.
@@ -32,7 +32,7 @@ For this scenario, you must have Network Security Group Flow Logging enabled on
32
32
33
33
### Setup considerations
34
34
35
-
In this example Grafana, ElasticSearch, and Logstash are configured on an Ubuntu LTS Server deployed in Azure. This minimal setup is used for running all three components - they are all running on the same VM. This setup should only be used for testing and non-critical workloads. Logstash, Elasticsearch, and Grafana can all be architected to scale independently across many instances. For more information, see the documentation for each of these components.
35
+
In this example Grafana, Elasticsearch, and Logstash are configured on an Ubuntu LTS Server deployed in Azure. This minimal setup is used for running all three components - they are all running on the same VM. This setup should only be used for testing and non-critical workloads. Logstash, Elasticsearch, and Grafana can all be architected to scale independently across many instances. For more information, see the documentation for each of these components.
36
36
37
37
### Install Logstash
38
38
@@ -47,7 +47,7 @@ The following instructions are used to install Logstash in Ubuntu. For instructi
47
47
sudo dpkg -i logstash-5.2.0.deb
48
48
```
49
49
50
-
2. Configure Logstash to parse the flow logs and send them to ElasticSearch. Create a Logstash.conf file using:
50
+
2. Configure Logstash to parse the flow logs and send them to Elasticsearch. Create a Logstash.conf file using:
51
51
52
52
```bash
53
53
sudo touch /etc/logstash/conf.d/logstash.conf
@@ -137,7 +137,7 @@ The input section designates the input source of the logs that Logstash will pro
137
137
138
138
The filter section then flattens each flow log file so that each individual flow tuple and its associated properties becomes a separate Logstash event.
139
139
140
-
Finally, the output section forwards each Logstash event to the ElasticSearch server. Feel free to modify the Logstash config file to suit your specific needs.
140
+
Finally, the output section forwards each Logstash event to the Elasticsearch server. Feel free to modify the Logstash config file to suit your specific needs.
141
141
142
142
### Install the Logstash input plugin for Azure Blob storage
For more information about this plug in, see [Logstash input plugin for Azure Storage Blobs](https://github.com/Azure/azure-diagnostics-tools/tree/master/Logstash/logstash-input-azureblob).
151
151
152
-
### Install ElasticSearch
152
+
### Install Elasticsearch
153
153
154
-
You can use the following script to install ElasticSearch. For information about installing ElasticSearch, see [Elastic Stack](https://www.elastic.co/guide/en/elastic-stack/current/index.html).
154
+
You can use the following script to install Elasticsearch. For information about installing Elasticsearch, see [Elastic Stack](https://www.elastic.co/guide/en/elastic-stack/current/index.html).
@@ -177,21 +177,21 @@ sudo service grafana-server start
177
177
178
178
For additional installation information, see [Installing on Debian / Ubuntu](https://docs.grafana.org/installation/debian/).
179
179
180
-
#### Add the ElasticSearch server as a data source
180
+
#### Add the Elasticsearch server as a data source
181
181
182
-
Next, you need to add the ElasticSearch index containing flow logs as a data source. You can add a data source by selecting **Add data source** and completing the form with the relevant information. A sample of this configuration can be found in the following screenshot:
182
+
Next, you need to add the Elasticsearch index containing flow logs as a data source. You can add a data source by selecting **Add data source** and completing the form with the relevant information. A sample of this configuration can be found in the following screenshot:
183
183
184
184

185
185
186
186
#### Create a dashboard
187
187
188
-
Now that you have successfully configured Grafana to read from the ElasticSearch index containing NSG flow logs, you can create and personalize dashboards. To create a new dashboard, select**Create your first dashboard**. The following sample graph configuration shows flows segmented by NSG rule:
188
+
Now that you have successfully configured Grafana to read from the Elasticsearch index containing NSG flow logs, you can create and personalize dashboards. To create a new dashboard, select**Create your first dashboard**. The following sample graph configuration shows flows segmented by NSG rule:
By integrating Network Watcher with ElasticSearch and Grafana, you now have a convenient and centralized way to manage and visualize NSG flow logs as well as other data. Grafana has a number of other powerful graphing features that can also be used to further manage flow logs and better understand your network traffic. Now that you have a Grafana instance set up and connected to Azure, feel free to continue to explore the other functionality that it offers.
194
+
By integrating Network Watcher with Elasticsearch and Grafana, you now have a convenient and centralized way to manage and visualize NSG flow logs as well as other data. Grafana has a number of other powerful graphing features that can also be used to further manage flow logs and better understand your network traffic. Now that you have a Grafana instance set up and connected to Azure, feel free to continue to explore the other functionality that it offers.
0 commit comments