Skip to content

Commit 8e37c56

Browse files
Apoorvkudesia-sumologicjpipkin1amee-sumo
authored
SUMO-265484 | Apoorv | Update. Azure Event Hub doc and add dashboards… (#5620)
* SUMO-265484 | Apoorv | Update. Azure Event Hub doc and add dashboards and monitors * SUMO-265484 | Apoorv | Fix. small typos in VM doc * Fix spelling error * SUMO-265484 | Apoorv | Add. screenshot links * Revert "fix Typos in Azure VM doc" This reverts commit 45f9fb4. # Conflicts: # docs/integrations/microsoft-azure/azure-virtual-machine.md * SUMO-265484 | Apoorv | unrevert the spelling fix removed * Update azure-virtual-machine.md * Update docs/integrations/microsoft-azure/azure-event-hubs.md Co-authored-by: John Pipkin (Sumo Logic) <[email protected]> * Update docs/integrations/microsoft-azure/azure-event-hubs.md Co-authored-by: John Pipkin (Sumo Logic) <[email protected]> * Update docs/integrations/microsoft-azure/azure-event-hubs.md Co-authored-by: John Pipkin (Sumo Logic) <[email protected]> * Update docs/integrations/microsoft-azure/azure-event-hubs.md Co-authored-by: John Pipkin (Sumo Logic) <[email protected]> * Update docs/integrations/microsoft-azure/azure-event-hubs.md Co-authored-by: John Pipkin (Sumo Logic) <[email protected]> * Update docs/integrations/microsoft-azure/azure-event-hubs.md Co-authored-by: John Pipkin (Sumo Logic) <[email protected]> * Update docs/integrations/microsoft-azure/azure-event-hubs.md Co-authored-by: John Pipkin (Sumo Logic) <[email protected]> * Update docs/integrations/microsoft-azure/azure-event-hubs.md Co-authored-by: John Pipkin (Sumo Logic) <[email protected]> --------- Co-authored-by: John Pipkin <[email protected]> Co-authored-by: Amee Lepcha <[email protected]>
1 parent 9b1b978 commit 8e37c56

File tree

1 file changed

+136
-14
lines changed

1 file changed

+136
-14
lines changed

docs/integrations/microsoft-azure/azure-event-hubs.md

Lines changed: 136 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -29,33 +29,155 @@ For more information on supported metrics, refer to [Azure documentation](https:
2929
Azure service sends monitoring data to Azure Monitor, which can then [stream data to Eventhub](https://learn.microsoft.com/en-us/azure/azure-monitor/essentials/stream-monitoring-data-event-hubs). Sumo Logic supports:
3030

3131
* Logs collection from [Azure Monitor](https://docs.microsoft.com/en-us/azure/monitoring-and-diagnostics/monitoring-get-started) using our [Azure Event Hubs source](/docs/send-data/collect-from-other-data-sources/azure-monitoring/ms-azure-event-hubs-source/).
32-
* Metrics collection using our [HTTP Logs and Metrics source](/docs/send-data/collect-from-other-data-sources/azure-monitoring/collect-metrics-azure-monitor/) via Azure Functions deployed using the ARM template.
32+
* Metrics collection using our [Azure Metrics Source](/docs/send-data/hosted-collectors/microsoft-source/azure-metrics-source).
3333

3434
You must explicitly enable diagnostic settings for each Event Hub Namespace you want to monitor. You can forward logs to the same event hub provided they satisfy the limitations and permissions as described [here](https://learn.microsoft.com/en-us/azure/azure-monitor/essentials/diagnostic-settings?tabs=portal#destination-limitations).
3535

3636
When you configure the event hubs source or HTTP source, plan your source category to ease the querying process. A hierarchical approach allows you to make use of wildcards. For example: `Azure/EventHub/Logs`, `Azure/EventHub/Metrics`.
3737

3838
### Configure metrics collection
3939

40-
In this section, you will configure a pipeline for shipping metrics from Azure Monitor to an Event Hub, onto an Azure Function, and finally to an HTTP Source on a hosted collector in Sumo Logic.
40+
import MetricsSourceBeta from '../../reuse/metrics-source-beta.md';
4141

42-
1. Create a hosted collector and tag the `tenant_name` field. You can get the tenant name using the instructions [here](https://learn.microsoft.com/en-us/azure/active-directory-b2c/tenant-management-read-tenant-name#get-your-tenant-name). <br/><img src={useBaseUrl('img/integrations/microsoft-azure/Azure-Storage-Tag-Tenant-Name.png')} alt="Azure Tag Tenant Name" style={{border: '1px solid gray'}} width="500" />
43-
1. [Configure an HTTP Source](/docs/send-data/collect-from-other-data-sources/azure-monitoring/collect-metrics-azure-monitor/#step-1-configure-an-http-source).
44-
1. [Configure and deploy the ARM Template](/docs/send-data/collect-from-other-data-sources/azure-monitoring/collect-metrics-azure-monitor/#step-2-configure-azure-resources-using-arm-template).
45-
1. [Export metrics to Event Hub](/docs/send-data/collect-from-other-data-sources/azure-monitoring/collect-metrics-azure-monitor/#step-3-export-metrics-for-a-particular-resource-to-event-hub). Perform the steps below for each Event Hub Namespaces that you want to monitor.
46-
* Choose `Stream to an event hub` as the destination.
47-
* Select `AllMetrics`.
48-
* Use the Event hub namespace created by the ARM template in Step 2 above. You can create a new Event hub or use the one created by the ARM template. You can use the default policy `RootManageSharedAccessKey` as the policy name.
42+
<MetricsSourceBeta/>
4943

5044
### Configure logs collection
5145

5246
In this section, you will configure a pipeline for shipping diagnostic logs from Azure Monitor to an Event Hub.
5347

54-
1. To set up the Azure Event Hubs source in Sumo Logic, refer to [Azure Event Hubs Source for Logs](/docs/send-data/collect-from-other-data-sources/azure-monitoring/ms-azure-event-hubs-source/).
55-
2. To create the Diagnostic settings in the Azure portal, refer to the [Azure documentation](https://learn.microsoft.com/en-us/azure/azure-monitor/essentials/diagnostic-settings?tabs=portal#create-diagnostic-settings). Perform the steps below for each Event Hub Namespaces that you want to monitor.
56-
* Choose `Stream to an event hub` as the destination.
57-
* Select `allLogs`.
58-
* Use the Event hub namespace and Event hub name configured in the previous step in the destination details section. You can use the default policy `RootManageSharedAccessKey` as the policy name.
48+
#### Diagnostic logs
49+
50+
1. To set up the Azure Event Hubs source in Sumo Logic, refer to the [Azure Event Hubs Source for Logs](/docs/send-data/collect-from-other-data-sources/azure-monitoring/ms-azure-event-hubs-source/).
51+
1. To create the diagnostic settings in Azure portal, refer to the [Azure documentation](https://learn.microsoft.com/en-us/azure/azure-monitor/essentials/diagnostic-settings?tabs=portal#create-diagnostic-settings). Perform the steps below for each Azure Event Hubs namespace that you want to monitor.
52+
1. Choose `Stream to an event hub` as the destination.
53+
1. Select `allLogs`.
54+
1. Use the Event Hub namespace and Event Hub name configured in the previous step in the destination details section. You can use the default policy `RootManageSharedAccessKey` as the policy name.<br/><img src={useBaseUrl('img/send-data/azure-eventgrid-logs.png')} alt="Azure Event Grid logs" style={{border: '1px solid gray'}} width="800" />
55+
1. Tag the location field in the source with the right location value. <br/><img src={useBaseUrl('img/integrations/microsoft-azure/Azure-Storage-Tag-Location.png')} alt="Azure Virtual Machine Tag Location" style={{border: '1px solid gray'}} width="400" />
56+
57+
#### Activity Logs
58+
59+
To collect activity logs, refer to the [Collecting Logs for the Azure Audit App from Event Hub](/docs/integrations/microsoft-azure/audit) section in the Azure Audit documentation. Do not perform this step in case you are already collecting activity logs for a subscription.
60+
61+
:::note
62+
Since this source contains logs from multiple regions, make sure that you do not tag this source with the location tag.
63+
:::
64+
65+
## Installing the Azure Event Hubs app
66+
67+
import AppInstallIndexV2 from '../../reuse/apps/app-install-index-option.md';
68+
69+
<AppInstallIndexV2/>
70+
71+
As part of the app installation process, the following fields will be created by default:
72+
73+
- `tenant_name`. This field is tagged at the collector level. You can get the tenant name using the instructions [here](https://learn.microsoft.com/en-us/azure/active-directory-b2c/tenant-management-read-tenant-name#get-your-tenant-name).
74+
- `location`. The region the resource name belongs to.
75+
- `subscription_id`. ID associated with a subscription where the resource is present.
76+
- `resource_group`. The resource group name where the Azure resource is present.
77+
- `provider_name`. Azure resource provider name (for example, Microsoft.Network).
78+
- `resource_type`. Azure resource type (for example, storage accounts).
79+
- `resource_name`. The name of the resource (for example, storage account name).
80+
- `service_type`. Type of the service that can be accessed with an Azure resource.
81+
- `service_name`. Services that can be accessed with an Azure resource. (For example, in Azure Container Instances the service is Subscriptions.)
82+
83+
## Viewing the Azure Event Hubs dashboards
84+
85+
import ViewDashboardsIndex from '../../reuse/apps/view-dashboards-index.md';
86+
87+
<ViewDashboardsIndex/>
88+
89+
### Overview
90+
91+
The **Azure Event Hubs - Overview** dashboard provides comprehensive details on Eventhubs and details such as overall number of requests, namespaces and instances, size by eventhubs, operation types, ingress and egress of data
92+
<img src={useBaseUrl('https://sumologic-app-data-v2.s3.us-east-1.amazonaws.com/dashboards/AzureEventHubs/Azure-Event-Hubs-Overview.png')} alt="Azure Event Hubs - Overview dashboard" style={{border: '1px solid gray'}} width="800" />
93+
94+
### Operations
95+
96+
The **Azure Event Hubs - Operations** dashboard provides details over the recent create, read, delete or update operations done by the event hubs.
97+
<img src={useBaseUrl('https://sumologic-app-data-v2.s3.us-east-1.amazonaws.com/dashboards/AzureEventHubs/Azure-Event-Hubs-Operations.png')} alt="Azure Event Hubs - Operations" style={{border: '1px solid gray'}} width="800" />
98+
99+
### Performance
100+
101+
The **Azure Event Hubs - Performance** dashboard provides insights into the performance of your Azure Event Hubs. This includes metrics on Replication lag and count, cluster CPU usage and memory usage.
102+
<img src={useBaseUrl('https://sumologic-app-data-v2.s3.us-east-1.amazonaws.com/dashboards/AzureEventHubs/Azure-Event-Hubs-Performance.png')} alt="Azure Event Hubs - Performance" style={{border: '1px solid gray'}} width="800" />
103+
104+
### Network
105+
106+
The **Azure Event Hubs - Network** dashboard provides details on network traffic and connectivity related to your Azure Event Hubs. This includes data on inbound and outbound traffic in bytes and message, connections and requests.
107+
<img src={useBaseUrl('https://sumologic-app-data-v2.s3.us-east-1.amazonaws.com/dashboards/AzureEventHubs/Azure-Event-Hubs-Network.png')} alt="Azure Event Hubs - Network" style={{border: '1px solid gray'}} width="800" />
108+
109+
### Kafka Overview
110+
111+
The **Azure Event Hubs - Kafka Overview** dashboard provides details on Kafka Coordinator events based on different operations count, Kafka Coordinator operations based on namespaces and clients, last 10 log messages and heartbeat events.
112+
<img src={useBaseUrl('https://sumologic-app-data-v2.s3.us-east-1.amazonaws.com/dashboards/AzureEventHubs/Azure-Event-Hubs-Kafka-Overview.png')} alt="Azure Event Hubs - Kafka Overview" style={{border: '1px solid gray'}} width="800" />
113+
114+
### Kafka Errors
115+
116+
The **Azure Event Hubs - Kafka Errors** dashboard provides information about Kafka related errors in Event Hubs including error count, errors by object and error messages, error by namespaces and last 10 Kafka error messages
117+
<img src={useBaseUrl('https://sumologic-app-data-v2.s3.us-east-1.amazonaws.com/dashboards/AzureEventHubs/Azure-Event-Hubs-Kafka-Errors.png')} alt="Azure Event Hubs - Kafka Errors" style={{border: '1px solid gray'}} width="800" />
118+
119+
### Errors
120+
121+
The **Azure Event Hubs - Errors** dashboard provides information about errors in Event Hubs including user errors, diagnostic errors, operation errors, top 10 error numbers and error messages, error trend and comparison analyses by types of activity, operation result and entity.
122+
<img src={useBaseUrl('https://sumologic-app-data-v2.s3.us-east-1.amazonaws.com/dashboards/AzureEventHubs/Azure-Event-Hubs-Errors.png')} alt="Azure Event Hubs - Errors" style={{border: '1px solid gray'}} width="800" />
123+
124+
### Audit
125+
126+
The **Azure Event Hubs - Audit** dashboard provides audit information on namespace level events, and cluster level events such as audit failures, auth failures, auth protocols, status and connections.
127+
<img src={useBaseUrl('https://sumologic-app-data-v2.s3.us-east-1.amazonaws.com/dashboards/AzureEventHubs/Azure-Event-Hubs-Audit.png')} alt="Azure Event Hubs - Operations" style={{border: '1px solid gray'}} width="800" />
128+
129+
### Administrative Operations
130+
131+
The **Azure Event Hubs - Administrative Operations** dashboard provides details on the operational activities and status of your Azure Event Hubs resources.
132+
133+
Use this dashboard to:
134+
* Monitor the distribution of operation types and their success rates to ensure proper functioning of your Event Hubs.
135+
* Identify potential issues by analyzing the top operations causing errors and correlating them with specific users or applications.
136+
* Track recent write and delete operations to maintain an audit trail of changes made to your Event Hubs.
137+
138+
<img src={useBaseUrl('https://sumologic-app-data-v2.s3.us-east-1.amazonaws.com/dashboards/AzureEventHubs/Azure-Event-Hubs-Administrative-Operations.png')} alt="Azure Event Hubs - Administrative Operations dashboard" style={{border: '1px solid gray'}} width="800" />
139+
140+
### Policy and Recommendations
141+
142+
The **Azure Event Hubs - Policy and Recommendations** dashboard provides details on policy events and recommendations for your Azure Event Hubs resources.
143+
144+
Use this dashboard to:
145+
* Monitor the success and failure rates of policy events to ensure proper configuration and compliance.
146+
* Track and analyse recent recommendations to improve the performance and security of your Event Hubs setup.
147+
* Identify trends in policy events and recommendations over time to proactively address potential issues.
148+
149+
<img src={useBaseUrl('https://sumologic-app-data-v2.s3.us-east-1.amazonaws.com/dashboards/AzureEventHubs/Azure-Event-Hubs-Policy-and-Recommendations.png')} alt="Azure Event Hubs - Policy and Recommendations dashboard" style={{border: '1px solid gray'}} width="800" />
150+
151+
## Create monitors for Azure Event Hubs
152+
153+
import CreateMonitors from '../../reuse/apps/create-monitors.md';
154+
155+
<CreateMonitors/>
156+
157+
### Azure Event Hubs alerts
158+
159+
These alerts are metric based and will work for all Azure Storage.
160+
161+
| Alert Name | Alert Description and Conditions | Alert Condition | Recover Condition |
162+
|:-----------------------------------------------------|:----------------------------------------------------------------------------------------------------------------------------------------------------------|:----------------|:------------------|
163+
| `Azure Event Hubs - Available Memory (Cluster Only)` | This alert is triggered when Average Available Memory Percentage is less than 10% and a warning alert is triggered at 20% available memory. | Count < 10 | Count > = 10 |
164+
| `Azure Event Hubs - CPU Usage (Cluster Only)` | This alert is triggered when Average CPU Usage is greater than 80% and a warning alert is triggered at 70% CPU usage. | Count > 80 | Count < = 80 |
165+
| `Azure Event Hubs - Incoming Messages` | This alert is triggered when Total Incoming Messages Count is greater than 1000. | Count > 1000 | Count < = 1000 |
166+
| `Azure Event Hubs - Server Errors` | This alert is triggered when Total Server Errors Count is greater than 1. | Count > 1 | Count < = 1 |
167+
| `Azure Event Hubs - Throttled Requests` | This alert is triggered when Total Throttled Requests Count is greater than 1. | Count > 1 | Count < = 1 |
168+
| `Azure Event Hubs - User Errors` | This alert is triggered when Total User Errors Count is greater than 1. | Count > 1 | Count < = 1 |
169+
170+
## Upgrade/Downgrade the Azure Event Hubs app (optional)
171+
172+
import AppUpdate from '../../reuse/apps/app-update.md';
173+
174+
<AppUpdate/>
175+
176+
## Uninstalling the Azure Event Hubs app (optional)
177+
178+
import AppUninstall from '../../reuse/apps/app-uninstall.md';
179+
180+
<AppUninstall/>
59181

60182
## Troubleshooting
61183

0 commit comments

Comments
 (0)