Skip to content

Commit 2f60374

Browse files
committed
PR review changes
1 parent 8901f5c commit 2f60374

File tree

2 files changed

+46
-40
lines changed

2 files changed

+46
-40
lines changed

articles/data-factory/how-to-configure-shir-for-log-analytics-collection.md

Lines changed: 34 additions & 34 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
---
2-
title: Configuring the self-hosted integration runtime for log analytics collection
2+
title: Configure the self-hosted integration runtime for log analytics collection
33
titleSuffix: Azure Data Factory & Azure Synapse
4-
description: This topic describes how to instrument the self-hosted integration runtime for log analytics collection.
4+
description: This article describes how to instrument the self-hosted integration runtime for log analytics collection.
55
author: jonburchel
66
ms.service: data-factory
77
ms.subservice:
@@ -11,64 +11,61 @@ ms.date: 02/22/2022
1111
ms.author: jburchel
1212
---
1313

14-
# Configuring the self-hosted integration runtime (SHIR) for log analytics collection
14+
# Configure the self-hosted integration runtime (SHIR) for log analytics collection
1515
[!INCLUDE[appliesto-adf-asa-md](includes/appliesto-adf-asa-md.md)]
1616

1717
## Prerequisites
1818

19-
An available Log Analytics workspace is required for this approach.
20-
21-
- We recommended you note down the workspace ID and authentication Key of your Log Analytics workspace as you might need it for certain scenarios.
22-
- This solution will increase the data that will be sent to the Log Analytics workspace and will have a small impact on overall cost. Please read on for details on how to keep the amount of data to a minimum.
19+
An available Log Analytics workspace is required for this approach. We recommended you note down the workspace ID and authentication Key of your Log Analytics workspace as you might need it for certain scenarios. This solution will increase the data that will be sent to the Log Analytics workspace and will have a small impact on overall cost. Read on for details on how to keep the amount of data to a minimum.
2320

2421
## Objectives and scenarios
2522

26-
Centralize the events and the performance counter data to your Log Analytics workspace, first the virtual machine hosting the SHIR must be appropriately instrumented. Please choose between two main scenarios below.
23+
Centralize the events and the performance counter data to your Log Analytics workspace, first the virtual machine hosting the SHIR must be appropriately instrumented. Choose between two main scenarios below.
2724

2825
### Instrumenting on-premises virtual machines
2926

30-
The article [Install Log Analytics agent on Windows computers](../azure-monitor/agents/agent-windows.md) describes how to install the client on a virtual machine typically hosted on-premises. This can be either a physical server or a virtual machine hosted on a customer managed hypervisor. As mentioned in the prerequisite section, when installing the Log Analytics agent, you will have to provide the Log Analytics workspace ID and Workspace Key to finalize the connection.
27+
The article [Install Log Analytics agent on Windows computers](../azure-monitor/agents/agent-windows.md) describes how to install the client on a virtual machine typically hosted on-premises. This can be either a physical server or a virtual machine hosted on a customer managed hypervisor. As mentioned in the prerequisite section, when installing the Log Analytics agent, you'll have to provide the Log Analytics workspace ID and Workspace Key to finalize the connection.
3128

32-
### Instrumenting Azure virtual machines
29+
### Instrument Azure virtual machines
3330

34-
The recommended approach to instrument an Azure virtual machine based SHIR is to use virtual machine insights as described in the article [Enable VM insights overview](../azure-monitor/vm/vminsights-enable-overview.md). Note that there are multiple ways to configure the Log Analytics agent when the SHIR is hosted in an Azure virtual machine. All the options are described in the article [Log Analytics agent overview](../azure-monitor/agents/log-analytics-agent.md#installation-options).
31+
The recommended approach to instrument an Azure virtual machine based SHIR is to use virtual machine insights as described in the article [Enable VM insights overview](../azure-monitor/vm/vminsights-enable-overview.md). There are multiple ways to configure the Log Analytics agent when the SHIR is hosted in an Azure virtual machine. All the options are described in the article [Log Analytics agent overview](../azure-monitor/agents/log-analytics-agent.md#installation-options).
3532

36-
## Configuring event log and performance counter capture
33+
## Configure event log and performance counter capture
3734

38-
This step will highlight how to configure both Event viewer logs and performance counters to be captured and sent over to Log Analytics. The steps described below are common regardless of how the agent was deployed.
35+
This step will highlight how to configure both Event viewer logs and performance counters to be sent to Log Analytics. The steps described below are common regardless of how the agent was deployed.
3936

40-
### Selecting event viewer journals
37+
### Select event viewer journals
4138

4239
First you must collect event viewer journals relevant to the SHIR as described in the article [Collect Windows event log data sources with Log Analytics agent in Azure Monitor](../azure-monitor/agents/data-sources-windows-events.md).
4340

44-
It's important to note that when choosing the event logs using the interface, it is normal that you will not see all journals that can possibly exist on a machine. Consequently, the two journals that we need for SHIR monitoring will not show up in this list. If you type the journal name exactly as it appears on the local virtual machine, it will be captured and sent to your Log analytics workspace.
41+
It's important to note that when choosing the event logs using the interface, it's normal that you won't see all journals that can possibly exist on a machine. So, the two journals that we need for SHIR monitoring won't show up in this list. If you type the journal name exactly as it appears on the local virtual machine, it will be captured and sent to your Log analytics workspace.
4542

4643
The event journal name we must configure are:
4744

4845
- Connectors – Integration Runtime
4946
- Integration Runtime
5047

51-
:::image type="content" source="media/how-to-configure-shir-for-log-analytics-collection/configure-journals-for-collection.png" alt-text="Shows the selection of the SHIR relevant logs with errors and warnings checked.":::
48+
:::image type="content" source="media/how-to-configure-shir-for-log-analytics-collection/configure-journals-for-collection.png" alt-text="Screenshot of the selection of the SHIR relevant logs with errors and warnings checked.":::
5249

5350
> [!IMPORTANT]
5451
> Leaving the **Information** level checked will increase the volume of data significantly if you have many SHIR hosts deployed and a larger number of scans. We strongly suggest you keep only Error and Warning.
5552
56-
### Selecting Performance counters
53+
### Select Performance counters
5754

58-
In the same configuration pane, you can click on **Windows Performance Counters** to select individual performance counters to send up to log analytics. Keep in mind that performance counters are, by their nature, a continuous data stream. Therefore, it’s crucial that you consider the impact of data collection on the total cost of your Azure Monitor/Log Analytics deployment. Unless an allowed data ingestion budget has been granted and a constant ingestion of data has been allowed and budgeted for, gathering performance counters should only be configured for a defined period to establish a performance baseline.
55+
In the same configuration pane, you can select on **Windows Performance Counters** to select individual performance counters to send up to log analytics. Keep in mind that performance counters are, by their nature, a continuous data stream. Therefore, it’s crucial that you consider the impact of data collection on the total cost of your Azure Monitor/Log Analytics deployment. Unless an allowed data ingestion budget has been granted and a constant ingestion of data has been allowed and budgeted for, gathering performance counters should only be configured for a defined period to establish a performance baseline.
5956

60-
In the interface, when first configuring it, a suggested counter set will be recommended. Select those that apply to the type of performance analysis you want to perform. **%CPU** and **Available memory** are commonly monitored counters but others like **Network bandwidth consumption** can be useful in scenarios where the data volume are large, and bandwidth or execution time are constrained.
57+
In the interface, when first configuring it, a suggested counter set will be recommended. Select those that apply to the type of performance analysis you want to perform. **%CPU** and **Available memory** are commonly monitored counters but others like **Network bandwidth consumption** can be useful in scenarios where the data volume is large, and bandwidth or execution time are constrained.
6158

62-
:::image type="content" source="media/how-to-configure-shir-for-log-analytics-collection/select-counters.png" alt-text="Shows the counter selection interface in the Azure portal.":::
59+
:::image type="content" source="media/how-to-configure-shir-for-log-analytics-collection/select-counters.png" alt-text="Screenshot of the counter selection interface in the Azure portal.":::
6360

64-
## Viewing Events and Performance counter data in Log Analytics
61+
## View Events and Performance counter data in Log Analytics
6562

66-
Please consult this tutorial on [How to query data in Log Analytics](../azure-monitor/logs/log-analytics-tutorial.md).
63+
Consult this tutorial on [How to query data in Log Analytics](../azure-monitor/logs/log-analytics-tutorial.md).
6764
The two tables where the telemetry is saved are called Perf and Event respectively. The following query will check the row count to see if we have data flowing in. This would confirm if the instrumentation described above is working.
6865

6966
### Sample KQL queries
7067

71-
#### Checking row counts
68+
#### Check row counts
7269

7370
```kusto
7471
(
@@ -84,23 +81,23 @@ The two tables where the telemetry is saved are called Perf and Event respective
8481
)
8582
```
8683

87-
#### Querying events
84+
#### Query events
8885

89-
##### Retrieving the first 10 event rows
86+
##### Retrieve the first 10 event rows
9087

9188
```kusto
9289
Event
9390
| take 10
9491
```
9592

96-
##### Retrieving the event count by message severity
93+
##### Retrieve the event count by message severity
9794

9895
```kusto
9996
Event
10097
| summarize count() by EventLevelName
10198
```
10299

103-
##### Rendering a pie chart of count by message severity
100+
##### Render a pie chart of count by message severity
104101

105102
```kusto
106103
Event
@@ -110,7 +107,7 @@ Event
110107

111108
##### Retrieve all errors with a particular text string
112109

113-
Here we are searching for all message that have the word _disconnected_ in them.
110+
Here we are searching for all messages that have the word _disconnected_ in them.
114111

115112
```kusto
116113
Event
@@ -119,7 +116,7 @@ Event
119116

120117
##### Multi-table search for a keyword without knowing the schema
121118

122-
The search command is useful when one does not know which column the information is contained in. This query returns all rows from the specified tables that has at least one column that contains the search term. The word is _disconnected_ in this example.
119+
The search command is useful when one does not know which column the information is contained in. This query returns all rows from the specified tables that have at least one column that contains the search term. The word is _disconnected_ in this example.
123120

124121
```kusto
125122
search in (Perf, Event) "disconnected"
@@ -134,7 +131,7 @@ Event
134131
| where EventLog == "Connectors – Integration Runtime"
135132
```
136133

137-
##### Using timespans to restrict query results
134+
##### Use timespans to restrict query results
138135

139136
This query uses the same query as above but restricts results to those occurring 2 days ago or more recently.
140137

@@ -146,14 +143,14 @@ Event
146143

147144
#### Query performance counter data
148145

149-
##### Getting the first 10 performance counter readings
146+
##### Retrieve the first 10 performance counter readings
150147

151148
```kusto
152149
Perf
153150
| take 10
154151
```
155152

156-
##### Getting a specific counter with time constraints
153+
##### Retrieve a specific counter with time constraints
157154

158155
```kusto
159156
Perf
@@ -165,7 +162,7 @@ Perf
165162

166163
Performance counters are hierarchical in nature, so be mindful to have enough _where_ predicates in your query to select only the specific counter you need.
167164

168-
##### 95th percentile for a given counter binned by 30 minute slices of the last 24 hours
165+
##### Retrieve 95th percentile for a given counter binned by 30 minute slices of the last 24 hours
169166

170167
This example is all the counters for a specific network adapter.
171168

@@ -178,7 +175,7 @@ Perf
178175
| summarize percentile(CounterValue, 95) by bin(TimeGenerated, 30m), Computer, ObjectName, InstanceName, CounterName
179176
```
180177

181-
##### Using variables for code reusability
178+
##### Use variables for code reusability
182179

183180
Here we are making the object name and counter name a variable so we do not have to change the KQL query body to make changes to those selections later.
184181

@@ -192,5 +189,8 @@ Perf
192189
| summarize Value=max(CounterValue) by CounterName, TimeStamps=TimeGenerated
193190
```
194191

192+
## Next Steps
195193

194+
- [Review integration runtime concepts in Azure Data Factory.](concepts-integration-runtime.md)
195+
- Learn how to [create a self-hosted integration runtime in the Azure portal.](create-self-hosted-integration-runtime.md)
196196

Lines changed: 12 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
---
2-
title: Monitoring self-hosted intgegration runtime (SHIR) virtual machines in Azure
2+
title: Monitor self-hosted integration runtime (SHIR) virtual machines in Azure
33
titleSuffix: Azure Data Factory & Azure Synapse
4-
description: This topic describes how to monitor Azure virtual machines hosting the self-hosted integration runtime.
4+
description: This article describes how to monitor Azure virtual machines hosting the self-hosted integration runtime.
55
author: jonburchel
66
ms.service: data-factory
77
ms.subservice:
@@ -11,19 +11,25 @@ ms.date: 02/22/2022
1111
ms.author: jburchel
1212
---
1313

14-
# Monitoring self-hosted intgegration runtime (SHIR) virtual machines in Azure
14+
# Monitor self-hosted integration runtime (SHIR) virtual machines in Azure
1515
[!INCLUDE[appliesto-adf-asa-md](includes/appliesto-adf-asa-md.md)]
1616

1717
By default, the Self Hosted Integration Runtime’s diagnostic and performance telemetry is saved locally on the virtual or physical machine that is hosting it. Two broad categories of telemetry are of interest for monitoring the Self Hosted Integration Runtime.
1818

1919
## Event logs
2020

21-
When logged on locally to the Self Hosted Integration Runtime, specific events can be viewed using the [event viewer](/windows/win32/eventlog/viewing-the-event-log.md). The relevant events are captured in two event viewer journals named: **Connectors – Integration Runtime** and **Integration Runtime** respectively. While it’s possible to log in to the Self Hosted Integration Runtime hosts individually to view these events, it is also possible to stream these events to a Log Analytics workspace in Azure monitor for ease of query and centralization purposes.
21+
When logged on locally to the Self Hosted Integration Runtime, specific events can be viewed using the [event viewer](/windows/win32/eventlog/viewing-the-event-log.md). The relevant events are captured in two event viewer journals named: **Connectors – Integration Runtime** and **Integration Runtime** respectively. While it’s possible to log on to to the Self Hosted Integration Runtime hosts individually to view these events, it's also possible to stream these events to a Log Analytics workspace in Azure monitor for ease of query and centralization purposes.
2222

2323
## Performance counters
2424

2525
Performance counters in Windows and Linux provide insight into the performance of hardware components, operating systems, and applications such as the Self Hosted Integration Runtime. The performance counters can be viewed and collected locally on the VM using the performance monitor tool. See the article on [using performance counters](/windows/win32/perfctrs/using-performance-counters.md) for more details.
2626

27-
## Centralizing log collection and analysis
27+
## Centralize log collection and analysis
2828

29-
When a deployment requires a more in-depth level of analysis or has reached a certain scale, it becomes impractical to log on locally to each Self Hosted Integration Runtime host. Therefore, we recommend leveraging Azure Monitor and Azure Log Analytics specifically to collect that data and enable a single pane of glass monitoring for your Self Hosted Integration Runtimes. See the article on [Configuring the SHIR for log analytics collection](how-to-configure-shir-for-log-analytics-collection.md) for instructions on how to instrument your Self Hosted Integration Runtimes for Azure Monitor.
29+
When a deployment requires a more in-depth level of analysis or has reached a certain scale, it becomes impractical to log on to locally to each Self Hosted Integration Runtime host. Therefore, we recommend using Azure Monitor and Azure Log Analytics specifically to collect that data and enable a single pane of glass monitoring for your Self Hosted Integration Runtimes. See the article on [Configuring the SHIR for log analytics collection](how-to-configure-shir-for-log-analytics-collection.md) for instructions on how to instrument your Self Hosted Integration Runtimes for Azure Monitor.
30+
31+
## Next Steps
32+
33+
- [How to configure SHIR for log analytics collection](how-to-configure-shir-for-log-analytics-collection.md)
34+
- [Review integration runtime concepts in Azure Data Factory.](concepts-integration-runtime.md)
35+
- Learn how to [create a self-hosted integration runtime in the Azure portal.](create-self-hosted-integration-runtime.md)

0 commit comments

Comments
 (0)