You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
An available Log Analytics workspace is required for this approach.
20
-
21
-
- We recommended you note down the workspace ID and authentication Key of your Log Analytics workspace as you might need it for certain scenarios.
22
-
- This solution will increase the data that will be sent to the Log Analytics workspace and will have a small impact on overall cost. Please read on for details on how to keep the amount of data to a minimum.
19
+
An available Log Analytics workspace is required for this approach. We recommended you note down the workspace ID and authentication Key of your Log Analytics workspace as you might need it for certain scenarios. This solution will increase the data that will be sent to the Log Analytics workspace and will have a small impact on overall cost. Read on for details on how to keep the amount of data to a minimum.
23
20
24
21
## Objectives and scenarios
25
22
26
-
Centralize the events and the performance counter data to your Log Analytics workspace, first the virtual machine hosting the SHIR must be appropriately instrumented. Please choose between two main scenarios below.
23
+
Centralize the events and the performance counter data to your Log Analytics workspace, first the virtual machine hosting the SHIR must be appropriately instrumented. Choose between two main scenarios below.
27
24
28
25
### Instrumenting on-premises virtual machines
29
26
30
-
The article [Install Log Analytics agent on Windows computers](../azure-monitor/agents/agent-windows.md) describes how to install the client on a virtual machine typically hosted on-premises. This can be either a physical server or a virtual machine hosted on a customer managed hypervisor. As mentioned in the prerequisite section, when installing the Log Analytics agent, you will have to provide the Log Analytics workspace ID and Workspace Key to finalize the connection.
27
+
The article [Install Log Analytics agent on Windows computers](../azure-monitor/agents/agent-windows.md) describes how to install the client on a virtual machine typically hosted on-premises. This can be either a physical server or a virtual machine hosted on a customer managed hypervisor. As mentioned in the prerequisite section, when installing the Log Analytics agent, you'll have to provide the Log Analytics workspace ID and Workspace Key to finalize the connection.
31
28
32
-
### Instrumenting Azure virtual machines
29
+
### Instrument Azure virtual machines
33
30
34
-
The recommended approach to instrument an Azure virtual machine based SHIR is to use virtual machine insights as described in the article [Enable VM insights overview](../azure-monitor/vm/vminsights-enable-overview.md). Note that there are multiple ways to configure the Log Analytics agent when the SHIR is hosted in an Azure virtual machine. All the options are described in the article [Log Analytics agent overview](../azure-monitor/agents/log-analytics-agent.md#installation-options).
31
+
The recommended approach to instrument an Azure virtual machine based SHIR is to use virtual machine insights as described in the article [Enable VM insights overview](../azure-monitor/vm/vminsights-enable-overview.md). There are multiple ways to configure the Log Analytics agent when the SHIR is hosted in an Azure virtual machine. All the options are described in the article [Log Analytics agent overview](../azure-monitor/agents/log-analytics-agent.md#installation-options).
35
32
36
-
## Configuring event log and performance counter capture
33
+
## Configure event log and performance counter capture
37
34
38
-
This step will highlight how to configure both Event viewer logs and performance counters to be captured and sent over to Log Analytics. The steps described below are common regardless of how the agent was deployed.
35
+
This step will highlight how to configure both Event viewer logs and performance counters to be sent to Log Analytics. The steps described below are common regardless of how the agent was deployed.
39
36
40
-
### Selecting event viewer journals
37
+
### Select event viewer journals
41
38
42
39
First you must collect event viewer journals relevant to the SHIR as described in the article [Collect Windows event log data sources with Log Analytics agent in Azure Monitor](../azure-monitor/agents/data-sources-windows-events.md).
43
40
44
-
It's important to note that when choosing the event logs using the interface, it is normal that you will not see all journals that can possibly exist on a machine. Consequently, the two journals that we need for SHIR monitoring will not show up in this list. If you type the journal name exactly as it appears on the local virtual machine, it will be captured and sent to your Log analytics workspace.
41
+
It's important to note that when choosing the event logs using the interface, it's normal that you won't see all journals that can possibly exist on a machine. So, the two journals that we need for SHIR monitoring won't show up in this list. If you type the journal name exactly as it appears on the local virtual machine, it will be captured and sent to your Log analytics workspace.
45
42
46
43
The event journal name we must configure are:
47
44
48
45
- Connectors – Integration Runtime
49
46
- Integration Runtime
50
47
51
-
:::image type="content" source="media/how-to-configure-shir-for-log-analytics-collection/configure-journals-for-collection.png" alt-text="Shows the selection of the SHIR relevant logs with errors and warnings checked.":::
48
+
:::image type="content" source="media/how-to-configure-shir-for-log-analytics-collection/configure-journals-for-collection.png" alt-text="Screenshot of the selection of the SHIR relevant logs with errors and warnings checked.":::
52
49
53
50
> [!IMPORTANT]
54
51
> Leaving the **Information** level checked will increase the volume of data significantly if you have many SHIR hosts deployed and a larger number of scans. We strongly suggest you keep only Error and Warning.
55
52
56
-
### Selecting Performance counters
53
+
### Select Performance counters
57
54
58
-
In the same configuration pane, you can click on **Windows Performance Counters** to select individual performance counters to send up to log analytics. Keep in mind that performance counters are, by their nature, a continuous data stream. Therefore, it’s crucial that you consider the impact of data collection on the total cost of your Azure Monitor/Log Analytics deployment. Unless an allowed data ingestion budget has been granted and a constant ingestion of data has been allowed and budgeted for, gathering performance counters should only be configured for a defined period to establish a performance baseline.
55
+
In the same configuration pane, you can select on **Windows Performance Counters** to select individual performance counters to send up to log analytics. Keep in mind that performance counters are, by their nature, a continuous data stream. Therefore, it’s crucial that you consider the impact of data collection on the total cost of your Azure Monitor/Log Analytics deployment. Unless an allowed data ingestion budget has been granted and a constant ingestion of data has been allowed and budgeted for, gathering performance counters should only be configured for a defined period to establish a performance baseline.
59
56
60
-
In the interface, when first configuring it, a suggested counter set will be recommended. Select those that apply to the type of performance analysis you want to perform. **%CPU** and **Available memory** are commonly monitored counters but others like **Network bandwidth consumption** can be useful in scenarios where the data volume are large, and bandwidth or execution time are constrained.
57
+
In the interface, when first configuring it, a suggested counter set will be recommended. Select those that apply to the type of performance analysis you want to perform. **%CPU** and **Available memory** are commonly monitored counters but others like **Network bandwidth consumption** can be useful in scenarios where the data volume is large, and bandwidth or execution time are constrained.
61
58
62
-
:::image type="content" source="media/how-to-configure-shir-for-log-analytics-collection/select-counters.png" alt-text="Shows the counter selection interface in the Azure portal.":::
59
+
:::image type="content" source="media/how-to-configure-shir-for-log-analytics-collection/select-counters.png" alt-text="Screenshot of the counter selection interface in the Azure portal.":::
63
60
64
-
## Viewing Events and Performance counter data in Log Analytics
61
+
## View Events and Performance counter data in Log Analytics
65
62
66
-
Please consult this tutorial on [How to query data in Log Analytics](../azure-monitor/logs/log-analytics-tutorial.md).
63
+
Consult this tutorial on [How to query data in Log Analytics](../azure-monitor/logs/log-analytics-tutorial.md).
67
64
The two tables where the telemetry is saved are called Perf and Event respectively. The following query will check the row count to see if we have data flowing in. This would confirm if the instrumentation described above is working.
68
65
69
66
### Sample KQL queries
70
67
71
-
#### Checking row counts
68
+
#### Check row counts
72
69
73
70
```kusto
74
71
(
@@ -84,23 +81,23 @@ The two tables where the telemetry is saved are called Perf and Event respective
84
81
)
85
82
```
86
83
87
-
#### Querying events
84
+
#### Query events
88
85
89
-
##### Retrieving the first 10 event rows
86
+
##### Retrieve the first 10 event rows
90
87
91
88
```kusto
92
89
Event
93
90
| take 10
94
91
```
95
92
96
-
##### Retrieving the event count by message severity
93
+
##### Retrieve the event count by message severity
97
94
98
95
```kusto
99
96
Event
100
97
| summarize count() by EventLevelName
101
98
```
102
99
103
-
##### Rendering a pie chart of count by message severity
100
+
##### Render a pie chart of count by message severity
104
101
105
102
```kusto
106
103
Event
@@ -110,7 +107,7 @@ Event
110
107
111
108
##### Retrieve all errors with a particular text string
112
109
113
-
Here we are searching for all message that have the word _disconnected_ in them.
110
+
Here we are searching for all messages that have the word _disconnected_ in them.
114
111
115
112
```kusto
116
113
Event
@@ -119,7 +116,7 @@ Event
119
116
120
117
##### Multi-table search for a keyword without knowing the schema
121
118
122
-
The search command is useful when one does not know which column the information is contained in. This query returns all rows from the specified tables that has at least one column that contains the search term. The word is _disconnected_ in this example.
119
+
The search command is useful when one does not know which column the information is contained in. This query returns all rows from the specified tables that have at least one column that contains the search term. The word is _disconnected_ in this example.
123
120
124
121
```kusto
125
122
search in (Perf, Event) "disconnected"
@@ -134,7 +131,7 @@ Event
134
131
| where EventLog == "Connectors – Integration Runtime"
135
132
```
136
133
137
-
##### Using timespans to restrict query results
134
+
##### Use timespans to restrict query results
138
135
139
136
This query uses the same query as above but restricts results to those occurring 2 days ago or more recently.
140
137
@@ -146,14 +143,14 @@ Event
146
143
147
144
#### Query performance counter data
148
145
149
-
##### Getting the first 10 performance counter readings
146
+
##### Retrieve the first 10 performance counter readings
150
147
151
148
```kusto
152
149
Perf
153
150
| take 10
154
151
```
155
152
156
-
##### Getting a specific counter with time constraints
153
+
##### Retrieve a specific counter with time constraints
157
154
158
155
```kusto
159
156
Perf
@@ -165,7 +162,7 @@ Perf
165
162
166
163
Performance counters are hierarchical in nature, so be mindful to have enough _where_ predicates in your query to select only the specific counter you need.
167
164
168
-
##### 95th percentile for a given counter binned by 30 minute slices of the last 24 hours
165
+
##### Retrieve 95th percentile for a given counter binned by 30 minute slices of the last 24 hours
169
166
170
167
This example is all the counters for a specific network adapter.
Here we are making the object name and counter name a variable so we do not have to change the KQL query body to make changes to those selections later.
184
181
@@ -192,5 +189,8 @@ Perf
192
189
| summarize Value=max(CounterValue) by CounterName, TimeStamps=TimeGenerated
193
190
```
194
191
192
+
## Next Steps
195
193
194
+
-[Review integration runtime concepts in Azure Data Factory.](concepts-integration-runtime.md)
195
+
- Learn how to [create a self-hosted integration runtime in the Azure portal.](create-self-hosted-integration-runtime.md)
By default, the Self Hosted Integration Runtime’s diagnostic and performance telemetry is saved locally on the virtual or physical machine that is hosting it. Two broad categories of telemetry are of interest for monitoring the Self Hosted Integration Runtime.
18
18
19
19
## Event logs
20
20
21
-
When logged on locally to the Self Hosted Integration Runtime, specific events can be viewed using the [event viewer](/windows/win32/eventlog/viewing-the-event-log.md). The relevant events are captured in two event viewer journals named: **Connectors – Integration Runtime** and **Integration Runtime** respectively. While it’s possible to log in to the Self Hosted Integration Runtime hosts individually to view these events, it is also possible to stream these events to a Log Analytics workspace in Azure monitor for ease of query and centralization purposes.
21
+
When logged on locally to the Self Hosted Integration Runtime, specific events can be viewed using the [event viewer](/windows/win32/eventlog/viewing-the-event-log.md). The relevant events are captured in two event viewer journals named: **Connectors – Integration Runtime** and **Integration Runtime** respectively. While it’s possible to log on to to the Self Hosted Integration Runtime hosts individually to view these events, it's also possible to stream these events to a Log Analytics workspace in Azure monitor for ease of query and centralization purposes.
22
22
23
23
## Performance counters
24
24
25
25
Performance counters in Windows and Linux provide insight into the performance of hardware components, operating systems, and applications such as the Self Hosted Integration Runtime. The performance counters can be viewed and collected locally on the VM using the performance monitor tool. See the article on [using performance counters](/windows/win32/perfctrs/using-performance-counters.md) for more details.
26
26
27
-
## Centralizing log collection and analysis
27
+
## Centralize log collection and analysis
28
28
29
-
When a deployment requires a more in-depth level of analysis or has reached a certain scale, it becomes impractical to log on locally to each Self Hosted Integration Runtime host. Therefore, we recommend leveraging Azure Monitor and Azure Log Analytics specifically to collect that data and enable a single pane of glass monitoring for your Self Hosted Integration Runtimes. See the article on [Configuring the SHIR for log analytics collection](how-to-configure-shir-for-log-analytics-collection.md) for instructions on how to instrument your Self Hosted Integration Runtimes for Azure Monitor.
29
+
When a deployment requires a more in-depth level of analysis or has reached a certain scale, it becomes impractical to log on to locally to each Self Hosted Integration Runtime host. Therefore, we recommend using Azure Monitor and Azure Log Analytics specifically to collect that data and enable a single pane of glass monitoring for your Self Hosted Integration Runtimes. See the article on [Configuring the SHIR for log analytics collection](how-to-configure-shir-for-log-analytics-collection.md) for instructions on how to instrument your Self Hosted Integration Runtimes for Azure Monitor.
30
+
31
+
## Next Steps
32
+
33
+
-[How to configure SHIR for log analytics collection](how-to-configure-shir-for-log-analytics-collection.md)
34
+
-[Review integration runtime concepts in Azure Data Factory.](concepts-integration-runtime.md)
35
+
- Learn how to [create a self-hosted integration runtime in the Azure portal.](create-self-hosted-integration-runtime.md)
0 commit comments