Skip to content

Commit e18b57b

Browse files
committed
edit pass: selective-logging-analysis
1 parent c83e475 commit e18b57b

File tree

1 file changed

+27
-27
lines changed

1 file changed

+27
-27
lines changed

articles/hdinsight/selective-logging-analysis.md

Lines changed: 27 additions & 27 deletions
Original file line numberDiff line numberDiff line change
@@ -1,23 +1,23 @@
11
---
2-
title: Use the selective logging feature with a script action in Azure HDInsight clusters
2+
title: Use selective logging with a script action in Azure HDInsight clusters
33
description: Learn how to use the selective logging feature with a script action to monitor logs.
44
ms.service: hdinsight
55
ms.topic: how-to
66
ms.custom: seoapr2020, devx-track-azurepowershell, references-regions, devx-track-azurecli
77
ms.date: 07/31/2022
88
---
99

10-
# Use the selective logging feature with a script action in Azure HDInsight
10+
# Use selective logging with a script action in Azure HDInsight
1111

1212
[Azure Monitor Logs](../azure-monitor/logs/log-query-overview.md) is an Azure Monitor service that monitors your cloud and on-premises environments. The monitoring helps maintain their availability and performance.
1313

14-
Azure Monitor Logs collects data generated by resources in your cloud, resources in on-premises environments, and other monitoring tools. It uses the data to provide analysis across multiple sources. To get the analysis, you enable the selective logging feature by using a script action in the Azure portal in HDInsight.
14+
Azure Monitor Logs collects data generated by resources in your cloud, resources in on-premises environments, and other monitoring tools. It uses the data to provide analysis across multiple sources. To get the analysis, you enable the selective logging feature by using a script action for HDInsight in the Azure portal.
1515

1616
## About selective logging
1717

18-
Selective logging is a part of Azure overall monitoring system. After you connect your cluster to a Log Analytics workspace and enable selective logging, you can see logs and metrics like HDInsight security logs, Yarn Resource Manager, and system metrics. You can monitor workloads and see how they're affecting cluster stability.
18+
Selective logging is a part of the overall monitoring system in Azure. After you connect your cluster to a Log Analytics workspace and enable selective logging, you can see logs and metrics like HDInsight security logs, Yarn Resource Manager, and system metrics. You can monitor workloads and see how they're affecting cluster stability.
1919

20-
Selective logging allows you to enable or disable all the tables of selected tables in the Log Analytics workspace. You can adjust the source type for each table.
20+
Selective logging allows you to enable or disable all the tables, or enable selected tables, in the Log Analytics workspace. You can adjust the source type for each table.
2121

2222
> [!NOTE]
2323
> If Log Analytics is reinstalled in a cluster, you'll have to disable all the tables and log types again. Reinstallation resets all the configuration files to their original state.
@@ -55,7 +55,7 @@ For instructions on how to create an HDInsight cluster, see [Get started with Az
5555
1. Select all the node types that apply for the cluster. The options are head node, worker node, and Zookeeper node.
5656
1. Define the parameters. For example:
5757
- Spark: `spark HDInsightSparkLogs:SparkExecutorLog --disable`
58-
- Interactivehive: `interactivehive HDInsightSparkLogs:SparkExecutorLog --enable`
58+
- Interactive Query: `interactivehive HDInsightSparkLogs:SparkExecutorLog --enable`
5959
- Hadoop: `hadoop HDInsightSparkLogs:SparkExecutorLog --disable`
6060
- HBase: `hbase HDInsightSparkLogs: HDInsightHBaseLogs --enable`
6161

@@ -64,7 +64,7 @@ For instructions on how to create an HDInsight cluster, see [Get started with Az
6464
1. Select **Create**.
6565
1. After a few minutes, a green check mark appears next to your script action history. It means the script has successfully run.
6666

67-
:::image type="content" source="./media/hdinsight-hadoop-oms-selective-log-analytics-tutorial/enable-table-and-log-types.png" alt-text="Screenshot showing enable table and log types.":::
67+
:::image type="content" source="./media/hdinsight-hadoop-oms-selective-log-analytics-tutorial/enable-table-and-log-types.png" alt-text="Screenshot that shows a successful run of a script to enable tables and log types.":::
6868

6969
You'll see the desired changes in the Log Analytics workspace.
7070

@@ -101,7 +101,7 @@ If you submit your script action but there are no changes in the Log Analytics w
101101
> [!NOTE]
102102
> For the script to run in the latest cluster, the script must persist.
103103
104-
1. Make sure that you seleted all the node types that you wanted for the script action.
104+
1. Make sure that you selected all the node types that you wanted for the script action.
105105

106106
:::image type="content" source="./media/hdinsight-hadoop-oms-selective-log-analytics-tutorial/select-node-types.png" alt-text="Screenshot that shows selected node types.":::
107107

@@ -116,7 +116,7 @@ If the script action shows a **Failed** status in the script action history:
116116

117117
### Spark cluster
118118

119-
Different log types(sources) inside **Spark** tables
119+
The following table names are for different log types (sources) inside Spark tables.
120120

121121
| Source number | Table name | Log types | Description |
122122
| --- | --- | --- | --- |
@@ -128,38 +128,38 @@ Different log types(sources) inside **Spark** tables
128128
| 6. | HDInsightHadoopAnd YarnMetrics | No log types | This table contains JMX metrics from the Hadoop and YARN frameworks. It contains all the same JMX metrics as the old Custom Logs tables, plus more metrics that we considered important. We added Timeline Server, Node Manager, and Job History Server metrics. It contains one metric per record. |
129129
| 7. | HDInsightOozieLogs | Oozie | This table contains all logs generated from the Oozie framework. |
130130

131-
### Interactive query cluster
131+
### Interactive Query cluster
132132

133-
Different log types(sources) inside **interactive query** tables
133+
The following table names are for different log types (sources) inside Interactive Query tables.
134134

135135
| Source number | Table name | Log types | Description |
136136
| --- | --- | --- | --- |
137137
| 1. | HDInsightAmbariClusterAlerts | No log types | This table contains Ambari cluster alerts from each node in the cluster (except for edge nodes). Each alert is a record in this table. |
138138
| 2. | HDInsightAmbariSystem Metrics | No log types | This table contains system metrics collected from Ambari. The metrics now come from each node in the cluster (except for edge nodes) instead of just the two head nodes. Each metric is now a column, and each metric is reported once per record. |
139-
| 3. | HDInsightHadoopAndYarnLogs | **Head node** : MRJobSummary, Resource Manager, TimelineServer **Worker node:** NodeManager | This table contains all logs generated from the Hadoop and YARN frameworks. |
139+
| 3. | HDInsightHadoopAndYarnLogs | **Head node**: MRJobSummary, Resource Manager, TimelineServer **Worker node**: NodeManager | This table contains all logs generated from the Hadoop and YARN frameworks. |
140140
| 4. | HDInsightHadoopAndYarnMetrics | No log types | This table contains JMX metrics from the Hadoop and YARN frameworks. It contains all the same JMX metrics as the old Custom Logs tables, plus more metrics that we considered important. We added Timeline Server, Node Manager, and Job History Server metrics. It contains one metric per record. |
141141
| 5. | HDInsightHiveAndLLAPLogs | **Head node**: InteractiveHiveHSILog, InteractiveHiveMetastoreLog, ZeppelinLog | This table contains logs generated from Hive, LLAP, and their related components: WebHCat and Zeppelin. |
142142
| 6. | HDInsightHiveAndLLAPmetrics | No log types | This table contains JMX metrics from the Hive and LLAP frameworks. It contains all the same JMX metrics as the old Custom Logs tables. It contains one metric per record. |
143143
| 7. | HDInsightHiveTezAppStats | No log types |
144-
| 8. | HDInsightSecurityLogs | **Head node**: AmbariAuditLog, AuthLog **Zookeeper node, worker node**: AuthLog | This table contains records from the Ambari audit and authentication Logs. |
144+
| 8. | HDInsightSecurityLogs | **Head node**: AmbariAuditLog, AuthLog **Zookeeper node, worker node**: AuthLog | This table contains records from the Ambari audit and authentication logs. |
145145

146146
### HBase cluster
147147

148-
Different log types(sources) inside **HBase** tables
148+
The following table names are for different log types (sources) inside HBase tables.
149149

150150
| Source number | Table name | Log types | Description |
151151
| --- | --- | --- | --- |
152152
| 1. | HDInsightAmbariClusterAlerts | No other log types | This table contains Ambari cluster alerts from each node in the cluster (except for edge nodes). Each alert is a record in this table.
153153
| 2. | HDInsightAmbariSystem Metrics | No other log types | This table contains system metrics collected from Ambari. The metrics now come from each node in the cluster (except for edge nodes) instead of just the two head nodes. Each metric is now a column, and each metric is reported once per record. |
154-
| 3. | HDInsightHadoopAndYarnLogs | **Head node** : MRJobSummary, Resource Manager, TimelineServer **Worker node:** NodeManager | This table contains all logs generated from the Hadoop and YARN frameworks. |
155-
| 4. | HDInsightSecurityLogs | **Head node:** AmbariAuditLog, AuthLog **Worker node:** AuthLog **ZooKeper node:** AuthLog | This table contains records from the Ambari audit and authentication logs. |
156-
| 5. | HDInsightHBaseLogs | **Head node** : HDFSGarbageCollectorLog, HDFSNameNodeLog **Worker node:** PhoenixServerLog, HBaseRegionServerLog, HBaseRestServerLog **Zookeeper node:** HBaseMasterLog | This table contains logs from HBase and its related components: Phoenix and HDFS. |
157-
| 6. | HDInsightHBaseMetrics | No log types | This table contains JMX metrics from HBase. It contains all the same JMX metrics from the tables listed in the Old Schema column. In contrast from the old tables, each row contains one metric. |
154+
| 3. | HDInsightHadoopAndYarnLogs | **Head node**: MRJobSummary, Resource Manager, TimelineServer **Worker node**: NodeManager | This table contains all logs generated from the Hadoop and YARN frameworks. |
155+
| 4. | HDInsightSecurityLogs | **Head node**: AmbariAuditLog, AuthLog **Worker node**: AuthLog **Zookeeper node**: AuthLog | This table contains records from the Ambari audit and authentication logs. |
156+
| 5. | HDInsightHBaseLogs | **Head node**: HDFSGarbageCollectorLog, HDFSNameNodeLog **Worker node**: PhoenixServerLog, HBaseRegionServerLog, HBaseRestServerLog **Zookeeper node**: HBaseMasterLog | This table contains logs from HBase and its related components: Phoenix and HDFS. |
157+
| 6. | HDInsightHBaseMetrics | No log types | This table contains JMX metrics from HBase. It contains all the same JMX metrics from the tables listed in the **Old Schema** column. In contrast from the old tables, each row contains one metric. |
158158
| 7. | HDInsightHadoopAndYarn Metrics | No log types | This table contains JMX metrics from the Hadoop and YARN frameworks. It contains all the same JMX metrics as the old Custom Logs tables, plus more metrics that we considered important. We added Timeline Server, Node Manager, and Job History Server metrics. It contains one metric per record. |
159159

160160
### Hadoop cluster
161161

162-
Different log types(sources) inside Hadoop tables
162+
The following table names are for different log types (sources) inside Hadoop tables.
163163

164164
| Source number | Table name | Log types | Description |
165165
| --- | --- | --- | --- |
@@ -169,7 +169,7 @@ Different log types(sources) inside Hadoop tables
169169
| 4. | HDInsightHadoopAndYarnMetrics | No Log Types | This table contains JMX metrics from the Hadoop and YARN frameworks. It contains all the same JMX metrics as the old Custom Logs tables, plus more metrics that we considered important. We added Timeline Server, Node Manager, and Job History Server metrics. It contains one metric per record. |
170170
| 5. | HDInsightHiveAndLLAPLogs | **Head node**: HiveMetastoreLog, HiveServer2Log, WebHcatLog | This table contains logs generated from Hive, LLAP, and their related components: WebHCat and Zeppelin. |
171171
| 6. | HDInsight Hive And LLAP Metrics | No log types | This table contains JMX metrics from the Hive and LLAP frameworks. It contains all the same JMX metrics as the old Custom Logs tables. It contains one metric per record. |
172-
| 7. | HDInsight Security Logs | **Head node**: AmbariAuditLog, AuthLog **Zookeeper node**: AuthLog | This table contains records from the Ambari audit and authentication Logs. |
172+
| 7. | HDInsight Security Logs | **Head node**: AmbariAuditLog, AuthLog **Zookeeper node**: AuthLog | This table contains records from the Ambari audit and authentication logs. |
173173

174174
## Parameter syntax
175175

@@ -184,9 +184,7 @@ A parameter contains three parts:
184184

185185
### Syntax for multiple tables
186186

187-
When you have multiple tables, they're separated with a comma.
188-
189-
For example:
187+
When you have multiple tables, they're separated with a comma. For example:
190188

191189
`spark HDInsightSecurityLogs, HDInsightAmbariSystemMetrics --disable`
192190

@@ -197,21 +195,23 @@ For example:
197195
When you have multiple source types or log types, they're separated with a space.
198196

199197
To disable a source, write the table name that contains the log types, followed by a colon and then the real log type name:
200-
`TableName : LogTypeName`
198+
199+
`TableName : LogTypeName`
201200

202201
For example, assume that `spark HDInsightSecurityLogs` is a table that has two log types: `AmbariAuditLog` and `AuthLog`. To disable both the log types, the correct syntax would be:
203202

204203
`spark HDInsightSecurityLogs: AmbariAuditLog AuthLog --disable`
205204

206-
### Multiple tables and source types
207-
If there are two tables and two source types that you need to disable, use the following syntax:
205+
### Syntax for multiple tables and source types
206+
207+
If you need to disable two tables and two source types, use the following syntax:
208208

209209
- Spark: `InteractiveHiveMetastoreLog` log type in the `HDInsightHiveAndLLAPLogs` table
210210
- Hbase: `InteractiveHiveHSILog` log type in the `HDInsightHiveAndLLAPLogs` table
211211
- Hadoop: `HDInsightHiveAndLLAPMetrics` table
212212
- Hadoop: `HDInsightHiveTezAppStats` table
213213

214-
Separate the tables with a comma. Denote sources by uwing a colon after the table name in which they reside.
214+
Separate the tables with a comma. Denote sources by using a colon after the table name in which they reside.
215215

216216
The correct parameter syntax for these cases would be:
217217

0 commit comments

Comments
 (0)