Skip to content

Commit 207873f

Browse files
committed
Curate for Azure Monitor duplication
1 parent d577562 commit 207873f

8 files changed

+50
-77
lines changed

articles/synapse-analytics/guidance/implementation-success-perform-monitoring-review.md

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -19,13 +19,13 @@ Using your solution requirements and other data collected during the [assessment
1919

2020
You can use [Azure Monitor](../../azure-monitor/overview.md) to provide base-level infrastructure metrics, alerts, and logs for most Azure services. Azure diagnostic logs are emitted by a resource to provide rich, frequent data about the operation of that resource. Azure Synapse can write diagnostic logs in Azure Monitor.
2121

22-
For more information, see [Use Azure Monitor with your Azure Synapse Analytics workspace](../monitoring/how-to-monitor-using-azure-monitor.md).
22+
For more information, see [Use Azure Monitor with your Azure Synapse Analytics workspace](../monitor-synapse-analytics.md).
2323

2424
## Monitor dedicated SQL pools
2525

26-
You can monitor a dedicated SQL pool by using Azure Monitor, altering, dynamic management views (DMVs), and Log Analytics.
26+
You can monitor a dedicated SQL pool by using Azure Monitor, alerting, dynamic management views (DMVs), and Log Analytics.
2727

28-
- **Alerts:** You can set up alerts that send you an email or call a webhook when a certain metric reaches a predefined threshold. For example, you can receive an alert email when the database size grows too large. For more information, see [Create alerts for Azure SQL Database and Azure Synapse Analytics using the Azure portal](/azure/azure-sql/database/alerts-insights-configure-portal).
28+
- **Alerts:** You can set up alerts that send you an email or call a webhook when a certain metric reaches a predefined threshold. For example, you can receive an alert email when the database size grows too large. For more information, see [Alerts](../monitor-synapse-analytics.md#alerts).
2929
- **DMVs:** You can use [DMVs](../sql-data-warehouse/sql-data-warehouse-manage-monitor.md) to monitor workloads to help investigate query executions in SQL pools.
3030
- **Log Analytics:** [Log Analytics](../../azure-monitor/logs/log-analytics-tutorial.md) is a tool in the Azure portal that you can use to edit and run log queries from data collected by Azure Monitor. For more information, see [Monitor workload - Azure portal](../sql-data-warehouse/sql-data-warehouse-monitor-workload-portal.md).
3131

articles/synapse-analytics/guidance/implementation-success-perform-operational-readiness-review.md

Lines changed: 1 addition & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -56,15 +56,7 @@ Set and document expectations for monitoring readiness with your business. These
5656

5757
Consider using [Azure Monitor](../../azure-monitor/overview.md) to collect, analyze, and act on telemetry data from your Azure and on-premises environments. Azure Monitor helps you maximize performance and availability of your applications by proactively identify problems in seconds.
5858

59-
List all the important metrics to monitor for each service in your solution along with their acceptable thresholds. For example, the following list includes important metrics to monitor for a dedicated SQL pool:
60-
61-
- `DWULimit`
62-
- `DWUUsed`
63-
- `AdaptiveCacheHitPercent`
64-
- `AdaptiveCacheUsedPercent`
65-
- `LocalTempDBUsedPercent`
66-
- `ActiveQueries`
67-
- `QueuedQueries`
59+
List all the important metrics to monitor for each service in your solution along with their acceptable thresholds. For example, to view metrics to monitor for a dedicated SQL pool, see [Supported metrics for Microsoft.Synapse/workspaces/sqlPools](../monitor-synapse-analytics-reference.md#supported-metrics-for-microsoftsynapseworkspacessqlpools).
6860

6961
Consider using [Azure Service Health](https://azure.microsoft.com/features/service-health/) to notify you about Azure service incidents and planned maintenance. That way, you can take action to mitigate downtime. You can set up customizable cloud alerts and use a personalized dashboard to analyze health issues, monitor the impact to your cloud resources, get guidance and support, and share details and updates.
7062

articles/synapse-analytics/monitor-synapse-analytics-reference.md

Lines changed: 35 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -22,6 +22,18 @@ The following table lists the metrics available for the Microsoft.Synapse/worksp
2222
[!INCLUDE [horz-monitor-ref-metrics-tableheader](~/reusable-content/ce-skilling/azure/includes/azure-monitor/horizontals/horz-monitor-ref-metrics-tableheader.md)]
2323
[!INCLUDE [Microsoft.Synapse/workspaces](~/azure-reference-other-repo/azure-monitor-ref/supported-metrics/includes/microsoft-synapse-workspaces-metrics-include.md)]
2424

25+
### Azure Synapse Link metrics
26+
27+
Azure Synapse Link emits the following metrics to Azure Monitor:
28+
29+
| **Metric** | **Aggregation types** | **Description** |
30+
|---------|---------|---------|
31+
| Link connection events | Sum | Number of Synapse Link connection events, including start, stop, and failure |
32+
| Link latency in seconds | Max, Min, Avg | Synapse Link data processing latency in seconds |
33+
| Link processed data volume (bytes) | Sum | Data volume in bytes processed by Synapse Link |
34+
| Link processed rows | Sum | Row counts processed by Synapse Link |
35+
| Link table events | Sum | Number of Synapse Link table events, including snapshot, removal, and failure |
36+
2537
### Supported metrics for Microsoft.Synapse/workspaces/bigDataPools
2638
The following table lists the metrics available for the Microsoft.Synapse/workspaces/bigDataPools resource type.
2739
[!INCLUDE [horz-monitor-ref-metrics-tableheader](~/reusable-content/ce-skilling/azure/includes/azure-monitor/horizontals/horz-monitor-ref-metrics-tableheader.md)]
@@ -108,6 +120,8 @@ For more information on these logs, see the following information:
108120
- [sys.dm_pdw_waits](/sql/relational-databases/system-dynamic-management-views/sys-dm-pdw-waits-transact-sql?view=azure-sqldw-latest&preserve-view=true)
109121
- [sys.dm_pdw_sql_requests](/sql/relational-databases/system-dynamic-management-views/sys-dm-pdw-sql-requests-transact-sql?view=azure-sqldw-latest&preserve-view=true)
110122

123+
To view the list of DMVs that apply to Synapse SQL, see [System views supported in Synapse SQL](./sql/reference-tsql-system-views.md#dedicated-sql-pool-dynamic-management-views-dmvs).
124+
111125
[!INCLUDE [horz-monitor-ref-logs-tables](~/reusable-content/ce-skilling/azure/includes/azure-monitor/horizontals/horz-monitor-ref-logs-tables.md)]
112126

113127
### Synapse Workspaces
@@ -138,6 +152,27 @@ Microsoft.Synapse/workspaces
138152
- [SynapseDXTableUsageStatistics](/azure/azure-monitor/reference/tables/SynapseDXTableUsageStatistics#columns)
139153
- [SynapseDXTableDetails](/azure/azure-monitor/reference/tables/SynapseDXTableDetails#columns)
140154

155+
### Available Apache Spark configurations
156+
157+
| Configuration name | Default value | Description |
158+
| ------------------ | ------------- | ----------- |
159+
| spark.synapse.logAnalytics.enabled | false | To enable the Log Analytics sink for the Spark applications, true. Otherwise, false. |
160+
| spark.synapse.logAnalytics.workspaceId | - | The destination Log Analytics workspace ID. |
161+
| spark.synapse.logAnalytics.secret | - | The destination Log Analytics workspace secret. |
162+
| spark.synapse.logAnalytics.keyVault.linkedServiceName | - | The Key Vault linked service name for the Log Analytics workspace ID and key. |
163+
| spark.synapse.logAnalytics.keyVault.name | - | The Key Vault name for the Log Analytics ID and key. |
164+
| spark.synapse.logAnalytics.keyVault.key.workspaceId | SparkLogAnalyticsWorkspaceId | The Key Vault secret name for the Log Analytics workspace ID. |
165+
| spark.synapse.logAnalytics.keyVault.key.secret | SparkLogAnalyticsSecret | The Key Vault secret name for the Log Analytics workspace |
166+
| spark.synapse.logAnalytics.uriSuffix | ods.opinsights.azure.com | The destination Log Analytics workspace [URI suffix](../azure-monitor/logs/data-collector-api.md#request-uri). If your workspace isn't in Azure global, you need to update the URI suffix according to the respective cloud. |
167+
| spark.synapse.logAnalytics.filter.eventName.match | - | Optional. The comma-separated spark event names, you can specify which events to collect. For example: `SparkListenerJobStart,SparkListenerJobEnd` |
168+
| spark.synapse.logAnalytics.filter.loggerName.match | - | Optional. The comma-separated log4j logger names, you can specify which logs to collect. For example: `org.apache.spark.SparkContext,org.example.Logger` |
169+
| spark.synapse.logAnalytics.filter.metricName.match | - | Optional. The comma-separated spark metric name suffixes, you can specify which metrics to collect. For example: `jvm.heap.used`|
170+
171+
> [!NOTE]
172+
> - For Microsoft Azure operated by 21Vianet, the `spark.synapse.logAnalytics.uriSuffix` parameter should be `ods.opinsights.azure.cn`.
173+
> - For Azure Government, the `spark.synapse.logAnalytics.uriSuffix` parameter should be `ods.opinsights.azure.us`.
174+
> - For any cloud except Azure, the `spark.synapse.logAnalytics.keyVault.name` parameter should be the fully qualified domain name (FQDN) of the Key Vault. For example, `AZURE_KEY_VAULT_NAME.vault.usgovcloudapi.net` for AzureUSGovernment.
175+
141176
[!INCLUDE [horz-monitor-ref-activity-log](~/reusable-content/ce-skilling/azure/includes/azure-monitor/horizontals/horz-monitor-ref-activity-log.md)]
142177

143178
- [Microsoft.Sql resource provider operations](/azure/role-based-access-control/permissions/databases#microsoftsql)

articles/synapse-analytics/spark/apache-spark-azure-log-analytics.md

Lines changed: 1 addition & 23 deletions
Original file line numberDiff line numberDiff line change
@@ -103,29 +103,7 @@ spark.synapse.logAnalytics.keyVault.key.secret <AZURE_KEY_VAULT_SECRET_KEY_NAME>
103103
spark.synapse.logAnalytics.keyVault.linkedServiceName <LINKED_SERVICE_NAME>
104104
```
105105

106-
#### Available Apache Spark configuration
107-
108-
| Configuration name | Default value | Description |
109-
| ------------------ | ------------- | ----------- |
110-
| spark.synapse.logAnalytics.enabled | false | To enable the Log Analytics sink for the Spark applications, true. Otherwise, false. |
111-
| spark.synapse.logAnalytics.workspaceId | - | The destination Log Analytics workspace ID. |
112-
| spark.synapse.logAnalytics.secret | - | The destination Log Analytics workspace secret. |
113-
| spark.synapse.logAnalytics.keyVault.linkedServiceName | - | The Key Vault linked service name for the Log Analytics workspace ID and key. |
114-
| spark.synapse.logAnalytics.keyVault.name | - | The Key Vault name for the Log Analytics ID and key. |
115-
| spark.synapse.logAnalytics.keyVault.key.workspaceId | SparkLogAnalyticsWorkspaceId | The Key Vault secret name for the Log Analytics workspace ID. |
116-
| spark.synapse.logAnalytics.keyVault.key.secret | SparkLogAnalyticsSecret | The Key Vault secret name for the Log Analytics workspace |
117-
| spark.synapse.logAnalytics.uriSuffix | ods.opinsights.azure.com | The destination Log Analytics workspace [URI suffix][uri_suffix]. If your workspace isn't in Azure global, you need to update the URI suffix according to the respective cloud. |
118-
| spark.synapse.logAnalytics.filter.eventName.match | - | Optional. The comma-separated spark event names, you can specify which events to collect. For example: `SparkListenerJobStart,SparkListenerJobEnd` |
119-
| spark.synapse.logAnalytics.filter.loggerName.match | - | Optional. The comma-separated log4j logger names, you can specify which logs to collect. For example: `org.apache.spark.SparkContext,org.example.Logger` |
120-
| spark.synapse.logAnalytics.filter.metricName.match | - | Optional. The comma-separated spark metric name suffixes, you can specify which metrics to collect. For example: `jvm.heap.used`|
121-
122-
> [!NOTE]
123-
> - For Microsoft Azure operated by 21Vianet, the `spark.synapse.logAnalytics.uriSuffix` parameter should be `ods.opinsights.azure.cn`.
124-
> - For Azure Government, the `spark.synapse.logAnalytics.uriSuffix` parameter should be `ods.opinsights.azure.us`.
125-
> - For any cloud except Azure, the `spark.synapse.logAnalytics.keyVault.name` parameter should be the fully qualified domain name (FQDN) of the Key Vault. For example, `AZURE_KEY_VAULT_NAME.vault.usgovcloudapi.net` for AzureUSGovernment.
126-
127-
[uri_suffix]: ../../azure-monitor/logs/data-collector-api.md#request-uri
128-
106+
For a list of Apache Spark configurations, see [Available Apache Spark configurations](../monitor-synapse-analytics-reference.md#available-apache-spark-configurations)
129107

130108
### Step 3: Upload your Apache Spark configuration to an Apache Spark pool
131109

articles/synapse-analytics/sql-data-warehouse/sql-data-warehouse-monitor-workload-portal.md

Lines changed: 2 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -31,13 +31,9 @@ For more information on workspaces, see [Create a Log Analytics workspace](../..
3131

3232
## Turn on Resource logs
3333

34-
Configure diagnostic settings to emit logs from your SQL pool. Logs consist of telemetry views equivalent to the most commonly used performance troubleshooting DMVs. Currently the following views are supported:
34+
Configure diagnostic settings to emit logs from your SQL pool. Logs consist of telemetry views equivalent to the most commonly used performance troubleshooting DMVs.
3535

36-
- [sys.dm_pdw_exec_requests](/sql/relational-databases/system-dynamic-management-views/sys-dm-pdw-exec-requests-transact-sql?toc=/azure/synapse-analytics/sql-data-warehouse/toc.json&bc=/azure/synapse-analytics/sql-data-warehouse/breadcrumb/toc.json&view=azure-sqldw-latest&preserve-view=true)
37-
- [sys.dm_pdw_request_steps](/sql/relational-databases/system-dynamic-management-views/sys-dm-pdw-request-steps-transact-sql?toc=/azure/synapse-analytics/sql-data-warehouse/toc.json&bc=/azure/synapse-analytics/sql-data-warehouse/breadcrumb/toc.json&view=azure-sqldw-latest&preserve-view=true)
38-
- [sys.dm_pdw_dms_workers](/sql/relational-databases/system-dynamic-management-views/sys-dm-pdw-dms-workers-transact-sql?toc=/azure/synapse-analytics/sql-data-warehouse/toc.json&bc=/azure/synapse-analytics/sql-data-warehouse/breadcrumb/toc.json&view=azure-sqldw-latest&preserve-view=true)
39-
- [sys.dm_pdw_waits](/sql/relational-databases/system-dynamic-management-views/sys-dm-pdw-waits-transact-sql?toc=/azure/synapse-analytics/sql-data-warehouse/toc.json&bc=/azure/synapse-analytics/sql-data-warehouse/breadcrumb/toc.json&view=azure-sqldw-latest&preserve-view=true)
40-
- [sys.dm_pdw_sql_requests](/sql/relational-databases/system-dynamic-management-views/sys-dm-pdw-sql-requests-transact-sql?toc=/azure/synapse-analytics/sql-data-warehouse/toc.json&bc=/azure/synapse-analytics/sql-data-warehouse/breadcrumb/toc.json&view=azure-sqldw-latest&preserve-view=true)
36+
For a list of views that are currently supported, see [Dynamic Management Views](../monitor-synapse-analytics-reference.md#dynamic-management-views-dmvs).
4137

4238
:::image type="content" source="./media/sql-data-warehouse-monitor-workload-portal/enable_diagnostic_logs.png" alt-text="Screenshot of the page to create a diagnostic setting in the Azure portal.":::
4339

0 commit comments

Comments
 (0)