You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: articles/storage-mover/log-monitoring.md
+15-49Lines changed: 15 additions & 49 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -8,32 +8,20 @@ ms.topic: how-to
8
8
ms.date: 03/20/2023
9
9
---
10
10
11
-
# Monitoring Azure Storage Mover copy and job logs
11
+
# How to enable Azure Storage Mover copy and job logs
12
12
13
-
When you use a migration tool to move your critical data from on-premises sources to Azure destination targets, you want to monitor copy operations for outages and errors. Monitoring includes metrics and logs that provide visibility into the success of your migration. **Copy logs** and **Job run logs**are especially useful because they allow you to trace the migration result of individual files. Metrics relating to **Job run metrics** are also available, and can be analyzed to help pinpoint periods of low performance and other trends.
13
+
When you use a migration tool to move your critical data from on-premises sources to Azure destination targets, you want to be able to monitor operations for potential issues. The data relating to the operations performed during your migration can be stored as either logs entries or metrics. When configured, Azure Storage Mover can provide **Copy logs** and **Job run logs**. These logs are especially useful because they allow you to trace the migration result of job runs and of individual files.
14
14
15
-
Logs and metrics can be sent or streamed to the destinations listed in the following table. To ensure the security of data in transit, we strongly encourage you to configure Transport Layer Security (TLS). All destination endpoints support TLS 1.2.
15
+
Both the copy and job run logs can be sent to an Azure Analytics Workspace. Analytics workspaces are storage units where log data generated by Azure services are stored. Log Analytics is also integrated into the Storage Mover portal experience. This integration allows you to create and run log queries from multiple logs and interactively analyze their results.
| **Log Analytics workspace** | Metrics are converted to log form. This option might not be available for all resource types. Sending them to the Azure Monitor Logs store (which is searchable via Log Analytics) helps you to integrate them into queries, alerts, and visualizations with existing log data.
20
-
|**Azure Storage account**| Archiving logs and metrics to a Storage account is useful for audit, static analysis, or backup. Compared to using Azure Monitor Logs or a Log Analytics workspace, Storage is less expensive, and logs can be kept there indefinitely. |
21
-
|**Azure Event Hubs**| When you send logs and metrics to Event Hubs, you can stream data to external systems such as third-party SIEMs and other Log Analytics solutions. |
22
-
|**Azure Monitor partner integrations**| Specialized integrations can be made between Azure Monitor and other non-Microsoft monitoring platforms. Integration is useful when you're already using one of the partners. |
17
+
> [!IMPORTANT]
18
+
> Before you can access migrations's log data, you need to ensure that you've created an Azure Analytics Workspace and configured your Storage Mover instance to use it. It's important to complete the steps described within this article before running your migration jobs to collect the resulting data. Any migration log data generated prior to configuration will not be accessible.
23
19
24
-
This article describes the monitoring data reported from Azure Storage Mover, and how you can use the features of Azure Monitor to analyze errors during the migration. All other destinations are outside the scope of this article, but can be referenced in the [Diagnostic settings in Azure Monitor](/azure/azure-monitor/essentials/diagnostic-settings?WT.mc_id=Portal-Microsoft_Azure_Monitoring&tabs=portal) article.
20
+
This article describes the steps involved to create an analytics workspace and to configure a diagnostic setting within a storage mover resource.
25
21
26
-
## What is Azure Monitor?
22
+
## Configuring Azure Log Analytics and Storage Mover
27
23
28
-
Azure Storage Mover sends copy and job run monitoring data to Azure Monitor, Azure's full stack monitoring service. The Azure Monitor service provides a complete set of features to monitor your Azure resources and resources in other clouds and on-premises.
29
-
30
-
One of Azure Monitor's most useful features is the Log Analytics tool. The Log Analytics tool is designed to help you monitor your migration projects by enabling you to edit and run log queries against your copy and job logs.
31
-
32
-
You can read more about Azure Monitor by visiting the [Azure Monitor overview](../azure-monitor/overview.md) article.
33
-
34
-
## Configuring Azure Monitor and Storage Mover
35
-
36
-
Before you can view the logs generated during your migration, you need to ensure that you've configured both Azure Monitor and your Storage Mover instance. This section briefly describes how to configure an Azure Monitor Log Analytics Workspace and Storage Mover diagnostic settings. After completing the following steps, you'll be able to query the data provided by your Storage Mover resource.
24
+
This section briefly describes how to configure an Azure Monitor Log Analytics Workspace and Storage Mover diagnostic settings. After completing the following steps, you'll be able to query the data provided by your Storage Mover resource.
37
25
38
26
### Create a Log Analytics workspace
39
27
@@ -65,17 +53,9 @@ After your Storage Mover diagnostic setting has been saved, it will be reflected
65
53
66
54
## Analyzing logs
67
55
68
-
All resource logs in Azure Monitor have the same fields, and are followed by service-specific fields. The common schema is outlined in [Azure Monitor resource log schema](../azure-monitor/essentials/resource-logs-schema.md).
69
-
70
-
Storage Mover generates two logs, copy logs and job run logs.
71
-
72
-
Storage Mover generates two tables, StorageMoverCopyLogsFailed and StorageMoverJobRunLogs.
56
+
All resource logs in Azure Monitor have the same fields, and are followed by service-specific fields. The common schema is outlined in [Azure Monitor resource log schema](../azure-monitor/essentials/resource-logs-schema.md).
73
57
74
-
The schema for StorageMoverCopyLogsFailed is found in [Azure Storage Copy log data reference](/azure/azure-monitor/reference/tables/StorageMoverCopyLogsFailed).
75
-
76
-
The schema for StorageMoverJobRunLogs is found in [Azure Storage Job run log data reference](/azure/azure-monitor/reference/tables/StorageMoverJobRunLogs).
77
-
78
-
Log entries are created only if there are requests made against the service endpoint. For example, if a storage account has activity in its file endpoint but not in its table or queue endpoints, only logs that pertain to the Azure Blob Storage service are created. Azure Storage logs contain detailed information about successful and failed requests to a storage service. This information can be used to monitor individual requests and to diagnose issues with a storage service. Requests are logged on a best-effort basis.
58
+
Storage Mover generates two tables, StorageMoverCopyLogsFailed and StorageMoverJobRunLogs. The schema for **StorageMoverCopyLogsFailed** is found in the [Azure Storage Copy log data reference](/azure/azure-monitor/reference/tables/StorageMoverCopyLogsFailed), and the schema for **StorageMoverJobRunLogs** is found in [Azure Storage Job run log data reference](/azure/azure-monitor/reference/tables/StorageMoverJobRunLogs).
79
59
80
60
### Sample Kusto queries
81
61
@@ -118,26 +98,12 @@ The sample queries provided below can be entered in the **Log search** bar to he
118
98
| render piechart
119
99
```
120
100
121
-
## Alerts
122
-
123
-
Azure Monitor alerts proactively notify you when important conditions are found in your monitoring data. They allow you to identify and address issues in your system before your customers notice them. You can set alerts on [metrics](../azure-monitor/alerts/alerts-metric-overview.md), [logs](../azure-monitor/alerts/alerts-unified-log.md), and the [activity log](../azure-monitor/alerts/activity-log-alerts.md).
124
-
125
-
The following table lists some example scenarios to monitor and the proper metric to use for the alert:
126
-
127
-
| Scenario | Metric to use for alert |
128
-
|-|-|
129
-
| Storage Mover service is throttled. | Metric: Transactions<br>Dimension name: Response type |
130
-
| Storage Mover requests are successful 99% of the time. | Metric: Availability<br>Dimension names: Geo type, API name, Authentication |
131
-
| Storage Mover egress has exceeded 500 GiB in one day. | Metric: Egress<br>Dimension names: Geo type, API name, Authentication |
132
-
133
101
## Next steps
134
102
135
103
Get started with any of these guides.
136
104
137
-
| Guide | Description |
138
-
|---|---|
139
-
| [Getting started with Azure Metrics Explorer](../azure-monitor/essentials/metrics-getting-started.md) | A tour of Metrics Explorer.
140
-
| [Overview of Log Analytics in Azure Monitor](../azure-monitor/logs/log-analytics-overview.md) | A tour of Log Analytics. |
141
-
| [Azure Monitor Metrics overview](../azure-monitor/essentials/data-platform-metrics.md) | The basics of metrics and metric dimensions |
142
-
| [Azure Monitor Logs overview](../azure-monitor/logs/data-platform-logs.md)| The basics of logs and how to collect and analyze them |
143
-
| [Troubleshoot Storage Mover issues](log-monitoring.md)| Common performance issues and guidance about how to troubleshoot them. |
0 commit comments