You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: articles/site-recovery/azure-to-azure-troubleshoot-replication.md
+52-41Lines changed: 52 additions & 41 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -4,30 +4,33 @@ description: Troubleshoot replication in Azure VM disaster recovery with Azure S
4
4
author: sideeksh
5
5
manager: rochakm
6
6
ms.topic: troubleshooting
7
-
ms.date: 8/2/2019
8
-
7
+
ms.date: 04/03/2020
9
8
---
9
+
10
10
# Troubleshoot replication in Azure VM disaster recovery
11
11
12
-
This article describes common problems in Azure Site Recovery when you're replicating and recovering Azure virtual machines from one region to another region. It also explains how to troubleshoot the common problems. For more information about supported configurations, see the [support matrix for replicating Azure VMs](site-recovery-support-matrix-azure-to-azure.md).
12
+
This article describes common problems in Azure Site Recovery when you're replicating and recovering Azure virtual machines (VM) from one region to another region. It also explains how to troubleshoot the common problems. For more information about supported configurations, see the [support matrix for replicating Azure VMs](site-recovery-support-matrix-azure-to-azure.md).
13
13
14
14
Azure Site Recovery consistently replicates data from the source region to the disaster recovery region. It also creates a crash-consistent recovery point every 5 minutes. If Site Recovery can't create recovery points for 60 minutes, it alerts you with this information:
15
15
16
-
Error message: "No crash consistent recovery point available for the VM in the last 60 minutes."</br>
16
+
```plaintext
17
+
Error message: "No crash consistent recovery point available for the VM in the last 60 minutes."
18
+
17
19
Error ID: 153007
20
+
```
18
21
19
22
The following sections describe causes and solutions.
20
23
21
-
## <aname="high-data-change-rate-on-the-source-virtal-machine"></a>High data change rate on the source virtual machine
24
+
## High data change rate on the source virtual machine
22
25
23
26
Azure Site Recovery creates an event if the data change rate on the source virtual machine is higher than the supported limits. To see whether the problem is because of high churn, go to **Replicated items** > **VM** > **Events - last 72 hours**.
24
-
You should see the event "Data change rate beyond supported limits":
27
+
You should see the event **Data change rate beyond supported limits**:
25
28
26
-

29
+
:::image type="content" source="./media/site-recovery-azure-to-azure-troubleshoot/data_change_event.png" alt-text="Azure Site Recovery page that shows a high data change rate that is too high.":::
27
30
28
31
If you select the event, you should see the exact disk information:
29
32
30
-

33
+
:::image type="content" source="./media/site-recovery-azure-to-azure-troubleshoot/data_change_event2.png" alt-text="Page that shows the data change rate event details.":::
31
34
32
35
### Azure Site Recovery limits
33
36
@@ -48,96 +51,104 @@ Premium P20 or P30 or P40 or P50 disk | 16 KB or greater |20 MB/s | 1684 GB per
48
51
49
52
Azure Site Recovery has limits on data change rates, depending on the type of disk. To see if this problem is recurring or temporary, find the data change rate of the affected virtual machine. Go to the source virtual machine, find the metrics under **Monitoring**, and add the metrics as shown in this screenshot:
50
53
51
-

54
+
:::image type="content" source="./media/site-recovery-azure-to-azure-troubleshoot/churn.png" alt-text="Page that shows the three-step process for finding the data change rate.":::
52
55
53
56
1. Select **Add metric**, and add **OS Disk Write Bytes/Sec** and **Data Disk Write Bytes/Sec**.
54
57
1. Monitor the spike as shown in the screenshot.
55
58
1. View the total write operations happening across OS disks and all data disks combined. These metrics might not give you information at the per-disk level, but they indicate the total pattern of data churn.
56
59
57
60
A spike in data change rate might come from an occasional data burst. If the data change rate is greater than 10 MB/s (for Premium) or 2 MB/s (for Standard) and comes down, replication will catch up. If the churn is consistently well beyond the supported limit, consider one of these options:
58
61
59
-
- Exclude the disk that's causing a high data-change rate: First, disable the replication. Then you can exclude the disk by using [PowerShell](./azure-to-azure-exclude-disks.md).
60
-
- Change the tier of the disaster recovery storage disk: This option is possible only if the disk data churn is less than 20 MB/s. Let's say a VM with a P10 disk has a data churn of greater than 8 MB/s but less than 10 MB/s. If the customer can use a P30 disk for target storage during protection, the problem can be solved. This solution is only possible for machines that are using Premium-Managed Disks. Follow these steps:
62
+
- Exclude the disk that's causing a high data-change rate: First, disable the replication. Then you can exclude the disk by using [PowerShell](azure-to-azure-exclude-disks.md).
63
+
- Change the tier of the disaster recovery storage disk: This option is possible only if the disk data churn is less than 20 MB/s. For example, a VM with a P10 disk has a data churn of greater than 8 MB/s but less than 10 MB/s. If the customer can use a P30 disk for target storage during protection, the problem can be solved. This solution is only possible for machines that are using Premium-Managed Disks. Follow these steps:
61
64
62
-
1. Go to **Disks** of the affected replicated machine and copy the replica disk name.
63
-
1. Go to this replica of the managed disk.
64
-
1. You might see a banner in **Overview** that says an SAS URL has been generated. Select this banner and cancel the export. Ignore this step if you don't see the banner.
65
-
1. As soon as the SAS URL is revoked, go to **Configuration** for the managed disk. Increase the size so that Site Recovery supports the observed churn rate on the source disk.
65
+
1. Go to **Disks** of the affected replicated machine and copy the replica disk name.
66
+
1. Go to this replica of the managed disk.
67
+
1. You might see a banner in **Overview** that says an SAS URL has been generated. Select this banner and cancel the export. Ignore this step if you don't see the banner.
68
+
1. As soon as the SAS URL is revoked, go to **Configuration** for the managed disk. Increase the size so that Site Recovery supports the observed churn rate on the source disk.
Site Recovery sends replicated data to the cache storage account. You might experience network latency if uploading the data from a virtual machine to the cache storage account is slower than 4 MB in 3 seconds.
72
75
73
-
To check for a problem related to latency, use [AzCopy](https://docs.microsoft.com/azure/storage/common/storage-use-azcopy). You can use this command-line utility to upload data from the virtual machine to the cache storage account. If the latency is high, check whether you're using a network virtual appliance (NVA) to control outbound network traffic from VMs. The appliance might get throttled if all the replication traffic passes through the NVA.
76
+
To check for a problem related to latency, use [AzCopy](/azure/storage/common/storage-use-azcopy). You can use this command-line utility to upload data from the virtual machine to the cache storage account. If the latency is high, check whether you're using a network virtual appliance (NVA) to control outbound network traffic from VMs. The appliance might get throttled if all the replication traffic passes through the NVA.
74
77
75
-
We recommend creating a network service endpoint in your virtual network for "Storage" so that the replication traffic doesn't go to the NVA. For more information, see [Network virtual appliance configuration](https://docs.microsoft.com/azure/site-recovery/azure-to-azure-about-networking#network-virtual-appliance-configuration).
78
+
We recommend creating a network service endpoint in your virtual network for "Storage" so that the replication traffic doesn't go to the NVA. For more information, see [Network virtual appliance configuration](azure-to-azure-about-networking.md#network-virtual-appliance-configuration).
76
79
77
80
### Network connectivity
78
81
79
-
For Site Recovery replication to work, it needs the VM to provide outbound connectivity to specific URLs or IP ranges. You might have your VM behind a firewall or use network security group (NSG) rules to control outbound connectivity. If so, you might experience issues. To make sure all the URLs are connected, see [Outbound connectivity for Site Recovery URLs](https://docs.microsoft.com/azure/site-recovery/azure-to-azure-about-networking#outbound-connectivity-for-urls).
82
+
For Site Recovery replication to work, it needs the VM to provide outbound connectivity to specific URLs or IP ranges. You might have your VM behind a firewall or use network security group (NSG) rules to control outbound connectivity. If so, you might experience issues. To make sure all the URLs are connected, see [Outbound connectivity for URLs](azure-to-azure-about-networking.md#outbound-connectivity-for-urls).
80
83
81
84
## Error ID 153006 - No app-consistent recovery point available for the VM in the past "X" minutes
82
85
83
86
Following are some of the most common issues.
84
87
85
-
####Known issue in SQL server 2008/2008 R2
88
+
### Known issue in SQL server 2008/2008 R2
86
89
87
-
**How to fix**: There's a known issue with SQL server 2008/2008 R2. Refer to the article [Azure Site Recovery Agent or other non-component VSS backup fails for a server hosting SQL Server 2008 R2](https://support.microsoft.com/help/4504103/non-component-vss-backup-fails-for-server-hosting-sql-server-2008-r2).
90
+
**How to fix:** There's a known issue with SQL server 2008/2008 R2. Refer to the article [Azure Site Recovery Agent or other non-component VSS backup fails for a server hosting SQL Server 2008 R2](https://support.microsoft.com/help/4504103/non-component-vss-backup-fails-for-server-hosting-sql-server-2008-r2).
88
91
89
-
####Azure Site Recovery jobs fail on servers hosting any version of SQL Server instances with AUTO_CLOSE DBs
92
+
### Azure Site Recovery jobs fail on servers hosting any version of SQL Server instances with AUTO_CLOSE DBs
90
93
91
-
**How to fix**: Refer to the article [Non-component VSS backups such as Azure Site Recovery jobs fail on servers hosting SQL Server instances with AUTO_CLOSE DBs](https://support.microsoft.com/help/4504104/non-component-vss-backups-such-as-azure-site-recovery-jobs-fail-on-ser).
94
+
**How to fix:** Refer to the article [Non-component VSS backups such as Azure Site Recovery jobs fail on servers hosting SQL Server instances with AUTO_CLOSE DBs](https://support.microsoft.com/help/4504104/non-component-vss-backups-such-as-azure-site-recovery-jobs-fail-on-ser).
92
95
96
+
### Known issue in SQL Server 2016 and 2017
93
97
94
-
#### Known issue in SQL Server 2016 and 2017
98
+
**How to fix**: Refer to the article [Cumulative Update 16 for SQL Server 2017](https://support.microsoft.com/help/4508218/cumulative-update-16-for-sql-server-2017).
95
99
96
-
**How to fix**: Refer to the article [Error occurs when you back up a virtual machine with non-component based backup in SQL Server 2016 and 2017](https://support.microsoft.com/en-us/help/4508218/cumulative-update-16-for-sql-server-2017).
100
+
### You're using Azure Storage Spaces Direct Configuration
97
101
98
-
#### You're using Azure Storage Spaces Direct Configuration
99
-
100
-
**How to fix**: Azure Site Recovery can't create application consistent recovery point for Storage Spaces Direct Configuration. [Configure the replication policy](https://docs.microsoft.com/azure/site-recovery/azure-to-azure-how-to-enable-replication-s2d-vms).
102
+
**How to fix**: Azure Site Recovery can't create application consistent recovery point for Storage Spaces Direct Configuration. [Configure the replication policy](azure-to-azure-how-to-enable-replication-s2d-vms.md).
101
103
102
104
### More causes because of VSS-related issues:
103
105
104
106
To troubleshoot further, check the files on the source machine to get the exact error code for failure:
105
107
106
-
C:\Program Files (x86)\Microsoft Azure Site Recovery\agent\Application Data\ApplicationPolicyLogs\vacp.log
108
+
`C:\Program Files (x86)\Microsoft Azure Site Recovery\agent\Application Data\ApplicationPolicyLogs\vacp.log`
107
109
108
-
To locate the errors in the file, search for the string "vacpError" by opening the vacp.log file in an editor.
110
+
To locate the errors, open the _vacp.log_file in a text editor search for the string **vacpError**.
109
111
110
-
Ex: vacpError:220#Following disks are in FilteringStopped state [\\.\PHYSICALDRIVE1=5, ]#220|^|224#FAILED: CheckWriterStatus().#2147754994|^|226#FAILED to revoke tags.FAILED: CheckWriterStatus().#2147754994|^|
112
+
```plaintext
113
+
Ex: vacpError:220#Following disks are in FilteringStopped state [\\.\PHYSICALDRIVE1=5, ]#220|^|224#FAILED: CheckWriterStatus().#2147754994|^|226#FAILED to revoke tags.FAILED: CheckWriterStatus().#2147754994|^|
114
+
```
111
115
112
116
In the preceding example, **2147754994** is the error code that tells you about the failure following this sentence.
113
117
114
118
#### VSS writer is not installed - Error 2147221164
115
119
116
-
**How to fix**: To generate application consistency tag, Azure Site Recovery uses Volume Shadow Copy Service (VSS). Site Recovery installs a VSS Provider for its operation to take app consistency snapshots. Azure Site Recovery installs this VSS Provider as a service. If VSS Provider isn't installed, the application consistency snapshot creation fails. It shows the error ID 0x80040154 "Class not registered." Refer to the article for [VSS writer installation troubleshooting](https://docs.microsoft.com/azure/site-recovery/vmware-azure-troubleshoot-push-install#vss-installation-failures).
120
+
**How to fix**: To generate application consistency tag, Azure Site Recovery uses Volume Shadow Copy Service (VSS). Site Recovery installs a VSS Provider for its operation to take app consistency snapshots. Azure Site Recovery installs this VSS Provider as a service. If VSS Provider isn't installed, the application consistency snapshot creation fails. It shows the **error ID 0x80040154 Class not registered**. Refer to the article for [VSS writer installation troubleshooting](vmware-azure-troubleshoot-push-install.md#vss-installation-failures).
117
121
118
122
#### VSS writer is disabled - Error 2147943458
119
123
120
-
**How to fix**: To generate the application consistency tag, Azure Site Recovery uses VSS. Site Recovery installs a VSS Provider for its operation to take app consistency snapshots. This VSS Provider is installed as a service. If you don't have the VSS Provider service enabled, the application consistency snapshot creation fails. It shows the error "The specified service is disabled and cannot be started (0x80070422)."
124
+
**How to fix**: To generate the application consistency tag, Azure Site Recovery uses VSS. Site Recovery installs a VSS Provider for its operation to take app consistency snapshots. This VSS Provider is installed as a service. If you don't have the VSS Provider service enabled, the application consistency snapshot creation fails. It shows the error: **The specified service is disabled and cannot be started (0x80070422)**.
121
125
122
126
If VSS is disabled:
123
127
124
128
- Verify that the startup type of the VSS Provider service is set to **Automatic**.
**How to fix**: To generate the application consistency tag, Azure Site Recovery uses VSS. Check whether the Azure Site Recovery VSS Provider service is installed.
133
137
134
138
Use the following commands to reinstall VSS Provider:
135
-
1. Uninstall existing provider: C:\Program Files (x86)\Microsoft Azure Site Recovery\agent\InMageVSSProvider_Uninstall.cmd
136
-
1. Reinstall VSS Provider: C:\Program Files (x86)\Microsoft Azure Site Recovery\agent\InMageVSSProvider_Install.cmd
139
+
140
+
1. Uninstall existing provider:
141
+
142
+
`"C:\Program Files (x86)\Microsoft Azure Site Recovery\agent\InMageVSSProvider_Uninstall.cmd"`
143
+
144
+
1. Reinstall VSS Provider:
145
+
146
+
`"C:\Program Files (x86)\Microsoft Azure Site Recovery\agent\InMageVSSProvider_Install.cmd"`
137
147
138
148
Verify that the startup type of the VSS Provider service is set to **Automatic**.
0 commit comments