Skip to content

Commit 9c49a9b

Browse files
authored
Merge pull request #209187 from JnHs/jh-arcvm-dr
add troubleshooting topic
2 parents 1731bb6 + 6dbcc0c commit 9c49a9b

File tree

4 files changed

+169
-9
lines changed

4 files changed

+169
-9
lines changed

articles/azure-arc/resource-bridge/troubleshoot-resource-bridge.md

Lines changed: 5 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -185,7 +185,7 @@ This is usually caused when trying to run commands from remote PowerShell, which
185185

186186
To install Azure Arc resource bridge on an Azure Stack HCI cluster, `az arcappliance` commands must be run locally on a node in the cluster. Sign in to the node through Remote Desktop Protocol (RDP) or use a console session to run these commands.
187187

188-
## Azure Arc-enabled VMWare VCenter issues
188+
## Azure Arc-enabled VMware VCenter issues
189189

190190
### `az arcappliance prepare` failure
191191

@@ -229,7 +229,7 @@ When deploying the resource bridge on VMware vCenter, you specify the folder in
229229

230230
### Insufficient permissions
231231

232-
When deploying the resource bridge on VMWare Vcenter, you may get an error saying that you have insufficient permission. To resolve this issue, make sure that your user account has all of the following privileges in VMware vCenter and then try again.
232+
When deploying the resource bridge on VMware Vcenter, you may get an error saying that you have insufficient permission. To resolve this issue, make sure that your user account has all of the following privileges in VMware vCenter and then try again.
233233

234234
```
235235
"Datastore.AllocateSpace"
@@ -346,10 +346,12 @@ When deploying the resource bridge on VMWare Vcenter, you may get an error sayin
346346

347347
## Next steps
348348

349+
[Understand recovery operations for resource bridge in Azure Arc-enabled VMware vSphere disaster scenarios](../vmware-vsphere/disaster-recovery.md)
350+
349351
If you don't see your problem here or you can't resolve your issue, try one of the following channels for support:
350352

351353
* Get answers from Azure experts through [Microsoft Q&A](/answers/topics/azure-arc.html).
352354

353355
* Connect with [@AzureSupport](https://twitter.com/azuresupport), the official Microsoft Azure account for improving customer experience. Azure Support connects the Azure community to answers, support, and experts.
354356

355-
* [Open an Azure support request](../../azure-portal/supportability/how-to-create-azure-support-request.md).
357+
* [Open an Azure support request](../../azure-portal/supportability/how-to-create-azure-support-request.md).

articles/azure-arc/vmware-vsphere/day2-operations-resource-bridge.md

Lines changed: 7 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -2,15 +2,15 @@
22
title: Perform ongoing administration for Arc-enabled VMware vSphere
33
description: Learn how to perform day 2 administrator operations related to Azure Arc-enabled VMware vSphere
44
ms.topic: how-to
5-
ms.date: 03/28/2022
5+
ms.date: 08/25/2022
66

77
---
88

99
# Perform ongoing administration for Arc-enabled VMware vSphere
1010

1111
In this article, you'll learn how to perform various administrative operations related to Azure Arc-enabled VMware vSphere (preview):
1212

13-
- Upgrading the Arc resource bridge
13+
- Upgrading the Azure Arc resource bridge (preview)
1414
- Updating the credentials
1515
- Collecting logs from the Arc resource bridge
1616

@@ -21,7 +21,7 @@ Each of these operations requires either SSH key to the resource bridge VM or th
2121
Azure Arc-enabled VMware vSphere requires the Arc resource bridge to connect your VMware vSphere environment with Azure. Periodically, new images of Arc resource bridge will be released to include security and feature updates.
2222

2323
> [!NOTE]
24-
> To upgrade the arc resource bridge VM to the latest version, you will need to perform the onboarding again with the **same resource IDs**. This will cause some downtime as operations performed through Arc during this time might fail.
24+
> To upgrade the Arc resource bridge VM to the latest version, you will need to perform the onboarding again with the **same resource IDs**. This will cause some downtime as operations performed through Arc during this time might fail.
2525
2626
To upgrade to the latest version of the resource bridge, perform the following steps:
2727

@@ -63,7 +63,7 @@ Azure Arc-enabled VMware vSphere uses the vSphere account credentials you provid
6363
6464
As part of your security practices, you might need to rotate credentials for your vCenter accounts. As credentials are rotated, you must also update the credentials provided to Azure Arc to ensure the functioning of Azure Arc-enabled VMware services.
6565
66-
There are two different sets of credentials stored on the Arc resource bridge. But you can use the same account credentials for both.
66+
There are two different sets of credentials stored on the Arc resource bridge. You can use the same account credentials for both.
6767
6868
- **Account for Arc resource bridge**. This account is used for deploying the Arc resource bridge VM and will be used for upgrade.
6969
- **Account for VMware cluster extension**. This account is used to discover inventory and perform all VM operations through Azure Arc-enabled VMware vSphere
@@ -100,12 +100,13 @@ az arcappliance logs <provider> --out-dir <path to specified output directory> -
100100

101101
During initial onboarding, SSH keys are saved to the workstation. If you're running this command from the workstation that was used for onboarding, no other steps are required.
102102

103-
If you're running this command from a different workstation, you must make sure the following files are copied to the new workstation in the same location.
103+
If you're running this command from a different workstation, make sure the following files are copied to the new workstation in the same location.
104104

105105
- For a Windows workstation, `C:\ProgramData\kva\.ssh\logkey` and `C:\ProgramData\kva\.ssh\logkey.pub`
106106

107107
- For a Linux workstation, `$HOME\.KVA\.ssh\logkey` and `$HOME\.KVA\.ssh\logkey.pub`
108108

109109
## Next steps
110110

111-
[Troubleshoot common issues related to resource bridge](../resource-bridge/troubleshoot-resource-bridge.md)
111+
- [Troubleshoot common issues related to resource bridge](../resource-bridge/troubleshoot-resource-bridge.md)
112+
- [Understand disaster recovery operations for resource bridge](disaster-recovery.md)
Lines changed: 154 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,154 @@
1+
---
2+
title: Perform disaster recovery operations
3+
description: Learn how to perform recovery operations for the Azure Arc resource bridge VM in Azure Arc-enabled VMware vSphere disaster scenarios.
4+
ms.topic: how-to
5+
ms.date: 08/16/2022
6+
7+
---
8+
9+
# Perform disaster recovery operations
10+
11+
In this article, you'll learn how to perform recovery operations for the Azure Arc resource bridge (preview) VM in Azure Arc-enabled VMware vSphere disaster scenarios.
12+
13+
## Disaster scenarios & recovery goals
14+
15+
In disaster scenarios for the Azure Arc resource bridge virtual machine (VM), including accidental deletion and hardware failure, the resource bridge Azure resource will have a status of `offline`. This means that the connection between on-premises infrastructure and Azure is lost, and previously managed Arc-enabled resources are disconnected from their on-premises counterparts.
16+
17+
By performing recovery options, you can recreate a healthy Arc resource bridge and automatically reenable disconnected Arc-enabled resources.
18+
19+
## Recovering the Arc resource bridge
20+
21+
> [!NOTE]
22+
> When prompted for names for the Arc resource bridge, custom locations, and vCenter Azure resources, you'll need to provide the **same resource IDs** as the original resources in Azure.
23+
24+
To recover the Arc resource bridge VM, you'll need to:
25+
26+
- Delete the existing Arc resource bridge.
27+
- Create a new Arc resource bridge.
28+
- Recreate necessary custom extensions and custom locations.
29+
- Reconnect the new Arc resource bridge to existing resources in Azure.
30+
31+
Follow the [Perform manual recovery for Arc resource bridge](#perform-manual-recovery-for-arc-resource-bridge) if any of the following apply:
32+
33+
- The Arc resource bridge VM template is still present in vSphere.
34+
- The old Arc resource bridge contained multiple cluster extensions.
35+
- The old Arc resource bridge contained multiple custom locations.
36+
37+
If none of the above apply, you can use the automated recovery process described in [Use a script to recover Arc resource bridge](#use-a-script-to-recover-arc-resource-bridge).
38+
39+
## Perform manual recovery for Arc resource bridge
40+
41+
1. Copy the Azure region and resource IDs of the Arc resource bridge, custom location, and vCenter Azure resources.
42+
43+
1. If the original configuration files for setting up Arc-enabled VMware vSphere are still present, move to the next step.
44+
45+
Otherwise, recreate the configuration files and validate them. vSphere-related configurations can be changed from the original settings, but any Azure-related configurations (resource groups, Azure IDs, location) must be the same as in the original setup.
46+
47+
```azurecli
48+
az arcappliance createconfig vmware --resource-group <resource group of original Arc resource bridge> --name <name of original Arc resource bridge> --location <Azure region of original Arc resource bridge>
49+
```
50+
51+
```azurecli
52+
az arcappliance validate vmware --config-file <path to configuration "name-appliance.yaml" file>
53+
```
54+
55+
1. If the original Arc resource bridge VM template for setting up Arc-enabled VMware vSphere is still present in vSphere, move to the next step.
56+
57+
Otherwise, prepare a new VM template:
58+
59+
```azurecli
60+
az arcappliance prepare vmware --config-file <path to configuration "name-appliance.yaml" file>
61+
```
62+
63+
1. Delete the existing Arc resource bridge. This command will delete both the on-premises VM in vSphere and the associated Azure resource.
64+
65+
```azurecli
66+
az arcappliance delete vmware --config-file <path to configuration "name-appliance.yaml" file>
67+
```
68+
69+
1. Deploy a new Arc resource bridge VM.
70+
71+
```azurecli
72+
az arcappliance deploy vmware --config-file <path to configuration "name-appliance.yaml" file>
73+
```
74+
75+
1. Create a new Arc resource bridge Azure resource and establish the connection between vCenter and Azure.
76+
77+
```azurecli
78+
az arcappliance create vmware --config-file <path to configuration "name-appliance.yaml" file> --kubeconfig <path to kubeconfig file>
79+
```
80+
81+
1. Wait for the new Arc resource bridge to have a status of "Running". This process can take up to 5 minutes. Check the status in the Azure portal or use the following command:
82+
83+
```azurecli
84+
az arcappliance show --resource-group <resource-group-name> --name <Arc-resource-bridge-name>
85+
```
86+
87+
1. Recreate necessary custom extensions. For Arc-enabled VMware vSphere:
88+
89+
```azurecli
90+
az k8s-extension create --resource-group <resource-group-name> --name azure-vmwareoperator --cluster-name <cluster-name> --cluster-type appliances --scope cluster --extension-type Microsoft.VMWare --release-train stable --release-namespace azure-vmwareoperator --auto-upgrade true --config Microsoft.CustomLocation.ServiceAccount=azure-vmwareoperator 
91+
```
92+
93+
1. Recreate original custom locations. The name must be the same as the resource ID of the existing custom location in Azure. This method will allow the newly created custom location to automatically connect to the existing Azure resource.
94+
95+
```azurecli
96+
az customlocation create --name <name of existing custom location resource in Azure> --namespace azure-vmwareoperator --resource-group <resource group of the existing custom location> --host-resource-id <extension-name>
97+
```
98+
99+
1. Reconnect to the existing vCenter Azure resource. The name must be the same as the resource ID of the existing vCenter resource in Azure.
100+
101+
```azurecli
102+
az connectedvmware vcenter connect --custom-location <custom-location-name> --location <Azure-region> --name <name of existing vCenter resource in Azure> --resource-group <resource group of the existing vCenter resource> --username <username to the vSphere account> --password <password to the vSphere account>
103+
```
104+
105+
1. Once the above commands are successfully completed, the resource bridge should be recovered, and the previously disconnected Arc-enabled resources will be manageable in Azure again.
106+
107+
## Use a script to recover Arc resource bridge
108+
109+
> [!NOTE]
110+
> The script used in this automated recovery process will also upgrade the resource bridge to the latest version.
111+
112+
To recover the Arc resource bridge, perform the following steps:
113+
114+
1. Copy the Azure region and resource IDs of the Arc resource bridge, custom location, and vCenter Azure resources.
115+
116+
1. Find and delete the old Arc resource bridge **template** from your vCenter.
117+
118+
1. Download the [onboarding script](../vmware-vsphere/quick-start-connect-vcenter-to-arc-using-script.md#run-the-script) from the Azure portal and update the following section in the script, using the **same information** as the original resources in Azure.
119+
120+
```powershell
121+
$location = <Azure region of the resources>
122+
123+
$applianceSubscriptionId = <subscription-id>
124+
$applianceResourceGroupName = <resource-group-name>
125+
$applianceName = <resource-bridge-name>
126+
127+
$customLocationSubscriptionId = <subscription-id>
128+
$customLocationResourceGroupName = <resource-group-name>
129+
$customLocationName = <custom-location-name>
130+
131+
$vCenterSubscriptionId = <subscription-id>
132+
$vCenterResourceGroupName = <resource-group-name>
133+
$vCenterName = <vcenter-name-in-azure>
134+
```
135+
136+
1. [Run the onboarding script](../vmware-vsphere/quick-start-connect-vcenter-to-arc-using-script.md#run-the-script) again with the `--force` parameter.
137+
138+
``` powershell-interactive
139+
./resource-bridge-onboarding-script.ps1 --force
140+
```
141+
142+
1. [Provide the inputs](../vmware-vsphere/quick-start-connect-vcenter-to-arc-using-script.md#inputs-for-the-script) as prompted.
143+
144+
1. Once the script successfully finishes, the resource bridge should be recovered, and the previously disconnected Arc-enabled resources will be manageable in Azure again.
145+
146+
## Next steps
147+
148+
[Troubleshoot Azure Arc resource bridge (preview) issues](../resource-bridge/troubleshoot-resource-bridge.md)
149+
150+
If the recovery steps above are unsuccessful in restoring Arc resource bridge to its original state, try one of the following channels for support:
151+
152+
- Get answers from Azure experts through [Microsoft Q&A](/answers/topics/azure-arc.html).
153+
- Connect with [@AzureSupport](https://twitter.com/azuresupport), the official Microsoft Azure account for improving customer experience. Azure Support connects the Azure community to answers, support, and experts.
154+
- [Open an Azure support request](../../azure-portal/supportability/how-to-create-azure-support-request.md).

articles/azure-arc/vmware-vsphere/toc.yml

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -22,6 +22,9 @@
2222
href: day2-operations-resource-bridge.md
2323
- name: Remove vCenter from Azure Arc
2424
href: remove-vcenter-from-arc-vmware.md
25+
- name: Perform recovery operations
26+
displayName: disaster
27+
href: disaster-recovery.md
2528
- name: Reference
2629
items:
2730
- name: Azure CLI

0 commit comments

Comments
 (0)