Merge pull request #193176 from snehithm/day2-ops-preview

v-regandowner · web-flow · commit 4cf15fec0f49 · 2022-04-01T10:34:22.000-04:00
Day2 operations docs
diff --git a/articles/azure-arc/overview.md b/articles/azure-arc/overview.md
@@ -28,6 +28,7 @@ Currently, Azure Arc allows you to manage the following resource types hosted ou
 * [Azure data services](data/overview.md): Run Azure data services on-premises, at the edge, and in public clouds using Kubernetes and the infrastructure of your choice. SQL Managed Instance
 and PostgreSQL Hyperscale (preview) services are currently available.
 * [SQL Server](/sql/sql-server/azure-arc/overview): Extend Azure services to SQL Server instances hosted outside of Azure.
+* Virtual machines (preview): Provision, resize, delete and manage virtual machines based on [VMware vSphere](/vmware-vsphere/overview.md) or [Azure Stack HCI](/azure-stack/hci/manage/azure-arc-enabled-virtual-machines) and enable VM self-service through role-based access.
 
 ## Key features and benefits
 
@@ -47,6 +48,8 @@ Some of the key scenarios that Azure Arc supports are:
 
 * Create [custom locations](./kubernetes/custom-locations.md) on top of your [Azure Arc-enabled Kubernetes](./kubernetes/overview.md) clusters, using them as target locations for deploying Azure services instances. Deploy your Azure service cluster extensions for [Azure Arc-enabled Data Services](./data/create-data-controller-direct-azure-portal.md), [App Services on Azure Arc](../app-service/overview-arc-integration.md) (including web, function, and logic apps) and [Event Grid on Kubernetes](../event-grid/kubernetes/overview.md).
 
+* Perform virtual machine lifecycle and management operations for [VMware vSphere](/vmware-vsphere/overview.md) and [Azure Stack HCI](/azure-stack/hci/manage/azure-arc-enabled-virtual-machines) environments.
+
 * A unified experience viewing your Azure Arc-enabled resources, whether you are using the Azure portal, the Azure CLI, Azure PowerShell, or Azure REST API.
 
 ## Pricing
@@ -81,4 +84,5 @@ For information, see the [Azure pricing page](https://azure.microsoft.com/pricin
 * Learn about [Azure Arc-enabled Kubernetes](./kubernetes/overview.md).
 * Learn about [Azure Arc-enabled data services](https://azure.microsoft.com/services/azure-arc/hybrid-data-services/).
 * Learn about [SQL Server on Azure Arc-enabled servers](/sql/sql-server/azure-arc/overview).
+* Learn about [Azure Arc-enabled VMware vSphere](vmware-vsphere/overview.md) and [Azure Arc-enabled Azure Stack HCI](https://docs.microsoft.com/azure-stack/hci/manage/azure-arc-enabled-virtual-machines)
 * Experience Azure Arc-enabled services by exploring the [Jumpstart proof of concept](https://azurearcjumpstart.io/azure_arc_jumpstart/).
diff --git a/articles/azure-arc/vmware-vsphere/day2-operations-resource-bridge.md b/articles/azure-arc/vmware-vsphere/day2-operations-resource-bridge.md
@@ -0,0 +1,111 @@
+---
+title:  Perform ongoing administration for Arc-enabled VMware vSphere
+description: Learn how to perform day 2 administrator operations related to Azure Arc-enabled VMware vSphere
+ms.topic: how-to 
+ms.date: 03/28/2022
+
+---
+
+# Perform ongoing administration for Arc-enabled VMware vSphere
+
+In this article, you'll learn how to perform various administrative operations related to Azure Arc-enabled VMware vSphere (preview):
+
+- Upgrading the Arc resource bridge
+- Updating the credentials
+- Collecting logs from the Arc resource bridge
+
+Each of these operations requires either SSH key to the resource bridge VM or the kubeconfig that provides access to the Kubernetes cluster on the resource bridge VM.
+
+## Upgrading the Arc resource bridge
+
+Azure Arc-enabled VMware vSphere requires the Arc resource bridge to connect your VMware vSphere environment with Azure. Periodically, new images of Arc resource bridge will be released to include security and feature updates.
+
+> [!NOTE]
+> To upgrade the arc resource bridge VM to the latest version, you will need to perform the onboarding again with the **same resource IDs**. This will cause some downtime as operations performed through Arc during this time might fail.
+
+To upgrade to the latest version of the resource bridge, perform the following steps:
+
+1. Copy the Azure region and resource IDs of the Arc resource bridge, custom location and vCenter Azure resources
+
+2. Find and delete the old Arc resource bridge **template** from your vCenter
+
+3. Download the script from the portal and update the following section in the script
+
+    ```powershell
+    $location = <Azure region of the resources>
+    
+    $applianceSubscriptionId = <subscription-id>
+    $applianceResourceGroupName = <resourcegroup-name>
+    $applianceName = <resource-bridge-name>
+    
+    $customLocationSubscriptionId = <subscription-id>
+    $customLocationResourceGroupName = <resourcegroup-name>
+    $customLocationName = <custom-location-name>
+    
+    $vCenterSubscriptionId = <subscription-id>
+    $vCenterResourceGroupName = <resourcegroup-name>
+    $vCenterName = <vcenter-name-in-azure>
+    ```
+
+4. [Run the onboarding script](quick-start-connect-vcenter-to-arc-using-script.md#run-the-script) again with the `--force` parameter
+
+    ``` powershell-interactive
+    ./resource-bridge-onboarding-script.ps1 --force
+    ```
+
+5. [Provide the inputs](quick-start-connect-vcenter-to-arc-using-script.md#inputs-for-the-script) as prompted.
+
+6. Once the onboarding is successfully completed, the resource bridge is upgraded to the latest version.
+
+## Updating the vSphere account credentials
+
+Azure Arc-enabled VMware vSphere uses the vSphere account credentials you provided during the onboarding to communicate with your vCenter server. These credentials are only persisted locally on the Arc resource bridge VM.
+
+As part of your security practices, you might need to rotate credentials for your vCenter accounts. As credentials are rotated, you must also update the credentials provided to Azure Arc to ensure the functioning of Azure Arc-enabled VMware services.
+
+There are two different sets of credentials stored on the Arc resource bridge. But you can use the same account credentials for both.
+
+- **Account for Arc resource bridge**. This account is used for deploying the Arc resource bridge VM and will be used for upgrade.
+- **Account for VMware cluster extension**. This account is used to discover inventory and perform all VM operations through Azure Arc-enabled VMware vSphere
+
+To update the credentials of the account for Arc resource bridge, run the following command from a workstation that can access cluster configuration IP address of the Arc resource bridge locally:
+
+```azurecli
+az arcappliance setcredential vmware --kubeconfig <kubeconfig>
+```
+
+To update the credentials used by the VMware cluster extension on the resource bridge. This command can be run from anywhere with `connectedvmware` CLI extension installed.
+
+```azurecli
+az connectedvmware vcenter connect --custom-location <name of the custom location> --location <Azure region>  --name <name of the vCenter resource in Azure>       --resource-group <resource group for the vCenter resource>  --username   <username for the vSphere account>  --password  <password to the vSphere account>
+```
+
+## Collecting logs from the Arc resource bridge
+
+For any issues encountered with the Azure Arc resource bridge, you can collect logs for further investigation. To collect the logs, use the Azure CLI [`Az arcappliance log`](https://docs.microsoft.com/cli/azure/arcappliance/logs?#az-arcappliance-logs-vmware) command.
+
+The `az arcappliance log` command must be run from a workstation that can communicate with the Arc resource bridge either via the cluster configuration IP address or the IP address of the Arc resource bridge VM.
+
+To save the logs to a destination folder, run the following command. This command requires connectivity to cluster configuration IP address.
+
+```azurecli
+az arcappliance logs <provider> --kubeconfig <path to kubeconfig> --out-dir <path to specified output directory>
+```
+
+If the Kubernetes cluster on the resource bridge isn't in functional state, you can use the following command. This command requires connectivity to IP address of the Azure Arc resource bridge VM via SSH
+
+```azurecli
+az arcappliance logs <provider> --out-dir <path to specified output directory> --ip XXX.XXX.XXX.XXX
+```
+
+During initial onboarding, SSH keys are saved to the workstation. If you're running this command from the workstation that was used for onboarding, no other steps are required.
+
+If you're running this command from a different workstation,  you must make sure the following files are copied to the new workstation in the same location.
+
+- For a Windows workstation, `C:\ProgramData\kva\.ssh\logkey` and `C:\ProgramData\kva\.ssh\logkey.pub`
+  
+- For a Linux workstation, `$HOME\.KVA\.ssh\logkey` and `$HOME\.KVA\.ssh\logkey.pub`
+
+## Next steps
+
+[Troubleshoot common issues related to resource bridge](../resource-bridge/troubleshoot-resource-bridge.md)
diff --git a/articles/azure-arc/vmware-vsphere/quick-start-connect-vcenter-to-arc-using-script.md b/articles/azure-arc/vmware-vsphere/quick-start-connect-vcenter-to-arc-using-script.md
@@ -146,7 +146,7 @@ After the command finishes running, your setup is complete. You can now use the
 ## Save SSH keys and kubeconfig
 
 > [!IMPORTANT]
-> Performing some day 2 operations on the Arc resource bridge will require the SSH key to the resource bridge VM and kubeconfig to the Kubernetes cluster on it. It is important to store them to a secure location as it is not possible to retrieve them if the workstation used for the onboarding is deleted.
+> Performing [day 2 operations on the Arc resource bridge](day2-operations-resource-bridge.md) will require the SSH key to the resource bridge VM and kubeconfig to the Kubernetes cluster on it. It is important to store them to a secure location as it is not possible to retrieve them if the workstation used for the onboarding is deleted.
 
 You will find the kubeconfig file with the name `kubeconfig` in the folder where  the onboarding script is downloaded and run.
 
diff --git a/articles/azure-arc/vmware-vsphere/remove-vcenter-from-arc-vmware.md b/articles/azure-arc/vmware-vsphere/remove-vcenter-from-arc-vmware.md
@@ -0,0 +1,129 @@
+---
+title: Remove your VMware vCenter environment from Azure Arc
+description: This article explains the steps to cleanly remove your VMware vCenter environment from Azure Arc-enabled VMware vSphere and delete related Azure Arc resources from Azure.
+author: snehithm
+ms.author: snmuvva
+ms.topic: how-to 
+ms.date: 3/28/2022
+# Customer intent: As an infrastructure admin, I want to cleanly remove my VMware vCenter environment from Azure Arc-enabled VMware vSphere.
+---
+
+# Remove your VMware vCenter  environment from Azure Arc
+
+In this article, you'll learn how to cleanly remove your VMware vCenter environment from Azure Arc-enabled VMware vSphere. For VMware vSphere environments that you no longer want to manage with Azure Arc-enabled VMware vSphere, follow the steps in the article to:
+
+- Remove guest management from VMware virtual machines
+- Remove VMware vSphere resource from Azure Arc
+- Remove Arc resource bridge related items in your vCenter
+
+## Remove guest management from VMware virtual machines
+
+To prevent continued billing of Azure management services after you remove the vSphere environment from Azure Arc, you must first cleanly remove guest management from all Arc-enabled VMware vSphere virtual machines where it was enabled.
+
+When you enable guest management on Arc-enabled VMware vSphere virtual machines, the Arc connected machine agent is installed on them.  Once guest management is enabled, you can install VM extensions on them and use Azure management services like the Log Analytics on them.
+
+To cleanly remove guest management, you must follow the steps below to remove any VM extensions from the virtual machine, disconnect the agent, and uninstall the software from your virtual machine. It's important to complete each of the three steps to fully remove all related software components from your virtual machines.
+
+### Step 1: Remove VM extensions
+
+If you have deployed Azure VM extensions to an Azure Arc-enabled VMware vSphere VM, you must uninstall the extensions before disconnecting the agent or uninstalling the software. Uninstalling the Azure Connected Machine agent doesn't automatically remove extensions, and they won't be recognized if you late connect the VM to Azure Arc again.
+Uninstall extensions using following steps:
+
+1. Go to [Azure Arc center in Azure portal](https://portal.azure.com/#blade/Microsoft_Azure_HybridCompute/AzureArcCenterBlade/overview)
+
+2. Select **VMware vCenters**.
+
+3. Search and select the vCenter you want to remove from Azure Arc.
+
+    ![Browse your VMware Inventory ](./media/browse-vmware-inventory.png)
+
+4. Select **Virtual machines** under **vCenter inventory**.
+
+5. Search and select the virtual machine where you have Guest Management enabled.
+
+6. Select **Extensions**.
+
+7. Select the extensions and select **Uninstall**
+
+### Step 2: Disconnect the agent from Azure Arc
+
+Disconnecting the agent clears the local state of the agent and removes agent information from our systems. To disconnect the agent, sign-in and run the following command as an administrator/root account on the virtual machine.
+
+```powershell
+    azcmagent disconnect --force-local-only
+```
+
+### Step 3: Uninstall the agent
+
+#### For Windows virtual machines
+
+To uninstall the Windows agent from the machine, do the following:
+
+1. Sign in to the computer with an account that has administrator permissions.
+2. In Control Panel, select Programs and Features.
+3. In Programs and Features, select Azure Connected Machine Agent, select Uninstall, and then select Yes.
+4. Delete the `C:\Program Files\AzureConnectedMachineAgent` folder
+
+#### For Linux virtual machines
+
+To uninstall the Linux agent, the command to use depends on the Linux operating system. You must have `root` access permissions or your account must have elevated rights using sudo.
+
+- For Ubuntu, run the following command:
+
+  ```bash
+  sudo apt purge azcmagent
+  ```
+
+- For RHEL, CentOS, Oracle Linux run the following command:
+
+    ```bash
+    sudo yum remove azcmagent
+    ```
+
+- For SLES, run the following command:
+
+     ```bash
+    sudo zypper remove azcmagent
+    ```
+
+## Remove VMware vSphere resources from Azure
+
+When you enable VMware vSphere resources in Azure, an Azure resource representing them is created. Before you can delete the vCenter resource in Azure, you must delete all the Azure resources that represent your related vSphere resources.
+
+1. Go to [Azure Arc center in Azure portal](https://portal.azure.com/#blade/Microsoft_Azure_HybridCompute/AzureArcCenterBlade/overview)
+
+2. Select **VMware vCenters**.
+
+3. Search and select the vCenter you remove from Azure Arc.
+
+4. Select **Virtual machines** under **vCenter inventory**.
+
+5. Select all the VMs that have **Azure Enabled** value as **Yes**.
+
+6. Select **Remove from Azure**.
+
+    This action will only remove these resource representations from Azure. The resources will continue to remain in your vCenter. 
+
+7. Perform the steps 4,5 and 6 for **Resources pools/clusters/hosts**, **Templates**, **Networks**, and **Datastores**
+
+8. Once the deletion is complete, select **Overview**.
+
+9. Note the **Custom location** and the **Azure Arc Resource bridge** resource in the **Essentials** section.
+
+10. Select **Remove from Azure** to remove the vCenter resource from Azure.
+
+11. Go to the **Custom location** resource and click **Delete**
+
+12. Go to the **Azure Arc Resource bridge** resource and click **Delete**
+
+At this point, all your Arc-enabled VMware vSphere resources are removed from Azure.
+
+## Remove Arc resource bridge related items in your vCenter
+
+During onboarding, to create a connection between your VMware vCenter and Azure, an Azure Arc resource bridge is deployed into your VMware vSphere environment. As the last step, you must delete the resource bridge VM as well the VM template created during the onboarding.
+
+You can find both the virtual machine and the template on the resource pool/cluster/host that you provided during [Azure Arc-enabled VMware vSphere onboarding](quick-start-connect-vcenter-to-arc-using-script.md).
+
+## Next steps
+
+- [Connect the vCenter to Azure Arc again](quick-start-connect-vcenter-to-arc-using-script.md)
diff --git a/articles/azure-arc/vmware-vsphere/toc.yml b/articles/azure-arc/vmware-vsphere/toc.yml
@@ -8,14 +8,25 @@
   items:
   - name: Connect vCenter to Azure Arc
     href: quick-start-connect-vcenter-to-arc-using-script.md
+  - name: Create a virtual machine on VMware vCenter
+    href: quick-start-create-a-vm.md 
 - name: How-to guides
   items:
   - name: Enable VMware vCenter resources in Azure
     href: browse-and-enable-vcenter-resources-in-azure.md
   - name: Manage access to VMware resources through Azure RBAC
     href: manage-access-to-arc-vmware-resources.md
-  - name: Create a virtual machine on VMware vCenter
-    href: quick-start-create-a-vm.md   
   - name: Perform operations on VMware VMs in Azure
     href: manage-vmware-vms-in-azure.md
-
+  - name: Administer Arc-enabled VMware vSphere
+    href: day2-operations-resource-bridge.md
+  - name: Remove vCenter from Azure Arc
+    href: remove-vcenter-from-arc-vmware.md
+- name: Reference
+  items:
+  - name: Azure CLI
+    items:
+    - name: Azure Arc-enabled VMware vSphere
+      href: /cli/azure/connectedvmware
+    - name: Azure Arc resource bridge
+      href:  /cli/azure/arcappliance
