|
| 1 | +--- |
| 2 | +title: Azure VMware Solution - Host maintenance best practices |
| 3 | +description: Understand the best practices and recommendations to maintain Azure VMware Solution Software-Defined Data Center |
| 4 | +ms.topic: conceptual |
| 5 | +ms.service: azure-vmware |
| 6 | +ms.date: 03/25/2025 |
| 7 | +--- |
| 8 | + |
| 9 | +# Azure VMware Solution Software-Defined Data Center (SDDC) maintenance best practices |
| 10 | + |
| 11 | +Azure VMware Solution undertakes periodic maintenance of the private cloud. This maintenance includes security patches, minor and major updates to VMware software stack. This page describes the host monitoring, remediation, and recommended best practices that help keep the private cloud ready for maintenance. |
| 12 | + |
| 13 | +## Host maintenance and lifecycle management |
| 14 | + |
| 15 | +[!INCLUDE [vmware-software-update-frequency](includes/vmware-software-update-frequency.md)] |
| 16 | + |
| 17 | +## Host monitoring and remediation |
| 18 | + |
| 19 | +Azure VMware Solution continuously monitors the health of both the VMware components and underlay. When Azure VMware Solution detects a failure, it takes action to repair the failed components. When Azure VMware Solution detects a degradation or failure on an Azure VMware Solution node, it triggers the host remediation process. |
| 20 | + |
| 21 | +Host remediation involves replacing the faulty node with a new healthy node in the cluster. Then, when possible, the faulty host is placed in VMware vSphere maintenance mode. VMware vSphere vMotion moves the VMs off the faulty host to other available servers in the cluster, potentially allowing zero downtime for live migration of workloads. If the faulty host can't be placed in maintenance mode, the host is removed from the cluster. Before the faulty host is removed, the customer workloads are migrated to a newly added host. |
| 22 | + |
| 23 | +> [!TIP] |
| 24 | +> **Customer communication:** An email is sent to the customer's email address before the replacement is initiated and again after the replacement is successful. |
| 25 | +> |
| 26 | +> To receive emails related to host replacement, you need to be added to any of the following Azure Role-Based Access Control (RBAC) roles in the subscription: 'ServiceAdmin', 'CoAdmin', 'Owner', 'Contributor'. |
| 27 | +
|
| 28 | +Azure VMware Solution monitors the following conditions on the host: |
| 29 | + |
| 30 | +- Processor status |
| 31 | +- Memory status |
| 32 | +- Connection and power state |
| 33 | +- Hardware fan status |
| 34 | +- Network connectivity loss |
| 35 | +- Hardware system board status |
| 36 | +- Errors occurred on one or more disks of a vSAN host |
| 37 | +- Hardware voltage |
| 38 | +- Hardware temperature status |
| 39 | +- Hardware power status |
| 40 | +- Storage status |
| 41 | +- Connection failure |
| 42 | + |
| 43 | +## Maintenance Operations Best Practices |
| 44 | +The following actions are always recommended for ensuring host maintenance operations are carried out successfully: |
| 45 | +- **vSAN storage utilization:** To maintain Service Level Agreement (SLA), ensure that your vSphere cluster's storage space utilization remains below 75%. If the utilization exceeds 75%, upgrades may take longer than expected or fail entirely. If your storage utilization exceeds 75%, consider adding a node to expand the cluster and prevent potential downtime during upgrades. |
| 46 | +- **Distributed Resource Scheduler (DRS) rules:** DRS VM-VM anti-affinity rules must be configured in a way to have at least (N+1) hosts in the cluster, where N is the number of VMs part of DRS rule. |
| 47 | +- **Failures To Tolerate (FTT) violation:** To prevent data loss, change VMs configured with a vSAN storage policy for Failures to Tolerate (FTT) of 0 to a vSAN storage policy compliant with [Microsoft SLA](https://www.microsoft.com/licensing/docs/view/Service-Level-Agreements-SLA-for-Online-Services?lang=1) (FTT=1 for up to five hosts in a cluster and FTT=2 for six or more hosts in a cluster) and ensure host maintenance operations can carried out seamlessly. |
| 48 | +- **Remove VM CD-ROM mounts:** VMs mounted with "Emulate mode" CD-ROMs block host maintenance. Ensure CD-ROMs are mounted in "Passthrough mode". |
| 49 | +- **Serial/parallel port or external device:** If you're using an image file (ISO, FLP, etc.), ensure that it's accessible from all ESXi hosts in the cluster. Store the files on a datastore that are shared between all ESXi Servers that participate in the vMotion of the virtual machine. For more information, see [Broadcom KB article](https://knowledge.broadcom.com/external/article/324829/vmotion-fails-with-the-compatibility-err.html). |
| 50 | +- **Orphaned VMs:** In the case of an orphaned virtual machine, the Virtual Machine (VM) needs to be either re-registered if possible (if it hasn't been deleted) or removed from inventory. For more information, see [Broadcom KB article](https://knowledge.broadcom.com/external/article/312831/virtual-machines-appear-as-invalid-or-or.html). |
| 51 | +- **SCSI shared controller:** When using SCSI bus sharing use with bus type as "Physical" for VMs. VMs connected to Virtual SCSCI controllers will be powered-off. For more information, see [Broadcom KB article](https://knowledge.broadcom.com/external/article?legacyId=2147661). |
| 52 | +- **Third-party VMs & applications:** For third-party VMs & applications: |
| 53 | + - Ensure that third-party solutions deployed on Azure VMware Solution are compliant and don't interfere with maintenance operations. |
| 54 | + - Ensure that the VM isn’t installed with a VM-Host "Must run" DRS rule. Additionally, verify that these applications are compatible with upcoming versions of the VMware stack. |
| 55 | + - Consult with your solution vendor and update in advance if necessary to maintain compatibility post-upgrade. |
| 56 | + |
| 57 | + |
| 58 | +## Alert Codes and Remediation Table |
| 59 | +| Error Code | Error Details | Recommended Action | |
| 60 | +|--------------------|---------------------------------|---------------------| |
| 61 | +| EPC_CDROM_EMULATEMODE | This error is encountered when CD-ROM on the Virtual Machine uses emulate mode, whose ISO image isn't accessible | Follow [this KB article](https://knowledge.broadcom.com/external/article?legacyId=79306) for the removal of any CDROM mounted on customer's workload Virtual Machines in emulate mode or detach ISO. It's recommended to use "Passthrough mode" for mounting any CD-ROM. | |
| 62 | +| EPC_DRSOVERRIDERULE | This error is encountered when there's a Virtual Machine with DRS Override set to "Disabled" mode. | VM shouldn't block vMotion while putting host into maintenance. Set Partially Automated DRS rules for the VM. Refer to [this document](/azure/azure-vmware/create-placement-policy#enable-restrict-vm-movement-for-specific-vms) to know more about VM placement policies. | |
| 63 | +| EPC_SCSIDEVICE_SHARINGMODE | This error is encountered when a Virtual Machine is configured to use a device that prevents a maintenance operation: A device that is a SCSI controller which is engaged in bus-sharing | Follow [this KB article](https://knowledge.broadcom.com/external/article?legacyId=79910) for the removal of any SCSI controller engaged in bus-sharing attached to VMs | |
| 64 | +| EPC_DATASTORE_INACCESSIBLE | This error is encountered when any external Datastore attached to AVS Private Cloud becomes inaccessible | Follow [this article](/azure/azure-vmware/attach-azure-netapp-files-to-azure-vmware-solution-hosts?tabs=azure-portal#performance-best-practices) for the removal of any stale Datastore attached to cluster | |
| 65 | +| EPC_NWADAPTER_STALE | This error is encountered when connected Network interface on the Virtual Machine uses network adapter which becomes inaccessible | Follow [this KB article](https://knowledge.broadcom.com/external/article/318738/troubleshooting-the-migration-compatibil.html) for the removal of any stale N/W adapters attached to Virtual Machines | |
| 66 | +| EPC_SERIAL_PORT | This error is encountered when a Virtual Machine’s serial port is connected to a device that can't be accessed on the destination host. | If you're using an image file (ISO, FLP, and so on), ensure that it's accessible from all ESXi servers on the cluster. Store the files on a data store that is shared between all ESXi servers that participate in vMotion of the virtual machine. Refer to [this KB article](https://knowledge.broadcom.com/external/article/324829/vmotion-fails-with-the-compatibility-err.html) from Broadcom for more information. | |
| 67 | +| EPC_HARDWARE_DEVICE | This error is encountered when a Virtual Machine’s parallel Port/USB Device is connected to a device can't be accessed on the destination host. | If you're using an image file (ISO, FLP, and so on), ensure that it's accessible from all ESXi servers of the cluster. Store the files on a data store that is shared between all ESXi servers that participate in the vMotion of the virtual machine. Refer to [this KB article](https://knowledge.broadcom.com/external/article/324829/vmotion-fails-with-the-compatibility-err.html) from Broadcom for more information. | |
| 68 | +| EPC_INVALIDVM / EPC_ORPHANVM | This error is encountered when there's an orphaned or Invalid VM in the inventory | Ensure all your Virtual Machines are accessible to the vCenter. Refer to [this KB article](https://knowledge.broadcom.com/external/article/312831/virtual-machines-appear-as-invalid-or-or.html) for more information | |
| 69 | + |
| 70 | + |
| 71 | +> [!NOTE] |
| 72 | +> Azure VMware Solution tenant admins must not edit or delete the previously defined VMware vCenter Server alarms because they're managed by the Azure VMware Solution control plane on vCenter Server. These alarms are used by Azure VMware Solution monitoring to trigger the Azure VMware Solution host remediation process. |
| 73 | +
|
| 74 | + |
| 75 | +## Next steps |
| 76 | + |
| 77 | +Now that you've covered Azure VMware Solution private cloud maintenance best practices, you might want to learn about: |
| 78 | + |
| 79 | +- [Azure VMware Solution networking and interconnectivity concepts](architecture-networking.md) |
| 80 | +- [Azure VMware Solution storage concepts](architecture-storage.md) |
| 81 | +- [How to configure Azure Alerts in Azure VMware Solution](configure-alerts-for-azure-vmware-solution.md) |
| 82 | + |
| 83 | +<!-- LINKS - internal --> |
| 84 | +[concepts-networking]: ./concepts-networking.md |
| 85 | + |
| 86 | +<!-- LINKS - external--> |
| 87 | +[vCSA versions]: https://kb.vmware.com/s/article/2143838 |
| 88 | + |
| 89 | +[ESXi versions]: https://kb.vmware.com/s/article/2143832 |
| 90 | + |
| 91 | +[vSAN versions]: https://kb.vmware.com/s/article/2150753 |
0 commit comments