Merge pull request #175260 from nicjohn79/patch-2

Update hdinsight-upgrade-cluster.md

Author: v-ccolin

articles/hdinsight/hdinsight-upgrade-cluster.md

@@ -45,6 +45,21 @@ The following documents provide guidance on how to migrate specific workloads:
 
 For more information about database backup and restore, see [Recover a database in Azure SQL Database by using automated database backups](../azure-sql/database/recovery-using-backups.md).
 
+## Upgrade scenarios
+
+As mentioned above, we recommend that HDInsight clusters be regularly migrated to the latest version in order to take advantage of new features and fixes. Please see the below list of reasons we would request that a cluster be deleted and redeployed:
+
+* The cluster version is [Retired](https://docs.microsoft.com/azure/hdinsight/hdinsight-retired-versions) or in [Basic support](https://docs.microsoft.com/azure/hdinsight/hdinsight-36-component-versioning) and you are having a cluster issue that would be resolved with a newer version.
+* The root cause of a cluster issue is determined to be related to an undersized VM. [View the engineering team's recommended node configuration](https://docs.microsoft.com/azure/hdinsight/hdinsight-supported-node-configuration#all-supported-regions-except-brazil-south-and-japan-west).
+* A customer opens a support case and the Engineering team determines the issue has already been fixed in a newer cluster version.
+* A default metastore database (Ambari, Hive, Oozie, Ranger) has reached it’s utilization limit. We will ask the customer to recreate the cluster using a [custom metastore](https://docs.microsoft.com/azure/hdinsight/hdinsight-use-external-metadata-stores#custom-metastore) database.
+* The root cause of a cluster issue is due to an **Unsupported Operation**. Here are some of the common unsupported operations:
+     * **Moving or Adding a service in Ambari**. When viewing information on the cluster services in Ambari, one of the actions available from the Service Actions menu is **Move [Service Name]**. Another action is **Add [Service Name]**. Both of these options are unsupported.
+     * **Python package corruption**. HDInsight clusters depend on the built-in Python environments, Python 2.7 and Python 3.5. Directly installing custom packages in those default built-in environments may cause unexpected library version changes and break the cluster. Learn how to [safely install custom external Python packages](https://docs.microsoft.com/azure/hdinsight/spark/apache-spark-python-package-installation#safely-install-external-python-packages) for your Spark applications.
+     * **Third-party software**. Customers have the ability to install third-party software on their HDInsight clusters; however, we will recommend recreating the cluster if it breaks the existing functionality.
+     * **Multiple workloads on the same cluster**. In HDInsight 4.0, the Hive Warehouse Connector needs separate clusters for Spark and Interactive Query workloads. [Follow these steps to set up both clusters in Azure HDInsight](https://docs.microsoft.com/azure/hdinsight/interactive-query/apache-hive-warehouse-connector). Similarly, integrating [Spark with HBASE](https://docs.microsoft.com/azure/hdinsight/hdinsight-using-spark-query-hbase) requires 2 different clusters.
+     * **Custom Ambari DB password changed**. The Ambari DB password is set during cluster creation and there is no current mechanism to update it. If a customer deploys the cluster with a [custom Ambari DB](https://docs.microsoft.com/azure/hdinsight/hdinsight-custom-ambari-db), they will have the ability to change the DB password on the SQL DB; however, there is no way to update this password for a running HDInsight cluster.
+
 ## Next steps
 
 * [Learn how to create Linux-based HDInsight clusters](hdinsight-hadoop-provision-linux-clusters.md)