Merge pull request #90450 from hrasheed-msft/hdi_custom_ambaridb

HdInsight custom ambaridb

Author: American-Dipper

articles/hdinsight/TOC.yml

@@ -136,7 +136,9 @@
       - name: Autoscale clusters
         href: ./hdinsight-autoscale-clusters.md
       - name: Use external metadata stores
-        href: ./hdinsight-use-external-metadata-stores.md     
+        href: ./hdinsight-use-external-metadata-stores.md
+      - name: Custom Ambari DB
+        href: ./hdinsight-custom-ambari-db.md          
       - name: Manage logs for an HDInsight cluster
         href: ./hdinsight-log-management.md
       - name: Add storage accounts

articles/hdinsight/hdinsight-custom-ambari-db.md

@@ -0,0 +1,63 @@
+---
+title: Custom Apache Ambari database on Azure HDInsight
+description: Learn how to create HDInsight clusters with your own custom Apache Ambari database.
+author: hrasheed-msft
+ms.reviewer: jasonh
+ms.service: hdinsight
+ms.custom: hdinsightactive
+ms.topic: conceptual
+ms.date: 10/29/2019
+ms.author: hrasheed
+---
+# Set up HDInsight clusters with a custom Ambari DB
+
+Apache Ambari simplifies the management and monitoring of an Apache Hadoop cluster. Ambari provides an easy to use web UI and REST API. Ambari is included on HDInsight clusters, and is used to monitor the cluster and make configuration changes.
+
+In normal cluster creation, as described in other articles such as [Set up clusters in HDInsight](hdinsight-hadoop-provision-linux-clusters.md), Ambari is deployed in an [S0 Azure SQL database](../sql-database/sql-database-dtu-resource-limits-single-databases.md#standard-service-tier) that is managed by HDInsight and is not accessible to users.
+
+The custom Ambari DB feature allows you to deploy a new cluster and setup Ambari in an external database that you manage. The deployment is done with an Azure Resource Manager template. This feature has the following benefits:
+
+- Customization - you choose the size and processing capacity of the database. If you have large clusters processing intensive workloads, an Ambari database with lower specifications could become a bottleneck for management operations.
+- Flexibility - you can scale the database as needed to suit your requirements.
+- Control - you can manage backups and security for your database in a way that fits with your organizations requirements.
+
+The remainder of this article discusses the following points:
+
+- requirements to use the custom Ambari DB feature
+- the steps necessary to provision HDInsight clusters using your own external database for Apache Ambari
+
+## Custom Ambari DB requirements
+
+You can deploy a custom Ambari DB with all cluster types and versions. Multiple clusters cannot use the same Ambari DB.
+
+The custom Ambari DB has the following other requirements:
+
+- You must have an existing Azure SQL DB server and database.
+- The database that you provide for Ambari setup must be empty. There should be no tables in the default dbo schema.
+- The user used to connect to the database should have SELECT, CREATE TABLE, and INSERT permissions on the database.
+- Turn on the option to [Allow access to Azure services](../sql-database/sql-database-vnet-service-endpoint-rule-overview.md#azure-portal-steps) on the Azure SQL server where you will host Ambari.
+- Management IP addresses from HDInsight service need to be allowed in the SQL Server. See [HDInsight management IP addresses](hdinsight-management-ip-addresses.md) for a list of the IP addresses that must be added to the SQL server firewall.
+
+When you host your Apache Ambari DB in an external database, remember the following points:
+
+- You're responsible for the additional costs of the Azure SQL DB that holds Ambari.
+- Back up your custom Ambari DB periodically. Azure SQL Database generates backups automatically, but the backup retention time-frame varies. For more information, see [Learn about automatic SQL Database backups](../sql-database/sql-database-automated-backups.md).
+
+## Deploy clusters with a custom Ambari DB
+
+To create an HDInsight cluster that uses your own external Ambari database, use the [custom Ambari DB Quickstart template](https://github.com/Azure/azure-quickstart-templates/tree/master/101-hdinsight-custom-ambari-db).
+
+Edit the parameters in the `azuredeploy.parameters.json` to specify information about your new cluster and the database that will hold Ambari.
+
+You can begin the deployment using the Azure CLI. Replace `<RESOURCEGROUPNAME>` with the resource group where you want to deploy your cluster.
+
+```azure-cli
+az group deployment create --name HDInsightAmbariDBDeployment \
+    --resource-group <RESOURCEGROUPNAME> \
+    --template-file azuredeploy.json \
+    --parameters azuredeploy.parameters.json
+```
+
+## Next steps
+
+- [Use external metadata stores in Azure HDInsight](hdinsight-use-external-metadata-stores.md)

articles/hdinsight/hdinsight-use-external-metadata-stores.md

@@ -7,11 +7,13 @@ ms.reviewer: jasonh
 ms.service: hdinsight
 ms.custom: hdinsightactive
 ms.topic: conceptual
-ms.date: 10/17/2019
+ms.date: 10/29/2019
 ---
 
 # Use external metadata stores in Azure HDInsight
 
+HDInsight allows you to take control of your data and metadata by deploying key metadata solutions and management databases to external data stores. This feature is currently available for [Apache Hive metastore](#custom-metastore), [Apache Oozie metastore](#apache-oozie-metastore) and [Apache Ambari database](#custom-ambari-db).
+
 The Apache Hive metastore in HDInsight is an essential part of the Apache Hadoop architecture. A metastore is the central schema repository that can be used by other big data access tools such as Apache Spark, Interactive Query (LLAP), Presto, or Apache Pig. HDInsight uses an Azure SQL Database as the Hive metastore.
 
 ![HDInsight Hive Metadata Store Architecture](./media/hdinsight-use-external-metadata-stores/metadata-store-architecture.png)
@@ -88,6 +90,10 @@ Apache Oozie is a workflow coordination system that manages Hadoop jobs.  Oozie
 
 For instructions on creating an Oozie metastore with Azure SQL Database, see [Use Apache Oozie for workflows](hdinsight-use-oozie-linux-mac.md).
 
+## Custom Ambari DB
+
+To use your own external database with Apache Ambari on HDInsight, see [Custom Apache Ambari database](hdinsight-custom-ambari-db.md).
+
 ## Next steps
 
 * [Set up clusters in HDInsight with Apache Hadoop, Apache Spark, Apache Kafka, and more](./hdinsight-hadoop-provision-linux-clusters.md)

articles/hdinsight/index.yml

@@ -42,6 +42,8 @@ landingContent:
             url: ./hdinsight-hadoop-use-data-lake-storage-gen2.md
       - linkListType: whats-new
         links:
+          - text: Custom Ambari DB
+            url: ./hdinsight-custom-ambari-db.md
           - text: 'Autoscale: automatically scale hadoop clusters'
             url: ./hdinsight-autoscale-clusters.md
           - text: Monitoring with HDInsight