Merge pull request #114321 from hrasheed-msft/hdiaprilpublicprs

Hdiaprilpublicprs

Author: PRMerger20

articles/hdinsight/hdinsight-use-external-metadata-stores.md

@@ -94,6 +94,8 @@ You can point your cluster to a previously created Azure SQL Database at any tim
 
 * In HDInsight 4.0, Spark and Hive use independent catalogs for accessing SparkSQL or Hive tables. A table created by Spark lives in the Spark catalog. A table created by Hive lives in the Hive catalog. This behavior is different than HDInsight 3.6 where Hive and Spark shared common catalog. Hive and Spark Integration in HDInsight 4.0 relies on Hive Warehouse Connector (HWC). HWC works as a bridge between Spark and Hive. [Learn about Hive Warehouse Connector](../hdinsight/interactive-query/apache-hive-warehouse-connector.md).
 
+* In HDInsight 4.0 if you would like to Share the metastore between Hive and Spark, you can do so by changing the property metastore.catalog.default to hive in your Spark cluster. You can find this property in Ambari Advanced spark2-hive-site-override. It’s important to understand that sharing of metastore only works for external hive tables, this will not work if you have internal/managed hive tables or ACID tables.  
+
 ## Apache Oozie metastore
 
 Apache Oozie is a workflow coordination system that manages Hadoop jobs. Oozie supports Hadoop jobs for Apache MapReduce, Pig, Hive, and others.  Oozie uses a metastore to store details about workflows. To increase performance when using Oozie, you can use Azure SQL Database as a custom metastore. The metastore provides access to Oozie job data after you delete your cluster.

articles/hdinsight/hdinsight-version-release.md

@@ -88,8 +88,12 @@ There's no supported upgrade path from previous versions of HDInsight to HDInsig
 * HDInsight 4.0 doesn't support Apache Storm.
 * Hive View is no longer available in HDInsight 4.0.
 * Shell interpreter in Apache Zeppelin isn't supported in Spark and Interactive Query clusters.
-* You can't *disable* LLAP on a Spark-LLAP cluster. You can only turn off LLAP.
-* Azure Data Lake Storage Gen2 can't save Jupyter notebooks in a Spark cluster.
+* You can't *disable* LLAP on a Spark-LLAP cluster. You can only turn LLAP off.
+* Azure Data Lake Storage Gen2 can't save Juypter notebooks in a Spark cluster.
+* Apache pig runs on Tez by default, However you can change it to Mapreduce
+* Spark SQL Ranger integration for row and column security is deprecated
+* Spark 2.4 and Kafka 2.1 are available in HDInsight 4.0, so Spark 2.3 and Kafka 1.1 are no longer supported. We recommend using Spark 2.4 & Kafka 2.3 and above in HDInsight 4.0.
+
 
 ## Next steps