20230502 synapse whats new

Author: WilliamDAssafMSFT

articles/synapse-analytics/whats-new.md

@@ -4,7 +4,7 @@ description: Learn about the new features and documentation improvements for Azu
 author: WilliamDAssafMSFT
 ms.author: wiassaf
 ms.reviewer: rymajidi, sngun
-ms.date: 04/04/2023
+ms.date: 05/02/2023
 ms.service: synapse-analytics
 ms.subservice: overview
 ms.topic: conceptual
@@ -25,18 +25,19 @@ The following table lists the features of Azure Synapse Analytics that are curre
 
 | **Feature** |  **Learn more**|
 |:-- |:-- |
-| **Spark Advisor for Azure Synapse Notebook** | The [Spark Advisor for Azure Synapse Notebook](monitoring/apache-spark-advisor.md) analyzes code run by Spark and displays real-time advice for Notebooks. The Spark advisor offers recommendations for code optimization based on built-in common patterns, performs error analysis, and locates the root cause of failures.  |
 | **Apache Spark Delta Lake tables in serverless SQL pools** | The ability to for serverless SQL pools to access Delta Lake tables created in Spark databases is in preview. For more information, see [Azure Synapse Analytics shared metadata tables](metadata/table.md).|
 | **Apache Spark elastic pool storage** | Azure Synapse Analytics Spark pools now support elastic pool storage in preview. Elastic pool storage allows the Spark engine to monitor worker node temporary storage and attach more disks if needed. No action is required, and you should see fewer job failures as a result. For more information, see [Blog: Azure Synapse Analytics Spark elastic pool storage is available for public preview](https://techcommunity.microsoft.com/t5/azure-synapse-analytics-blog/azure-synapse-analytics-august-update-2022/ba-p/3535126#TOCREF_8).|
 | **Apache Spark Optimized Write** | [Optimize Write is a Delta Lake on Azure Synapse](spark/optimize-write-for-apache-spark.md) feature reduces the number of files written by Apache Spark 3 (3.1 and 3.2) and aims to increase individual file size of the written data.|
 | **Apache Spark R language support** | Built-in [R support for Apache Spark](spark/apache-spark-r-language.md) is now in preview. |
 | **Azure Synapse Data Explorer** | The [Azure Synapse Data Explorer](./data-explorer/data-explorer-overview.md) provides an interactive query experience to unlock insights from log and telemetry data. Connectors for Azure Data Explorer are available for Synapse Data Explorer. For more news, see [Azure Synapse Data Explorer (preview)](#azure-synapse-data-explorer-preview).|
 | **Browse ADLS Gen2 folders in the Azure Synapse Analytics workspace** | You can now browse an Azure Data Lake Storage Gen2 (ADLS Gen2) container or folder in your Azure Synapse Analytics workspace in Synapse Studio. To learn more, see [Browse an ADLS Gen2 folder with ACLs in Azure Synapse Analytics](how-to-access-container-with-access-control-lists.md).|
+| **Capture changed data from Cosmos DB analytical store** | Azure Cosmos DB analytical store now supports change data capture (CDC) for Azure Cosmos DB API for NoSQL and Azure Cosmos DB API for MongoDB. For more information, see [Capture Changed Data from your Cosmos DB analytical store](https://techcommunity.microsoft.com/t5/azure-data-factory-blog/capture-changed-data-from-your-cosmos-db-analytical-store/ba-p/3783530) and [DevBlog: Change Data Capture (CDC) with Azure Cosmos DB analytical store](https://devblogs.microsoft.com/cosmosdb/now-in-preview-change-data-capture-cdc-with-azure-cosmos-db-analytical-store/).| 
 | **Data flow improvements to Data Preview** | To learn more, see [Data Preview and debug improvements in Mapping Data Flows](https://techcommunity.microsoft.com/t5/azure-data-factory-blog/data-preview-and-debug-improvements-in-mapping-data-flows/ba-p/3268254?wt.mc_id=azsynapseblog_mar2022_blog_azureeng). |
 | **Distribution Advisor**| The Distribution Advisor is a new preview feature in Azure Synapse dedicated SQL pools Gen2 that analyzes queries and recommends the best distribution strategies for tables to improve query performance. For more information, see [Distribution Advisor in Azure Synapse SQL](sql/distribution-advisor.md).|
 | **Distributed Deep Neural Network Training**  | Learn more about new distributed training libraries like Horovod, Petastorm, TensorFlow, and PyTorch in [Deep learning tutorials](./machine-learning/concept-deep-learning.md). |
 | **Embed ADX dashboards** | Azure Data Explorer dashboards be [embedded in an IFrame and hosted in third party apps](/azure/data-explorer/kusto/api/monaco/host-web-ux-in-iframe). |
 | **Reject options for delimited text files** | Reject options for CREATE EXTERNAL TABLE on delimited files is in preview. |
+| **Spark Advisor for Azure Synapse Notebook** | The [Spark Advisor for Azure Synapse Notebook](monitoring/apache-spark-advisor.md) analyzes code run by Spark and displays real-time advice for Notebooks. The Spark advisor offers recommendations for code optimization based on built-in common patterns, performs error analysis, and locates the root cause of failures.|
 | **Time-To-Live in managed virtual network (VNet)** | Reserve compute for the time-to-live (TTL) in managed virtual network TTL period, saving time and improving efficiency. For more information on this preview, see [Announcing public preview of Time-To-Live (TTL) in managed virtual network](https://techcommunity.microsoft.com/t5/azure-data-factory-blog/announcing-public-preview-of-time-to-live-ttl-in-managed-virtual/ba-p/3552879).|
 | **User-Assigned managed identities** | Now you can use user-assigned managed identities in linked services for authentication in Synapse Pipelines and Dataflows.To learn more, see [Credentials in Azure Data Factory and Azure Synapse](../data-factory/credentials.md?context=%2Fazure%2Fsynapse-analytics%2Fcontext%2Fcontext&tabs=data-factory).|
 
@@ -84,6 +85,7 @@ This section summarizes recent new features and capabilities of [Apache Spark fo
 
 |**Month** | **Feature** |  **Learn more**|
 |:-- |:-- | :-- |
+| April 2023 | **Delta Lake - Low Shuffle Merge** | [Low Shuffle Merge optimization for Delta tables](spark/low-shuffle-merge-for-apache-spark.md) is now available in Apache Spark 3.2 and 3.3 pools. You can now update a Delta table with advanced conditions using the Delta Lake MERGE command. |
 | March 2023 | **Library management new ability: in-line installation** | `%pip` and `%conda` are now available in Apache Spark for Synapse! `%pip` and `%conda` are commands that can be used on Notebooks to install Python packages. For more information, see [Manage session-scoped Python packages through %pip and %conda commands](spark/apache-spark-manage-session-packages.md#manage-session-scoped-python-packages-through-pip-and-conda-commands). |
 | March 2023 | **Increasing Azure Synapse Analytics Spark performance up to 77%** | More regions are receiving the [performance increase for Azure Synapse Spark workloads](https://azure.microsoft.com/updates/increasing-azure-synapse-analytics-spark-performance-by-up-to-77/), including most recently Korea Central, Central India, and Australia Southeast. |
 | March 2023 | **Azure Synapse Spark Notebook – Unit Testing** | Learn how to [test and create unit test cases for Spark jobs developed using Synapse Notebook](https://techcommunity.microsoft.com/t5/azure-synapse-analytics-blog/azure-synapse-spark-notebook-unit-testing/ba-p/3725137). |
@@ -106,6 +108,7 @@ This section summarizes recent new features and capabilities of Azure Synapse An
 
 |**Month** | **Feature** |  **Learn more**|
 |:-- |:-- | :-- |
+| April 2023 | **Capture changed data from Cosmos DB analytical store (Public Preview)** | Azure Cosmos DB analytical store now supports change data capture (CDC) for Azure Cosmos DB API for NoSQL and Azure Cosmos DB API for MongoDB. For more information, see [Capture Changed Data from your Cosmos DB analytical store](https://techcommunity.microsoft.com/t5/azure-data-factory-blog/capture-changed-data-from-your-cosmos-db-analytical-store/ba-p/3783530) and [DevBlog: Change Data Capture (CDC) with Azure Cosmos DB analytical store](https://devblogs.microsoft.com/cosmosdb/now-in-preview-change-data-capture-cdc-with-azure-cosmos-db-analytical-store/).|
 | March 2023 | **Deep dive: Synapse pipelines storage event trigger security** | This Customer Success Engineering blog post is a deep dive into [Azure Synapse pipelines storage event trigger security](https://techcommunity.microsoft.com/t5/azure-synapse-analytics-blog/synapse-pipelines-storage-event-trigger-security-deep-dive/ba-p/3778250). ADF and Synapse Pipelines offer a feature that allows pipeline execution to be triggered based on various events, such as storage blob creation or deletion. This can be used by customers to implement event-driven pipeline orchestration.|
 | January 2023 | **SQL CDC incremental extract now supports numeric columns** | Enabling incremental [extract from SQL Server CDC in dataflows](../data-factory/connector-sql-server.md?tabs=data-factory#native-change-data-capture) allows you to only process rows that have changed since the last time that pipeline was executed. Supported incremental column types now include date/time and numeric columns. |
 | December 2022 | **Express virtual network injection** | Both the standard and express methods to [inject your SSIS Integration Runtime (IR) into a VNet](https://techcommunity.microsoft.com/t5/sql-server-integration-services/vnet-or-no-vnet-secure-data-access-from-ssis-in-azure-data/ba-p/1062056) are generally available now. For more information, see [General Availability of Express Virtual Network injection for SSIS in Azure Data Factory](https://techcommunity.microsoft.com/t5/sql-server-integration-services/general-availability-of-express-virtual-network-injection-for/ba-p/3699993).| 
@@ -214,6 +217,8 @@ Azure Data Explorer (ADX) is a fast and highly scalable data exploration service
 
 |**Month** | **Feature** |  **Learn more**|
 |:-- |:-- | :-- |
+| April 2023 | **ARM template to deploy Azure Data Explorer DB with Cosmos DB connection** | An [ARM template is now available to quickly deploy an Azure Data Explorer cluster](/samples/azure/azure-quickstart-templates/kusto-cosmos-db/) with System Assigned Identity, a database, an Azure Cosmos DB account (NoSql), an Azure Cosmos DB database, an Azure Cosmos DB container, and a data connection between the Cosmos DB container and the Kusto database (using the system assigned identity). |
+| April 2023 | **Ingest data from Azure Events Hub to ADX free tier** | Azure Data Explorer now supports integration with Events Hub in ADX free tier. For more information, see [Free Event Hub data analysis with Azure Data Explorer](https://techcommunity.microsoft.com/t5/azure-data-explorer-blog/free-event-hub-data-analysis-with-azure-data-explorer/ba-p/3775034). |
 | March 2023 | **View cluster history in Kusto Data Explorer** | It is now easier to track the history of queries and commands run on a Kusto cluster using [`.show queries`](/azure/data-explorer/kusto/management/queries) and [`.show commands-and-queries`](/azure/data-explorer/kusto/management/commands-and-queries). |
 | March 2023 | **Amazon S3 support in Kusto Web Explorer** | You can now [ingest data from Amazon S3](/azure/data-explorer/kusto/api/connection-strings/storage-connection-strings) seamlessly via the Ingestion Hub in Kusto Web Explorer (KWE). |
 | March 2023 | **Plotly visuals support** | Use the [Plotly graphing library](https://techcommunity.microsoft.com/t5/azure-data-explorer-blog/plotly-visualizations-in-azure-data-explorer/ba-p/3717768) to create visualizations for [a KQL query using 'render' operator](/azure/data-explorer/kusto/query/renderoperator?pivots=azuredataexplorer) or interactively when [building ADX dashboards](/azure/data-explorer/azure-data-explorer-dashboards). |