Merge pull request #290822 from whhender/november-2024-synapse-freshness

November 2024 synapse freshness

Author: v-dirichards

articles/synapse-analytics/cicd/media/cross-tenant-aad.png

@@ -1,29 +1,33 @@
 ---
 title: 'Quickstart: Get started analyzing with Spark' 
-description: In this tutorial, you'll learn to analyze data with Apache Spark.
+description: In this tutorial, you'll learn to analyze some sample data with Apache Spark in Azure Synapse Analytics.
 author: whhender
 ms.author: whhender
 ms.reviewer: whhender
 ms.service: azure-synapse-analytics
 ms.subservice: spark
-ms.topic: tutorial
-ms.date: 11/18/2022
+ms.topic: quickstart
+ms.date: 11/15/2024
 ---
 
-# Analyze with Apache Spark
+# Quickstart: Analyze with Apache Spark
 
 In this tutorial, you'll learn the basic steps to load and analyze data with Apache Spark for Azure Synapse.
 
+## Prerequisites
+
+Make sure you have [placed the sample data in the primary storage account](get-started-create-workspace.md#place-sample-data-into-the-primary-storage-account).
+
 ## Create a serverless Apache Spark pool
 
 1. In Synapse Studio, on the left-side pane, select **Manage** > **Apache Spark pools**.
-1. Select **New** 
+1. Select **New**
 1. For **Apache Spark pool name** enter **Spark1**.
 1. For **Node size** enter **Small**.
 1. For **Number of nodes** Set the minimum to 3 and the maximum to 3
 1. Select **Review + create** > **Create**. Your Apache Spark pool will be ready in a few seconds.
 
-## Understanding serverless Apache Spark pools
+## Understand serverless Apache Spark pools
 
 A serverless Spark pool is a way of indicating how a user wants to work with Spark. When you start using a pool, a Spark session is created if needed. The pool controls how many Spark resources will be used by that session and how long the session will last before it automatically pauses. You pay for spark resources used during that session and not for the pool itself. This way a Spark pool lets you use Apache Spark without managing clusters. This is similar to how a serverless SQL pool works.
 
@@ -63,6 +67,7 @@ Data is available via the dataframe named **df**. Load it into a Spark database
     spark.sql("CREATE DATABASE IF NOT EXISTS nyctaxi")
     df.write.mode("overwrite").saveAsTable("nyctaxi.trip")
     ```
+
 ## Analyze the NYC Taxi data using Spark and notebooks
 
 1. Create a new code cell and enter the following code. 
@@ -93,7 +98,7 @@ Data is available via the dataframe named **df**. Load it into a Spark database
 
 1. In the cell results, select **Chart** to see the data visualized.
 
-## Next steps
+## Next step
 
 > [!div class="nextstepaction"]
 > [Analyze data with dedicated SQL pool](get-started-analyze-sql-pool.md)

articles/synapse-analytics/cicd/media/cross-tenant-sign-in.png

@@ -7,43 +7,43 @@ ms.reviewer: whhender
 ms.service: azure-synapse-analytics
 ms.subservice: pipeline
 ms.topic: tutorial
-ms.date: 12/31/2020
+ms.date: 11/20/2024
 ---
 
-# Integrate with pipelines
+# Tutorial: Integrate with pipelines
 
 In this tutorial, you'll learn how to integrate pipelines and activities using Synapse Studio. 
 
 ## Create a pipeline and add a notebook activity
 
 1. In Synapse Studio, go to the **Integrate** hub.
-1. Select **+** > **Pipeline** to create a new pipeline. Click on the new pipeline object to open the Pipeline designer.
+1. Select **+** > **Pipeline** to create a new pipeline. Select the new pipeline object to open the Pipeline designer.
 1. Under **Activities**, expand the **Synapse** folder, and drag a **Notebook** object into the designer.
 1. Select the **Settings** tab of the Notebook activity properties. Use the drop-down list to select a notebook from your current Synapse workspace.
 
 ## Schedule the pipeline to run every hour
 
 1. In the pipeline, select **Add trigger** > **New/edit**.
 1. In **Choose trigger**, select **New**, and set the **Recurrence** to "every 1 hour".
-1. Select **OK**. 
-1. Select **Publish All**. 
+1. Select **OK**.
+1. Select **Publish All**.
 
 ## Forcing a pipeline to run immediately
 
-Once the pipeline is published, you may want to run it immediately without waiting for an hour to pass.
+Once the pipeline is published, you might want to run it immediately without waiting for an hour to pass.
 
 1. Open the pipeline.
-1. Click **Add trigger** > **Trigger now**.
-1. Select **OK**. 
+1. Select **Add trigger** > **Trigger now**.
+1. Select **OK**.
 
 ## Monitor pipeline execution
 
 1. Go to the **Monitor** hub.
 1. Select **Pipeline runs** to monitor pipeline execution progress.
-1. In this view you can switch between tabular **List** display a graphical **Gantt** chart. 
-1. Click on a pipeline name to see the status of activities in that pipeline.
+1. In this view you can switch between tabular **List** display a graphical **Gantt** chart.
+1. Select a pipeline name to see the status of activities in that pipeline.
 
-## Next steps
+## Next step
 
 > [!div class="nextstepaction"]
 > [Visualize data with Power BI](get-started-visualize-power-bi.md)

articles/synapse-analytics/cicd/media/use-another-account.png

@@ -4,14 +4,15 @@ description: Learn how to use text analytics in Azure Synapse Analytics.
 ms.service: azure-synapse-analytics
 ms.subservice: machine-learning
 ms.topic: tutorial
-ms.date: 11/02/2021
+ms.date: 11/19/2024
 author: ruixinxu
 ms.author: ruxu
+# customer intent: As a Synapse Analytics user, I want to be able to analyze my text using Azure AI services.
 ---
 
 # Tutorial: Text Analytics with Azure AI services
 
-[Text Analytics](/azure/ai-services/language-service/) is an [Azure AI services](/azure/ai-services/) that enables you to perform  text mining and text analysis with Natural Language Processing (NLP) features. In this tutorial, you'll learn how to use [Text Analytics](/azure/ai-services/language-service/) to analyze unstructured text on Azure Synapse Analytics.
+In this tutorial, you learn how to use [Text Analytics](/azure/ai-services/language-service/) to analyze unstructured text on Azure Synapse Analytics. [Text Analytics](/azure/ai-services/language-service/) is an [Azure AI services](/azure/ai-services/) that enables you to perform  text mining and text analysis with Natural Language Processing (NLP) features.
 
 This tutorial demonstrates using text analytics with [SynapseML](https://github.com/microsoft/SynapseML) to:
 
@@ -29,34 +30,35 @@ If you don't have an Azure subscription, [create a free account before you begin
 
 - [Azure Synapse Analytics workspace](../get-started-create-workspace.md) with an Azure Data Lake Storage Gen2 storage account configured as the default storage. You need to be the *Storage Blob Data Contributor* of the Data Lake Storage Gen2 file system that you work with.
 - Spark pool in your Azure Synapse Analytics workspace. For details, see [Create a Spark pool in Azure Synapse](../quickstart-create-sql-pool-studio.md).
-- Pre-configuration steps described in the tutorial [Configure Azure AI services in Azure Synapse](tutorial-configure-cognitive-services-synapse.md).
-
+- Preconfiguration steps described in the tutorial [Configure Azure AI services in Azure Synapse](tutorial-configure-cognitive-services-synapse.md).
 
 ## Get started
-Open Synapse Studio and create a new notebook. To get started, import [SynapseML](https://github.com/microsoft/SynapseML). 
+
+Open Synapse Studio and create a new notebook. To get started, import [SynapseML](https://github.com/microsoft/SynapseML).
 
 ```python
 import synapse.ml
-from synapse.ml.cognitive import *
+from synapse.ml.services import *
 from pyspark.sql.functions import col
 ```
 
 ## Configure text analytics
 
-Use the linked text analytics you configured in the [pre-configuration steps](tutorial-configure-cognitive-services-synapse.md) . 
+Use the linked text analytics you configured in the [preconfiguration steps](tutorial-configure-cognitive-services-synapse.md).
 
 ```python
-ai_service_name = "<Your linked service for text analytics>"
+linked_service_name = "<Your linked service for text analytics>"
 ```
 
 ## Text Sentiment
-The Text Sentiment Analysis provides a way for detecting the sentiment labels (such as "negative", "neutral" and "positive") and confidence scores at the sentence and document-level. See the [Supported languages in Text Analytics API](/azure/ai-services/language-service/language-detection/overview?tabs=sentiment-analysis) for the list of enabled languages.
+
+The Text Sentiment Analysis provides a way for detecting the sentiment labels (such as "negative", "neutral", and "positive") and confidence scores at the sentence and document-level. See the [Supported languages in Text Analytics API](/azure/ai-services/language-service/language-detection/overview?tabs=sentiment-analysis) for the list of enabled languages.
 
 ```python
 
 # Create a dataframe that's tied to it's column names
 df = spark.createDataFrame([
-  ("I am so happy today, its sunny!", "en-US"),
+  ("I am so happy today, it's sunny!", "en-US"),
   ("I am frustrated by this rush hour traffic", "en-US"),
   ("The Azure AI services on spark aint bad", "en-US"),
 ], ["text", "language"])
@@ -77,13 +79,14 @@ display(results
     .select("text", "sentiment"))
 
 ```
+
 ### Expected results
 
 |text|sentiment|
 |---|---|
-|I am so happy today, its sunny!|positive|
-|I am frustrated by this rush hour traffic|negative|
-|The Azure AI services on spark aint bad|positive|
+|I'm so happy today, it's sunny!|positive|
+|I'm frustrated by this rush hour traffic|negative|
+|The Azure AI services on spark aint bad|neutral|
 
 ---
 
@@ -186,12 +189,15 @@ ner = (NER()
 
 display(ner.transform(df).select("text", col("replies").getItem("document").getItem("entities").alias("entities")))
 ```
+
 ### Expected results
+
 ![Expected results for named entity recognition v3.1](./media/tutorial-text-analytics-use-mmlspark/expected-output-ner-v-31.png)
 
 ---
 
 ## Personally Identifiable Information (PII) V3.1
+
 The PII feature is part of NER and it can identify and redact sensitive entities in text that are associated with an individual person such as: phone number, email address, mailing address, passport number. See the [Supported languages in Text Analytics API](/azure/ai-services/language-service/language-detection/overview?tabs=pii) for the list of enabled languages.
 
 ```python
@@ -209,17 +215,20 @@ pii = (PII()
 
 display(pii.transform(df).select("text", col("replies").getItem("document").getItem("entities").alias("entities")))
 ```
+
 ### Expected results
+
 ![Expected results for personal identifiable information v3.1](./media/tutorial-text-analytics-use-mmlspark/expected-output-pii-v-31.png)
 
 ---
 
 ## Clean up resources
+
 To ensure the Spark instance is shut down, end any connected sessions(notebooks). The pool shuts down when the **idle time** specified in the Apache Spark pool is reached. You can also select **stop session** from the status bar at the upper right of the notebook.
 
 ![Screenshot showing the Stop session button on the status bar.](./media/tutorial-build-applications-use-mmlspark/stop-session.png)
 
-## Next steps
+## Related content
 
 * [Check out Synapse sample notebooks](https://github.com/Azure-Samples/Synapse/tree/main/MachineLearning) 
 * [SynapseML GitHub Repo](https://github.com/microsoft/SynapseML)

articles/synapse-analytics/cicd/source-control.md

@@ -3,18 +3,14 @@ title: Managed private endpoints
 description: An article that explains Managed private endpoints in Azure Synapse Analytics
 author: ashinMSFT
 ms.service: azure-synapse-analytics
-ms.topic: overview
+ms.topic: concept-article
 ms.subservice: security
-ms.date: 01/12/2020
+ms.date: 11/15/2024
 ms.author: seshin
 ms.reviewer: whhender
 ---
 
-# Synapse Managed private endpoints
-
-This article will explain Managed private endpoints in Azure Synapse Analytics.
-
-## Managed private endpoints
+# Azure Synapse Analytics managed private endpoints
 
 Managed private endpoints are private endpoints created in a Managed Virtual Network associated with your Azure Synapse workspace. Managed private endpoints establish a private link to Azure resources. Azure Synapse manages these private endpoints on your behalf. You can create Managed private endpoints from your Azure Synapse workspace to access Azure services (such as Azure Storage or Azure Cosmos DB) and Azure hosted customer/partner services.
 
@@ -28,14 +24,13 @@ Learn more about [private links and private endpoints](../../private-link/index.
 >[!NOTE]
 >When creating an Azure Synapse workspace, you can choose to associate a Managed Virtual Network to it. If you choose to have a Managed Virtual Network associated to your workspace, you can also choose to limit outbound traffic from your workspace to only approved targets. You must create Managed private endpoints to these targets. 
 
-
 A private endpoint connection is created in a "Pending" state when you create a Managed private endpoint in Azure Synapse. An approval workflow is started. The private link resource owner is responsible to approve or reject the connection. If the owner approves the connection, the private link is established. But, if the owner doesn't approve the connection, then the private link won't be established. In either case, the Managed private endpoint will be updated with the status of the connection. Only a Managed private endpoint in an approved state can be used to send traffic to the private link resource that is linked to the Managed private endpoint.
 
 ## Managed private endpoints for dedicated SQL pool and serverless SQL pool
 
-Dedicated SQL pool and serverless SQL pool are analytic capabilities in your Azure Synapse workspace. These capabilities use multi-tenant infrastructure that isn't deployed into the [Managed workspace Virtual Network](./synapse-workspace-managed-vnet.md).
+Dedicated SQL pool and serverless SQL pool are analytic capabilities in your Azure Synapse workspace. These capabilities use multitenant infrastructure that isn't deployed into the [Managed workspace Virtual Network](./synapse-workspace-managed-vnet.md).
 
-When a workspace is created, Azure Synapse creates two Managed private endpoints in the workspace, one for dedicated SQL pool and one for serverless SQL pool. 
+When a workspace is created, Azure Synapse creates two Managed private endpoints in the workspace, one for dedicated SQL pool and one for serverless SQL pool.
 
 These two Managed private endpoints are listed in Synapse Studio. Select **Manage** in the left navigation, then select **Managed private endpoints** to see them in the Studio.
 
@@ -45,7 +40,6 @@ The Managed private endpoint that targets SQL pool is called *synapse-ws-sql--\<
 
 These two Managed private endpoints are automatically created for you when you create your Azure Synapse workspace. You aren't charged for these two Managed private endpoints.
 
-
 ## Supported data sources
 
 Azure Synapse Spark supports over 25 data sources to connect to using managed private endpoints. Users need to specify the resource identifier, which can be found in the **Properties** settings page of their data source in the Azure portal.
@@ -81,6 +75,6 @@ Azure Synapse Spark supports over 25 data sources to connect to using managed pr
 | Azure App Services | /subscriptions/{subscription-id}/resourceGroups/{resource-group-name}/providers/Microsoft.Web/sites/{app-service-name}
 
 
-## Next steps
+## Get started
 
-To learn more, advance to the [Create Managed private endpoints to your data sources](./how-to-create-managed-private-endpoints.md) article.
+To learn more, advance to the [create managed private endpoints to your data sources](./how-to-create-managed-private-endpoints.md) article.

articles/synapse-analytics/get-started-analyze-spark.md

@@ -3,9 +3,9 @@ title: Manage Apache Spark packages
 description: Learn how to add and manage libraries used by Apache Spark in Azure Synapse Analytics.
 author: shuaijunye
 ms.service: azure-synapse-analytics
-ms.reviewer: whhender, whhender, eskot
+ms.reviewer: whhender, eskot
 ms.topic: how-to
-ms.date: 04/15/2023
+ms.date: 11/15/2024
 ms.author: shuaijunye
 ms.subservice: spark
 ms.custom: kr2b-contr-experiment, devx-track-azurepowershell
@@ -120,7 +120,6 @@ To learn more about how to manage session-scoped packages, see the following art
 
 - [R session packages](./apache-spark-manage-session-packages.md#session-scoped-r-packages-preview): Within your session, you can install packages across all nodes within your Spark pool by using `install.packages` or `devtools`.
 
-
 ## Automate the library management process through Azure PowerShell cmdlets and REST APIs
 
 If your team wants to manage libraries without visiting the package management UIs, you have the option to manage the workspace packages and pool-level package updates through Azure PowerShell cmdlets or REST APIs for Azure Synapse Analytics.
@@ -130,7 +129,7 @@ For more information, see the following articles:
 - [Manage your Spark pool libraries through REST APIs](apache-spark-manage-packages-outside-ui.md#manage-packages-through-rest-apis)
 - [Manage your Spark pool libraries through Azure PowerShell cmdlets](apache-spark-manage-packages-outside-ui.md#manage-packages-through-azure-powershell-cmdlets)
 
-## Next steps
+## Related content
 
 - [View the default libraries and supported Apache Spark versions](apache-spark-version-support.md)
 - [Troubleshoot library installation errors](apache-spark-troubleshoot-library-errors.md)