Merge pull request #284944 from MicrosoftDocs/main

Publish to live, Monday 4 AM PST, 8/19

Author: ttorble

articles/hdinsight/azure-monitor-agent.md

@@ -3,7 +3,7 @@ title: Azure Monitor Agent (AMA) migration guide for Azure HDInsight clusters
 description: Learn how to migrate to Azure Monitor Agent (AMA) in Azure HDInsight clusters.
 ms.service: azure-hdinsight
 ms.topic: how-to
-ms.date: 07/31/2024
+ms.date: 08/14/2024
 ---
 
 # Azure Monitor Agent (AMA) migration guide for Azure HDInsight clusters
@@ -84,13 +84,29 @@ Activate the new integration by going to your cluster's portal page and scrollin
 1. Then, select Enable and you can choose the Log Analytics workspace that you want your logs to be sent to.  
     :::image type="content" source="./media/azure-monitor-agent/monitor-integration.png" alt-text=" Screenshot showing Azure monitor integration." border="true" lightbox="./media/azure-monitor-agent/monitor-integration.png":::
 
-1. Enable Azure Monitor Agent Integration with Log Analytics and select your workspace (existing workspace when you're migrating from your previous image to newer image) 
+1. Enable Azure Monitor Agent Integration with Log Analytics and select your workspace (existing workspace when you're migrating from your previous image to newer image).
 
 1. Once you confirm the workspace selection, precondition steps commence. 
 
     :::image type="content" source="./media/azure-monitor-agent/pre-condition.png" alt-text="Screenshot showing preconditions." border="true" lightbox="./media/azure-monitor-agent/pre-condition.png":::
 
-1. Select Save once precondition steps are complete. 
+1. Select Save once precondition steps are complete.
+
+### Enable Azure Monitor Agent logging for Spark cluster  
+
+Azure HDInsight Spark clusters control AMA integration using a Spark configuration `spark.hdi.ama.enabled`, by default the value is set to false. This configuration controls whether the Spark specific logs will come up in the Log Analytics workspace. If you want to enable AMA in your Spark clusters and retrieve the Spark event logs in their LA workspaces, you need to perform an additional step to enable AMA for spark specific logs.
+
+The following steps describe how customers can enable the new Azure Monitor Agent logging for their spark workloads.
+
+1. Go to Ambari -> Spark Configs.
+
+1. Navigate to **Custom Spark defaults** and search for config `spark.hdi.ama.enabled`, the default value of this config will be false. Set this value as  **true**. 
+
+    :::image type="content" source="./media/azure-monitor-agent/enable-spark.png" alt-text="Screenshot showing how to enable Azure Monitor Agent logging for Spark cluster." border="true" lightbox="./media/azure-monitor-agent/enable-spark.png":::
+
+1. Click **save** and restart Spark services on all nodes. 
+
+1. Access the tables in LA workspace.  
 
 ### Access the new tables 
 

articles/hdinsight/media/azure-monitor-agent/enable-spark.png

@@ -337,17 +337,22 @@
     },
     {
       "source_path_from_root": "/articles/iot-central/howto-create-and-manage-applications-as-CSP.md",
-      "redirect_url": "/azure/iot-central/core/howto-create-and-manage-applications-csp",
+      "redirect_url": "/azure/iot-central/core/howto-create-iot-central-application",
       "redirect_document_id": false
     },
     {
       "source_path_from_root": "/articles/iot-central/howto-create-and-manage-applications-csp.md",
-      "redirect_url": "/azure/iot-central/core/howto-create-and-manage-applications-csp/",
+      "redirect_url": "/azure/iot-central/core/howto-create-iot-central-application/",
       "redirect_document_id": false
     },
     {
       "source_path_from_root": "/articles/iot-central/preview/howto-create-and-manage-applications-csp.md",
-      "redirect_url": "/azure/iot-central/core/howto-create-and-manage-applications-csp/",
+      "redirect_url": "/azure/iot-central/core/howto-create-iot-central-application/",
+      "redirect_document_id": false
+    },
+    {
+      "source_path_from_root": "/articles/iot-central/core/howto-create-and-manage-applications-csp.md",
+      "redirect_url": "/azure/iot-central/core/howto-create-iot-central-application/",
       "redirect_document_id": false
     },
     {

articles/iot-central/.openpublishing.redirection.iot-central.json

@@ -215,8 +215,6 @@
           href: core/howto-integrate-with-devops.md
         - name: Manage and monitor applications
           href: core/howto-manage-and-monitor-iot-central.md
-        - name: Manage from CSP portal
-          href: core/howto-create-and-manage-applications-csp.md
         - name: Create a private endpoint for devices
           href: core/howto-create-private-endpoint.md
         - name: Export to a secure VNET

articles/iot-central/TOC.yml

@@ -20,7 +20,7 @@ Whichever approach you choose, the configuration options are the same, and the p
 Other approaches, not described in this article include:
 
 - [Use the REST API to create and manage IoT Central applications.](../core/howto-manage-iot-central-with-rest-api.md).
-- [Create and manage an Azure IoT Central application from the Microsoft Cloud Solution Provider portal](howto-create-and-manage-applications-csp.md).
+- [Create and manage an Azure IoT Central application from the Microsoft Cloud Solution Provider portal](https://partner.microsoft.com/cloud-solution-provider).
 
 ## Parameters