Merge pull request #97417 from dagiro/freshness84

freshness84

Author: GitHubber17

articles/hdinsight/spark/apache-spark-job-debugging.md

@@ -5,37 +5,38 @@ author: hrasheed-msft
 ms.author: hrasheed
 ms.reviewer: jasonh
 ms.service: hdinsight
-ms.custom: hdinsightactive
 ms.topic: conceptual
-ms.date: 12/05/2018
+ms.custom: hdinsightactive
+ms.date: 11/29/2019
 ---
 
 # Debug Apache Spark jobs running on Azure HDInsight
 
 In this article, you learn how to track and debug [Apache Spark](https://spark.apache.org/) jobs running on HDInsight clusters using the [Apache Hadoop YARN](https://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/YARN.html) UI, Spark UI, and the Spark History Server. You start a Spark job using a notebook available with the Spark cluster, **Machine learning: Predictive analysis on food inspection data using MLLib**. You can use the following steps to track an application that you submitted using any other approach as well, for example, **spark-submit**.
 
-## Prerequisites
+If you don't have an Azure subscription, create a [free account](https://azure.microsoft.com/free/?WT.mc_id=A261C142F) before you begin.
 
-You must have the following:
+## Prerequisites
 
-* An Azure subscription. See [Get Azure free trial](https://azure.microsoft.com/documentation/videos/get-azure-free-trial-for-testing-hadoop-in-hdinsight/).
 * An Apache Spark cluster on HDInsight. For instructions, see [Create Apache Spark clusters in Azure HDInsight](apache-spark-jupyter-spark-sql.md).
+
 * You should have started running the notebook, **[Machine learning: Predictive analysis on food inspection data using MLLib](apache-spark-machine-learning-mllib-ipython.md)**. For instructions on how to run this notebook, follow the link.  
 
 ## Track an application in the YARN UI
 
-1. Launch the YARN UI. Click **Yarn** under **Cluster dashboards**.
+1. Launch the YARN UI. Select **Yarn** under **Cluster dashboards**.
 
     ![Azure portal launch YARN UI](./media/apache-spark-job-debugging/launch-apache-yarn-ui.png)
 
    > [!TIP]  
-   > Alternatively, you can also launch the YARN UI from the Ambari UI. To launch the Ambari UI, click **Ambari home** under **Cluster dashboards**. From the Ambari UI, click **YARN**, click **Quick Links**, click the active Resource Manager, and then click **Resource Manager UI**.
+   > Alternatively, you can also launch the YARN UI from the Ambari UI. To launch the Ambari UI, select **Ambari home** under **Cluster dashboards**. From the Ambari UI, navigate to **YARN** > **Quick Links** > the active Resource Manager > **Resource Manager UI**.
 
-2. Because you started the Spark job using Jupyter notebooks, the application has the name **remotesparkmagics** (this is the name for all applications that are started from the notebooks). Click the application ID against the application name to get more information about the job. This launches the application view.
+2. Because you started the Spark job using Jupyter notebooks, the application has the name **remotesparkmagics** (this is the name for all applications that are started from the notebooks). Select the application ID against the application name to get more information about the job. This launches the application view.
 
     ![Spark history server Find Spark application ID](./media/apache-spark-job-debugging/find-application-id1.png)
 
     For such applications that are launched from the Jupyter notebooks, the status is always **RUNNING** until you exit the notebook.
+
 3. From the application view, you can drill down further to find out the containers associated with the application and the logs (stdout/stderr). You can also launch the Spark UI by clicking the linking corresponding to the **Tracking URL**, as shown below.
 
     ![Spark history server download container logs](./media/apache-spark-job-debugging/download-container-logs.png)
@@ -44,15 +45,15 @@ You must have the following:
 
 In the Spark UI, you can drill down into the Spark jobs that are spawned by the application you started earlier.
 
-1. To launch the Spark UI, from the application view, click the link against the **Tracking URL**, as shown in the screen capture above. You can see all the Spark jobs that are launched by the application running in the Jupyter notebook.
+1. To launch the Spark UI, from the application view, select the link against the **Tracking URL**, as shown in the screen capture above. You can see all the Spark jobs that are launched by the application running in the Jupyter notebook.
 
     ![Spark history server jobs tab](./media/apache-spark-job-debugging/view-apache-spark-jobs.png)
 
-2. Click the **Executors** tab to see processing and storage information for each executor. You can also retrieve the call stack by clicking on the **Thread Dump** link.
+2. Select the **Executors** tab to see processing and storage information for each executor. You can also retrieve the call stack by selecting the **Thread Dump** link.
 
     ![Spark history server executors tab](./media/apache-spark-job-debugging/view-spark-executors.png)
 
-3. Click the **Stages** tab to see the stages associated with the application.
+3. Select the **Stages** tab to see the stages associated with the application.
 
     ![Spark history server stages tab](./media/apache-spark-job-debugging/view-apache-spark-stages.png "View Spark stages")
 
@@ -77,8 +78,8 @@ In the Spark UI, you can drill down into the Spark jobs that are spawned by the
 
 6. Other tabs in the Spark UI provide useful information about the Spark instance as well.
 
-   * Storage tab - If your application creates an RDDs, you can find information about those in the Storage tab.
-   * Environment tab - This tab provides a lot of useful information about your Spark instance such as the:
+   * Storage tab - If your application creates an RDD, you can find information about those in the Storage tab.
+   * Environment tab - This tab provides useful information about your Spark instance such as the:
      * Scala version
      * Event log directory associated with the cluster
      * Number of executor cores for the application
@@ -88,14 +89,14 @@ In the Spark UI, you can drill down into the Spark jobs that are spawned by the
 
 Once a job is completed, the information about the job is persisted in the Spark History Server.
 
-1. To launch the Spark History Server, from the Overview blade, click **Spark history server** under **Cluster dashboards**.
+1. To launch the Spark History Server, from the **Overview** page, select **Spark history server** under **Cluster dashboards**.
 
     ![Azure portal launch Spark history server](./media/apache-spark-job-debugging/launch-spark-history-server.png "Launch Spark History Server1")
 
    > [!TIP]  
-   > Alternatively, you can also launch the Spark History Server UI from the Ambari UI. To launch the Ambari UI, from the Overview blade, click **Ambari home** under **Cluster dashboards**. From the Ambari UI, click **Spark**, click **Quick Links**, and then click **Spark History Server UI**.
+   > Alternatively, you can also launch the Spark History Server UI from the Ambari UI. To launch the Ambari UI, from the Overview blade, select **Ambari home** under **Cluster dashboards**. From the Ambari UI, navigate to **Spark2** > **Quick Links** > **Spark2 History Server UI**.
 
-2. You see all the completed applications listed. Click an application ID to drill down into an application for more info.
+2. You see all the completed applications listed. Select an application ID to drill down into an application for more info.
 
     ![Spark history server completed applications](./media/apache-spark-job-debugging/view-completed-applications.png "Launch Spark History Server2")