You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: articles/hdinsight/spark/apache-spark-deep-learning-caffe.md
+8-6Lines changed: 8 additions & 6 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -3,6 +3,7 @@ title: Use Caffe on Azure HDInsight Spark for distributed deep learning
3
3
description: Use Caffe on Apache Spark for distributed deep learning in Azure HDInsight.
4
4
author: hrasheed-msft
5
5
ms.author: hrasheed
6
+
ms.reviewer: jasonh
6
7
ms.service: hdinsight
7
8
ms.custom: hdinsightactive
8
9
ms.topic: conceptual
@@ -60,7 +61,7 @@ The second step is to download, compile, and install protobuf 2.5.0 for Caffe du
60
61
61
62
To get started, you can just run this script action against your cluster to all the worker nodes and head nodes (for HDInsight 3.5). You can either run the script actions on an existing cluster, or use script actions during the cluster creation. For more information on the script actions, see the documentation [here](https://docs.microsoft.com/azure/hdinsight/hdinsight-hadoop-customize-cluster-linux).
62
63
63
-

64
+

64
65
65
66
66
67
## Step 2: Build Caffe on Apache Spark for HDInsight on the head node
@@ -170,7 +171,8 @@ For this example, since you are using CPU rather than GPU, you should change the
For more information on how to define the network, check the [Caffe documentation on MNIST dataset](https://caffe.berkeleyvision.org/gathered/examples/mnist.html)
185
187
@@ -199,15 +201,15 @@ If you want to know what happened, you usually need to get the Spark driver's lo
You can take a look at how many resources are allocated for this particular application. You can click the "Scheduler" link, and then you will see that for this application, there are nine containers running. you ask YARN to provide eight executors, and another container is for driver process.
You may want to check the driver logs or container logs if there are failures. For driver logs, you can click the application ID in YARN UI, then click the "Logs" button. The driver logs are written into stderr.
0 commit comments