Cluster called un in piComputeClusterApp.py

[Marcel-Jan]

I wondered why I got an error on piComputeClusterApp.py. There's a clustername called spark://un:7077 in it, instead of local[]. I think you forgot to change it to local? I've tried the code with local[] and that works.

spark = SparkSession.builder.appName("PySparkPi on a cluster")
.master("spark://un:7077").getOrCreate()
#.master("local[*]").getOrCreate()

[jgperrin]

@rambabu-posa, can you help @Marcel-Jan ?

[rambabu-posa]

@jgperrin sure, thanks

Hi @Marcel-Jan, thanks to reaching out to me on this issue.
Actually piComputeClusterApp.py meant to run that application within a Spark Cluster (or Hadoop cluster) so that you can understand how a PySpark program works in a cluster of computers.

If you have a Spark/Hadoop cluster, please replace that url to match your cluster details.

If you run it with local[*] then it became same as piComputeApp.py. (Standard PySpark application).