-
Notifications
You must be signed in to change notification settings - Fork 15
User Guide 1.0.0 alpha
In order to use DDS in your Spark shell, you need to add it to your classpath and import the DDS core functions.
./bin/spark-shell --driver-class-path spawncamping-dds-1.0.0-alpha.jarimport de.frosner.dds.core.DDS._DDS comes with a lightweight web server that serves the results and charts to your browser. It pushes JSON objects to the JavaScript front-end that will then display them using HTML, CSS and SVG. The server needs to be started once after the Spark shell has loaded. It can be used for the entire session. However, you can stop and restart it as often as you like.
The server can be started by calling the start() function in the Spark shell.
The server can be stopped by calling the stop() function in the Spark shell.
summarize(numbers: RDD[N])(implicit num: Numeric[N])Shows some basic summary statistics for the given numerical RDD.
groupAndSummarize(readyToGroup: RDD[(K, N)])(implicit num: Numeric[N])Shows some basic summary statistics for each of the groups defined by the given key. It is assumed that there are key-value pairs in each input row, where the key can be used for grouping.
summarizeGroups(grouped: RDD[(K, Iterable[N])])(implicit num: Numeric[N])Shows some basic summary statistics for each of the given groups. It is assumed that there is one input row per group and is usually a result of a group-by operation on an RDD.
groupAndPie(readyToGroup: RDD[(K, N)])(implicit num: Numeric[N])Computes a pie chart vizualizing the sum of the numeric values per group. It is assumed that there are key-value pairs in each input row, where the key can be used for grouping. DDS will sum the values for each group.
pieGroups(grouped: RDD[(K, Iterable[N])])(implicit num: Numeric[N])Computes a pie chart visualizing the sum of the numeric values per group. It is assumed that there is one input row per group and is usually a result of a group-by operation on an RDD.