This repository was archived by the owner on Aug 13, 2024. It is now read-only.
explore handling minute or hour buckets with updatestatebykey #26
Description
explore
// Update the cumulative count using updateStateByKey
// This will give a Dstream made of state (which is the cumulative count of the events)
// instead of doing some in Dynamo
val stateDstream = eventDstream.updateStateByKey[Int](newUpdateFunc,
new HashPartitioner %28ssc.sparkContext.defaultParallelism%29, true, initialRDD)