Skip to content

Commit 704ad6b

Browse files
authored
Merge pull request #57331 from dhgoelmsft/exactlyonce
fixed formatting
2 parents 32d0141 + 63e3877 commit 704ad6b

File tree

1 file changed

+5
-1
lines changed

1 file changed

+5
-1
lines changed

articles/hdinsight/spark/apache-spark-streaming-exactly-once.md

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,7 @@ author: hrasheed-msft
77
ms.author: hrasheed
88
ms.custom: hdinsightactive
99
ms.topic: conceptual
10-
ms.date: 01/26/2018
10+
ms.date: 11/06/2018
1111
---
1212
# Create Spark Streaming jobs with exactly-once event processing
1313

@@ -55,17 +55,21 @@ Checkpoints are enabled in Spark Streaming in two steps.
5555

5656
1. In the StreamingContext object, configure the storage path for the checkpoints:
5757

58+
```Scala
5859
val ssc = new StreamingContext(spark, Seconds(1))
5960
ssc.checkpoint("/path/to/checkpoints")
61+
```
6062

6163
In HDInsight, these checkpoints should be saved to the default storage attached to your cluster, either Azure Storage or Azure Data Lake Store.
6264

6365
2. Next, specify a checkpoint interval (in seconds) on the DStream. At each interval, state data derived from the input event is persisted to storage. Persisted state data can reduce the computation needed when rebuilding the state from the source event.
6466

67+
```Scala
6568
val lines = ssc.socketTextStream("hostname", 9999)
6669
lines.checkpoint(30)
6770
ssc.start()
6871
ssc.awaitTermination()
72+
```
6973

7074
### Use idempotent sinks
7175

0 commit comments

Comments
 (0)