Skip to content

Commit e89f070

Browse files
authored
Update migrate-storm-to-spark.md
1 parent 2ab1fb3 commit e89f070

File tree

1 file changed

+4
-4
lines changed

1 file changed

+4
-4
lines changed

articles/hdinsight/storm/migrate-storm-to-spark.md

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -30,6 +30,10 @@ Apache Storm can provide different levels of guaranteed message processing. For
3030
|**Event time support**|[Yes](https://storm.apache.org/releases/2.0.0/Windowing.html)|No|[Yes](https://spark.apache.org/docs/latest/structured-streaming-programming-guide.html)|
3131
|**Languages**|Java, etc.|Scala, Java, Python|Python, R, Scala, Java, SQL|
3232

33+
### Spark streaming vs Spark structured streaming
34+
35+
Spark Structured Streaming is replacing Spark Streaming (DStreams). Going forward, Structured Streaming will receive enhancements and maintenance, while DStreams will be in maintenance mode only. Structured Streaming is currently not as feature-complete as DStreams for the sources and sinks that it supports out of the box, so evaluate your requirements to choose the appropriate Spark stream processing option.
36+
3337
## Streaming (Single event) processing vs Micro-Batch processing
3438

3539
Storm provides a model that processes each single event. This means that all incoming records will be processed as soon as they arrive. Spark Streaming applications must wait a fraction of a second to collect each micro-batch of events before sending that batch on for processing. In contrast, an event-driven application processes each event immediately. Spark Streaming latency is typically under a few seconds. The benefits of the micro-batch approach are more efficient data processing and simpler aggregate calculations.
@@ -157,10 +161,6 @@ val windowedCounts = words.groupBy(
157161

158162
If the built-in operations don't meet the data transformation requirements, you can use UDF (User-Defined Functions).
159163

160-
### Spark streaming vs Spark structured streaming
161-
162-
Spark Structured Streaming is replacing Spark Streaming (DStreams). Going forward, Structured Streaming will receive enhancements and maintenance, while DStreams will be in maintenance mode only. Structured Streaming is currently not as feature-complete as DStreams for the sources and sinks that it supports out of the box, so evaluate your requirements to choose the appropriate Spark stream processing option.
163-
164164
## General migration flow
165165

166166
Presumed current environment:

0 commit comments

Comments
 (0)