Skip to content

Commit 5c6ef09

Browse files
authored
Updating changes
1 parent f033b49 commit 5c6ef09

File tree

1 file changed

+1
-1
lines changed

1 file changed

+1
-1
lines changed

articles/hdinsight/hdinsight-apache-kafka-spark-structured-streaming.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -183,7 +183,7 @@ This example demonstrates how to use Spark Structured Streaming with Kafka on HD
183183
184184
4. Load packages used by the Notebook by entering the following information in a Notebook cell. Run the command by using **CTRL + ENTER**.
185185
186-
**We recommend that you disable dynamic allocation by setting spark.dynamicAllocation.enabled to false when running streaming applications.**
186+
Spark streaming has microbatching, which means data comes as batches and executers run on the batches of data. If the executor has idle timeout less than the time it takes to process the batch then the executors would be constantly added and removed. If the executors idle timeout is greater than the batch duration, the executor never gets removed. Hence **we recommend that you disable dynamic allocation by setting spark.dynamicAllocation.enabled to false when running streaming applications.**
187187
188188
```
189189
%%configure -f

0 commit comments

Comments
 (0)