Skip to content

Commit 4e44998

Browse files
author
Sreekanth Iyer (Ushta Te Consultancy Services)
committed
Added last point to the bullet list
1 parent b50c507 commit 4e44998

File tree

1 file changed

+1
-0
lines changed

1 file changed

+1
-0
lines changed

articles/hdinsight/spark/what-is-speculative-execution-in-spark.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -41,6 +41,7 @@ When job the finished, the original task 5 is killed by the driver.
4141
* When the data is skewed, the speculated task takes same time as that of the original task increasing the resource consumption and slows the execution time. The data skewness has to be addressed first.
4242
* When speculative execution is enabled, sometimes duplicate records are created if there's any unexpected executor or node failure. Make sure the job is idempotent.
4343
* Enabling speculation can impact performance as duplicate tasks are being created. Make sure it's disabled if the performance is a concern.
44+
* Speculative execution shouldn't be used for a long time period on production jobs. Extended use can result in failed tasks/capacity issues.
4445

4546
## Next Steps
4647

0 commit comments

Comments
 (0)