Skip to content
This repository was archived by the owner on Jan 9, 2020. It is now read-only.

Commit 76ee41f

Browse files
jiangxb1987cloud-fan
authored andcommitted
[SPARK-16251][SPARK-20200][CORE][TEST] Flaky test: org.apache.spark.rdd.LocalCheckpointSuite.missing checkpoint block fails with informative message
## What changes were proposed in this pull request? Currently we don't wait to confirm the removal of the block from the slave's BlockManager, if the removal takes too much time, we will fail the assertion in this test case. The failure can be easily reproduced if we sleep for a while before we remove the block in BlockManagerSlaveEndpoint.receiveAndReply(). ## How was this patch tested? N/A Author: Xingbo Jiang <[email protected]> Closes apache#18314 from jiangxb1987/LocalCheckpointSuite. (cherry picked from commit 7dc3e69) Signed-off-by: Wenchen Fan <[email protected]>
1 parent b5504f6 commit 76ee41f

File tree

1 file changed

+8
-0
lines changed

1 file changed

+8
-0
lines changed

core/src/test/scala/org/apache/spark/rdd/LocalCheckpointSuite.scala

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -17,6 +17,10 @@
1717

1818
package org.apache.spark.rdd
1919

20+
import scala.concurrent.duration._
21+
22+
import org.scalatest.concurrent.Eventually.{eventually, interval, timeout}
23+
2024
import org.apache.spark.{LocalSparkContext, SparkContext, SparkException, SparkFunSuite}
2125
import org.apache.spark.storage.{RDDBlockId, StorageLevel}
2226

@@ -168,6 +172,10 @@ class LocalCheckpointSuite extends SparkFunSuite with LocalSparkContext {
168172
// Collecting the RDD should now fail with an informative exception
169173
val blockId = RDDBlockId(rdd.id, numPartitions - 1)
170174
bmm.removeBlock(blockId)
175+
// Wait until the block has been removed successfully.
176+
eventually(timeout(1 seconds), interval(100 milliseconds)) {
177+
assert(bmm.getBlockStatus(blockId).isEmpty)
178+
}
171179
try {
172180
rdd.collect()
173181
fail("Collect should have failed if local checkpoint block is removed...")

0 commit comments

Comments
 (0)