Skip to content

Commit d35e81f

Browse files
WeichenXu123mengxr
authored andcommitted
[SPARK-27454][ML][SQL] Spark image datasource fail when encounter some illegal images
## What changes were proposed in this pull request? Fix in Spark image datasource fail when encounter some illegal images. This related to bugs inside `ImageIO.read` so in spark code I add exception handling for it. ## How was this patch tested? N/A Please review http://spark.apache.org/contributing.html before opening a pull request. Closes apache#24362 from WeichenXu123/fix_image_ds_bug. Authored-by: WeichenXu <[email protected]> Signed-off-by: Xiangrui Meng <[email protected]>
1 parent 3ab96d7 commit d35e81f

File tree

1 file changed

+7
-1
lines changed

1 file changed

+7
-1
lines changed

mllib/src/main/scala/org/apache/spark/ml/image/ImageSchema.scala

Lines changed: 7 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -133,7 +133,13 @@ object ImageSchema {
133133
*/
134134
private[spark] def decode(origin: String, bytes: Array[Byte]): Option[Row] = {
135135

136-
val img = ImageIO.read(new ByteArrayInputStream(bytes))
136+
val img = try {
137+
ImageIO.read(new ByteArrayInputStream(bytes))
138+
} catch {
139+
// Catch runtime exception because `ImageIO` may throw unexcepted `RuntimeException`.
140+
// But do not catch the declared `IOException` (regarded as FileSystem failure)
141+
case _: RuntimeException => null
142+
}
137143

138144
if (img == null) {
139145
None

0 commit comments

Comments
 (0)