Skip to content

Commit 3776fbd

Browse files
JoshRosendongjoon-hyun
authored andcommitted
[SPARK-28430][UI] Fix stage table rendering when some tasks' metrics are missing
## What changes were proposed in this pull request? The Spark UI's stages table misrenders the input/output metrics columns when some tasks are missing input metrics. See the screenshot below for an example of the problem: ![image](https://user-images.githubusercontent.com/50748/61420042-a3abc100-a8b5-11e9-8a92-7986563ee712.png) This is because those columns' are defined as ```scala {if (hasInput(stage)) { metricInfo(task) { m => ... <td>....</td> } } ``` where `metricInfo` renders the node returned by the closure in case metrics are defined or returns `Nil` in case metrics are not defined. If metrics are undefined then we'll fail to render the empty `<td></td>` tag, causing columns to become misaligned as shown in the screenshot. To fix this, this patch changes this to ```scala {if (hasInput(stage)) { <td>{ metricInfo(task) { m => ... Unparsed(...) } }</td> } ``` which is an idiom that's already in use for the shuffle read / write columns. ## How was this patch tested? It isn't. I'm arguing for correctness because the modifications are consistent with rendering methods that work correctly for other columns. Closes apache#25183 from JoshRosen/joshrosen/fix-task-table-with-partial-io-metrics. Authored-by: Josh Rosen <[email protected]> Signed-off-by: Dongjoon Hyun <[email protected]>
1 parent a0c2fa6 commit 3776fbd

File tree

1 file changed

+14
-10
lines changed

1 file changed

+14
-10
lines changed

core/src/main/scala/org/apache/spark/ui/jobs/StagePage.scala

Lines changed: 14 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -641,18 +641,22 @@ private[ui] class TaskPagedTable(
641641
<td>{accumulatorsInfo(task)}</td>
642642
}}
643643
{if (hasInput(stage)) {
644-
metricInfo(task) { m =>
645-
val bytesRead = Utils.bytesToString(m.inputMetrics.bytesRead)
646-
val records = m.inputMetrics.recordsRead
647-
<td>{bytesRead} / {records}</td>
648-
}
644+
<td>{
645+
metricInfo(task) { m =>
646+
val bytesRead = Utils.bytesToString(m.inputMetrics.bytesRead)
647+
val records = m.inputMetrics.recordsRead
648+
Unparsed(s"$bytesRead / $records")
649+
}
650+
}</td>
649651
}}
650652
{if (hasOutput(stage)) {
651-
metricInfo(task) { m =>
652-
val bytesWritten = Utils.bytesToString(m.outputMetrics.bytesWritten)
653-
val records = m.outputMetrics.recordsWritten
654-
<td>{bytesWritten} / {records}</td>
655-
}
653+
<td>{
654+
metricInfo(task) { m =>
655+
val bytesWritten = Utils.bytesToString(m.outputMetrics.bytesWritten)
656+
val records = m.outputMetrics.recordsWritten
657+
Unparsed(s"$bytesWritten / $records")
658+
}
659+
}</td>
656660
}}
657661
{if (hasShuffleRead(stage)) {
658662
<td class={TaskDetailsClassNames.SHUFFLE_READ_BLOCKED_TIME}>

0 commit comments

Comments
 (0)