Skip to content

Commit d52bf47

Browse files
lw-linzsxwing
authored andcommitted
[SPARK-12304][STREAMING] Make Spark Streaming web UI display more fri…
…endly Receiver graphs Currently, the Spark Streaming web UI uses the same maxY when displays 'Input Rate Times& Histograms' and 'Per-Receiver Times& Histograms'. This may lead to somewhat un-friendly graphs: once we have tens of Receivers or more, every 'Per-Receiver Times' line almost hits the ground. This issue proposes to calculate a new maxY against the original one, which is shared among all the `Per-Receiver Times& Histograms' graphs. Before: ![before-5](https://cloud.githubusercontent.com/assets/15843379/11761362/d790c356-a0fa-11e5-860e-4b834603de1d.png) After: ![after-5](https://cloud.githubusercontent.com/assets/15843379/11761361/cfabf692-a0fa-11e5-97d0-4ad124aaca2a.png) Author: proflin <[email protected]> Closes #10318 from proflin/SPARK-12304.
1 parent ca0690b commit d52bf47

File tree

1 file changed

+7
-1
lines changed

1 file changed

+7
-1
lines changed

streaming/src/main/scala/org/apache/spark/streaming/ui/StreamingPage.scala

Lines changed: 7 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -392,9 +392,15 @@ private[ui] class StreamingPage(parent: StreamingTab)
392392
maxX: Long,
393393
minY: Double,
394394
maxY: Double): Seq[Node] = {
395+
val maxYCalculated = listener.receivedEventRateWithBatchTime.values
396+
.flatMap { case streamAndRates => streamAndRates.map { case (_, eventRate) => eventRate } }
397+
.reduceOption[Double](math.max)
398+
.map(_.ceil.toLong)
399+
.getOrElse(0L)
400+
395401
val content = listener.receivedEventRateWithBatchTime.toList.sortBy(_._1).map {
396402
case (streamId, eventRates) =>
397-
generateInputDStreamRow(jsCollector, streamId, eventRates, minX, maxX, minY, maxY)
403+
generateInputDStreamRow(jsCollector, streamId, eventRates, minX, maxX, minY, maxYCalculated)
398404
}.foldLeft[Seq[Node]](Nil)(_ ++ _)
399405

400406
// scalastyle:off

0 commit comments

Comments
 (0)