Skip to content

[SPARK-15373][WEB UI] Spark UI should show consistent timezones. #13158

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 5 commits into from
Closed

[SPARK-15373][WEB UI] Spark UI should show consistent timezones. #13158

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
4f4e605
[SPARK-15373][WEB UI] Spark UI should show consistent timezones.
dongjoon-hyun May 17, 2016
67c608e
Use TimeZone.getDefault().
dongjoon-hyun May 18, 2016
e170041
Revert the file permission on vis.min.js and vis.min.css.
dongjoon-hyun May 18, 2016
0fa7d5b
Make UIUtils.getTimeZoneOffset function.
dongjoon-hyun May 18, 2016
206171a
Remove unused imports
dongjoon-hyun May 18, 2016
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 14 additions & 5 deletions core/src/main/resources/org/apache/spark/ui/static/timeline-view.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@
* limitations under the License.
*/

function drawApplicationTimeline(groupArray, eventObjArray, startTime) {
function drawApplicationTimeline(groupArray, eventObjArray, startTime, offset) {
var groups = new vis.DataSet(groupArray);
var items = new vis.DataSet(eventObjArray);
var container = $("#application-timeline")[0];
Expand All @@ -26,7 +26,10 @@ function drawApplicationTimeline(groupArray, eventObjArray, startTime) {
editable: false,
showCurrentTime: false,
min: startTime,
zoomable: false
zoomable: false,
moment: function (date) {
return vis.moment(date).utcOffset(offset);
Copy link
Member

@srowen srowen May 18, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does vis support an actual timezone, or just a UTC offset? some timezones' offsets aren't whole hours :(

Copy link
Member Author

@dongjoon-hyun dongjoon-hyun May 18, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unfortunately, moment (used by vis) only supports the followings. Actually, this PR is using the last one, minute format.

moment().utcOffset("+08:00");
moment().utcOffset(8);   // hour: -16 < x < 16
moment().utcOffset(480); // minutes

Copy link
Member

@srowen srowen May 18, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ah right, your formula gives minutes not hours. That seems fine.

}
};

var applicationTimeline = new vis.Timeline(container);
Expand Down Expand Up @@ -87,7 +90,7 @@ $(function (){
}
});

function drawJobTimeline(groupArray, eventObjArray, startTime) {
function drawJobTimeline(groupArray, eventObjArray, startTime, offset) {
var groups = new vis.DataSet(groupArray);
var items = new vis.DataSet(eventObjArray);
var container = $('#job-timeline')[0];
Expand All @@ -99,6 +102,9 @@ function drawJobTimeline(groupArray, eventObjArray, startTime) {
showCurrentTime: false,
min: startTime,
zoomable: false,
moment: function (date) {
return vis.moment(date).utcOffset(offset);
}
};

var jobTimeline = new vis.Timeline(container);
Expand Down Expand Up @@ -159,7 +165,7 @@ $(function (){
}
});

function drawTaskAssignmentTimeline(groupArray, eventObjArray, minLaunchTime, maxFinishTime) {
function drawTaskAssignmentTimeline(groupArray, eventObjArray, minLaunchTime, maxFinishTime, offset) {
var groups = new vis.DataSet(groupArray);
var items = new vis.DataSet(eventObjArray);
var container = $("#task-assignment-timeline")[0]
Expand All @@ -173,7 +179,10 @@ function drawTaskAssignmentTimeline(groupArray, eventObjArray, minLaunchTime, ma
showCurrentTime: false,
min: minLaunchTime,
max: maxFinishTime,
zoomable: false
zoomable: false,
moment: function (date) {
return vis.moment(date).utcOffset(offset);
}
};

var taskTimeline = new vis.Timeline(container)
Expand Down
2 changes: 1 addition & 1 deletion core/src/main/resources/org/apache/spark/ui/static/vis.min.css
View file
Open in desktop

Large diffs are not rendered by default.

41 changes: 24 additions & 17 deletions core/src/main/resources/org/apache/spark/ui/static/vis.min.js
View file
Open in desktop

Large diffs are not rendered by default.

5 changes: 4 additions & 1 deletion core/src/main/scala/org/apache/spark/ui/UIUtils.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ package org.apache.spark.ui

import java.net.URLDecoder
import java.text.SimpleDateFormat
import java.util.{Date, Locale}
import java.util.{Date, Locale, TimeZone}

import scala.util.control.NonFatal
import scala.xml._
Expand Down Expand Up @@ -502,4 +502,7 @@ private[spark] object UIUtils extends Logging {
}
param
}

def getTimeZoneOffset() : Int =
TimeZone.getDefault().getOffset(System.currentTimeMillis()) / 1000 / 60
}
2 changes: 1 addition & 1 deletion core/src/main/scala/org/apache/spark/ui/jobs/AllJobsPage.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -206,7 +206,7 @@ private[ui] class AllJobsPage(parent: JobsTab) extends WebUIPage("") {
</div> ++
<script type="text/javascript">
{Unparsed(s"drawApplicationTimeline(${groupJsonArrayAsStr}," +
s"${eventArrayAsStr}, ${startTime});")}
s"${eventArrayAsStr}, ${startTime}, ${UIUtils.getTimeZoneOffset()});")}
</script>
}

Expand Down
3 changes: 2 additions & 1 deletion core/src/main/scala/org/apache/spark/ui/jobs/JobPage.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -179,7 +179,8 @@ private[ui] class JobPage(parent: JobsTab) extends WebUIPage("job") {
</div>
</div> ++
<script type="text/javascript">
{Unparsed(s"drawJobTimeline(${groupJsonArrayAsStr}, ${eventArrayAsStr}, ${appStartTime});")}
{Unparsed(s"drawJobTimeline(${groupJsonArrayAsStr}, ${eventArrayAsStr}, " +
s"${appStartTime}, ${UIUtils.getTimeZoneOffset()});")}
</script>
}

Expand Down
5 changes: 3 additions & 2 deletions core/src/main/scala/org/apache/spark/ui/jobs/StagePage.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ import scala.xml.{Elem, Node, Unparsed}

import org.apache.commons.lang3.StringEscapeUtils

import org.apache.spark.{InternalAccumulator, SparkConf}
import org.apache.spark.SparkConf
import org.apache.spark.executor.TaskMetrics
import org.apache.spark.scheduler.{AccumulableInfo, TaskInfo, TaskLocality}
import org.apache.spark.ui._
Expand Down Expand Up @@ -746,7 +746,8 @@ private[ui] class StagePage(parent: StagesTab) extends WebUIPage("stage") {
</div> ++
<script type="text/javascript">
{Unparsed(s"drawTaskAssignmentTimeline(" +
s"$groupArrayStr, $executorsArrayStr, $minLaunchTime, $maxFinishTime)")}
s"$groupArrayStr, $executorsArrayStr, $minLaunchTime, $maxFinishTime, " +
s"${UIUtils.getTimeZoneOffset()})")}
</script>
}

Expand Down