[SPARK-20275][UI] Do not display "Completed" column for in-progress applications

[jerryshao]

What changes were proposed in this pull request?

Current HistoryServer will display completed date of in-progress application as 1969-12-31 23:59:59, which is not so meaningful. Instead of unnecessarily showing this incorrect completed date, here propose to make this column invisible for in-progress applications.

The purpose of only making this column invisible rather than deleting this field is that: this data is fetched through REST API, and in the REST API the format is like below shows, in which endTime matches endTimeEpoch. So instead of changing REST API to break backward compatibility, here choosing a simple solution to only make this column invisible.

[ {
  "id" : "local-1491805439678",
  "name" : "Spark shell",
  "attempts" : [ {
    "startTime" : "2017-04-10T06:23:57.574GMT",
    "endTime" : "1969-12-31T23:59:59.999GMT",
    "lastUpdated" : "2017-04-10T06:23:57.574GMT",
    "duration" : 0,
    "sparkUser" : "",
    "completed" : false,
    "startTimeEpoch" : 1491805437574,
    "endTimeEpoch" : -1,
    "lastUpdatedEpoch" : 1491805437574
  } ]
} ]%

Here is UI before changed:

And after:

How was this patch tested?

Manual verification.

[SparkQA]

Test build #75651 has finished for PR 17588 at commit 6828315.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[jerryshao]

Ping @srowen @ajbozarth do you have any further comment? Thanks.

[ajbozarth]

Nope, that was a LGTM

[jerryshao]

Ping @srowen again.

[jiangxb1987]

LGTM, cc @cloud-fan

[jiangxb1987]

retest this please

[srowen]

Oops, lost track of this one. i'll merge after tests.

[SparkQA]

Test build #77551 has finished for PR 17588 at commit 6828315.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[cloud-fan]

LGTM, merging to master/2.2/2.1!