[SPARK-3777] Display "Executor ID" for Tasks in Stage page

[zsxwing]

Now the Stage page only displays "Executor"(host) for tasks. However, there may be more than one Executors running in the same host. Currently, when some task is hung, I only know the host of the faulty executor. Therefore I have to check all executors in the host.

Adding "Executor ID" in the Tasks table. would be helpful to locate the faulty executor. Here is the new page:

[SparkQA]

QA tests have started for PR 2642 at commit 4bbe2c7.

• This patch merges cleanly.

[SparkQA]

QA tests have finished for PR 2642 at commit 4bbe2c7.

• This patch passes unit tests.
• This patch merges cleanly.
• This patch adds no public classes.

[AmplabJenkins]

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/21246/

[sryza]

As horizontal space is precious for including more metrics, might it make sense to combine "Address / Executor" and "Executor ID" into a single "Executor" column, with values like "1 / 10.37.129.2". Also, is including the port still worthwhile now that we have the ID?

[SparkQA]

QA tests have started for PR 2642 at commit 37945af.

• This patch merges cleanly.

[zsxwing]

As horizontal space is precious for including more metrics, might it make sense to combine "Address / Executor" and "Executor ID" into a single "Executor" column, with values like "1 / 10.37.129.2".

Agree. I updated to put them into one column. I use host because I feel Address should include the port number but here we only have host. The new screenshot is as follow:

Also, is including the port still worthwhile now that we have the ID?

TaskInfo does not have a port field, and I cannot find an easy way to add it. However, I think Executor ID is enough. If the executor id is provided, I can use ps -ef | grep spark | grep " <executor_id> " to find the process id. Looks port cannot help me find the process id more easily.

[SparkQA]

QA tests have finished for PR 2642 at commit 37945af.

• This patch passes unit tests.
• This patch merges cleanly.
• This patch adds no public classes.

[AmplabJenkins]

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/21295/Test PASSed.

[andrewor14]

LGTM. @sryza any other comments?

[sryza]

This LGTM too. Thanks for making those changes @zsxwing .

[andrewor14]

Alright, this is going into master and 1.1