Skip to content

[SPARK-2297][UI] Make task attempt and speculation more explicit in UI. #1236

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
wants to merge 6 commits into from
Closed

[SPARK-2297][UI] Make task attempt and speculation more explicit in UI. #1236

wants to merge 6 commits into from

Conversation

@rxin
Copy link
Contributor

@rxin rxin commented Jun 26, 2014

New UI:

screen shot 2014-06-26 at 1 43 52 pm

@rxin
Copy link
Contributor Author

rxin commented Jun 26, 2014

Note that I hacked the internal to get this UI because speculation doesn't launch tasks on the same host. I removed the constraint during testing -- that's why you see them on the same executor.

@AmplabJenkins
Copy link

AmplabJenkins commented Jun 26, 2014

Merged build triggered.

@AmplabJenkins
Copy link

AmplabJenkins commented Jun 26, 2014

Merged build started.

kayousterhout
kayousterhout reviewed Jun 26, 2014
View reviewed changes
core/src/main/scala/org/apache/spark/ui/jobs/StagePage.scala Outdated
Copy link
Contributor

@kayousterhout kayousterhout Jun 26, 2014

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hive -> hide?

@kayousterhout
Copy link
Contributor

kayousterhout commented Jun 26, 2014

Looks good except for a few naming nits. Also the Json tests are going to (or should!) fail -- you need to update the Json strings at the bottom to include the speculative thing

kayousterhout
kayousterhout reviewed Jun 26, 2014
View reviewed changes
core/src/test/scala/org/apache/spark/util/JsonProtocolSuite.scala Outdated
Copy link
Contributor

@kayousterhout kayousterhout Jun 26, 2014

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can you name "e" isSpeculative?

@AmplabJenkins
Copy link

AmplabJenkins commented Jun 26, 2014

Merged build finished.

@AmplabJenkins
Copy link

AmplabJenkins commented Jun 26, 2014

Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/16169/

@AmplabJenkins
Copy link

AmplabJenkins commented Jun 26, 2014

Merged build triggered.

@AmplabJenkins
Copy link

AmplabJenkins commented Jun 26, 2014

Merged build started.

@AmplabJenkins
Copy link

AmplabJenkins commented Jun 26, 2014

Merged build triggered.

@AmplabJenkins
Copy link

AmplabJenkins commented Jun 26, 2014

Merged build started.

@AmplabJenkins
Copy link

AmplabJenkins commented Jun 26, 2014

Merged build finished.

@AmplabJenkins
Copy link

AmplabJenkins commented Jun 26, 2014

Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/16171/

@AmplabJenkins
Copy link

AmplabJenkins commented Jun 26, 2014

Merged build finished. All automated tests passed.

@AmplabJenkins
Copy link

AmplabJenkins commented Jun 26, 2014

All automated tests passed.
Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/16172/

@rxin
Copy link
Contributor Author

rxin commented Jun 27, 2014

Ok merging this in master. Thanks for reviewing!

@asfgit asfgit closed this in d1636dd Jun 27, 2014
@kayousterhout
Copy link
Contributor

kayousterhout commented Jun 27, 2014

Hey @rxin I was wondering -- is the TID useless now that the UI shows the attempt #?

@rxin
Copy link
Contributor Author

rxin commented Jun 27, 2014

It's going to be useless if we stop using them in logs. I think right now they might still be useful since they can be used to correlate with log messages.

@kayousterhout
Copy link
Contributor

kayousterhout commented Jun 27, 2014

Oh good point that makes sense

On Thu, Jun 26, 2014 at 10:21 PM, Reynold Xin [email protected]
wrote:

It's going to be useless if we stop using them in logs. I think right now
they might still be useful since they can be used to correlate with log
messages.

—
Reply to this email directly or view it on GitHub
#1236 (comment).

xiliu82 pushed a commit to xiliu82/spark that referenced this pull request Sep 4, 2014
@rxin @conviva-zz
[SPARK-2297][UI] Make task attempt and speculation more explicit in UI.
e748f26 
New UI:

![screen shot 2014-06-26 at 1 43 52 pm](https://cloud.githubusercontent.com/assets/323388/3404643/82b9ddc6-fd73-11e3-96f9-f7592a7aee79.png)

Author: Reynold Xin <[email protected]>

Closes apache#1236 from rxin/ui-task-attempt and squashes the following commits:

3b645dd [Reynold Xin] Expose attemptId in Stage.
c0474b1 [Reynold Xin] Beefed up unit test.
c404bdd [Reynold Xin] Fix ReplayListenerSuite.
f56be4b [Reynold Xin] Fixed JsonProtocolSuite.
e29e0f7 [Reynold Xin] Minor update.
5e4354a [Reynold Xin] [SPARK-2297][UI] Make task attempt and speculation more explicit in UI.
@schlosna schlosna mentioned this pull request Feb 8, 2018
Closed
wangyum pushed a commit that referenced this pull request May 26, 2023
@xingchaozh
[CARMEL-6553] Backport [SPARK-35673][SQL] Fix user-defined hint and u…
5f91f87 
…nrecognized hint in subquery (#1236)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@rxin @AmplabJenkins @kayousterhout