-
Notifications
You must be signed in to change notification settings - Fork 28.9k
[SPARK-14265] Get attempId of stage and transfer it to web #12059
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
|
Test build #54506 has finished for PR 12059 at commit
|
|
My guess is that the PySpark test failure was unrelated, jenkins retest this please. |
|
cc @ajbozarth who has been working on a number of Spark UI changes lately. |
|
Test build #56178 has finished for PR 12059 at commit
|
|
@ajbozarth help me check, thanks |
|
I'll take a look this afternoon |
|
I'll have to take a look tomorrow, I've been blocked by local env issues all afternoon |
|
I can't seem to recreate the problem you're fixing so my input is limited. Just looking over your code though I don't see any glaring issues. Sorry I can't be more help. |
|
Test build #62866 has finished for PR 12059 at commit
|
|
@KaiXinXiaoLei this pr has been stale for a long time, is this still an issue you wan to fix? If not could you close it? |
What changes were proposed in this pull request?
if a stage is failed and attempt to start again, DAG visualization does not render correctly for this stage. Because in spark-dag-viz.js, the link of stage is ("#stage-" + stageId + "-" + attemptId), and the value of attemptId is 0 . So I think the attemptId of stage sholike this: stageopening spark web, the attemptId of stage should be transfer to web, and make the link of ("#stage-" + stageId + "-" + attemptId) is correct.
How was this patch tested?
Please test as fellow. Run spark app, and kill executor to make stage reSubmitted. Then check DAG visualization in web.