Skip to content

[SPARK-23133][K8S] Fix passing java options to Executor #20296

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
wants to merge 1 commit into from
Closed

[SPARK-23133][K8S] Fix passing java options to Executor #20296

wants to merge 1 commit into from

Conversation

@andrusha
Copy link
Contributor

@andrusha andrusha commented Jan 17, 2018

What changes were proposed in this pull request?

Pass through spark java options to the executor in context of docker image.

How was this patch tested?

Deployed two version of containers to local k8s, checked that java options were present in the updated image on the running executor.

@foxish
Copy link
Contributor

foxish commented Jan 18, 2018

cc/ @vanzin @felixcheung @liyinan926

@foxish
Copy link
Contributor

foxish commented Jan 18, 2018

ok to test

@liyinan926
Copy link
Contributor

liyinan926 commented Jan 18, 2018

Thanks for fixing this! LGTM.

@foxish
Copy link
Contributor

foxish commented Jan 18, 2018

LGTM, looks like we missed this when unifying the docker images. Would be good to get this into 2.3.0 as well.

@ssuchter
Copy link
Contributor

ssuchter commented Jan 18, 2018

ok to test

@vanzin
Copy link
Contributor

vanzin commented Jan 18, 2018

@ssuchter should I wait for your tests to run? Normal PRB won't touch this file, so I'm leaning towards just pushing this.

@jiangxb1987
Copy link
Contributor

jiangxb1987 commented Jan 18, 2018

Why is this PR against 2.3 but not master?

@foxish
Copy link
Contributor

foxish commented Jan 18, 2018

Good point. @andrusha, can you target it to master instead?

@foxish
Copy link
Contributor

foxish commented Jan 18, 2018 •
edited
Loading

That would explain also why the tests aren't running.
@sameeragarwal, @vanzin, can someone with manual merge powers retarget this to the master branch?

@jiangxb1987
Copy link
Contributor

jiangxb1987 commented Jan 18, 2018

Normally you should close this and open another PR against the master branch.

@foxish
Copy link
Contributor

foxish commented Jan 18, 2018

I think one of us should do it then - in the interest of time and making the next RC.
It looks like the PR author may be in a different timezone.

@vanzin
Copy link
Contributor

vanzin commented Jan 18, 2018

Yes, we can't change the target branch of a PR.

foxish added a commit to foxish/spark that referenced this pull request Jan 18, 2018
@foxish
[SPARK-23133][K8S] Fix passing java options to Executor
903179c 
Supersedes apache#20296
@foxish foxish mentioned this pull request Jan 18, 2018
Closed
@foxish
Copy link
Contributor

foxish commented Jan 18, 2018

Opened #20322.

asfgit pushed a commit that referenced this pull request Jan 18, 2018
@andrusha
[SPARK-23133][K8S] Fix passing java options to Executor
b8c6d93 
Pass through spark java options to the executor in context of docker image.
Closes #20296

andrusha: Deployed two version of containers to local k8s, checked that java options were present in the updated image on the running executor.
Manual test

Author: Andrew Korzhuev <[email protected]>

Closes #20322 from foxish/patch-1.

(cherry picked from commit f568e9c)
Signed-off-by: Marcelo Vanzin <[email protected]>
@asfgit asfgit closed this in f568e9c Jan 18, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

6 participants

@andrusha @foxish @liyinan926 @ssuchter @vanzin @jiangxb1987