-
Notifications
You must be signed in to change notification settings - Fork 28.9k
[SPARK-22133][DOCS] Documentation for Mesos Reject Offer Configurations #19555
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
|
@susanxhuynh could you pls review this? |
ArtRand
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I would prefer that the spark.mem.max changes be a part of #19510 only. Just some nits w.r.t wording. Thanks for doing this!
docs/running-on-mesos.md
Outdated
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Could you please add these changes only in #19510?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@ArtRand Sure, I will move the documentation to 19510
docs/running-on-mesos.md
Outdated
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
reserve is probably not the correct term to use here. I would use consume, as Spark does not actually make resource reservations http://mesos.apache.org/documentation/latest/reservation/
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Agree. I will update it later on
docs/running-on-mesos.md
Outdated
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
As above, please add this in the separate PR.
docs/running-on-mesos.md
Outdated
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This doesn't sound correct. The mesos.proto (https://github.com/apache/mesos/blob/master/include/mesos/mesos.proto#L2310) states:
Time to consider unused resources refused. Note that all unused
resources will be considered refused and use the default value
(below) regardless of whether Filters was passed to
SchedulerDriver::launchTasks. You MUST pass Filters with this
field set to change this behavior (i.e., get another offer which
includes unused resources sooner or later than the default).
some simple word-smithing or a link should make it clearer.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for the comment, I will update it.
|
LGTM |
|
Hey @srowen can we get a merge on this? |
|
Test build #3984 has finished for PR 19555 at commit
|
|
Merged to master |
What changes were proposed in this pull request?
Documentation about Mesos Reject Offer Configurations
Related PR
#19510 for
spark.mem.max