[SPARK-4949]shutdownCallback in SparkDeploySchedulerBackend should be enclosed by synchronized block.

[sarutak]

A variable shutdownCallback in SparkDeploySchedulerBackend can be accessed from multiple threads so it should be enclosed by synchronized block.

[SparkQA]

Test build #24759 has finished for PR 3781 at commit 5942765.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[srowen]

On the one hand, this sounds like it could be an AtomicBoolean. On another hand -- this whole mechanism could be replaced by something more robust in java.util.concurrent

[SparkQA]

Test build #24782 has finished for PR 3781 at commit 1b60fd1.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[SparkQA]

Test build #25235 has finished for PR 3781 at commit 3e3631d.

• This patch fails Spark unit tests.
• This patch merges cleanly.
• This patch adds no public classes.

[sarutak]

retest this please.

[SparkQA]

Test build #25244 has finished for PR 3781 at commit 3e3631d.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[srowen]

This looks good to me. Most of it is making some fields private that look like they should be, simplifying one synchronization primitive, and does fix a theoretical concurrency problem. Tests pass. I will merge soon if there are no objections.

[srowen]

@sarutak would this not be simpler as a @volatile reference? that's how appId is handled below.

[sarutak]

@srowen Thank you for picking up this PR. I've changed the shutdownCallback to be a simple volatile variable.

[SparkQA]

Test build #27667 has finished for PR 3781 at commit 3c1c018.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[SparkQA]

Test build #27668 has finished for PR 3781 at commit 42ca528.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds the following public classes (experimental):
  • class Partitioner(object):

[srowen]

OK, but why do you need this setter now?

[sarutak]

It's no longer needed so I remove it soon.

[SparkQA]

Test build #27678 has finished for PR 3781 at commit c146c93.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.