[SPARK-6602][Core] Update MapOutputTrackerMasterActor to MapOutputTrackerMasterEndpoint

[zsxwing]

This is the second PR for [SPARK-6602]. It updated MapOutputTrackerMasterActor and its unit tests.

cc @rxin

[AmplabJenkins]

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/29730/
Test PASSed.

[rxin]

Does Akka always serialize exceptions? There are exceptions that cannot be serialized, and we should be careful there.

Can you add a unit test calling sendFailure with an unserializable exception to make sure things still work?

[rxin]

This also brings another question: what's the semantics in the case of exceptions? Are they ignored, logged and ignored, or propagated back to the caller?

[zsxwing]

There are exceptions that cannot be serialized, and we should be careful there.

Because Throwable implements java.io.Serializable, I assume all exceptions can be serialized. Do you mean that some exception may throw an exception in writeObject(java.io.ObjectOutputStream out) to prevent it from being serialized?

[rxin]

In an ideal world all exceptions should be serializable, but I've seen in the past an exception that was not serializable (because it contained some field that was not serializable).

[zsxwing]

Such exception will be swallowed by Akka by default. I added ErrorMonitor to handle errors caught by Akka. Although they can not be serialized, at least we can log them.

[rxin]

One more thing - can you update the RpcEnv doc to specify what happens in the case of uncaught exceptions?

[zsxwing]

Done.

[SparkQA]

Test build #29736 has started for PR 5371 at commit 93c6c20.

[rxin]

LGTM.

[SparkQA]

Test build #29737 has started for PR 5371 at commit 4013a22.

[SparkQA]

Test build #29737 has finished for PR 5371 at commit 4013a22.

• This patch fails Scala style tests.
• This patch merges cleanly.
• This patch adds no public classes.
• This patch does not change any dependencies.

[AmplabJenkins]

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/29737/
Test FAILed.

[SparkQA]

Test build #29738 has started for PR 5371 at commit fcf3816.

[SparkQA]

Test build #29736 has finished for PR 5371 at commit 93c6c20.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.
• This patch does not change any dependencies.

[AmplabJenkins]

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/29736/
Test PASSed.

[SparkQA]

Test build #29738 has finished for PR 5371 at commit fcf3816.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.
• This patch does not change any dependencies.

[AmplabJenkins]

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/29738/
Test PASSed.

[rxin]

Merging in master. Thanks.