[SPARK-29292][YARN][K8S][MESOS] Fix Scala 2.13 compilation for remaining modules

[srowen]

What changes were proposed in this pull request?

See again the related PRs like #28971
This completes fixing compilation for 2.13 for all but repl, which is a separate task.

Why are the changes needed?

Eventually, we need to support a Scala 2.13 build, perhaps in Spark 3.1.

Does this PR introduce any user-facing change?

No

How was this patch tested?

Existing tests. (2.13 was not tested; this is about getting it to compile without breaking 2.12)

[srowen]

Oh, is this a known issue?

++ /usr/bin/R CMD INSTALL --library=/home/jenkins/workspace/SparkPullRequestBuilder-K8s/R/lib /home/jenkins/workspace/SparkPullRequestBuilder-K8s/R/pkg/
ERROR: this R is version 3.4.4, package 'SparkR' requires R >= 3.5

[SparkQA]

Test build #126061 has finished for PR 29147 at commit e4223f6.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[dongjoon-hyun]

I didn't notice that. It seems that we update the Job to use R 3.5+.

++ /usr/bin/R CMD INSTALL --library=/home/jenkins/workspace/SparkPullRequestBuilder-K8s/R/lib /home/jenkins/workspace/SparkPullRequestBuilder-K8s/R/pkg/
ERROR: this R is version 3.4.4, package 'SparkR' requires R >= 3.5

[dongjoon-hyun]

+1, LGTM. Thank you, @srowen .
Merged to master branch for Apache Spark 3.1.0 on December 2020.