Skip to content

[SPARK-7399][Spark Core]Fixed compilation error in scala 2.11 #5966

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
wants to merge 1 commit into from
Closed

[SPARK-7399][Spark Core]Fixed compilation error in scala 2.11 #5966

wants to merge 1 commit into from

Conversation

@tijoparacka
Copy link

@tijoparacka tijoparacka commented May 7, 2015

scala has deterministic naming-scheme for the generated methods which return default arguments . here one of the default argument of overloaded method has to be removed

@tijoparacka tijoparacka changed the title Fixed compilation error in scala 2.11 [SPARK-7399][Spark Core]Fixed compilation error in scala 2.11 May 7, 2015
@AmplabJenkins
Copy link

AmplabJenkins commented May 7, 2015

Can one of the admins verify this patch?

@srowen
Copy link
Member

srowen commented May 7, 2015

ok to test

@AmplabJenkins
Copy link

AmplabJenkins commented May 7, 2015

Merged build triggered.

@AmplabJenkins
Copy link

AmplabJenkins commented May 7, 2015

Merged build started.

@SparkQA
Copy link

SparkQA commented May 7, 2015

Test build #32093 has started for PR 5966 at commit c90bba8.

@skyluc
Copy link

skyluc commented May 7, 2015

Verified that the patch fixes the compilation problem.

A few comments:

  • The method withScope with 3 parameters seems to be only used in the tests. Is there any other reason to keep this method?
  • The fix is required following this fix in Scala 2.11.

@SparkQA
Copy link

SparkQA commented May 7, 2015

Test build #32093 has finished for PR 5966 at commit c90bba8.

  • This patch fails Spark unit tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@AmplabJenkins
Copy link

AmplabJenkins commented May 7, 2015

Merged build finished. Test FAILed.

@AmplabJenkins
Copy link

AmplabJenkins commented May 7, 2015

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/32093/
Test FAILed.

@pwendell
Copy link
Contributor

pwendell commented May 7, 2015

@andrewor14 FYI

@andrewor14
Copy link
Contributor

andrewor14 commented May 7, 2015

retest this please

@andrewor14
Copy link
Contributor

andrewor14 commented May 7, 2015

This is a surprise since the two methods have different number of non-default arguments. Thanks for fixing this @tijoparacka. We need to keep the second method for later.

@AmplabJenkins
Copy link

AmplabJenkins commented May 7, 2015

Merged build triggered.

@AmplabJenkins
Copy link

AmplabJenkins commented May 7, 2015

Merged build started.

@SparkQA
Copy link

SparkQA commented May 7, 2015

Test build #32119 has started for PR 5966 at commit c90bba8.

@pwendell
Copy link
Contributor

pwendell commented May 7, 2015

LGTM

@SparkQA
Copy link

SparkQA commented May 7, 2015

Test build #32119 has finished for PR 5966 at commit c90bba8.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@AmplabJenkins
Copy link

AmplabJenkins commented May 7, 2015

Merged build finished. Test PASSed.

@AmplabJenkins
Copy link

AmplabJenkins commented May 7, 2015

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/32119/
Test PASSed.

@andrewor14
Copy link
Contributor

andrewor14 commented May 7, 2015

Merging into master 1.4

asfgit pushed a commit that referenced this pull request May 7, 2015
[SPARK-7399] [SPARK CORE] Fixed compilation error in scala 2.11
d4e31bf 
scala has deterministic naming-scheme for the generated methods which return default arguments . here one of the default argument of overloaded method has to be removed

Author: Tijo Thomas <[email protected]>

Closes #5966 from tijoparacka/fix_compilation_error_in_scala2.11 and squashes the following commits:

c90bba8 [Tijo Thomas] Fixed compilation error in scala 2.11

(cherry picked from commit 0c33bf8)
Signed-off-by: Andrew Or <[email protected]>
andrewor14
andrewor14 reviewed May 7, 2015
View reviewed changes
core/src/main/scala/org/apache/spark/rdd/RDDOperationScope.scala
Copy link
Contributor

@andrewor14 andrewor14 May 7, 2015

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I reverted the whitespace change here

@asfgit asfgit closed this in 0c33bf8 May 7, 2015
@dragos
Copy link
Contributor

dragos commented May 13, 2015

@andrewor14 I don't see this commit in master (i.e., it still fails on 2.11). What is the policy w.r.t. to master vs. the 1.4 branch?

@srowen
Copy link
Member

srowen commented May 13, 2015

Ah, looks like this might have gotten inadvertently reverted when merging bd61f07 ?

@dragos
Copy link
Contributor

dragos commented May 13, 2015

I would say good catch!

@andrewor14
Copy link
Contributor

andrewor14 commented May 13, 2015

Ugh, my mistake. I will merge this now.

@andrewor14
Copy link
Contributor

andrewor14 commented May 13, 2015

Honestly I don't understand why scala 2.11 can't compile this when 2.10 can. The number of non-default arguments here is not at all ambiguous. Anyway I'm looking into this ATM.

@andrewor14 andrewor14 mentioned this pull request May 13, 2015
Closed
@andrewor14
Copy link
Contributor

andrewor14 commented May 13, 2015

By the way I suspect this is also an issue in branch-1.4 because that patch was also merged there.

@dragos
Copy link
Contributor

dragos commented May 13, 2015

Scala is too conservative, indeed it could compile this, but it decided to forbid overloads + default arguments altogether because of the way they are encoded (and potential conflicts).

Indeed, the same problem is present in brach-1.4

asfgit pushed a commit that referenced this pull request May 13, 2015
[SPARK-7399] Spark compilation error for scala 2.11
f88ac70 
Subsequent fix following #5966. I tried this out locally.

Author: Andrew Or <[email protected]>

Closes #6129 from andrewor14/211-compilation and squashes the following commits:

713868f [Andrew Or] Fix compilation issue for scala 2.11
asfgit pushed a commit that referenced this pull request May 13, 2015
[SPARK-7399] Spark compilation error for scala 2.11
e6b8cef 
Subsequent fix following #5966. I tried this out locally.

Author: Andrew Or <[email protected]>

Closes #6129 from andrewor14/211-compilation and squashes the following commits:

713868f [Andrew Or] Fix compilation issue for scala 2.11

(cherry picked from commit f88ac70)
Signed-off-by: Andrew Or <[email protected]>
jeanlyn pushed a commit to jeanlyn/spark that referenced this pull request May 28, 2015
@jeanlyn
[SPARK-7399] [SPARK CORE] Fixed compilation error in scala 2.11
5b3ec04 
scala has deterministic naming-scheme for the generated methods which return default arguments . here one of the default argument of overloaded method has to be removed

Author: Tijo Thomas <[email protected]>

Closes apache#5966 from tijoparacka/fix_compilation_error_in_scala2.11 and squashes the following commits:

c90bba8 [Tijo Thomas] Fixed compilation error in scala 2.11
jeanlyn pushed a commit to jeanlyn/spark that referenced this pull request May 28, 2015
@jeanlyn
[SPARK-7399] Spark compilation error for scala 2.11
69f268c 
Subsequent fix following apache#5966. I tried this out locally.

Author: Andrew Or <[email protected]>

Closes apache#6129 from andrewor14/211-compilation and squashes the following commits:

713868f [Andrew Or] Fix compilation issue for scala 2.11
jeanlyn pushed a commit to jeanlyn/spark that referenced this pull request Jun 12, 2015
@jeanlyn
[SPARK-7399] [SPARK CORE] Fixed compilation error in scala 2.11
d4dc451 
scala has deterministic naming-scheme for the generated methods which return default arguments . here one of the default argument of overloaded method has to be removed

Author: Tijo Thomas <[email protected]>

Closes apache#5966 from tijoparacka/fix_compilation_error_in_scala2.11 and squashes the following commits:

c90bba8 [Tijo Thomas] Fixed compilation error in scala 2.11
jeanlyn pushed a commit to jeanlyn/spark that referenced this pull request Jun 12, 2015
@jeanlyn
[SPARK-7399] Spark compilation error for scala 2.11
7f8672b 
Subsequent fix following apache#5966. I tried this out locally.

Author: Andrew Or <[email protected]>

Closes apache#6129 from andrewor14/211-compilation and squashes the following commits:

713868f [Andrew Or] Fix compilation issue for scala 2.11
nemccarthy pushed a commit to nemccarthy/spark that referenced this pull request Jun 19, 2015
@nemccarthy
[SPARK-7399] [SPARK CORE] Fixed compilation error in scala 2.11
74d01c6 
scala has deterministic naming-scheme for the generated methods which return default arguments . here one of the default argument of overloaded method has to be removed

Author: Tijo Thomas <[email protected]>

Closes apache#5966 from tijoparacka/fix_compilation_error_in_scala2.11 and squashes the following commits:

c90bba8 [Tijo Thomas] Fixed compilation error in scala 2.11
nemccarthy pushed a commit to nemccarthy/spark that referenced this pull request Jun 19, 2015
@nemccarthy
[SPARK-7399] Spark compilation error for scala 2.11
5a310db 
Subsequent fix following apache#5966. I tried this out locally.

Author: Andrew Or <[email protected]>

Closes apache#6129 from andrewor14/211-compilation and squashes the following commits:

713868f [Andrew Or] Fix compilation issue for scala 2.11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

8 participants

@tijoparacka @AmplabJenkins @srowen @SparkQA @skyluc @pwendell @andrewor14 @dragos