Skip to content

Conversation

@scwf
Copy link
Contributor

@scwf scwf commented Aug 17, 2014

https://issues.apache.org/jira/browse/SPARK-3010

this pr is to fix redundant conditional in spark, such as
1.
private[spark] def codegenEnabled: Boolean =
if (getConf(CODEGEN_ENABLED, "false") == "true") true else false
2.
x => if (x == 2) true else false
...

@AmplabJenkins
Copy link

Can one of the admins verify this patch?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you really want to trim redundancy, the parens are now unnecessary -- as are the braces and the 'x'. So, you could pare this all the way down to:

val prunedRDD2 = PartitionPruningRDD.create(rdd, _ == 2)

@markhamstra
Copy link
Contributor

Something seems to be amiss in this PR: Several of your changes are neither in master already nor are they showing up in Files changed. Mismerged?

And some of your changes don't really go all the way toward cleaning up the logic. For example,

(aclsEnabled() && (user != null) && (!viewAcls.contains(user))) == false

would be better as

!aclsEnabled || user == null || viewAcls.contains(user)

@scwf
Copy link
Contributor Author

scwf commented Aug 17, 2014

@markhamstra, thanks for comments. I have merged SecurityManager right now and updated with your comments.

@mateiz
Copy link
Contributor

mateiz commented Aug 30, 2014

Jenkins, test this please

@mateiz
Copy link
Contributor

mateiz commented Aug 30, 2014

Looks good to me pending tests

@SparkQA
Copy link

SparkQA commented Aug 30, 2014

QA tests have started for PR 1992 at commit b2a044a.

  • This patch merges cleanly.

@SparkQA
Copy link

SparkQA commented Aug 31, 2014

QA tests have finished for PR 1992 at commit b2a044a.

  • This patch fails unit tests.
  • This patch merges cleanly.
  • This patch adds the following public classes (experimental):
    • class ExecutorClassLoader(conf: SparkConf, classUri: String, parent: ClassLoader,

@scwf
Copy link
Contributor Author

scwf commented Aug 31, 2014

@mateiz , retest this please, tests failed due to forked process exit code is not zero.
see #2108

@mateiz
Copy link
Contributor

mateiz commented Aug 31, 2014

Jenkins, test this please

@SparkQA
Copy link

SparkQA commented Aug 31, 2014

QA tests have started for PR 1992 at commit b2a044a.

  • This patch merges cleanly.

@SparkQA
Copy link

SparkQA commented Aug 31, 2014

QA tests have finished for PR 1992 at commit b2a044a.

  • This patch fails unit tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@scwf
Copy link
Contributor Author

scwf commented Aug 31, 2014

@mateiz retest this again, tests failed in sparkstreaming, thanks.

@pwendell
Copy link
Contributor

Jenkins, test this please.

@scwf
Copy link
Contributor Author

scwf commented Aug 31, 2014

hi @pwendell, jenkins fetch error.

@JoshRosen
Copy link
Contributor

Jenkins, retest this please.

@SparkQA
Copy link

SparkQA commented Aug 31, 2014

QA tests have started for PR 1992 at commit b2a044a.

  • This patch merges cleanly.

@SparkQA
Copy link

SparkQA commented Aug 31, 2014

QA tests have finished for PR 1992 at commit b2a044a.

  • This patch passes unit tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@mateiz
Copy link
Contributor

mateiz commented Aug 31, 2014

Merged this, thanks!

@asfgit asfgit closed this in 725715c Aug 31, 2014
xiliu82 pushed a commit to xiliu82/spark that referenced this pull request Sep 4, 2014
https://issues.apache.org/jira/browse/SPARK-3010

this pr is to fix redundant conditional in spark, such as
1.
private[spark] def codegenEnabled: Boolean =
if (getConf(CODEGEN_ENABLED, "false") == "true") true else false
2.
x => if (x == 2) true else false
...

Author: scwf <[email protected]>
Author: wangfei <[email protected]>

Closes apache#1992 from scwf/condition and squashes the following commits:

b2a044a [scwf] merge SecurityManager
e16239c [scwf] fix confilct
6811401 [scwf] fix merge confilct
0824df4 [scwf] Merge branch 'master' of https://github.com/apache/spark into patch-4
e274515 [scwf] fix redundant conditions
d032bf9 [wangfei] [SQL]Excess judgment
@scwf scwf deleted the condition branch September 24, 2014 07:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

7 participants