[SPARK-9710] [test] Fix RPackageUtilsSuite when R is not available.

[vanzin]

RUtils.isRInstalled throws an exception if R is not installed,
instead of returning false. Fix that.

[SparkQA]

Test build #40084 has finished for PR 8008 at commit df72d8c.

• This patch fails Spark unit tests.
• This patch merges cleanly.
• This patch adds no public classes.

[vanzin]

retest this please

[SparkQA]

Test build #40134 has finished for PR 8008 at commit df72d8c.

• This patch fails Spark unit tests.
• This patch merges cleanly.
• This patch adds no public classes.

[shivaram]

Jenkins, retest this please

[shivaram]

LGTM. Thanks @vanzin for the fix. cc @brkyvz

[brkyvz]

LGTM. Thanks @vanzin

[SparkQA]

Test build #40173 has finished for PR 8008 at commit df72d8c.

• This patch fails PySpark unit tests.
• This patch merges cleanly.
• This patch adds no public classes.

[shivaram]

Sigh -- our tests are too flaky.

Jenkins, retest this please

[shivaram]

Jenkins, retest this please

[SparkQA]

Test build #40204 has finished for PR 8008 at commit df72d8c.

• This patch fails Spark unit tests.
• This patch merges cleanly.
• This patch adds no public classes.

[shivaram]

Sigh, I guess today is not a good day to get tests passed.

[shivaram]

Jenkins, retest this please

[SparkQA]

Test build #40242 has finished for PR 8008 at commit df72d8c.

• This patch fails Spark unit tests.
• This patch merges cleanly.
• This patch adds no public classes.

[shivaram]

Jenkins, retest this please

[SparkQA]

Test build #40250 has finished for PR 8008 at commit df72d8c.

• This patch fails Spark unit tests.
• This patch merges cleanly.
• This patch adds no public classes.

[shivaram]

@JoshRosen @rxin This PR has failed Jenkins 6 (!) times with unrelated failures. Are there any known issues or flaky tests ? I'd like to get this fix into 1.5, but can't merge this right now

[rxin]

I started 4 different Jenkins run for this just now.

[SparkQA]

Test build #1415 has finished for PR 8008 at commit df72d8c.

• This patch fails Spark unit tests.
• This patch merges cleanly.
• This patch adds no public classes.

[shivaram]

I give up. I'm going to manually patch this on a clean branch on my laptop and run dev/run-tests and report back if it passes

[SparkQA]

Test build #1418 timed out for PR 8008 at commit df72d8c after a configured wait of 175m.

[shivaram]

Jenkins, retest this please

[shivaram]

@tdas @rxin I've see the test SPARK-6222: Do not clear received block data too soon fail a bunch of times (on Jenkins and on local runs) - Example stack trace at [1]. There is already an open JIRA saying this is a flaky test at https://issues.apache.org/jira/browse/SPARK-7420 -- Can we disable the test for now as its becoming really hard to merge PRs ?

[1] https://gist.github.com/shivaram/84a2b47ada5509c707fd

[rxin]

I disabled the test.

[SparkQA]

Test build #40272 has finished for PR 8008 at commit df72d8c.

• This patch fails Spark unit tests.
• This patch merges cleanly.
• This patch adds no public classes.

[shivaram]

Jenkins, retest this please

[SparkQA]

Test build #40278 has finished for PR 8008 at commit df72d8c.

• This patch fails Spark unit tests.
• This patch merges cleanly.
• This patch adds no public classes.

[shivaram]

Jenkins, retest this please

[AmplabJenkins]

Merged build finished. Test FAILed.

[vanzin]

So, we really need to start disabling these flaky tests and filing bugs, because waiting for jenkins to pass a build lately has been such a huge waste of time.

On the other hand, the actual tests that use this code have all passed in all builds, so can we just push this change?

[shivaram]

Yeah I was wondering the same thing. In fact I think the tests have managed to pass piece-wise (i.e. core sometimes, streaming sometimes, sql sometimes etc.) that if you aggregate all of them everything has passed at this point.

The function isRInstalled is only used in RPackageUtilsSuite and SparkSubmitSuite, both of which are in core and have passed tests multiple times. I'm going to merge this change because I'm tired of waiting for jenkins, but adding up build scores across builds might be an idea for the future (cc @JoshRosen ?)

[rxin]

Let's merge them.

On Monday, August 10, 2015, Shivaram Venkataraman [email protected]
wrote:

Yeah I was wondering the same thing. In fact I think the tests have
managed to pass piece-wise (i.e. core sometimes, streaming sometimes, sql
sometimes etc.) that if you aggregate all of them everything has passed at
this point.

The function isRInstalled is only used in RPackageUtilsSuite and
SparkSubmitSuite, both of which are in core and have passed tests
multiple times. I'm going to merge this change because I'm tired of waiting
for jenkins, but adding up build scores across builds might be an idea for
the future (cc @JoshRosen https://github.com/JoshRosen ?)

—
Reply to this email directly or view it on GitHub
#8008 (comment).

[robbinspg]

My 1.5 branch build is failing with as described in SPARK-9710 and I notice that this merge didn't make it into that branch. Any chance this will be backported?