[SPARK-24152][R][TESTS] Disable check-cran from run-tests.sh #26375

dongjoon-hyun · 2019-11-03T02:49:26Z

What changes were proposed in this pull request?

This PR aims to remove check-cran from run-tests.sh.
We had better add an independent Jenkins job to run check-cran.

Why are the changes needed?

CRAN instability has been a blocker for our daily dev process.
The following simple check causes consecutive failures in 4 of 9 Jenkins
jobs + PR builder.

* checking CRAN incoming feasibility ...Error in
.check_package_CRAN_incoming(pkgdir) :
  dims [product 24] do not match the length of object [0]

spark-branch-2.4-test-sbt-hadoop-2.6
spark-branch-2.4-test-sbt-hadoop-2.7
spark-master-test-sbt-hadoop-2.7
spark-master-test-sbt-hadoop-3.2
PRBuilder

Does this PR introduce any user-facing change?

No.

How was this patch tested?

Currently, PR builder is failing due to the above issue. This PR should pass the Jenkins.

dongjoon-hyun · 2019-11-03T02:53:08Z

@shivaram , @felixcheung , @HyukjinKwon , @viirya , @srowen .
I made a PR to recover our PRBuilder and 4 Jenkins jobs. How do you think about this?
In fact, CRAN failure occurs more frequently than PG&E power shutdown in this year. 😄

SparkQA · 2019-11-03T03:19:44Z

Test build #113149 has finished for PR 26375 at commit 0f2ce08.

This patch fails SparkR unit tests.
This patch merges cleanly.
This patch adds no public classes.

shivaram · 2019-11-03T03:45:42Z

Do we know what the reason for the failure is? Also what is the work around in terms of detecting CRAN problems in a timely fashion? Can we say run this just for PRs that touch R code?

viirya · 2019-11-03T03:49:21Z

R/run-tests.sh


-NO_TESTS=1 NO_MANUAL=1 $FWDIR/check-cran.sh 2>&1 | tee -a $CRAN_CHECK_LOG_FILE
-FAILED=$((PIPESTATUS[0]||$FAILED))
+# SPARK-24152 We will add this test back as a separate Jenkins job


Do we plan to upgrade R version? I remember that a higher R version can avoid this kind of failure. cc @HyukjinKwon

I guess the higher R version is going to hit this issue again after one year, but that sounds a good solution for now. How do you think about that, @shivaram and @felixcheung ?

SparkQA · 2019-11-03T03:57:59Z

Test build #113150 has finished for PR 26375 at commit 235c3af.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

dongjoon-hyun · 2019-11-03T04:06:45Z

Thanks, @shivaram .

Do we know what the reason for the failure is?

It's always happening their side changes. We don't know why this fails suddenly. As you see on SPARK-24152. This happens frequently in this year time to time . And, everytime, @viirya or @felixcheung contacted CRAN maintainers .

Also what is the work around in terms of detecting CRAN problems in a timely fashion?

We can have separate jobs for linter and publishing snapshot. It's non-blocking jobs.

Can we say run this just for PRs that touch R code?

That also can reduce this kind of massive outage. But, it's still be a blocker for all R PRs. CRAN check is only required only a few time when we do the release.

dongjoon-hyun · 2019-11-03T04:10:43Z

I'm okay for both (1) invoking CRAN check for R PR only and (2) upgrading Jenkins R and raised the minimum R versions. However, let's unblock the PRBuilder first.

Both (1) and (2) also needs some time to develop and validate.

dongjoon-hyun · 2019-11-03T04:11:58Z

Inside the new PR for one of (1) and (2), we can reenable this. How do you think about that, @shivaram and @viirya ? Currently, this blocks the almost whole community for 21 hours since Nov 1st midnight (PST).

shivaram · 2019-11-03T04:22:56Z

I'm not sure running it just on releases is enough as that makes it hard to go back and figure out what change broke the CRAN test. So running it on R PRs seems necessary to me.

I'm fine with unblocking the build for now if you can take care of opening a new JIRA / PR to track (1) and (2) above

viirya · 2019-11-03T04:30:35Z

It sounds fine to me as this actually blocks all PR dev. And because of weekend, CRAN sysadmins might not response quickly.

dongjoon-hyun · 2019-11-03T04:31:07Z

Thank you all. Sure!

dongjoon-hyun · 2019-11-03T04:35:50Z

I created SPARK-29732 first.

https://issues.apache.org/jira/browse/SPARK-29732

For the minimum version of R, could you raise an email to dev? If there is the best person to set it, I believe it is you, @shivaram , in the community. :)

dongjoon-hyun

As a kind of HOTFIX, I'm merging this PR to unblock all the other PRs. Thank you for the swift reviews! Merged to master/2.4.

### What changes were proposed in this pull request? This PR aims to remove `check-cran` from `run-tests.sh`. We had better add an independent Jenkins job to run `check-cran`. ### Why are the changes needed? CRAN instability has been a blocker for our daily dev process. The following simple check causes consecutive failures in 4 of 9 Jenkins jobs + PR builder. ``` * checking CRAN incoming feasibility ...Error in .check_package_CRAN_incoming(pkgdir) : dims [product 24] do not match the length of object [0] ``` - spark-branch-2.4-test-sbt-hadoop-2.6 - spark-branch-2.4-test-sbt-hadoop-2.7 - spark-master-test-sbt-hadoop-2.7 - spark-master-test-sbt-hadoop-3.2 - PRBuilder ### Does this PR introduce any user-facing change? No. ### How was this patch tested? Currently, PR builder is failing due to the above issue. This PR should pass the Jenkins. Closes #26375 from dongjoon-hyun/SPARK-24152. Authored-by: Dongjoon Hyun <[email protected]> Signed-off-by: Dongjoon Hyun <[email protected]> (cherry picked from commit 91d9901) Signed-off-by: Dongjoon Hyun <[email protected]>

HyukjinKwon · 2019-11-03T09:22:41Z

Hm, once we drop R 3.4 (deprecated from 3.0), we can deal with this issue permanently. I think we dont have to disable this but upgrade R version in Jenkins to 3.4.

felixcheung · 2019-11-03T09:23:17Z

+1 on what Hyukjin said instead of disabling tests

…

________________________________ From: Hyukjin Kwon <[email protected]> Sent: Sunday, November 3, 2019 1:37:53 AM To: apache/spark <[email protected]> Cc: Felix Cheung <[email protected]>; Mention <[email protected]> Subject: Re: [apache/spark] [SPARK-24152][R][TESTS] Disable check-cran from run-tests.sh (#26375) @shaneknapp<https://github.com/shaneknapp>, can we upgrade R version from 3.1 to 3.4 in Jenkins (since we deprecated it at #23012<#23012>)? I think we should re-enable it as soon as possible. The specific CRAN failure will be permanently fixed — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub<#26375?email_source=notifications&email_token=ACENZ67WU3Y7EMG4TFNYNXTQR2EWDA5CNFSM4JIJORXKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEC5NO5I#issuecomment-549115765>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/ACENZ67RGB3MMTDFLGXV3C3QR2EWDANCNFSM4JIJORXA>.

HyukjinKwon · 2019-11-03T09:23:53Z

Also CRAN check is pretty important for R package summision. It also checks misusages like require or library which should be requireNamespace for instance. This is error-prone and I think PR builders should catch.

HyukjinKwon · 2019-11-03T09:30:30Z

For instance, see the CRAN check failures at R vectorization - #23760.

https://github.com/apache/spark/pull/23760/files#diff-508641a8bd6c6b59f3e77c80cdcfa6a9R1221

Unlike pip packaging test, CRAN includes a set of linter-ish rules related to packages. It also verifies documentation as well.

HyukjinKwon · 2019-11-03T09:35:44Z

@shaneknapp, can we upgrade R version from 3.1 to 3.4 in Jenkins (since we deprecated it at #23012)? I think we should re-enable it as soon as possible. The specific CRAN failure will be permanently fixed

srowen · 2019-11-03T12:53:15Z

I think it's OK to disable temporarily to fix the PR builders. If updating R fixes it, then it can come back. Is this step important for PR builders vs the release process? only the release process pushes to CRAN. Would we not find a lot of errors until release time if we did not?

HyukjinKwon · 2019-11-03T13:04:48Z

Yeah, it has a lint-like feature unlike pip packaging test. I myself faced a lot when I worked on R vectoriation with Arrow optimization.

HyukjinKwon · 2019-11-03T13:06:40Z

Maybe we can upgrade and bring it back quick instead of reverting .. ? I dont know how hard it is to upgrade R to 3.4 and how much it takes in Jenkins cc @shaneknapp

dongjoon-hyun · 2019-11-03T16:38:24Z

Of course, we also can revert this if everything goes green. I'm +1 for all efforts if that is able to protect our PR builder from CRAN instability.

viirya · 2019-11-03T16:48:11Z

Looks like it was fixed at CRAN side.

dongjoon-hyun · 2019-11-03T18:04:07Z

Thank you for checking. Then, if you want, you can make a PR for reverting this~ @viirya .

HyukjinKwon · 2019-11-04T04:30:54Z

Thanks all!

[SPARK-24152][R] Disable check-cran from run-tests.sh

0f2ce08

dongjoon-hyun added the SPARKR label Nov 3, 2019

dongjoon-hyun changed the title ~~[SPARK-24152][R] Disable check-cran from run-tests.sh~~ [SPARK-24152][R][TESTS] Disable check-cran from run-tests.sh Nov 3, 2019

dongjoon-hyun added the TESTS label Nov 3, 2019

dongjoon-hyun removed the TESTS label Nov 3, 2019

Add empty result file

235c3af

dongjoon-hyun added the TESTS label Nov 3, 2019

viirya reviewed Nov 3, 2019

View reviewed changes

dongjoon-hyun commented Nov 3, 2019

View reviewed changes

dongjoon-hyun closed this in 91d9901 Nov 3, 2019

dongjoon-hyun deleted the SPARK-24152 branch November 3, 2019 04:38

[SPARK-24152][R][TESTS] Disable check-cran from run-tests.sh #26375

[SPARK-24152][R][TESTS] Disable check-cran from run-tests.sh #26375

Uh oh!

Conversation

dongjoon-hyun commented Nov 3, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What changes were proposed in this pull request?

Why are the changes needed?

Does this PR introduce any user-facing change?

How was this patch tested?

Uh oh!

dongjoon-hyun commented Nov 3, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

SparkQA commented Nov 3, 2019

Uh oh!

shivaram commented Nov 3, 2019

Uh oh!

viirya Nov 3, 2019

Choose a reason for hiding this comment

Uh oh!

dongjoon-hyun Nov 3, 2019

Choose a reason for hiding this comment

Uh oh!

SparkQA commented Nov 3, 2019

Uh oh!

dongjoon-hyun commented Nov 3, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

dongjoon-hyun commented Nov 3, 2019

Uh oh!

dongjoon-hyun commented Nov 3, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

shivaram commented Nov 3, 2019

Uh oh!

viirya commented Nov 3, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

dongjoon-hyun commented Nov 3, 2019

Uh oh!

dongjoon-hyun commented Nov 3, 2019

Uh oh!

dongjoon-hyun left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

HyukjinKwon commented Nov 3, 2019

Uh oh!

felixcheung commented Nov 3, 2019 via email

Uh oh!

HyukjinKwon commented Nov 3, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

HyukjinKwon commented Nov 3, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

HyukjinKwon commented Nov 3, 2019

Uh oh!

srowen commented Nov 3, 2019

Uh oh!

HyukjinKwon commented Nov 3, 2019

Uh oh!

HyukjinKwon commented Nov 3, 2019

Uh oh!

dongjoon-hyun commented Nov 3, 2019

Uh oh!

viirya commented Nov 3, 2019

Uh oh!

dongjoon-hyun commented Nov 3, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

HyukjinKwon commented Nov 4, 2019

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

dongjoon-hyun commented Nov 3, 2019 •

edited

Loading

dongjoon-hyun commented Nov 3, 2019 •

edited

Loading

dongjoon-hyun commented Nov 3, 2019 •

edited

Loading

dongjoon-hyun commented Nov 3, 2019 •

edited

Loading

viirya commented Nov 3, 2019 •

edited

Loading

dongjoon-hyun left a comment •

edited

Loading

HyukjinKwon commented Nov 3, 2019 •

edited

Loading

HyukjinKwon commented Nov 3, 2019 •

edited

Loading

dongjoon-hyun commented Nov 3, 2019 •

edited

Loading