[SPARK-3479] [Build] Report failed test category #2606

nchammas · 2014-09-30T22:40:37Z

This PR allows SparkQA (i.e. Jenkins) to report in its posts to GitHub what category of test failed, if one can be determined.

The failure categories are:

general failure
RAT checks failed
Scala style checks failed
Python style checks failed
Build failed
Spark unit tests failed
PySpark unit tests failed
MiMa checks failed

This PR also fixes the diffing logic used to determine if a patch introduces new classes.

nchammas · 2014-09-30T22:41:12Z

I'm gonna post several test commits here to try out the various failure modes and confirm that they are reported correctly.

SparkQA · 2014-09-30T22:49:32Z

QA tests have started for PR 2606 at commit 935f8c0.

This patch merges cleanly.

SparkQA · 2014-09-30T22:49:41Z

QA tests have finished for PR 2606 at commit 935f8c0.

This patch fails some tests.
This patch merges cleanly.
This patch adds no public classes.

AmplabJenkins · 2014-09-30T22:49:42Z

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/21072/

AmplabJenkins · 2014-09-30T22:52:18Z

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/21070/

AmplabJenkins · 2014-10-01T02:02:19Z

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/21080/

nchammas · 2014-10-01T02:11:11Z

Jenkins failed to checkout the patch.

Jenkins, retest this please.

nchammas · 2014-10-01T02:37:51Z

Jenkins, retest this please.

AmplabJenkins · 2014-10-01T02:52:19Z

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/21081/

nchammas · 2014-10-01T02:54:52Z

Jenkins, retest this please.

SparkQA · 2014-10-01T02:59:53Z

QA tests have started for PR 2606 at commit d5624ac.

This patch merges cleanly.

SparkQA · 2014-10-01T03:01:02Z

QA tests have finished for PR 2606 at commit d5624ac.

This patch fails Python style tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):
- class PStatsParam(AccumulatorParam):

AmplabJenkins · 2014-10-01T03:01:03Z

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/21083/

AmplabJenkins · 2014-10-01T03:27:19Z

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/21084/

nchammas · 2014-10-01T03:28:27Z

Grr, Jenkins flakiness checking out stuff from git...

Oi Jenkins! Retest this please.

nchammas · 2014-10-01T03:35:40Z

Jenkins, retest this please.

SparkQA · 2014-10-01T03:39:35Z

QA tests have started for PR 2606 at commit 4f5f338.

This patch merges cleanly.

SparkQA · 2014-10-01T03:39:46Z

QA tests have finished for PR 2606 at commit 4f5f338.

This patch fails RAT tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):
- class PStatsParam(AccumulatorParam):

AmplabJenkins · 2014-10-01T03:39:47Z

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/21085/

SparkQA · 2014-10-01T03:54:32Z

QA tests have started for PR 2606 at commit 7289777.

This patch merges cleanly.

SparkQA · 2014-10-01T04:09:32Z

QA tests have started for PR 2606 at commit 833e064.

This patch merges cleanly.

SparkQA · 2014-10-01T04:45:58Z

QA tests have finished for PR 2606 at commit 7289777.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

AmplabJenkins · 2014-10-01T04:46:01Z

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/21086/

SparkQA · 2014-10-01T05:00:40Z

QA tests have finished for PR 2606 at commit 833e064.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

AmplabJenkins · 2014-10-01T05:00:44Z

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/21087/

SparkQA · 2014-10-01T14:48:58Z

QA tests have started for PR 2606 at commit 1fea6ce.

This patch merges cleanly.

SparkQA · 2014-10-01T14:55:10Z

QA tests have finished for PR 2606 at commit 1fea6ce.

This patch fails to build.
This patch merges cleanly.
This patch adds the following public classes (experimental):
- abstract class RDDD[T: ClassTag](

AmplabJenkins · 2014-10-01T14:55:11Z

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/21116/

SparkQA · 2014-10-01T15:29:40Z

QA tests have started for PR 2606 at commit 28ffbab.

This patch merges cleanly.

nchammas · 2014-10-01T18:41:32Z

sigh... more checkout troubles.

nchammas · 2014-10-01T18:41:40Z

Jenkins, retest this please.

SparkQA · 2014-10-01T18:44:39Z

QA tests have started for PR 2606 at commit f137249.

This patch merges cleanly.

SparkQA · 2014-10-01T19:35:18Z

QA tests have finished for PR 2606 at commit f137249.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):
- class Dummy()

AmplabJenkins · 2014-10-01T19:35:21Z

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/21143/

nchammas · 2014-10-01T19:36:57Z

Hmm, I expected a failure in the PySpark doctests, not here.

nchammas · 2014-10-01T19:37:03Z

Jenkins, retest this please.

AmplabJenkins · 2014-10-01T19:47:25Z

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/21147/

nchammas · 2014-10-01T19:47:38Z

Another checkout timeout.

nchammas · 2014-10-01T19:47:43Z

Jenkins, retest this please.

SparkQA · 2014-10-01T19:54:51Z

QA tests have started for PR 2606 at commit f137249.

This patch merges cleanly.

SparkQA · 2014-10-01T20:46:06Z

QA tests have finished for PR 2606 at commit f137249.

This patch fails PySpark unit tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):
- class Dummy()

AmplabJenkins · 2014-10-01T20:46:09Z

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/21152/

SparkQA · 2014-10-01T20:55:03Z

QA tests have started for PR 2606 at commit d67df03.

This patch merges cleanly.

SparkQA · 2014-10-01T22:08:00Z

QA tests have finished for PR 2606 at commit d67df03.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

AmplabJenkins · 2014-10-01T22:08:03Z

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/21154/

nchammas · 2014-10-02T01:02:01Z

cc @pwendell This PR is ready for review.

Here are examples of the messages posted when:

nchammas · 2014-10-02T01:03:45Z

dev/run-tests-jenkins

We can do a valid diff regardless of the merge-ability of the patch, so I moved this out of the if block.

JoshRosen · 2014-10-02T20:20:42Z

This looks great! Thanks for adding the extra comments to explain the $sha and $ghprbActualCommit variables, since they can be kind of confusing.

nchammas · 2014-10-06T21:01:29Z

@pwendell @JoshRosen Is this PR ready to be merged in?

JoshRosen · 2014-10-06T21:18:45Z

Sure, this looks fine to me so I'm going to merge it now. Thanks!

nchammas force-pushed the report-failed-test-category branch from 0f6dcd3 to cc243c9 Compare September 30, 2014 22:41

report what test category failed

d67df03

nchammas force-pushed the report-failed-test-category branch from f137249 to d67df03 Compare October 1, 2014 20:48

nchammas changed the title ~~[SPARK-3479] [Build] [WIP] Report failed test category~~ [SPARK-3479] [Build] Report failed test category Oct 2, 2014

nchammas reviewed Oct 2, 2014
View reviewed changes

nchammas mentioned this pull request Oct 2, 2014

[SPARK-3398] [EC2] Have spark-ec2 intelligently wait for specific cluster states #2339

Closed

asfgit closed this in 69c3f44 Oct 6, 2014

JoshRosen mentioned this pull request Oct 6, 2014

Show which tests failed databricks/spark-pr-dashboard#24

Closed

nchammas deleted the report-failed-test-category branch October 8, 2014 01:05

[SPARK-3479] [Build] Report failed test category #2606

[SPARK-3479] [Build] Report failed test category #2606

Uh oh!

Conversation

nchammas commented Sep 30, 2014

Uh oh!

nchammas commented Sep 30, 2014

Uh oh!

SparkQA commented Sep 30, 2014

Uh oh!

SparkQA commented Sep 30, 2014

Uh oh!

AmplabJenkins commented Sep 30, 2014

Uh oh!

AmplabJenkins commented Sep 30, 2014

Uh oh!

AmplabJenkins commented Oct 1, 2014

Uh oh!

nchammas commented Oct 1, 2014

Uh oh!

nchammas commented Oct 1, 2014

Uh oh!

AmplabJenkins commented Oct 1, 2014

Uh oh!

nchammas commented Oct 1, 2014

Uh oh!

SparkQA commented Oct 1, 2014

Uh oh!

SparkQA commented Oct 1, 2014

Uh oh!

AmplabJenkins commented Oct 1, 2014

Uh oh!

AmplabJenkins commented Oct 1, 2014

Uh oh!

nchammas commented Oct 1, 2014

Uh oh!

nchammas commented Oct 1, 2014

Uh oh!

SparkQA commented Oct 1, 2014

Uh oh!

SparkQA commented Oct 1, 2014

Uh oh!

AmplabJenkins commented Oct 1, 2014

Uh oh!

SparkQA commented Oct 1, 2014

Uh oh!

SparkQA commented Oct 1, 2014

Uh oh!

SparkQA commented Oct 1, 2014

Uh oh!

AmplabJenkins commented Oct 1, 2014

Uh oh!

SparkQA commented Oct 1, 2014

Uh oh!

AmplabJenkins commented Oct 1, 2014

Uh oh!

SparkQA commented Oct 1, 2014

Uh oh!

SparkQA commented Oct 1, 2014

Uh oh!

AmplabJenkins commented Oct 1, 2014

Uh oh!

SparkQA commented Oct 1, 2014

Uh oh!

nchammas commented Oct 1, 2014

Uh oh!

nchammas commented Oct 1, 2014

Uh oh!

SparkQA commented Oct 1, 2014

Uh oh!

SparkQA commented Oct 1, 2014

Uh oh!

AmplabJenkins commented Oct 1, 2014

Uh oh!

nchammas commented Oct 1, 2014

Uh oh!

nchammas commented Oct 1, 2014

Uh oh!

AmplabJenkins commented Oct 1, 2014

Uh oh!