[SPARK-33069][INFRA] Skip test result report if no JUnit XML files are found #29946

HyukjinKwon · 2020-10-05T11:05:00Z

What changes were proposed in this pull request?

This PR proposes to skip test reporting ("Report test results") if there are no JUnit XML files are found.

Currently, we're running and skipping the tests dynamically. For example,

if there are only changes in SparkR at the underlying commit, it only runs the SparkR tests, and skip the other tests and generate JUnit XML files for SparkR test cases.
if there are only changes in docs at the underlying commit, the build skips all tests except linters and do not generate any JUnit XML files.

When test reporting ("Report test results") job is triggered after the main build ("Build and test
") is finished, and there are no JUnit XML files found, it reports the case as a failure. See https://github.com/apache/spark/runs/1196184007 as an example.

This PR works around it by simply skipping the testing report when there are no JUnit XML files are found.
Please see #29906 (comment) for more details.

Why are the changes needed?

To avoid false alarm for test results.

Does this PR introduce any user-facing change?

No, dev-only.

How was this patch tested?

Manually tested in my fork.

Positive case:

https://github.com/HyukjinKwon/spark/runs/1208624679?check_suite_focus=true
https://github.com/HyukjinKwon/spark/actions/runs/288996327

Negative case:

https://github.com/HyukjinKwon/spark/runs/1208229838?check_suite_focus=true
https://github.com/HyukjinKwon/spark/actions/runs/289000058

HyukjinKwon · 2020-10-05T11:05:09Z

cc @tgravescs and @dongjoon-hyun

HyukjinKwon · 2020-10-05T11:07:47Z

.github/workflows/test_report.yml

+    - name: Check if JUnit report XML files exist
+      run: |
+        if ls **/target/test-reports/*.xml > /dev/null 2>&1; then
+          echo '::set-output name=FILE_EXISTS::true'


BTW, the way is what GitHub Actions community suggested.

SparkQA · 2020-10-05T11:49:57Z

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/34019/

SparkQA · 2020-10-05T12:10:14Z

Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/34019/

SparkQA · 2020-10-05T13:36:07Z

Test build #129412 has finished for PR 29946 at commit 9b3950b.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

srowen

Seems OK if it makes this more robust

tgravescs

looks ok to me, I can't think of any cases this would break.

dongjoon-hyun

Thank you for taking care of this, @HyukjinKwon . Ya. I also saw this failure at doc-only PRs, too.

HyukjinKwon · 2020-10-06T00:09:32Z

Thanks guys. Merged to master, branch-3.0 and branch-2.4.

…e found ### What changes were proposed in this pull request? This PR proposes to skip test reporting ("Report test results") if there are no JUnit XML files are found. Currently, we're running and skipping the tests dynamically. For example, - if there are only changes in SparkR at the underlying commit, it only runs the SparkR tests, and skip the other tests and generate JUnit XML files for SparkR test cases. - if there are only changes in `docs` at the underlying commit, the build skips all tests except linters and do not generate any JUnit XML files. When test reporting ("Report test results") job is triggered after the main build ("Build and test ") is finished, and there are no JUnit XML files found, it reports the case as a failure. See https://github.com/apache/spark/runs/1196184007 as an example. This PR works around it by simply skipping the testing report when there are no JUnit XML files are found. Please see #29906 (comment) for more details. ### Why are the changes needed? To avoid false alarm for test results. ### Does this PR introduce _any_ user-facing change? No, dev-only. ### How was this patch tested? Manually tested in my fork. Positive case: https://github.com/HyukjinKwon/spark/runs/1208624679?check_suite_focus=true https://github.com/HyukjinKwon/spark/actions/runs/288996327 Negative case: https://github.com/HyukjinKwon/spark/runs/1208229838?check_suite_focus=true https://github.com/HyukjinKwon/spark/actions/runs/289000058 Closes #29946 from HyukjinKwon/test-junit-files. Authored-by: HyukjinKwon <[email protected]> Signed-off-by: HyukjinKwon <[email protected]> (cherry picked from commit a0aa8f3) Signed-off-by: HyukjinKwon <[email protected]>

HyukjinKwon · 2020-10-19T08:12:18Z

After dawidd6/action-download-artifact#32, it aborts the test reporting when there are no JUnit XML files are found. This change is not needed anymore, and I am going to revert this. See also https://github.com/apache/spark/runs/1273838759?check_suite_focus=true

…e found ### What changes were proposed in this pull request? This PR proposes to skip test reporting ("Report test results") if there are no JUnit XML files are found. Currently, we're running and skipping the tests dynamically. For example, - if there are only changes in SparkR at the underlying commit, it only runs the SparkR tests, and skip the other tests and generate JUnit XML files for SparkR test cases. - if there are only changes in `docs` at the underlying commit, the build skips all tests except linters and do not generate any JUnit XML files. When test reporting ("Report test results") job is triggered after the main build ("Build and test ") is finished, and there are no JUnit XML files found, it reports the case as a failure. See https://github.com/apache/spark/runs/1196184007 as an example. This PR works around it by simply skipping the testing report when there are no JUnit XML files are found. Please see apache#29906 (comment) for more details. ### Why are the changes needed? To avoid false alarm for test results. ### Does this PR introduce _any_ user-facing change? No, dev-only. ### How was this patch tested? Manually tested in my fork. Positive case: https://github.com/HyukjinKwon/spark/runs/1208624679?check_suite_focus=true https://github.com/HyukjinKwon/spark/actions/runs/288996327 Negative case: https://github.com/HyukjinKwon/spark/runs/1208229838?check_suite_focus=true https://github.com/HyukjinKwon/spark/actions/runs/289000058 Closes apache#29946 from HyukjinKwon/test-junit-files. Authored-by: HyukjinKwon <[email protected]> Signed-off-by: HyukjinKwon <[email protected]> (cherry picked from commit a0aa8f3) Signed-off-by: HyukjinKwon <[email protected]>

Don't run the test report in case of no JUnit XML files

9b3950b

HyukjinKwon commented Oct 5, 2020

View reviewed changes

srowen reviewed Oct 5, 2020

View reviewed changes

tgravescs approved these changes Oct 5, 2020

View reviewed changes

dongjoon-hyun approved these changes Oct 5, 2020

View reviewed changes

HyukjinKwon closed this in a0aa8f3 Oct 6, 2020

HyukjinKwon mentioned this pull request Oct 6, 2020

Make no-JUnit test file case as successful case with a warning ScaCap/action-surefire-report#29

Closed

HyukjinKwon mentioned this pull request Oct 19, 2020

[SPARK-33180][INFRA] Enables 'fail_if_no_tests' when reporting test results in GitHub Actions #30091

Closed

HyukjinKwon deleted the test-junit-files branch December 7, 2020 02:07

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[SPARK-33069][INFRA] Skip test result report if no JUnit XML files are found #29946

[SPARK-33069][INFRA] Skip test result report if no JUnit XML files are found #29946

Uh oh!

HyukjinKwon commented Oct 5, 2020 •

edited

Loading

Uh oh!

HyukjinKwon commented Oct 5, 2020

Uh oh!

HyukjinKwon Oct 5, 2020 •

edited

Loading

Uh oh!

SparkQA commented Oct 5, 2020

Uh oh!

SparkQA commented Oct 5, 2020

Uh oh!

SparkQA commented Oct 5, 2020

Uh oh!

srowen left a comment

Uh oh!

tgravescs left a comment

Uh oh!

dongjoon-hyun left a comment

Uh oh!

HyukjinKwon commented Oct 6, 2020 •

edited

Loading

Uh oh!

HyukjinKwon commented Oct 19, 2020 •

edited

Loading

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

[SPARK-33069][INFRA] Skip test result report if no JUnit XML files are found #29946

[SPARK-33069][INFRA] Skip test result report if no JUnit XML files are found #29946

Uh oh!

Conversation

HyukjinKwon commented Oct 5, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What changes were proposed in this pull request?

Why are the changes needed?

Does this PR introduce any user-facing change?

How was this patch tested?

Uh oh!

HyukjinKwon commented Oct 5, 2020

Uh oh!

HyukjinKwon Oct 5, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

SparkQA commented Oct 5, 2020

Uh oh!

SparkQA commented Oct 5, 2020

Uh oh!

SparkQA commented Oct 5, 2020

Uh oh!

srowen left a comment

Choose a reason for hiding this comment

Uh oh!

tgravescs left a comment

Choose a reason for hiding this comment

Uh oh!

dongjoon-hyun left a comment

Choose a reason for hiding this comment

Uh oh!

HyukjinKwon commented Oct 6, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

HyukjinKwon commented Oct 19, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

HyukjinKwon commented Oct 5, 2020 •

edited

Loading

HyukjinKwon Oct 5, 2020 •

edited

Loading

HyukjinKwon commented Oct 6, 2020 •

edited

Loading

HyukjinKwon commented Oct 19, 2020 •

edited

Loading