Skip to content

Commit 4a67541

Browse files
HyukjinKwonJoshRosen
authored andcommitted
[SPARK-13023][PROJECT INFRA][FOLLOWUP][BRANCH-1.6] Unable to check root module ending up failure of Python tests
## What changes were proposed in this pull request? This PR fixes incorrect checking for `root` module (meaning all tests). I realised that #13806 is being failed due to this one. The PR corrects two files in `sql` and `core`. Since it seems fixing `core` module triggers all tests by `root` value from `determine_modules_for_files`. So, `changed_modules` becomes as below: ``` ['root', 'sql'] ``` and `module.dependent_modules` becaomes as below: ``` ['pyspark-mllib', 'pyspark-ml', 'hive-thriftserver', 'sparkr', 'mllib', 'examples', 'pyspark-sql'] ``` Now, `modules_to_test` does not include `root` and this checking is skipped but then both `changed_modules` and `modules_to_test` are being merged after that. So, this includes `root` module to test. This ends up with failing with the message below (e.g. https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/60990/consoleFull): ``` Error: unrecognized module 'root'. Supported modules: pyspark-core, pyspark-sql, pyspark-streaming, pyspark-ml, pyspark-mllib ``` ## How was this patch tested? N/A Author: hyukjinkwon <[email protected]> Closes #13845 from HyukjinKwon/fix-build-1.6.
1 parent 22a496d commit 4a67541

File tree

1 file changed

+2
-1
lines changed

1 file changed

+2
-1
lines changed

dev/run-tests.py

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -113,10 +113,11 @@ def determine_modules_to_test(changed_modules):
113113
modules_to_test = set()
114114
for module in changed_modules:
115115
modules_to_test = modules_to_test.union(determine_modules_to_test(module.dependent_modules))
116+
modules_to_test = modules_to_test.union(set(changed_modules))
116117
# If we need to run all of the tests, then we should short-circuit and return 'root'
117118
if modules.root in modules_to_test:
118119
return [modules.root]
119-
return modules_to_test.union(set(changed_modules))
120+
return modules_to_test
120121

121122

122123
def determine_tags_to_exclude(changed_modules):

0 commit comments

Comments
 (0)