Skip to content

Commit 8ab1306

Browse files
wangyumHyukjinKwon
authored andcommitted
[SPARK-23807][FOLLOW-UP][BUILD][TEST-HADOOP3.1] Add test-hadoop3.1 phrase
## What changes were proposed in this pull request? Add `test-hadoop3.1` phrase to test Spark against Spark’s Hadoop 3.1 profile. ## How was this patch tested? Tested on jenkins. This is output: ``` [info] Using build tool sbt with Hadoop profile hadoop3.1 under environment amplab_jenkins ... [info] Building Spark (w/Hive 1.2.1) using SBT with these arguments: -Phadoop-3.1 -Pkubernetes -Phive-thriftserver -Pkinesis-asl -Pyarn -Pspark-ganglia-lgpl -Phive -Pmesos test:package streaming-kinesis-asl-assembly/assembly ``` https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/103282/console Closes #24045 from wangyum/SPARK-23807. Authored-by: Yuming Wang <[email protected]> Signed-off-by: Hyukjin Kwon <[email protected]>
1 parent 93ea353 commit 8ab1306

File tree

2 files changed

+3
-0
lines changed

2 files changed

+3
-0
lines changed

dev/run-tests-jenkins.py

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -173,6 +173,8 @@ def main():
173173
os.environ["AMPLAB_JENKINS_BUILD_PROFILE"] = "hadoop2.6"
174174
if "test-hadoop2.7" in ghprb_pull_title:
175175
os.environ["AMPLAB_JENKINS_BUILD_PROFILE"] = "hadoop2.7"
176+
if "test-hadoop3.1" in ghprb_pull_title:
177+
os.environ["AMPLAB_JENKINS_BUILD_PROFILE"] = "hadoop3.1"
176178

177179
build_display_name = os.environ["BUILD_DISPLAY_NAME"]
178180
build_url = os.environ["BUILD_URL"]

dev/run-tests.py

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -274,6 +274,7 @@ def get_hadoop_profiles(hadoop_version):
274274

275275
sbt_maven_hadoop_profiles = {
276276
"hadoop2.7": ["-Phadoop-2.7"],
277+
"hadoop3.1": ["-Phadoop-3.1"],
277278
}
278279

279280
if hadoop_version in sbt_maven_hadoop_profiles:

0 commit comments

Comments
 (0)