Skip to content

Commit aff49a3

Browse files
srowenpwendell
authored andcommitted
SPARK-5172 [BUILD] spark-examples-***.jar shades a wrong Hadoop distribution
In addition to the `hadoop-2.x` profiles in the parent POM, there is actually another set of profiles in `examples` that has to be activated differently to get the right Hadoop 1 vs 2 flavor of HBase. This wasn't actually used in making Hadoop 2 distributions, hence the problem. To reduce complexity, I suggest merging them with the parent POM profiles, which is possible now. You'll see this changes appears to update the HBase version, but actually, the default 0.94 version was not being used. HBase is only used in examples, and the examples POM always chose one profile or the other that updated the version to 0.98.x anyway. Author: Sean Owen <[email protected]> Closes apache#3992 from srowen/SPARK-5172 and squashes the following commits: 17830d9 [Sean Owen] Control hbase hadoop1/2 flavor in the parent POM with existing hadoop-2.x profiles
1 parent a3978f3 commit aff49a3

File tree

2 files changed

+4
-24
lines changed

2 files changed

+4
-24
lines changed

examples/pom.xml

Lines changed: 0 additions & 23 deletions
Original file line numberDiff line numberDiff line change
@@ -392,29 +392,6 @@
392392
</dependency>
393393
</dependencies>
394394
</profile>
395-
<profile>
396-
<id>hbase-hadoop2</id>
397-
<activation>
398-
<property>
399-
<name>hbase.profile</name>
400-
<value>hadoop2</value>
401-
</property>
402-
</activation>
403-
<properties>
404-
<hbase.version>0.98.7-hadoop2</hbase.version>
405-
</properties>
406-
</profile>
407-
<profile>
408-
<id>hbase-hadoop1</id>
409-
<activation>
410-
<property>
411-
<name>!hbase.profile</name>
412-
</property>
413-
</activation>
414-
<properties>
415-
<hbase.version>0.98.7-hadoop1</hbase.version>
416-
</properties>
417-
</profile>
418395
<profile>
419396
<!-- We add a source directory specific to Scala 2.10 since Kafka
420397
only works with it -->

pom.xml

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -122,7 +122,7 @@
122122
<hadoop.version>1.0.4</hadoop.version>
123123
<protobuf.version>2.4.1</protobuf.version>
124124
<yarn.version>${hadoop.version}</yarn.version>
125-
<hbase.version>0.94.6</hbase.version>
125+
<hbase.version>0.98.7-hadoop1</hbase.version>
126126
<hbase.artifact>hbase</hbase.artifact>
127127
<flume.version>1.4.0</flume.version>
128128
<zookeeper.version>3.4.5</zookeeper.version>
@@ -1466,6 +1466,7 @@
14661466
<properties>
14671467
<hadoop.version>2.2.0</hadoop.version>
14681468
<protobuf.version>2.5.0</protobuf.version>
1469+
<hbase.version>0.98.7-hadoop2</hbase.version>
14691470
<avro.mapred.classifier>hadoop2</avro.mapred.classifier>
14701471
</properties>
14711472
</profile>
@@ -1476,6 +1477,7 @@
14761477
<hadoop.version>2.3.0</hadoop.version>
14771478
<protobuf.version>2.5.0</protobuf.version>
14781479
<jets3t.version>0.9.0</jets3t.version>
1480+
<hbase.version>0.98.7-hadoop2</hbase.version>
14791481
<commons.math3.version>3.1.1</commons.math3.version>
14801482
<avro.mapred.classifier>hadoop2</avro.mapred.classifier>
14811483
</properties>
@@ -1487,6 +1489,7 @@
14871489
<hadoop.version>2.4.0</hadoop.version>
14881490
<protobuf.version>2.5.0</protobuf.version>
14891491
<jets3t.version>0.9.0</jets3t.version>
1492+
<hbase.version>0.98.7-hadoop2</hbase.version>
14901493
<commons.math3.version>3.1.1</commons.math3.version>
14911494
<avro.mapred.classifier>hadoop2</avro.mapred.classifier>
14921495
</properties>

0 commit comments

Comments
 (0)