Skip to content

[SPARK-14893][SQL] Re-enable HiveSparkSubmitSuite SPARK-8489 test after HiveContext is removed #12924

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 2 commits into from
+8 −7
Closed

[SPARK-14893][SQL] Re-enable HiveSparkSubmitSuite SPARK-8489 test after HiveContext is removed #12924

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
bc5af46
[SPARK-14893] Re-enable HiveSparkSubmitSuite SPARK-8489 test after Hi…
dilipbiswal May 5, 2016
f9f80e7
review comments from andrew
dilipbiswal May 5, 2016
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 7 additions & 5 deletions sql/hive/src/test/resources/regression-test-SPARK-8489/Main.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,6 @@
* limitations under the License.
*/

import org.apache.spark.SparkContext
import org.apache.spark.sql.SparkSession

/**
Expand All @@ -33,15 +32,18 @@ object Main {
def main(args: Array[String]) {
// scalastyle:off println
println("Running regression test for SPARK-8489.")
val sc = new SparkContext("local", "testing")
val sparkSession = SparkSession.withHiveSupport(sc)
val spark = SparkSession.builder
.master("local")
.appName("testing")
.enableHiveSupport()
.getOrCreate()
// This line should not throw scala.reflect.internal.MissingRequirementError.
// See SPARK-8470 for more detail.
val df = sparkSession.createDataFrame(Seq(MyCoolClass("1", "2", "3")))
val df = spark.createDataFrame(Seq(MyCoolClass("1", "2", "3")))
df.collect()
println("Regression test for SPARK-8489 success!")
// scalastyle:on println
sc.stop()
spark.stop()
}
}

Binary file modified sql/hive/src/test/resources/regression-test-SPARK-8489/test-2.10.jar
View file
Open in desktop
Binary file not shown.
Binary file modified sql/hive/src/test/resources/regression-test-SPARK-8489/test-2.11.jar
View file
Open in desktop
Binary file not shown.
3 changes: 1 addition & 2 deletions sql/hive/src/test/scala/org/apache/spark/sql/hive/HiveSparkSubmitSuite.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -142,8 +142,7 @@ class HiveSparkSubmitSuite
runSparkSubmit(args)
}

// TODO: re-enable this after rebuilding the jar (HiveContext was removed)
ignore("SPARK-8489: MissingRequirementError during reflection") {
test("SPARK-8489: MissingRequirementError during reflection") {
// This test uses a pre-built jar to test SPARK-8489. In a nutshell, this test creates
// a HiveContext and uses it to create a data frame from an RDD using reflection.
// Before the fix in SPARK-8470, this results in a MissingRequirementError because
Expand Down