Skip to content

Commit c73caeb

Browse files
senthhwangyum
andcommitted
ODP-1302 [SPARK-43225][BUILD][SQL] Remove jackson-core-asl and jackson-mapper-asl from pre-built distribution (#39)
- Remove `jackson-core-asl` from maven dependency. - Change the scope of `jackson-mapper-asl` from compile to test. - Replace all `Hive.get(conf)` with `Hive.getWithoutRegisterFns(conf)`. To fix CVE issue: https://github.com/apache/spark/security/dependabot/50. No. manual test. Closes apache#40893 from wangyum/SPARK-43225. Lead-authored-by: Yuming Wang <[email protected]> Co-authored-by: Yuming Wang <[email protected]> Signed-off-by: Sean Owen <[email protected]> (cherry picked from commit 9c237d7) [SPARK-43868][SQL][TESTS] Remove `originalUDFs` from `TestHive` to ensure `ObjectHashAggregateExecBenchmark` can run successfully on Github Action This pr remove `originalUDFs` from `TestHive` to ensure `ObjectHashAggregateExecBenchmark` can run successfully on Github Action. After SPARK-43225, `org.codehaus.jackson:jackson-mapper-asl` becomes a test scope dependency, so when using GA to run benchmark, it is not in the classpath because GA uses https://github.com/apache/spark/blob/d61c77cac17029ee27319e6b766b48d314a4dd31/.github/workflows/benchmark.yml#L179-L183 iunstead of the sbt `Test/runMain`. `ObjectHashAggregateExecBenchmark` used `TestHive`, and `TestHive` will always call `org.apache.hadoop.hive.ql.exec.FunctionRegistry#getFunctionNames` to init `originalUDFs` before this pr, so when we run `ObjectHashAggregateExecBenchmark` on GitHub Actions, there will be the following exceptions: (cherry picked from commit 1c10e28caa54f32aa44a5406046894b8fcb335a4) # Conflicts: # pom.xml Co-authored-by: Yuming Wang <[email protected]>
1 parent 474d26f commit c73caeb

File tree

1 file changed

+0
-1
lines changed

1 file changed

+0
-1
lines changed

dev/deps/spark-deps-hadoop-3-hive-2.3

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -98,7 +98,6 @@ httpcore/4.4.16//httpcore-4.4.16.jar
9898
istack-commons-runtime/3.0.8//istack-commons-runtime-3.0.8.jar
9999
ivy/2.5.1//ivy-2.5.1.jar
100100
jackson-annotations/2.14.2//jackson-annotations-2.14.2.jar
101-
jackson-core-asl/1.9.13//jackson-core-asl-1.9.13.jar
102101
jackson-core/2.14.2//jackson-core-2.14.2.jar
103102
jackson-databind/2.14.2//jackson-databind-2.14.2.jar
104103
jackson-dataformat-cbor/2.14.2//jackson-dataformat-cbor-2.14.2.jar

0 commit comments

Comments
 (0)