You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
ODP-1302 [SPARK-43225][BUILD][SQL] Remove jackson-core-asl and jackson-mapper-asl from pre-built distribution (#39)
- Remove `jackson-core-asl` from maven dependency.
- Change the scope of `jackson-mapper-asl` from compile to test.
- Replace all `Hive.get(conf)` with `Hive.getWithoutRegisterFns(conf)`.
To fix CVE issue: https://github.com/apache/spark/security/dependabot/50.
No.
manual test.
Closesapache#40893 from wangyum/SPARK-43225.
Lead-authored-by: Yuming Wang <[email protected]>
Co-authored-by: Yuming Wang <[email protected]>
Signed-off-by: Sean Owen <[email protected]>
(cherry picked from commit 9c237d7)
[SPARK-43868][SQL][TESTS] Remove `originalUDFs` from `TestHive` to ensure `ObjectHashAggregateExecBenchmark` can run successfully on Github Action
This pr remove `originalUDFs` from `TestHive` to ensure `ObjectHashAggregateExecBenchmark` can run successfully on Github Action.
After SPARK-43225, `org.codehaus.jackson:jackson-mapper-asl` becomes a test scope dependency, so when using GA to run benchmark, it is not in the classpath because GA uses
https://github.com/apache/spark/blob/d61c77cac17029ee27319e6b766b48d314a4dd31/.github/workflows/benchmark.yml#L179-L183
iunstead of the sbt `Test/runMain`.
`ObjectHashAggregateExecBenchmark` used `TestHive`, and `TestHive` will always call `org.apache.hadoop.hive.ql.exec.FunctionRegistry#getFunctionNames` to init `originalUDFs` before this pr, so when we run `ObjectHashAggregateExecBenchmark` on GitHub Actions, there will be the following exceptions:
(cherry picked from commit 1c10e28caa54f32aa44a5406046894b8fcb335a4)
# Conflicts:
# pom.xml
Co-authored-by: Yuming Wang <[email protected]>
0 commit comments