Skip to content

Commit 67d2888

Browse files
committed
[SPARK-49226][SQL] Clean-up UDF code generation
### What changes were proposed in this pull request? This PR reduces the size of code generated for UDFs. ### Why are the changes needed? Simplification and code reduction. ### Does this PR introduce _any_ user-facing change? No ### How was this patch tested? Existing tests. ### Was this patch authored or co-authored using generative AI tooling? No Closes #47746 from hvanhovell/SPARK-49226. Authored-by: Herman van Hovell <[email protected]> Signed-off-by: Herman van Hovell <[email protected]>
1 parent 3fcf041 commit 67d2888

File tree

6 files changed

+380
-622
lines changed

6 files changed

+380
-622
lines changed

connector/connect/client/jvm/src/test/scala/org/apache/spark/sql/connect/client/CheckConnectJvmClientCompatibility.scala

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -196,7 +196,6 @@ object CheckConnectJvmClientCompatibility {
196196
ProblemFilters.exclude[MissingClassProblem](
197197
"org.apache.spark.sql.ExtendedExplainGenerator"),
198198
ProblemFilters.exclude[MissingClassProblem]("org.apache.spark.sql.UDTFRegistration"),
199-
ProblemFilters.exclude[MissingClassProblem]("org.apache.spark.sql.UDFRegistration$"),
200199
ProblemFilters.exclude[MissingClassProblem]("org.apache.spark.sql.DataSourceRegistration"),
201200

202201
// DataFrame Reader & Writer
@@ -297,6 +296,10 @@ object CheckConnectJvmClientCompatibility {
297296
ProblemFilters.exclude[MissingClassProblem](
298297
"org.apache.spark.sql.artifact.util.ArtifactUtils$"),
299298

299+
// UDFRegistration
300+
ProblemFilters.exclude[DirectMissingMethodProblem](
301+
"org.apache.spark.sql.UDFRegistration.register"),
302+
300303
// Datasource V2 partition transforms
301304
ProblemFilters.exclude[MissingClassProblem]("org.apache.spark.sql.PartitionTransform"),
302305
ProblemFilters.exclude[MissingClassProblem]("org.apache.spark.sql.PartitionTransform$"),

0 commit comments

Comments
 (0)