Skip to content

Commit 10b7244

Browse files
rdbluecloud-fan
authored andcommitted
[SPARK-17424] Fix unsound substitution bug in ScalaReflection.
## What changes were proposed in this pull request? This method gets a type's primary constructor and fills in type parameters with concrete types. For example, `MapPartitions[T, U] -> MapPartitions[Int, String]`. This Substitution fails when the actual type args are empty because they are still unknown. Instead, when there are no resolved types to subsitute, this returns the original args with unresolved type parameters. ## How was this patch tested? This doesn't affect substitutions where the type args are determined. This fixes our case where the actual type args are empty and our job runs successfully. Author: Ryan Blue <[email protected]> Closes #15062 from rdblue/SPARK-17424-fix-unsound-reflect-substitution. (cherry picked from commit b236933) Signed-off-by: Wenchen Fan <[email protected]>
1 parent 21764f8 commit 10b7244

File tree

1 file changed

+10
-2
lines changed

1 file changed

+10
-2
lines changed

sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/ScalaReflection.scala

Lines changed: 10 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -836,8 +836,16 @@ trait ScalaReflection {
836836
def getConstructorParameters(tpe: Type): Seq[(String, Type)] = {
837837
val formalTypeArgs = tpe.typeSymbol.asClass.typeParams
838838
val TypeRef(_, _, actualTypeArgs) = tpe
839-
constructParams(tpe).map { p =>
840-
p.name.toString -> p.typeSignature.substituteTypes(formalTypeArgs, actualTypeArgs)
839+
val params = constructParams(tpe)
840+
// if there are type variables to fill in, do the substitution (SomeClass[T] -> SomeClass[Int])
841+
if (actualTypeArgs.nonEmpty) {
842+
params.map { p =>
843+
p.name.toString -> p.typeSignature.substituteTypes(formalTypeArgs, actualTypeArgs)
844+
}
845+
} else {
846+
params.map { p =>
847+
p.name.toString -> p.typeSignature
848+
}
841849
}
842850
}
843851

0 commit comments

Comments
 (0)