[SPARK-8572] [SQL] Type coercion for ScalaUDFs

[piaozhexiu]

Implemented type coercion for udf arguments in Scala. The changes include-

• Add with ExpectsInputTypes to ScalaUDF class.
• Pass down argument types info from UDFRegistration and functions.

With this patch, the example query in SPARK-8572 no longer throws a type cast error at runtime.

Also added a unit test to UDFSuite in which a decimal type is passed to a udf that expects an int.

[SparkQA]

Test build #36457 has finished for PR 7203 at commit 066deed.

• This patch fails Spark unit tests.
• This patch merges cleanly.
• This patch adds the following public classes (experimental):
  • case class ScalaUDF(
  • case class UserDefinedFunction protected[sql] (

[rxin]

can you update the script earlier to include this new line?

[rxin]

it's just a few lines above this

[piaozhexiu]

@rxin Thank you for reviewing! I updated the script.

[rxin]

I think the convention is usually

a :: b :: c :: Nil

rather than

Nil :+ a :+ ...

Do you mind updating it?

[piaozhexiu]

Fixed.

[rxin]

Looks good. Just two nitpicks.

[rxin]

LGTM.

[SparkQA]

Test build #36479 has finished for PR 7203 at commit dce1efd.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds the following public classes (experimental):
  • case class ScalaUDF(
  • case class UserDefinedFunction protected[sql] (

[SparkQA]

Test build #36486 has finished for PR 7203 at commit 2d0ed15.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds the following public classes (experimental):
  • case class ScalaUDF(
  • case class UserDefinedFunction protected[sql] (

[cloud-fan]

lgtm

[rxin]

Thanks. I've merged this.