[SPARK-8209[SQL]Add function conv

[zhichao-li]

cc @chenghao-intel @adrian-wang

[SparkQA]

Test build #35099 has finished for PR 6872 at commit 0e35c54.

• This patch fails Scala style tests.
• This patch merges cleanly.
• This patch adds the following public classes (experimental):
  • abstract class TernaryExpression extends Expression with trees.TernaryNode[Expression]
  • class ConvUtil
  • case class Conv(numExpr: Expression, fromBaseExpr: Expression, toBaseExpr: Expression)
  • trait TernaryNode[BaseType <: TreeNode[BaseType]]
  • // Add jar to isolated hive (metadataHive) class loader.

[adrian-wang]

I think you should rewrite this part in a more functional way.

[chenghao-intel]

And be private[sql] object

[chenghao-intel]

Sorry, I mean trait.

[SparkQA]

Test build #35101 has finished for PR 6872 at commit d4f3c4d.

• This patch fails Spark unit tests.
• This patch merges cleanly.
• This patch adds no public classes.

[SparkQA]

Test build #35215 has finished for PR 6872 at commit ac09a0b.

• This patch fails MiMa tests.
• This patch merges cleanly.
• This patch adds the following public classes (experimental):
  • abstract class TernaryExpression extends Expression with trees.TernaryNode[Expression]
  • class ConvUtil
  • case class Conv(numExpr: Expression, fromBaseExpr: Expression, toBaseExpr: Expression)
  • trait TernaryNode[BaseType <: TreeNode[BaseType]]

[zhichao-li]

retest this please

[SparkQA]

Test build #35216 has finished for PR 6872 at commit ac09a0b.

• This patch fails Spark unit tests.
• This patch merges cleanly.
• This patch adds the following public classes (experimental):
  • abstract class TernaryExpression extends Expression with trees.TernaryNode[Expression]
  • class ConvUtil
  • case class Conv(numExpr: Expression, fromBaseExpr: Expression, toBaseExpr: Expression)
  • trait TernaryNode[BaseType <: TreeNode[BaseType]]

[SparkQA]

Test build #35223 has finished for PR 6872 at commit 4111d06.

• This patch fails MiMa tests.
• This patch merges cleanly.
• This patch adds the following public classes (experimental):
  • abstract class TernaryExpression extends Expression with trees.TernaryNode[Expression]
  • class ConvUtil
  • case class Conv(numExpr: Expression, fromBaseExpr: Expression, toBaseExpr: Expression)
  • trait TernaryNode[BaseType <: TreeNode[BaseType]]

[SparkQA]

Test build #35227 has finished for PR 6872 at commit c682481.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds the following public classes (experimental):
  • abstract class TernaryExpression extends Expression with trees.TernaryNode[Expression]
  • class ConvUtil
  • case class Conv(numExpr: Expression, fromBaseExpr: Expression, toBaseExpr: Expression)
  • trait TernaryNode[BaseType <: TreeNode[BaseType]]

[chenghao-intel]

Ternary is a very rare case, and probably most of the code in this class are needed to be overrided for different TernaryExpressions, so how about move the logic into the Conv?

[SparkQA]

Test build #35644 has finished for PR 6872 at commit d5c6249.

• This patch fails MiMa tests.
• This patch merges cleanly.
• This patch adds the following public classes (experimental):
  • case class Conv(numExpr: Expression, fromBaseExpr: Expression, toBaseExpr: Expression)

[zhichao-li]

retest this please.

[SparkQA]

Test build #35648 has finished for PR 6872 at commit d5c6249.

• This patch fails Spark unit tests.
• This patch merges cleanly.
• This patch adds the following public classes (experimental):
  • class LinearDataGenerator(object):
  • case class Conv(numExpr: Expression, fromBaseExpr: Expression, toBaseExpr: Expression)

[zhichao-li]

retest this please.
it's due to time out onorg.apache.spark.sql.hive.thriftserver.CliSuite.Single command with --database

[zhichao-li]

cc @chenghao-intel @rxin

[chenghao-intel]

Probably more user cases of fromBase and the toBase will be represented as the real value, than the column names? What do you think?
@rxin we actually have some cases like this, do you think if we need to support both constant string values and column names for cases like this?

[rxin]

yes i agree. i think we can always use real value here

[SparkQA]

Test build #37563 has finished for PR 6872 at commit e2bace3.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds the following public classes (experimental):
  • case class Conv(numExpr: Expression, fromBaseExpr: Expression, toBaseExpr: Expression)

[rxin]

we need a lot more test cases here. please think hard about corner cases.

[rxin]

@zhichao-li this is really close. Let's add some test cases and then merge it.

[rxin]

can you add documentation for Conv itself?

[rxin]

LGTM

[SparkQA]

Test build #37607 has finished for PR 6872 at commit 6ef3b37.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds the following public classes (experimental):
  • case class Conv(numExpr: Expression, fromBaseExpr: Expression, toBaseExpr: Expression)

[rxin]

Thanks. I've merged this.