[SPARK-8241][SQL] string function: concat_ws.

[rxin]

I also changed the semantics of concat w.r.t. null back to the same behavior as Hive.
That is to say, concat now returns null if any input is null.

[SparkQA]

Test build #37749 has finished for PR 7504 at commit 3c344a6.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds the following public classes (experimental):
  • case class UnresolvedAttribute(nameParts: Seq[String]) extends Attribute with Unevaluable
  • case class UnresolvedFunction(name: String, children: Seq[Expression])
  • case class UnresolvedStar(table: Option[String]) extends Star with Unevaluable
  • case class ResolvedStar(expressions: Seq[NamedExpression]) extends Star with Unevaluable
  • case class UnresolvedAlias(child: Expression)
  • case class Cast(child: Expression, dataType: DataType)
  • trait Unevaluable
  • case class SortOrder(child: Expression, direction: SortDirection)
  • trait AggregateExpression extends Expression with Unevaluable
  • case class Abs(child: Expression)
  • trait CodegenFallback
  • case class CreateArray(children: Seq[Expression]) extends Expression with CodegenFallback
  • case class CreateStruct(children: Seq[Expression]) extends Expression with CodegenFallback
  • case class CreateNamedStruct(children: Seq[Expression]) extends Expression with CodegenFallback
  • case class CurrentDate() extends LeafExpression with CodegenFallback
  • case class CurrentTimestamp() extends LeafExpression with CodegenFallback
  • case class Explode(child: Expression) extends UnaryExpression with Generator with CodegenFallback
  • case class Literal protected (value: Any, dataType: DataType)
  • case class Hex(child: Expression)
  • case class Unhex(child: Expression)
  • case class PrettyAttribute(name: String) extends Attribute with Unevaluable
  • case class In(value: Expression, list: Seq[Expression]) extends Predicate with CodegenFallback
  • case class NewSet(elementType: DataType) extends LeafExpression with CodegenFallback
  • case class AddItemToSet(item: Expression, set: Expression)
  • case class CombineSets(left: Expression, right: Expression)
  • case class CountSet(child: Expression) extends UnaryExpression with CodegenFallback
  • case class ConcatWs(children: Seq[Expression])
  • case class Upper(child: Expression)
  • case class StringFormat(children: Expression*) extends Expression with CodegenFallback
  • case class StringSpace(child: Expression)
  • case class Ascii(child: Expression)
  • case class Base64(child: Expression)
  • case class UnBase64(child: Expression)

[rxin]

Jenkins, retest this please.

[rxin]

cc @davies

[davies]

minor: Can we just ignore the null?

[rxin]

How? It won't match s. Also this thing doesn't compile if I do a wildcard match on s, e.g.

case s: _ => ...

[davies]

I mean we can return an empty Iterator

[rxin]

that won't work for the 1st value (separator), unless we special case handle that.

[davies]

That make sense, thanks!

[davies]

LGTM

[SparkQA]

Test build #37761 has finished for PR 7504 at commit dda1021.

• This patch fails Spark unit tests.
• This patch merges cleanly.
• This patch adds the following public classes (experimental):
  • case class ConcatWs(children: Seq[Expression])

[rxin]

cc @cloud-fan

This is changed to test sameType in order to support ArrayType(StringType)

[SparkQA]

Test build #37781 has finished for PR 7504 at commit 83fd950.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds the following public classes (experimental):
  • case class ConcatWs(children: Seq[Expression])