[SPARK-9871][SparkR] Add expression functions into SparkR which have a variable parameter

[yu-iskw]

Summary

• Add lit function
• Add concat, greatest, least functions

I think we need to improve collect function in order to implement struct function. Since collect doesn't work with arguments which includes a nested list variable. It seems that a list against struct still has jobj classes. So it would be better to solve this problem on another issue.

JIRA

[SPARK-9871] Add expression functions into SparkR which have a variable parameter - ASF JIRA

[SparkQA]

Test build #40858 has finished for PR 8194 at commit 2a5f084.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds the following public classes (experimental):
  • implicit class StringToColumn(val sc: StringContext)

[yu-iskw]

@shivaram could you review it?

[shivaram]

Any reason this is a S3 function unlike the others in this class ?

[yu-iskw]

Can we define a S4 function which has argument signitures which are like Any? As far as I know, setMethod requires signatures of argument objects.
The argument of lit in Scala is Any. Otherwise, we should define multiple S4 function as follows.

setMethod("lit", signature("numeric"), .....)
setMethod("lit", signature("character"), .....)
...

[yu-iskw]

I didn't know we can use ANY as a signature.

"There are two special classes that can be used in the signature: missing and ANY. missing matches the case where the argument is not supplied, and ANY is used for setting up default methods."
http://adv-r.had.co.nz/S4.html

[shivaram]

Thanks @yu-iskw. I left some comments inline. One more thing we need to do is to update our NAMESPACE file [1] with all the functions we have added so far (in this PR and in the previous one).

[1] https://github.com/apache/spark/blob/master/R/pkg/NAMESPACE

[SparkQA]

Test build #40938 has finished for PR 8194 at commit 50ed0e5.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[SparkQA]

Test build #40939 has finished for PR 8194 at commit dc98a13.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[SparkQA]

Test build #41002 has finished for PR 8194 at commit bf59a84.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[yu-iskw]

@shivaram could you review it?

• Define lit as a S4 function
• Add the functions into the NAMESPACE file
• Remove the unnecessary generic for struct

I have a question about column and col function in SparkR. Should we define them as S4 functions? They are currently defined as S3 functions.

[shivaram]

lit needs to be added to this list

[shivaram]

Thanks @yu-iskw - I had a minor comment about the NAMESPACE but otherwise this LGTM

[yu-iskw]

@shivaram thank you for your comment. I updated it!

[SparkQA]

Test build #41006 has finished for PR 8194 at commit 36407ef.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[shivaram]

LGTM Merging this

[yu-iskw]

Thank you for merging it!