[SPARK-8641][SPARK-12455][SQL] Native Spark Window functions - Follow-up (docs & tests)

[hvanhovell]

This PR is a follow-up for PR #9819. It adds documentation for the window functions and a couple of NULL tests.

The documentation was largely based on the documentation in (the source of) Hive and Presto:

• https://prestodb.io/docs/current/functions/window.html
• https://cwiki.apache.org/confluence/display/Hive/LanguageManual+WindowingAndAnalytics

I am not sure if we need to add the licenses of these two projects to the licenses directory. They are both under the ASL. @srowen any thoughts?

cc @yhuai

[hvanhovell]

ok to test

[SparkQA]

Test build #48080 has finished for PR 10402 at commit 2b806bb.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[yhuai]

@hvanhovell I just created https://issues.apache.org/jira/browse/SPARK-12455 for adding doc. Can you also put SPARK-12455 in the jira title?

[hvanhovell]

Done.

[gatorsmile]

I guess you also need to put extended like the example Upper given by @yhuai ?

[hvanhovell]

@gatorsmile extended is typically used to give an example of how the functions is used. See: https://github.com/apache/spark/blob/master/sql/catalyst/src/main/java/org/apache/spark/sql/catalyst/expressions/ExpressionDescription.java#L32-L33

I thought of adding this, but it is quite hard to create clear and concise examples for these functions. I am open to suggestions...

[gatorsmile]

+1

[yhuai]

Add some comments to explain the implementation of NTile?

[yhuai]

@hvanhovell Regarding the doc, we can acknowledge them (Hive and Presto) in the scala doc.

[SparkQA]

Test build #48294 has finished for PR 10402 at commit cf58954.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[SparkQA]

Test build #48362 has finished for PR 10402 at commit 767305a.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[SparkQA]

Test build #48363 has finished for PR 10402 at commit 13f9c95.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[yhuai]

Just a clarification question. For our current code, default will never be null, right?

[hvanhovell]

We currently always pass a literal(null) if no default is specified. However a user can pass a null value if (s)he wants to; there is no check to prevent this.

[hvanhovell]

BTW: This is not a problem, all code should be able to deal with a null default expression

[yhuai]

LGTM. Merging to master.