[SPARK-12989] [SQL] Delaying Alias Cleanup after ExtractWindowExpressions

[gatorsmile]

JIRA: https://issues.apache.org/jira/browse/SPARK-12989

In the rule ExtractWindowExpressions, we simply replace alias by the corresponding attribute. However, this will cause an issue exposed by the following case:

val data = Seq(("a", "b", "c", 3), ("c", "b", "a", 3)).toDF("A", "B", "C", "num")
  .withColumn("Data", struct("A", "B", "C"))
  .drop("A")
  .drop("B")
  .drop("C")

val winSpec = Window.partitionBy("Data.A", "Data.B").orderBy($"num".desc)
data.select($"*", max("num").over(winSpec) as "max").explain(true)

In this case, both Data.A and Data.B are alias in WindowSpecDefinition. If we replace these alias expression by their alias names, we are unable to know what they are since they will not be put in missingExpr too.

[SparkQA]

Test build #50246 has finished for PR 10963 at commit 468abca.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[marmbrus]

Just curious, why do we have to call toAttribute at all?

[gatorsmile]

The whole part is confusing. Let me rewrite this function to make it clear.

[marmbrus]

Well, wait a second. I would like to fix this in branch-1.6 so a minimal fix would be preferred (and we can refactor master later if there is a good reason).

[gatorsmile]

Sure, let me change it to ne without calling toAttribute and move the test case to DataFrameWindowSuite. Thanks!

[SparkQA]

Test build #50318 has finished for PR 10963 at commit 8025de0.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[SparkQA]

Test build #50481 has finished for PR 10963 at commit 476dcb4.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[marmbrus]

Thanks, merging to master and 1.6.