[SPARK-9241][SQL] Supporting multiple DISTINCT columns - follow-up (3)

[hvanhovell]

This PR is a 2nd follow-up for SPARK-9241. It contains the following improvements:

• Fix for a potential bug in distinct child expression and attribute alignment.
• Improved handling of duplicate distinct child expressions.
• Added test for distinct UDAF with multiple children.

cc @yhuai

[SparkQA]

Test build #45361 has finished for PR 9566 at commit ea4ea69.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[yhuai]

#9556 is in. Let's rebase this one. Also, can you explain more about the potential bug? So, it will be easier for a reader to get the context.

[hvanhovell]

I'll rebase.

The potential bug is caused by the fact that the attributes for the distinct columns and the expressions for the distinct columns can possibly be misaligned. This currently relies on the fact that a sequence maintains the same order as a (hash) map derived from it. This currently works, but is not guaranteed by the scala collections library. I'd rather fix this, than to wait for something to go wrong.

[hvanhovell]

By calling toMap I am potentially breaking the alignment between distinctAggChildren and distinctAggChildAttrs.

[yhuai]

LGTM

[SparkQA]

Test build #45558 has finished for PR 9566 at commit e61849b.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[yhuai]

Thanks! Merging to master and branch 1.6.