[MINOR] Follow up for SPARK-23936, fixed description of "map_concat" function

[asiunov]

What changes were proposed in this pull request?

The description of "map_concat" function does not correspond to the actual behavior. Unit tests, an example in shell, and the PR description (https://jira.apache.org/jira/browse/SPARK-23936, #21073) say that duplicated keys will be duplicated, but the function description says the opposite.

scala> spark.sql("SELECT map_concat(map(1, 'a', 2, 'b'), map(2, 'c', 3, 'd'))").show(20, false)
+--------------------------------------------+
|map_concat(map(1, a, 2, b), map(2, c, 3, d))|
+--------------------------------------------+
|[1 -> a, 2 -> b, 2 -> c, 3 -> d]            |
+--------------------------------------------+

Looks like it was implemented correctly in the original PR, but then incorrectly changed in this PR: #22437

[AmplabJenkins]

Can one of the admins verify this patch?

[HyukjinKwon]

which spark version did you use? Duplicated entries were fixed and it shows:

spark-sql> SELECT map_concat(map(1, 'a', 2, 'b'), map(2, 'c', 3, 'd'));
{1:"a",2:"c",3:"d"}

and

scala> spark.sql("SELECT map_concat(map(1, 'a', 2, 'b'), map(2, 'c', 3, 'd'))").show(20, false)
+--------------------------------------------+
|map_concat(map(1, a, 2, b), map(2, c, 3, d))|
+--------------------------------------------+
|[1 -> a, 2 -> c, 3 -> d]                    |
+--------------------------------------------+

in the current master.

[asiunov]

You are right. Looks like I've tested on v2.4.0 because my SPARK_HOME targeted to the old repo, and the logic was changed towards to exclude duplicates in #23124 .
Now I'm checking on the latest master and have the same results as yours.
Thank you!