Skip to content

Commit ddf4a50

Browse files
viiryaHyukjinKwon
authored andcommitted
[SPARK-28031][PYSPARK][TEST] Improve doctest on over function of Column
## What changes were proposed in this pull request? Just found the doctest on `over` function of `Column` is commented out. The window spec is also not for the window function used there. We should either remove the doctest, or improve it. Because other functions of `Column` have doctest generally, so this PR tries to improve it. ## How was this patch tested? Added doctest. Closes #24854 from viirya/column-test-minor. Authored-by: Liang-Chi Hsieh <[email protected]> Signed-off-by: HyukjinKwon <[email protected]>
1 parent 4f4829b commit ddf4a50

File tree

1 file changed

+10
-2
lines changed

1 file changed

+10
-2
lines changed

python/pyspark/sql/column.py

Lines changed: 10 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -668,9 +668,17 @@ def over(self, window):
668668
:return: a Column
669669
670670
>>> from pyspark.sql import Window
671-
>>> window = Window.partitionBy("name").orderBy("age").rowsBetween(-1, 1)
671+
>>> window = Window.partitionBy("name").orderBy("age") \
672+
.rowsBetween(Window.unboundedPreceding, Window.currentRow)
672673
>>> from pyspark.sql.functions import rank, min
673-
>>> # df.select(rank().over(window), min('age').over(window))
674+
>>> df.withColumn("rank", rank().over(window)) \
675+
.withColumn("min", min('age').over(window)).show()
676+
+---+-----+----+---+
677+
|age| name|rank|min|
678+
+---+-----+----+---+
679+
| 5| Bob| 1| 5|
680+
| 2|Alice| 1| 2|
681+
+---+-----+----+---+
674682
"""
675683
from pyspark.sql.window import WindowSpec
676684
if not isinstance(window, WindowSpec):

0 commit comments

Comments
 (0)