Skip to content

Commit c75b689

Browse files
ueshinHyukjinKwon
authored andcommitted
[SPARK-41233][CONNECT][PYTHON] Add array_prepend to Spark Connect Python client
### What changes were proposed in this pull request? This is a follow-up of #38947. Add `array_prepend` function to Spark Connect Python client. ### Why are the changes needed? `array_prepend` was added at #38947 without Spark Connect Python client. ### Does this PR introduce _any_ user-facing change? `array_prepend` will be available in Spark Connect Python client. ### How was this patch tested? Enabled the related test. Closes #40514 from ueshin/issues/SPARK-41233/array_prepend. Authored-by: Takuya UESHIN <[email protected]> Signed-off-by: Hyukjin Kwon <[email protected]>
1 parent 7ad1c80 commit c75b689

File tree

2 files changed

+21
-0
lines changed

2 files changed

+21
-0
lines changed

python/pyspark/sql/connect/functions.py

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1248,6 +1248,13 @@ def array_position(col: "ColumnOrName", value: Any) -> Column:
12481248
array_position.__doc__ = pysparkfuncs.array_position.__doc__
12491249

12501250

1251+
def array_prepend(col: "ColumnOrName", value: Any) -> Column:
1252+
return _invoke_function("array_prepend", _to_col(col), lit(value))
1253+
1254+
1255+
array_prepend.__doc__ = pysparkfuncs.array_prepend.__doc__
1256+
1257+
12511258
def array_remove(col: "ColumnOrName", element: Any) -> Column:
12521259
return _invoke_function("array_remove", _to_col(col), lit(element))
12531260

python/pyspark/sql/tests/connect/test_connect_function.py

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1088,6 +1088,20 @@ def test_collection_functions(self):
10881088
sdf.select(SF.array_append(sdf.a, sdf.f)).toPandas(),
10891089
)
10901090

1091+
# test array_prepend
1092+
self.assert_eq(
1093+
cdf.select(CF.array_prepend(cdf.a, "xyz")).toPandas(),
1094+
sdf.select(SF.array_prepend(sdf.a, "xyz")).toPandas(),
1095+
)
1096+
self.assert_eq(
1097+
cdf.select(CF.array_prepend(cdf.a, CF.lit("ab"))).toPandas(),
1098+
sdf.select(SF.array_prepend(sdf.a, SF.lit("ab"))).toPandas(),
1099+
)
1100+
self.assert_eq(
1101+
cdf.select(CF.array_prepend(cdf.a, cdf.f)).toPandas(),
1102+
sdf.select(SF.array_prepend(sdf.a, sdf.f)).toPandas(),
1103+
)
1104+
10911105
# test array_insert
10921106
self.assert_eq(
10931107
cdf.select(CF.array_insert(cdf.a, -5, "ab")).toPandas(),

0 commit comments

Comments
 (0)