Skip to content

Commit a381bce

Browse files
maropuHyukjinKwon
authored andcommitted
[SPARK-24673][SQL][PYTHON][FOLLOWUP] Support Column arguments in timezone of from_utc_timestamp/to_utc_timestamp
## What changes were proposed in this pull request? This pr supported column arguments in timezone of `from_utc_timestamp/to_utc_timestamp` (follow-up of #21693). ## How was this patch tested? Added tests. Author: Takeshi Yamamuro <[email protected]> Closes #21723 from maropu/SPARK-24673-FOLLOWUP.
1 parent 141953f commit a381bce

File tree

1 file changed

+23
-3
lines changed

1 file changed

+23
-3
lines changed

python/pyspark/sql/functions.py

Lines changed: 23 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1285,11 +1285,21 @@ def from_utc_timestamp(timestamp, tz):
12851285
that time as a timestamp in the given time zone. For example, 'GMT+1' would yield
12861286
'2017-07-14 03:40:00.0'.
12871287
1288-
>>> df = spark.createDataFrame([('1997-02-28 10:30:00',)], ['t'])
1289-
>>> df.select(from_utc_timestamp(df.t, "PST").alias('local_time')).collect()
1288+
:param timestamp: the column that contains timestamps
1289+
:param tz: a string that has the ID of timezone, e.g. "GMT", "America/Los_Angeles", etc
1290+
1291+
.. versionchanged:: 2.4
1292+
`tz` can take a :class:`Column` containing timezone ID strings.
1293+
1294+
>>> df = spark.createDataFrame([('1997-02-28 10:30:00', 'JST')], ['ts', 'tz'])
1295+
>>> df.select(from_utc_timestamp(df.ts, "PST").alias('local_time')).collect()
12901296
[Row(local_time=datetime.datetime(1997, 2, 28, 2, 30))]
1297+
>>> df.select(from_utc_timestamp(df.ts, df.tz).alias('local_time')).collect()
1298+
[Row(local_time=datetime.datetime(1997, 2, 28, 19, 30))]
12911299
"""
12921300
sc = SparkContext._active_spark_context
1301+
if isinstance(tz, Column):
1302+
tz = _to_java_column(tz)
12931303
return Column(sc._jvm.functions.from_utc_timestamp(_to_java_column(timestamp), tz))
12941304

12951305

@@ -1300,11 +1310,21 @@ def to_utc_timestamp(timestamp, tz):
13001310
zone, and renders that time as a timestamp in UTC. For example, 'GMT+1' would yield
13011311
'2017-07-14 01:40:00.0'.
13021312
1303-
>>> df = spark.createDataFrame([('1997-02-28 10:30:00',)], ['ts'])
1313+
:param timestamp: the column that contains timestamps
1314+
:param tz: a string that has the ID of timezone, e.g. "GMT", "America/Los_Angeles", etc
1315+
1316+
.. versionchanged:: 2.4
1317+
`tz` can take a :class:`Column` containing timezone ID strings.
1318+
1319+
>>> df = spark.createDataFrame([('1997-02-28 10:30:00', 'JST')], ['ts', 'tz'])
13041320
>>> df.select(to_utc_timestamp(df.ts, "PST").alias('utc_time')).collect()
13051321
[Row(utc_time=datetime.datetime(1997, 2, 28, 18, 30))]
1322+
>>> df.select(to_utc_timestamp(df.ts, df.tz).alias('utc_time')).collect()
1323+
[Row(utc_time=datetime.datetime(1997, 2, 28, 1, 30))]
13061324
"""
13071325
sc = SparkContext._active_spark_context
1326+
if isinstance(tz, Column):
1327+
tz = _to_java_column(tz)
13081328
return Column(sc._jvm.functions.to_utc_timestamp(_to_java_column(timestamp), tz))
13091329

13101330

0 commit comments

Comments
 (0)