Skip to content

Commit cc7f1eb

Browse files
yaooqinncloud-fan
authored andcommitted
[SPARK-29774][SQL][FOLLOWUP] Add a migration guide for date_add and date_sub
### What changes were proposed in this pull request? add a migration guide for date_add and date_sub to indicates their behavior change. It a followup for #26412 ### Why are the changes needed? add a migration guide ### Does this PR introduce any user-facing change? yes, doc change ### How was this patch tested? no Closes #26932 from yaooqinn/SPARK-29774-f. Authored-by: Kent Yao <[email protected]> Signed-off-by: Wenchen Fan <[email protected]>
1 parent b2baaa2 commit cc7f1eb

File tree

1 file changed

+2
-0
lines changed

1 file changed

+2
-0
lines changed

docs/sql-migration-guide.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -256,6 +256,8 @@ license: |
256256

257257
- Since Spark 3.0, day-time interval strings are converted to intervals with respect to the `from` and `to` bounds. If an input string does not match to the pattern defined by specified bounds, the `ParseException` exception is thrown. For example, `interval '2 10:20' hour to minute` raises the exception because the expected format is `[+|-]h[h]:[m]m`. In Spark version 2.4, the `from` bound was not taken into account, and the `to` bound was used to truncate the resulted interval. For instance, the day-time interval string from the showed example is converted to `interval 10 hours 20 minutes`. To restore the behavior before Spark 3.0, you can set `spark.sql.legacy.fromDayTimeString.enabled` to `true`.
258258

259+
- Since Spark 3.0, the `date_add` and `date_sub` functions only accepts int, smallint, tinyint as the 2nd argument, fractional and string types are not valid anymore, e.g. `date_add(cast('1964-05-23' as date), '12.34')` will cause `AnalysisException`. In Spark version 2.4 and earlier, if the 2nd argument is fractional or string value, it will be coerced to int value, and the result will be a date value of `1964-06-04`.
260+
259261
## Upgrading from Spark SQL 2.4 to 2.4.1
260262

261263
- The value of `spark.executor.heartbeatInterval`, when specified without units like "30" rather than "30s", was

0 commit comments

Comments
 (0)