Skip to content

[SPARK-31830][SQL] Consistent error handling for datetime formatting and parsing functions #28650

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 13 commits into from
Closed

[SPARK-31830][SQL] Consistent error handling for datetime formatting and parsing functions #28650

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
13 commits
Select commit Hold shift + click to select a range
f39dd48
[SPARK-31830][SQL] Consistent error handling for datetime formatting …
yaooqinn May 27, 2020
9971e98
Merge branch 'master' into SPARK-31830
yaooqinn May 28, 2020
33fedf8
parsing
yaooqinn May 28, 2020
82bb963
regen test results
yaooqinn May 28, 2020
a080397
address comments of style stuff
yaooqinn May 28, 2020
3c68c4e
import
yaooqinn May 28, 2020
ce2eff0
mv validatePatternString function
yaooqinn May 28, 2020
0680855
improve test coverage
yaooqinn May 28, 2020
01e5ce3
Merge branch 'master' into SPARK-31830
yaooqinn Jun 9, 2020
2e940bd
tests
yaooqinn Jun 9, 2020
daac8dc
rm tests
yaooqinn Jun 9, 2020
9d578f1
address comments
yaooqinn Jun 9, 2020
8139dfc
fix ut
yaooqinn Jun 9, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions docs/sql-migration-guide.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,8 @@ license: |
- In Spark 3.1, grouping_id() returns long values. In Spark version 3.0 and earlier, this function returns int values. To restore the behavior before Spark 3.0, you can set `spark.sql.legacy.integerGroupingId` to `true`.

- In Spark 3.1, SQL UI data adopts the `formatted` mode for the query plan explain results. To restore the behavior before Spark 3.0, you can set `spark.sql.ui.explainMode` to `extended`.

- In Spark 3.1, `from_unixtime`, `unix_timestamp`,`to_unix_timestamp`, `to_timestamp` and `to_date` will fail if the specified datetime pattern is invalid. In Spark 3.0 or earlier, they result `NULL`.

## Upgrading from Spark SQL 2.4 to 3.0

Expand Down
Loading