Skip to content

Commit bf5b6db

Browse files
MaxGekkHyukjinKwon
authored andcommitted
[MINOR][SQL][TESTS] Check file based V2 datasources on unsupported types
### What changes were proposed in this pull request? In the PR, I propose to modify `FileBasedDataSourceSuite` and check V2 datasources: JSON/CSV, ORC, Parquet on the unsupported data types: intervals and null. And remove TODOs. ### Why are the changes needed? To extend test coverage. ### Does this PR introduce _any_ user-facing change? No. ### How was this patch tested? By running the modified test suites: ``` $ build/sbt "test:testOnly *FileBasedDataSourceSuite" ``` ### Was this patch authored or co-authored using generative AI tooling? No. Closes #50356 from MaxGekk/time-off-fs-ds. Authored-by: Max Gekk <[email protected]> Signed-off-by: Hyukjin Kwon <[email protected]>
1 parent f137f6a commit bf5b6db

File tree

1 file changed

+2
-4
lines changed

1 file changed

+2
-4
lines changed

sql/core/src/test/scala/org/apache/spark/sql/FileBasedDataSourceSuite.scala

Lines changed: 2 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -503,8 +503,7 @@ class FileBasedDataSourceSuite extends QueryTest
503503
test("SPARK-24204 error handling for unsupported Interval data types - csv, json, parquet, orc") {
504504
withTempDir { dir =>
505505
val tempDir = new File(dir, "files").getCanonicalPath
506-
// TODO: test file source V2 after write path is fixed.
507-
Seq(true).foreach { useV1 =>
506+
Seq(true, false).foreach { useV1 =>
508507
val useV1List = if (useV1) {
509508
"csv,json,orc,parquet"
510509
} else {
@@ -572,8 +571,7 @@ class FileBasedDataSourceSuite extends QueryTest
572571
}
573572

574573
test("SPARK-24204 error handling for unsupported Null data types - csv, parquet, orc") {
575-
// TODO: test file source V2 after write path is fixed.
576-
Seq(true).foreach { useV1 =>
574+
Seq(true, false).foreach { useV1 =>
577575
val useV1List = if (useV1) {
578576
"csv,orc,parquet"
579577
} else {

0 commit comments

Comments
 (0)