Commit 6c628e7
[MINOR][SQL] Update the description of spark.sql.files.ignoreCorruptFiles and spark.sql.columnNameOfCorruptRecord
### What changes were proposed in this pull request?
1. The description of `spark.sql.files.ignoreCorruptFiles` is not accurate. When the file does not exist, we will issue the error message.
```
org.apache.spark.sql.AnalysisException: Path does not exist: file:/nonexist/path;
```
2. `spark.sql.columnNameOfCorruptRecord` also affects the CSV format. The current description only mentions JSON format.
### How was this patch tested?
N/A
Author: Xiao Li <[email protected]>
Closes #18184 from gatorsmile/updateMessage.
(cherry picked from commit 2a780ac)
Signed-off-by: Wenchen Fan <[email protected]>1 parent 0c42279 commit 6c628e7
File tree
1 file changed
+3
-3
lines changed- sql/catalyst/src/main/scala/org/apache/spark/sql/internal
1 file changed
+3
-3
lines changedLines changed: 3 additions & 3 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
345 | 345 | | |
346 | 346 | | |
347 | 347 | | |
348 | | - | |
| 348 | + | |
| 349 | + | |
349 | 350 | | |
350 | 351 | | |
351 | 352 | | |
| |||
535 | 536 | | |
536 | 537 | | |
537 | 538 | | |
538 | | - | |
539 | | - | |
| 539 | + | |
540 | 540 | | |
541 | 541 | | |
542 | 542 | | |
| |||
0 commit comments