Skip to content

Commit f1c951f

Browse files
committed
Add a comment that the default behaviour of line separator is from Hadoop
1 parent d6e9160 commit f1c951f

File tree

1 file changed

+3
-0
lines changed

1 file changed

+3
-0
lines changed

sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/HadoopFileLinesReader.scala

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -35,6 +35,9 @@ import org.apache.hadoop.mapreduce.task.TaskAttemptContextImpl
3535
* @param lineSeparator A line separator that should be used for each line. If the value is `None`,
3636
* it covers `\r`, `\r\n` and `\n`.
3737
* @param conf Hadoop configuration
38+
*
39+
* @note The behavior when `lineSeparator` is `None` (covering `\r`, `\r\n` and `\n`) is defined
40+
* by [[LineRecordReader]], not within Spark.
3841
*/
3942
class HadoopFileLinesReader(
4043
file: PartitionedFile,

0 commit comments

Comments
 (0)