Skip to content

Commit 53ae391

Browse files
committed
Fix for SPARK-27259
For compressed files, the size of split can be set to -1. This was allowed till Spark 2.1 but regressed starting spark 2.2.x This commit removes the validation on length being at least 0.
1 parent 857f109 commit 53ae391

File tree

1 file changed

+0
-1
lines changed

1 file changed

+0
-1
lines changed

core/src/main/scala/org/apache/spark/rdd/InputFileBlockHolder.scala

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -76,7 +76,6 @@ private[spark] object InputFileBlockHolder {
7676
def set(filePath: String, startOffset: Long, length: Long): Unit = {
7777
require(filePath != null, "filePath cannot be null")
7878
require(startOffset >= 0, s"startOffset ($startOffset) cannot be negative")
79-
require(length >= 0, s"length ($length) cannot be negative")
8079
inputBlock.get().set(new FileBlock(UTF8String.fromString(filePath), startOffset, length))
8180
}
8281

0 commit comments

Comments
 (0)