Skip to content

Commit 54cda0d

Browse files
Davies Liumarmbrus
authored andcommitted
[SPARK-10321] sizeInBytes in HadoopFsRelation
Having sizeInBytes in HadoopFsRelation to enable broadcast join. cc marmbrus Author: Davies Liu <[email protected]> Closes #8490 from davies/sizeInByte.
1 parent b3dd569 commit 54cda0d

File tree

1 file changed

+2
-0
lines changed

1 file changed

+2
-0
lines changed

sql/core/src/main/scala/org/apache/spark/sql/sources/interfaces.scala

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -518,6 +518,8 @@ abstract class HadoopFsRelation private[sql](maybePartitionSpec: Option[Partitio
518518

519519
override def inputFiles: Array[String] = cachedLeafStatuses().map(_.getPath.toString).toArray
520520

521+
override def sizeInBytes: Long = cachedLeafStatuses().map(_.getLen).sum
522+
521523
/**
522524
* Partition columns. Can be either defined by [[userDefinedPartitionColumns]] or automatically
523525
* discovered. Note that they should always be nullable.

0 commit comments

Comments
 (0)