Skip to content

Commit e5bff61

Browse files
committed
Spark SQL basicOperator add Intersect operator
Hi all, I want to submit a basic operator Intersect For example , in sql case select * from table1 intersect select * from table2 So ,i want use this operator support this function in Spark SQL This operator will return the the intersection of SparkPlan child table RDD .
1 parent 3249528 commit e5bff61

File tree

1 file changed

+13
-0
lines changed

1 file changed

+13
-0
lines changed

sql/core/src/main/scala/org/apache/spark/sql/execution/basicOperators.scala

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -204,3 +204,16 @@ case class ExistingRdd(output: Seq[Attribute], rdd: RDD[Row]) extends LeafNode {
204204
override def execute() = rdd
205205
}
206206

207+
208+
/**
209+
* :: DeveloperApi ::
210+
*/
211+
@DeveloperApi
212+
case class Intersect(children: Seq[SparkPlan])(@transient sc: SparkContext) extends SparkPlan {
213+
// TODO:
214+
override def output = children.head.output
215+
216+
override def execute() = {
217+
children.map(child => child.execute()).foldLeft(children(0).execute())((a, b) => a.intersection(b))
218+
219+
}

0 commit comments

Comments
 (0)