Skip to content

Commit 53d6ab5

Browse files
JoshRosenrxin
authored andcommitted
[SPARK-7686] [SQL] DescribeCommand is assigned wrong output attributes in SparkStrategies
In `SparkStrategies`, `RunnableDescribeCommand` is called with the output attributes of the table being described rather than the attributes for the `describe` command's output. I discovered this issue because it caused type conversion errors in some UnsafeRow conversion code that I'm writing. Author: Josh Rosen <[email protected]> Closes #6217 from JoshRosen/SPARK-7686 and squashes the following commits: 953a344 [Josh Rosen] Fix SPARK-7686 with a simple change in SparkStrategies. a4eec9f [Josh Rosen] Add failing regression test for SPARK-7686 (cherry picked from commit 5645628) Signed-off-by: Reynold Xin <[email protected]>
1 parent 6df71eb commit 53d6ab5

File tree

2 files changed

+8
-2
lines changed

2 files changed

+8
-2
lines changed

sql/core/src/main/scala/org/apache/spark/sql/execution/SparkStrategies.scala

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -354,10 +354,10 @@ private[sql] abstract class SparkStrategies extends QueryPlanner[SparkPlan] {
354354
case c: CreateTableUsingAsSelect if !c.temporary =>
355355
sys.error("Tables created with SQLContext must be TEMPORARY. Use a HiveContext instead.")
356356

357-
case LogicalDescribeCommand(table, isExtended) =>
357+
case describe @ LogicalDescribeCommand(table, isExtended) =>
358358
val resultPlan = self.sqlContext.executePlan(table).executedPlan
359359
ExecutedCommand(
360-
RunnableDescribeCommand(resultPlan, resultPlan.output, isExtended)) :: Nil
360+
RunnableDescribeCommand(resultPlan, describe.output, isExtended)) :: Nil
361361

362362
case _ => Nil
363363
}

sql/core/src/test/scala/org/apache/spark/sql/sources/DDLTestSuite.scala

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -99,4 +99,10 @@ class DDLTestSuite extends DataSourceTest {
9999
Row("arrayType", "array<string>", ""),
100100
Row("structType", "struct<f1:string,f2:int>", "")
101101
))
102+
103+
test("SPARK-7686 DescribeCommand should have correct physical plan output attributes") {
104+
val attributes = sql("describe ddlPeople").queryExecution.executedPlan.output
105+
assert(attributes.map(_.name) === Seq("col_name", "data_type", "comment"))
106+
assert(attributes.map(_.dataType).toSet === Set(StringType))
107+
}
102108
}

0 commit comments

Comments
 (0)