Skip to content

Conversation

@carsonwang
Copy link
Contributor

What changes were proposed in this pull request?

There is no Spark SQL UI when executing queries in Spark SQL CLI. It worked before in 2.1.0.
The method hiveResultString is used by SparkSQLDriver. It seems the line SQLExecution.withNewExecutionId was removed accidently. This PR is to bring the SQL UI back when using Spark SQL CLI.

How was this patch tested?

Manually ran bin/spark-sql and executed a query.

@SparkQA
Copy link

SparkQA commented Apr 5, 2017

Test build #75533 has finished for PR 17535 at commit 1d843af.

  • This patch fails Spark unit tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@srowen
Copy link
Member

srowen commented Apr 5, 2017

Looks closely related to #17540 ?

@carsonwang
Copy link
Contributor Author

Yes, it is closely related but two scenarios of adding SQLExecution.withNewExecutionId.
Now some tests fail because withNewExecutionId is called twice.

java.lang.IllegalArgumentException: spark.sql.execution.id is already set
[info]   	at org.apache.spark.sql.execution.SQLExecution$.withNewExecutionId(SQLExecution.scala:90)
[info]   	at org.apache.spark.sql.execution.datasources.FileFormatWriter$.write(FileFormatWriter.scala:164)
[info]   	at org.apache.spark.sql.hive.execution.InsertIntoHiveTable.run(InsertIntoHiveTable.scala:311)
[info]   	at org.apache.spark.sql.execution.command.ExecutedCommandExec.sideEffectResult$lzycompute(commands.scala:58)
[info]   	at org.apache.spark.sql.execution.command.ExecutedCommandExec.sideEffectResult(commands.scala:56)
[info]   	at org.apache.spark.sql.execution.command.ExecutedCommandExec.executeCollect(commands.scala:67)
[info]   	at org.apache.spark.sql.execution.SparkPlan.executeCollectPublic(SparkPlan.scala:295)
[info]   	at org.apache.spark.sql.execution.QueryExecution$$anonfun$hiveResultString$3.apply(QueryExecution.scala:137)
[info]   	at org.apache.spark.sql.execution.QueryExecution$$anonfun$hiveResultString$3.apply(QueryExecution.scala:136)
[info]   	at org.apache.spark.sql.execution.SQLExecution$.withNewExecutionId(SQLExecution.scala:65)
[info]   	at org.apache.spark.sql.execution.QueryExecution.hiveResultString(QueryExecution.scala:136)

@carsonwang
Copy link
Contributor Author

This fix will be included in #17540

@carsonwang carsonwang closed this May 9, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants