Skip to content

Commit fee695d

Browse files
wangyumdongjoon-hyun
authored andcommitted
[SPARK-27690][SQL] Remove materialized views first in HiveClientImpl.reset
## What changes were proposed in this pull request? We should remove materialized view first otherwise(note that Hive 3.1 could reproduce this issue): ```scala Cause: org.apache.derby.shared.common.error.DerbySQLIntegrityConstraintViolationException: DELETE on table 'TBLS' caused a violation of foreign key constraint 'MV_TABLES_USED_FK2' for key (4). The statement has been rolled back. at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source) at org.apache.derby.impl.jdbc.Util.generateCsSQLException(Unknown Source) at org.apache.derby.impl.jdbc.TransactionResourceImpl.wrapInSQLException(Unknown Source) at org.apache.derby.impl.jdbc.TransactionResourceImpl.handleException(Unknown Source) at org.apache.derby.impl.jdbc.EmbedConnection.handleException(Unknown Source) at org.apache.derby.impl.jdbc.ConnectionChild.handleException(Unknown Source) at org.apache.derby.impl.jdbc.EmbedStatement.executeStatement(Unknown Source) at org.apache.derby.impl.jdbc.EmbedPreparedStatement.executeBatchElement(Unknown Source) at org.apache.derby.impl.jdbc.EmbedStatement.executeLargeBatch(Unknown Source) ``` ## How was this patch tested? Existing test Closes #24592 from wangyum/SPARK-27690. Authored-by: Yuming Wang <[email protected]> Signed-off-by: Dongjoon Hyun <[email protected]>
1 parent a10608c commit fee695d

File tree

1 file changed

+13
-2
lines changed

1 file changed

+13
-2
lines changed

sql/hive/src/main/scala/org/apache/spark/sql/hive/client/HiveClientImpl.scala

Lines changed: 13 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -879,9 +879,20 @@ private[hive] class HiveClientImpl(
879879
}
880880

881881
def reset(): Unit = withHiveState {
882-
client.getAllTables("default").asScala.foreach { t =>
882+
val allTables = client.getAllTables("default")
883+
val (mvs, others) = allTables.asScala.map(t => client.getTable("default", t))
884+
.partition(_.getTableType.toString.equals("MATERIALIZED_VIEW"))
885+
886+
// Remove materialized view first, otherwise caused a violation of foreign key constraint.
887+
mvs.foreach { table =>
888+
val t = table.getTableName
889+
logDebug(s"Deleting materialized view $t")
890+
client.dropTable("default", t)
891+
}
892+
893+
others.foreach { table =>
894+
val t = table.getTableName
883895
logDebug(s"Deleting table $t")
884-
val table = client.getTable("default", t)
885896
try {
886897
client.getIndexes("default", t, 255).asScala.foreach { index =>
887898
shim.dropIndex(client, "default", t, index.getIndexName)

0 commit comments

Comments
 (0)