Skip to content

Commit 37ad33e

Browse files
committed
Do not use hiveQlTable at all.
1 parent 47cdb6d commit 37ad33e

File tree

1 file changed

+6
-19
lines changed

1 file changed

+6
-19
lines changed

sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveMetastoreCatalog.scala

Lines changed: 6 additions & 19 deletions
Original file line numberDiff line numberDiff line change
@@ -707,33 +707,20 @@ private[hive] case class MetastoreRelation
707707
hiveQlTable.getMetadata
708708
)
709709

710-
implicit class SchemaAttribute(f: FieldSchema) {
710+
implicit class SchemaAttribute(f: HiveColumn) {
711711
def toAttribute: AttributeReference = AttributeReference(
712-
f.getName,
713-
HiveMetastoreTypes.toDataType(f.getType),
712+
f.name,
713+
HiveMetastoreTypes.toDataType(f.hiveType),
714714
// Since data can be dumped in randomly with no validation, everything is nullable.
715715
nullable = true
716716
)(qualifiers = Seq(alias.getOrElse(tableName)))
717717
}
718718

719-
// Must be a stable value since new attributes are born here.
720-
// Since we create the TTable inside hiveQlTable manually, we can use TTbale's interface
721-
// to get Partition keys at here. We do this to make sure Hive will not try to use
722-
// any metastore utility functions. All of interactions between metastore and related
723-
// parts should be done through our ClientWrapper.
724-
// Without the guard of ClientWrapper, we cannot touch metastore client (e.g. Hive class) and
725-
// the conf associated with the metastore client (Hive.get().getConf()).
726719
/** PartitionKey attributes */
727-
val partitionKeys = hiveQlTable.getTTable.getPartitionKeys.map(_.toAttribute)
728-
729-
// Since we create the TTable inside hiveQlTable manually, we can use TTbale's interface
730-
// to get non-partition columns at here. We do this to make sure Hive will not try to use
731-
// any metastore utility functions. All of interactions between metastore and related
732-
// parts should be done through our ClientWrapper.
733-
// Without the guard of ClientWrapper, we cannot touch metastore client (e.g. Hive class) and
734-
// the conf associated with the metastore client (Hive.get().getConf()).
720+
val partitionKeys = table.partitionColumns.map(_.toAttribute)
721+
735722
/** Non-partitionKey attributes */
736-
val attributes = hiveQlTable.getTTable.getSd.getCols.map(_.toAttribute)
723+
val attributes = table.schema.map(_.toAttribute)
737724

738725
val output = attributes ++ partitionKeys
739726

0 commit comments

Comments
 (0)