Skip to content

Conversation

@southernriver
Copy link
Contributor

What changes were proposed in this pull request?

For SparkSQL,When we do some alter operations on hive table, the owner of hive table would be changed to someone who invoked the operation, it's unresonable. And in fact, the owner should not changed for the real prodcution environment, otherwise the authority check is out of order.

The problem can be reproduced as described in the below:

  1. First I create a table with username='xie' and then desc formatted table ,the owner is 'xiepengjie'
spark-sql> desc formatted bigdata_test.tt1;
col_name data_type comment
c int NULL
# Detailed Table Information
Database bigdata_test
Table tt1
Owner xie
Created Time Wed Sep 11 11:30:49 CST 2019
Last Access Thu Jan 01 08:00:00 CST 1970
Created By Spark 2.2 or prior
Type MANAGED
Provider hive
Table Properties [PART_LIMIT=10000, transient_lastDdlTime=1568172649, LEVEL=1, TTL=60]
Location hdfs://NS1/user/hive_admin/warehouse/bigdata_test.db/tt1
Serde Library org.apache.hadoop.hive.ql.io.orc.OrcSerde
InputFormat org.apache.hadoop.hive.ql.io.orc.OrcInputFormat
OutputFormat org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat
Storage Properties [serialization.format=1]
Partition Provider Catalog
Time taken: 0.371 seconds, Fetched 18 row(s)

  1. Then I use another username='johnchen' and execute alter table bigdata_test.tt1 set location 'hdfs://NS1/user/hive_admin/warehouse/bigdata_test.db/tt1', check the owner of hive table is 'johnchen', it's unresonable
spark-sql> desc formatted bigdata_test.tt1;
col_name        data_type       comment
c       int     NULL
 
# Detailed Table Information
Database        bigdata_test
Table   tt1
Owner   johnchen
Created Time    Wed Sep 11 11:30:49 CST 2019
Last Access     Thu Jan 01 08:00:00 CST 1970
Created By      Spark 2.2 or prior
Type    MANAGED
Provider        hive
Table Properties        [transient_lastDdlTime=1568871017, PART_LIMIT=10000, LEVEL=1, TTL=60]
Location        hdfs://NS1/user/hive_admin/warehouse/bigdata_test.db/tt1
Serde Library   org.apache.hadoop.hive.ql.io.orc.OrcSerde
InputFormat     org.apache.hadoop.hive.ql.io.orc.OrcInputFormat
OutputFormat    org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat
Storage Properties      [serialization.format=1]
Partition Provider      Catalog
Time taken: 0.041 seconds, Fetched 18 row(s)

Why are the changes needed?

In fact, the owner should not changed for the real prodcution environment, otherwise the authority check is out of order.

Does this PR introduce any user-facing change?

No

How was this patch tested?

Manual

@AmplabJenkins
Copy link

Can one of the admins verify this patch?

@southernriver southernriver changed the base branch from master to branch-2.4 December 14, 2019 07:32
@southernriver southernriver changed the base branch from branch-2.4 to branch-2.3 December 14, 2019 07:33
@southernriver southernriver changed the base branch from branch-2.3 to master December 14, 2019 07:33
@HeartSaVioR
Copy link
Contributor

HeartSaVioR commented Jan 9, 2020

Thanks for the contribution. @southernriver

  1. Could you resolve the conflict?
  2. Could we add UT for verifying the patch?

@HeartSaVioR
Copy link
Contributor

@cloud-fan
Copy link
Contributor

I roughly remember this has been fixed, cc @viirya @yaooqinn

@HeartSaVioR
Copy link
Contributor

HeartSaVioR commented Jan 9, 2020

Ah you're right. SPARK-29498 (#26160) fixed this and that seems to be the reason of conflict.
@southernriver Could you check SPARK-29498 and close this?

@yaooqinn
Copy link
Member

yaooqinn commented Jan 9, 2020

shouldn't be a problem any longer

@southernriver
Copy link
Contributor Author

Ah you're right. SPARK-29498 (#26160) fixed this and that seems to be the reason of conflict.
@southernriver Could you check SPARK-29498 and close this?

yes,this is the reason, i'll close this ,thank you all!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

7 participants