Skip to content

[SPARK-19970][SQL][BRANCH-1.6] Table owner should be USER instead of PRINCIPAL in kerberized clusters#17366

Closed
dongjoon-hyun wants to merge 1 commit into
apache:branch-1.6from
dongjoon-hyun:SPARK-19970-BRANCH-1.6
Closed

[SPARK-19970][SQL][BRANCH-1.6] Table owner should be USER instead of PRINCIPAL in kerberized clusters#17366
dongjoon-hyun wants to merge 1 commit into
apache:branch-1.6from
dongjoon-hyun:SPARK-19970-BRANCH-1.6

Conversation

@dongjoon-hyun

Copy link
Copy Markdown
Member

What changes were proposed in this pull request?

In the kerberized hadoop cluster, when Spark creates tables, the owner of tables are filled with PRINCIPAL strings instead of USER names. This is inconsistent with Hive and causes problems when using ROLE in Hive. We had better to fix this. For Apache Spark 1.6, it happens only with CREATE TABLE ... AS SELECT (CTAS) statement.

BEFORE

scala> sql("create table t_ctas as select 1").show
scala> sql("desc formatted t_ctas").show(false)
...
|Owner:                      |spark@EXAMPLE.COM                                         |       |

AFTER

scala> sql("create table t_ctas as select 1").show
scala> sql("desc formatted t_ctas").show(false)
...
|Owner:                      |spark                                         |       |

How was this patch tested?

Manually do create table and desc formatted because this happens in Kerberized clusters.

@SparkQA

SparkQA commented Mar 20, 2017

Copy link
Copy Markdown

Test build #74907 has finished for PR 17366 at commit 6c726dd.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@dongjoon-hyun

Copy link
Copy Markdown
Member Author

I'm closing this for branch-1.6 according to the advice.
Thank you!

@dongjoon-hyun dongjoon-hyun deleted the SPARK-19970-BRANCH-1.6 branch January 7, 2019 07:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants