Skip to content

ZEPPELIN-3395 Fix impersonate spark interpreter without proxy-user option#2916

Closed
iijima-satoshi wants to merge 1 commit into
apache:masterfrom
iijima-satoshi:fix-impersonate-spark
Closed

ZEPPELIN-3395 Fix impersonate spark interpreter without proxy-user option#2916
iijima-satoshi wants to merge 1 commit into
apache:masterfrom
iijima-satoshi:fix-impersonate-spark

Conversation

@iijima-satoshi
Copy link
Copy Markdown

@iijima-satoshi iijima-satoshi commented Apr 9, 2018

What is this PR for?

Fix impersonate spark interpreter.
Currently, impersonate spark interpreter always runs as zeppelin user, not impersonated user when disable proxy-user option.

What type of PR is it?

Bug Fix

What is the Jira issue?

https://issues.apache.org/jira/browse/ZEPPELIN-3395

How should this be tested?

  1. ZEPPELIN_IMPERSONATE_SPARK_PROXY_USER=false in zeppelin-env.sh
  2. set up user impersonation flag
  3. run some job using that spark interpreter
  4. spark interpreter process should be created with currently logged-in user

Questions:

  • Does the licenses files need update? No
  • Is there breaking changes for older versions? No
  • Does this needs documentation? No

@iijima-satoshi
Copy link
Copy Markdown
Author

iijima-satoshi commented Apr 10, 2018

Could someone review this?
Is it ok that this fix has no tests? This fix only includes interpreter.sh script. I manually tested it.

felixcheung
felixcheung reviewed Apr 11, 2018
View reviewed changes
Comment thread bin/interpreter.sh Outdated
addJarInDirForIntp "${LOCAL_INTERPRETER_REPO}"

if [[ ! -z "$ZEPPELIN_IMPERSONATE_USER" && "${INTERPRETER_ID}" != "spark" ]]; then
if [[ ! -z "$ZEPPELIN_IMPERSONATE_USER" && "$ZEPPELIN_IMPERSONATE_SPARK_PROXY_USER" == "false" ]]; then
Copy link
Copy Markdown
Member

@felixcheung felixcheung Apr 11, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hmm, I'm actually not sure this is the way to run spark job as the user

Copy link
Copy Markdown
Author

@iijima-satoshi iijima-satoshi Apr 11, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In case of ZEPPELIN_IMPERSONATE_SPARK_PROXY_USER = true, spark interpreter should run as zeppelin user with --proxy-user=[impersonated user].
In case of ZEPPELIN_IMPERSONATE_SPARK_PROXY_USER = false, spark interpreter should run as impersonated user with no --proxy-user option.

But Currently,
In case of ZEPPELIN_IMPERSONATE_SPARK_PROXY_USER = false, spark interpreter runs as zeppelin user with no --proxy-user option.
This is due to "${INTERPRETER_ID}" != "spark" of if condition.

If condition should include ZEPPELIN_IMPERSONATE_SPARK_PROXY_USER = false instead of "${INTERPRETER_ID}" != "spark".

Copy link
Copy Markdown
Author

@iijima-satoshi iijima-satoshi Apr 12, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@zjffdu Please could you review this?
This logic was recently modified by your PR #2705.

Copy link
Copy Markdown
Contributor

@zjffdu zjffdu Apr 13, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@iijima-satoshi Have you verified other interpreters like shell ?

Copy link
Copy Markdown
Author

@iijima-satoshi iijima-satoshi Apr 13, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@zjffdu I fixed this line. Please see https://github.com/apache/zeppelin/pull/2916/files
I have verified test cases including other interpreters.

@iijima-satoshi iijima-satoshi force-pushed the fix-impersonate-spark branch from 13d446d to a7465b6 Compare April 12, 2018 18:51
@iijima-satoshi
Copy link
Copy Markdown
Author

iijima-satoshi commented Apr 12, 2018

Tests for interpreters except spark one were droped in case of ZEPPELIN_IMPERSONATE_SPARK_PROXY_USER = true.
I fixed and manually tested.

@zjffdu
Copy link
Copy Markdown
Contributor

zjffdu commented Apr 13, 2018

Thanks @iijima-satoshi LGTM

@iijima-satoshi
Copy link
Copy Markdown
Author

iijima-satoshi commented Apr 16, 2018

@felixcheung Please let me know if there are any other comments. Thanks.

@felixcheung
Copy link
Copy Markdown
Member

felixcheung commented Apr 16, 2018

sure, I'd defer to @zjffdu
could you fix up the truncated PR title btw

@iijima-satoshi iijima-satoshi changed the title ZEPPELIN-3395 Fix: Spark interpreter does not run as impersonated user when disable… ZEPPELIN-3395 Fix impersonate spark interpreter without proxy-user option Apr 16, 2018
@iijima-satoshi
Copy link
Copy Markdown
Author

iijima-satoshi commented Apr 16, 2018

@felixcheung Fixed PR title. Thanks.

@asfgit asfgit closed this in aa3ea68 Apr 16, 2018
asfgit pushed a commit that referenced this pull request Apr 16, 2018
@iijima-satoshi @zjffdu
ZEPPELIN-3395 Fix impersonate spark interpreter without proxy-user op…
6376122 
…tion

### What is this PR for?
Fix impersonate spark interpreter.
Currently, impersonate spark interpreter always runs as zeppelin user, not impersonated user when disable proxy-user option.

### What type of PR is it?
Bug Fix

### What is the Jira issue?
https://issues.apache.org/jira/browse/ZEPPELIN-3395

### How should this be tested?
1. ZEPPELIN_IMPERSONATE_SPARK_PROXY_USER=false in zeppelin-env.sh
2. set up user impersonation flag
3. run some job using that spark interpreter
4. spark interpreter process should be created with currently logged-in user

### Questions:
* Does the licenses files need update? No
* Is there breaking changes for older versions? No
* Does this needs documentation? No

Author: iijima_satoshi <iijima_satoshi@cyberagent.co.jp>

Closes #2916 from iijima-satoshi/fix-impersonate-spark and squashes the following commits:

a7465b6 [iijima_satoshi] Fix: Spark interpreter does not run as impersonated user when disable proxy-user option

(cherry picked from commit aa3ea68)
Signed-off-by: Jeff Zhang <zjffdu@apache.org>
jwagun pushed a commit to jwagun/zeppelin that referenced this pull request Apr 23, 2018
@iijima-satoshi @jwagun
ZEPPELIN-3395 Fix impersonate spark interpreter without proxy-user op…
427c666 
…tion

### What is this PR for?
Fix impersonate spark interpreter.
Currently, impersonate spark interpreter always runs as zeppelin user, not impersonated user when disable proxy-user option.

### What type of PR is it?
Bug Fix

### What is the Jira issue?
https://issues.apache.org/jira/browse/ZEPPELIN-3395

### How should this be tested?
1. ZEPPELIN_IMPERSONATE_SPARK_PROXY_USER=false in zeppelin-env.sh
2. set up user impersonation flag
3. run some job using that spark interpreter
4. spark interpreter process should be created with currently logged-in user

### Questions:
* Does the licenses files need update? No
* Is there breaking changes for older versions? No
* Does this needs documentation? No

Author: iijima_satoshi <iijima_satoshi@cyberagent.co.jp>

Closes apache#2916 from iijima-satoshi/fix-impersonate-spark and squashes the following commits:

a7465b6 [iijima_satoshi] Fix: Spark interpreter does not run as impersonated user when disable proxy-user option
prabhjyotsingh pushed a commit to prabhjyotsingh/zeppelin that referenced this pull request Jul 4, 2018
@iijima-satoshi @zjffdu
ZEPPELIN-3395 Fix impersonate spark interpreter without proxy-user op…
fb57b10 
…tion

Fix impersonate spark interpreter.
Currently, impersonate spark interpreter always runs as zeppelin user, not impersonated user when disable proxy-user option.

Bug Fix

https://issues.apache.org/jira/browse/ZEPPELIN-3395

1. ZEPPELIN_IMPERSONATE_SPARK_PROXY_USER=false in zeppelin-env.sh
2. set up user impersonation flag
3. run some job using that spark interpreter
4. spark interpreter process should be created with currently logged-in user

* Does the licenses files need update? No
* Is there breaking changes for older versions? No
* Does this needs documentation? No

Author: iijima_satoshi <iijima_satoshi@cyberagent.co.jp>

Closes apache#2916 from iijima-satoshi/fix-impersonate-spark and squashes the following commits:

a7465b6 [iijima_satoshi] Fix: Spark interpreter does not run as impersonated user when disable proxy-user option

(cherry picked from commit aa3ea68)
Signed-off-by: Jeff Zhang <zjffdu@apache.org>
(cherry picked from commit 6376122)

Change-Id: I83bfdfed585bddabbf6a9ad5d1c65484c4573b3f
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@zjffdu zjffdu zjffdu left review comments

@felixcheung felixcheung felixcheung left review comments

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@iijima-satoshi @zjffdu @felixcheung