Skip to content

[SPARK-4242] [Core] Add SASL to external shuffle service#3108

Closed
aarondav wants to merge 5 commits into
apache:masterfrom
aarondav:sasl-client
Closed

[SPARK-4242] [Core] Add SASL to external shuffle service#3108
aarondav wants to merge 5 commits into
apache:masterfrom
aarondav:sasl-client

Conversation

@aarondav

@aarondav aarondav commented Nov 5, 2014

Copy link
Copy Markdown
Contributor

Does three things: (1) Adds SASL to ExternalShuffleClient, (2) puts SecurityManager in BlockManager's constructor, and (3) adds unit test.

@andrewor14 andrewor14 mentioned this pull request Nov 5, 2014
Closed
@SparkQA

SparkQA commented Nov 5, 2014

Copy link
Copy Markdown

Test build #22923 has started for PR 3108 at commit 2bf2908.

  • This patch merges cleanly.

andrewor14
andrewor14 reviewed Nov 5, 2014
View reviewed changes
Comment thread ...ork/shuffle/src/test/java/org/apache/spark/network/shuffle/ExternalShuffleSecuritySuite.java Outdated

@andrewor14 andrewor14 Nov 5, 2014

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

need new line above this

@rxin

rxin commented Nov 5, 2014

Copy link
Copy Markdown
Contributor

LGTM

1 similar comment
@andrewor14

andrewor14 commented Nov 5, 2014

Copy link
Copy Markdown
Contributor

LGTM

@aarondav

aarondav commented Nov 5, 2014

Copy link
Copy Markdown
Contributor Author

Addressed comments.

@SparkQA

SparkQA commented Nov 5, 2014

Copy link
Copy Markdown

Test build #22924 has started for PR 3108 at commit cbe451a.

  • This patch merges cleanly.

@SparkQA

SparkQA commented Nov 5, 2014

Copy link
Copy Markdown

Test build #22923 has finished for PR 3108 at commit 2bf2908.

  • This patch fails Spark unit tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@AmplabJenkins

AmplabJenkins commented Nov 5, 2014

Copy link
Copy Markdown

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/22923/
Test FAILed.

@SparkQA

SparkQA commented Nov 5, 2014

Copy link
Copy Markdown

Test build #22924 has finished for PR 3108 at commit cbe451a.

  • This patch fails Spark unit tests.
  • This patch merges cleanly.
  • This patch adds the following public classes (experimental):
    • class Matrices(object):
    • class ChiSqTestResult(JavaModelWrapper):

@AmplabJenkins

AmplabJenkins commented Nov 5, 2014

Copy link
Copy Markdown

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/22924/
Test FAILed.

aarondav
aarondav reviewed Nov 5, 2014
View reviewed changes
Comment thread network/common/pom.xml

@aarondav aarondav Nov 5, 2014

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@pwendell I don't know if this is kosher - it helps us compile in a YARN-compatible way, and Guava usually just adds and deprecates APIs.

@SparkQA

SparkQA commented Nov 5, 2014

Copy link
Copy Markdown

Test build #22930 has started for PR 3108 at commit b58518a.

  • This patch merges cleanly.

@SparkQA

SparkQA commented Nov 5, 2014

Copy link
Copy Markdown

Test build #22930 has finished for PR 3108 at commit b58518a.

  • This patch fails Spark unit tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@AmplabJenkins

AmplabJenkins commented Nov 5, 2014

Copy link
Copy Markdown

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/22930/
Test FAILed.

@aarondav

aarondav commented Nov 5, 2014

Copy link
Copy Markdown
Contributor Author

Jenkins, retest this please.

@SparkQA

SparkQA commented Nov 5, 2014

Copy link
Copy Markdown

Test build #22943 has started for PR 3108 at commit b58518a.

  • This patch merges cleanly.

@SparkQA

SparkQA commented Nov 5, 2014

Copy link
Copy Markdown

Test build #22943 has finished for PR 3108 at commit b58518a.

  • This patch fails Spark unit tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@AmplabJenkins

AmplabJenkins commented Nov 5, 2014

Copy link
Copy Markdown

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/22943/
Test FAILed.

@SparkQA

SparkQA commented Nov 5, 2014

Copy link
Copy Markdown

Test build #22944 has started for PR 3108 at commit 3543b70.

  • This patch merges cleanly.

@SparkQA

SparkQA commented Nov 5, 2014

Copy link
Copy Markdown

Test build #22944 has finished for PR 3108 at commit 3543b70.

  • This patch fails Spark unit tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@AmplabJenkins

AmplabJenkins commented Nov 5, 2014

Copy link
Copy Markdown

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/22944/
Test FAILed.

@SparkQA

SparkQA commented Nov 5, 2014

Copy link
Copy Markdown

Test build #22947 has started for PR 3108 at commit 48b622d.

  • This patch merges cleanly.

@SparkQA

SparkQA commented Nov 5, 2014

Copy link
Copy Markdown

Test build #22947 has finished for PR 3108 at commit 48b622d.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds the following public classes (experimental):
    • public final class LimitedInputStream extends FilterInputStream

@AmplabJenkins

AmplabJenkins commented Nov 5, 2014

Copy link
Copy Markdown

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/22947/
Test PASSed.

@pwendell

pwendell commented Nov 5, 2014

Copy link
Copy Markdown
Contributor

Okay thanks - pulling this in.

asfgit pushed a commit that referenced this pull request Nov 5, 2014
@aarondav @pwendell
[SPARK-4242] [Core] Add SASL to external shuffle service
e7f7356 
Does three things: (1) Adds SASL to ExternalShuffleClient, (2) puts SecurityManager in BlockManager's constructor, and (3) adds unit test.

Author: Aaron Davidson <aaron@databricks.com>

Closes #3108 from aarondav/sasl-client and squashes the following commits:

48b622d [Aaron Davidson] Screw it, let's just get LimitedInputStream
3543b70 [Aaron Davidson] Back out of pom change due to unknown test issue?
b58518a [Aaron Davidson] ByteStreams.limit() not available :(
cbe451a [Aaron Davidson] Address comments
2bf2908 [Aaron Davidson] [SPARK-4242] [Core] Add SASL to external shuffle service
@asfgit asfgit closed this in 4c42986 Nov 5, 2014
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

6 participants

@aarondav @SparkQA @rxin @andrewor14 @AmplabJenkins @pwendell