Skip to content

[SPARK-31655][BUILD][2.4] Upgrade snappy-java to 1.1.7.5#28506

Closed
AngersZhuuuu wants to merge 1 commit into
apache:branch-2.4from
AngersZhuuuu:spark-31655-2.4
Closed

[SPARK-31655][BUILD][2.4] Upgrade snappy-java to 1.1.7.5#28506
AngersZhuuuu wants to merge 1 commit into
apache:branch-2.4from
AngersZhuuuu:spark-31655-2.4

Conversation

@AngersZhuuuu

@AngersZhuuuu AngersZhuuuu commented May 12, 2020

Copy link
Copy Markdown
Contributor

What changes were proposed in this pull request?

snappy-java have release v1.1.7.5, upgrade to latest version.

Fixed in v1.1.7.4

Fixed in v1.1.7.5

  • Fixes java.lang.NoClassDefFoundError: org/xerial/snappy/pool/DefaultPoolFactory in 1.1.7.4

xerial/snappy-java@1.1.7.3...1.1.7.5

v 1.1.7.5 release note:
xerial/snappy-java@edc4ec2

Why are the changes needed?

Fix bug

Does this PR introduce any user-facing change?

No

How was this patch tested?

No need

@AngersZhuuuu

AngersZhuuuu commented May 12, 2020

Copy link
Copy Markdown
Contributor Author

cc @dongjoon-hyun @maropu

@HyukjinKwon

HyukjinKwon commented May 12, 2020

Copy link
Copy Markdown
Member

If we're going to port this back to 2.4, we should do it for 3.0 as well.

@AngersZhuuuu

AngersZhuuuu commented May 12, 2020

Copy link
Copy Markdown
Contributor Author

If we're going to port this back to 2.4, we should do it for 3.0 as well.

I think we should do that, some user have meeting this problem.

@SparkQA

SparkQA commented May 12, 2020

Copy link
Copy Markdown

Test build #122529 has finished for PR 28506 at commit 42f0c98.

  • This patch fails build dependency tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@AngersZhuuuu

AngersZhuuuu commented May 12, 2020

Copy link
Copy Markdown
Contributor Author

retest this please

dongjoon-hyun
dongjoon-hyun reviewed May 12, 2020
View reviewed changes

@dongjoon-hyun dongjoon-hyun left a comment •
edited
Loading

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you think we can have a valid unit test case for this? (non-ppc64le part)

@SparkQA

SparkQA commented May 12, 2020

Copy link
Copy Markdown

Test build #122534 has finished for PR 28506 at commit 42f0c98.

  • This patch fails build dependency tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@maropu

maropu commented May 12, 2020

Copy link
Copy Markdown
Member

@nblxa Could you check if your query could work correctly with this fix? #28472

@AngersZhuuuu

AngersZhuuuu commented May 12, 2020

Copy link
Copy Markdown
Contributor Author

Test deps failed since #28509 (comment)

@AngersZhuuuu

AngersZhuuuu commented May 15, 2020

Copy link
Copy Markdown
Contributor Author

Retest this please

@SparkQA

SparkQA commented May 15, 2020

Copy link
Copy Markdown

Test build #122654 has finished for PR 28506 at commit 42f0c98.

  • This patch fails due to an unknown error code, -9.
  • This patch merges cleanly.
  • This patch adds no public classes.

@maropu

maropu commented May 15, 2020

Copy link
Copy Markdown
Member

retest this please

@SparkQA

SparkQA commented May 15, 2020

Copy link
Copy Markdown

Test build #122656 has finished for PR 28506 at commit 42f0c98.

  • This patch fails Spark unit tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@maropu

maropu commented May 15, 2020

Copy link
Copy Markdown
Member

retest this please

@nblxa

nblxa commented May 15, 2020

Copy link
Copy Markdown

@maropu many thanks for backporting this fix! I wasn't expecting such a swift turnaround time :) I regret to say that at the moment I don't know yet when I can be able to test this due to certain constraints on my end. Apologies for that and many thanks again for your work put into this fix.

@SparkQA

SparkQA commented May 15, 2020

Copy link
Copy Markdown

Test build #122672 has finished for PR 28506 at commit 42f0c98.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

dongjoon-hyun
dongjoon-hyun approved these changes May 17, 2020
View reviewed changes

@dongjoon-hyun dongjoon-hyun left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1, LGTM. Merged to branch-2.4. Thanks.

dongjoon-hyun pushed a commit that referenced this pull request May 17, 2020
@AngersZhuuuu @dongjoon-hyun
[SPARK-31655][BUILD][2.4] Upgrade snappy-java to 1.1.7.5
6d8cef5 
### What changes were proposed in this pull request?

snappy-java have release v1.1.7.5, upgrade to latest version.

Fixed in v1.1.7.4
- Caching internal buffers for SnappyFramed streams #234
- Fixed the native lib for ppc64le to work with glibc 2.17 (Previously it depended on 2.22)

Fixed in v1.1.7.5
- Fixes java.lang.NoClassDefFoundError: org/xerial/snappy/pool/DefaultPoolFactory in 1.1.7.4

xerial/snappy-java@1.1.7.3...1.1.7.5

v 1.1.7.5 release note:
xerial/snappy-java@edc4ec2
### Why are the changes needed?
Fix bug

### Does this PR introduce _any_ user-facing change?
No

### How was this patch tested?
No need

Closes #28506 from AngersZhuuuu/spark-31655-2.4.

Authored-by: angerszhu <angers.zhu@gmail.com>
Signed-off-by: Dongjoon Hyun <dongjoon@apache.org>
@maropu

maropu commented May 17, 2020

Copy link
Copy Markdown
Member

Thanks, all!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@dongjoon-hyun dongjoon-hyun dongjoon-hyun approved these changes

Assignees

No one assigned

Labels

BUILD

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

6 participants

@AngersZhuuuu @HyukjinKwon @SparkQA @maropu @nblxa @dongjoon-hyun