Skip to content

[SPARK-32451][R][3.0] Support Apache Arrow 1.0.0#29462

Closed
HyukjinKwon wants to merge 1 commit into
apache:branch-3.0from
HyukjinKwon:SPARK-32451-3.0
Closed

[SPARK-32451][R][3.0] Support Apache Arrow 1.0.0#29462
HyukjinKwon wants to merge 1 commit into
apache:branch-3.0from
HyukjinKwon:SPARK-32451-3.0

Conversation

@HyukjinKwon

@HyukjinKwon HyukjinKwon commented Aug 18, 2020

Copy link
Copy Markdown
Member

What changes were proposed in this pull request?

This PR ports back #29252 to support Arrow 1.0.0.

Currently, SparkR with Arrow tests fails with the latest Arrow version in branch-3.0, see https://github.com/apache/spark/pull/29460/checks?check_run_id=996972267

Why are the changes needed?

To support higher Arrow R version with SparkR.

Does this PR introduce any user-facing change?

Yes, users will be able to use SparkR with Arrow 1.0.0+.

How was this patch tested?

Manually tested, GitHub Actions will test it.

@HyukjinKwon

Copy link
Copy Markdown
Member Author

cc @zhengruifeng FYI.

@SparkQA

SparkQA commented Aug 18, 2020

Copy link
Copy Markdown

Test build #127556 has finished for PR 29462 at commit e172fb0.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@viirya viirya left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, we didn't port back Arrow 1.0 support in previous PR. LGTM.

@dongjoon-hyun dongjoon-hyun left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1, LGTM. Thanks, @HyukjinKwon and @viirya .
Merged to branch-3.0.

dongjoon-hyun pushed a commit that referenced this pull request Aug 19, 2020
### What changes were proposed in this pull request?

This PR ports back #29252 to support Arrow 1.0.0.

Currently, SparkR with Arrow tests fails with the latest Arrow version in branch-3.0, see https://github.com/apache/spark/pull/29460/checks?check_run_id=996972267

### Why are the changes needed?

To support higher Arrow R version with SparkR.

### Does this PR introduce _any_ user-facing change?

Yes, users will be able to use SparkR with Arrow 1.0.0+.

### How was this patch tested?

Manually tested, GitHub Actions will test it.

Closes #29462 from HyukjinKwon/SPARK-32451-3.0.

Authored-by: HyukjinKwon <gurwls223@apache.org>
Signed-off-by: Dongjoon Hyun <dongjoon@apache.org>
@HyukjinKwon

Copy link
Copy Markdown
Member Author

Thanks!

@HyukjinKwon HyukjinKwon deleted the SPARK-32451-3.0 branch December 7, 2020 02:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants