Skip to content

ARROW-3761: [R] Bindings for CompressedInputStream, CompressedOutputStream#2962

Closed
romainfrancois wants to merge 8 commits into
apache:masterfrom
romainfrancois:ARROW-3761/CompressedStream
Closed

ARROW-3761: [R] Bindings for CompressedInputStream, CompressedOutputStream#2962
romainfrancois wants to merge 8 commits into
apache:masterfrom
romainfrancois:ARROW-3761/CompressedStream

Conversation

@romainfrancois

@romainfrancois romainfrancois commented Nov 14, 2018

Copy link
Copy Markdown
Contributor

No description provided.

@codecov-io

codecov-io commented Nov 14, 2018

Copy link
Copy Markdown

Codecov Report

Merging #2962 into master will increase coverage by 15.14%.
The diff coverage is n/a.

Impacted file tree graph

@@             Coverage Diff             @@
##           master    #2962       +/-   ##
===========================================
+ Coverage   74.31%   89.45%   +15.14%     
===========================================
  Files          68       12       -56     
  Lines        5423     1811     -3612     
===========================================
- Hits         4030     1620     -2410     
+ Misses       1297      191     -1106     
+ Partials       96        0       -96
Impacted Files Coverage Δ
go/arrow/array/table.go
go/arrow/math/uint64_amd64.go
go/arrow/internal/testing/tools/bool.go
go/arrow/internal/bitutil/bitutil.go
go/arrow/memory/memory_avx2_amd64.go
go/arrow/array/null.go
go/arrow/datatype_nested.go
go/arrow/array/string.go
go/arrow/math/uint64_avx2_amd64.go
go/arrow/array/builder.go
... and 46 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 60305d5...8ccc977. Read the comment docs.

@romainfrancois romainfrancois force-pushed the ARROW-3761/CompressedStream branch 3 times, most recently from 8ccc977 to 633d1cc Compare November 19, 2018 11:19
@romainfrancois romainfrancois force-pushed the ARROW-3761/CompressedStream branch from 633d1cc to 1013a1d Compare November 26, 2018 09:48
@wesm

wesm commented Nov 26, 2018

Copy link
Copy Markdown
Member

Are you working on this somewhere else?

@romainfrancois

Copy link
Copy Markdown
Contributor Author

I'm not.

@romainfrancois

Copy link
Copy Markdown
Contributor Author

I don't know what happened, but it looks I've lost some commits in that branch. this is thankfully minor.

@wesm

wesm commented Nov 26, 2018

Copy link
Copy Markdown
Member

Looks like you may need to git reset --hard to 8ccc977 and then rebase. It should be in your reflog

@wesm wesm reopened this Nov 26, 2018
@romainfrancois

Copy link
Copy Markdown
Contributor Author

Thanks @wesm

@wesm

wesm commented Nov 28, 2018

Copy link
Copy Markdown
Member

There are no tests here. What would you like to do?

@romainfrancois

Copy link
Copy Markdown
Contributor Author

I’m still unsure how to test this. I’ll look around in pyarrow next time i look at this branch

@wesm

wesm commented Nov 28, 2018

Copy link
Copy Markdown
Member

Got it. I think we just generate random bytes and write them to a compressed file, then read them back in and compare

@romainfrancois

Copy link
Copy Markdown
Contributor Author

Makes sense. IIRC I tried to serialize a record batch but that did not work out well.

@wesm

wesm commented Nov 28, 2018

Copy link
Copy Markdown
Member

Yeah, that requires random access. It should at least not crash, though

@wesm

wesm commented Nov 28, 2018

Copy link
Copy Markdown
Member

Well, to read using the *StreamReader should work. If it does not, that's buggy

@romainfrancois

Copy link
Copy Markdown
Contributor Author

Thanks. I’ll use that in testing.

@romainfrancois romainfrancois force-pushed the ARROW-3761/CompressedStream branch from 6002218 to efe4ae8 Compare December 3, 2018 08:43
@romainfrancois

Copy link
Copy Markdown
Contributor Author

This will likely need to be rebased once #3043 is merged

@romainfrancois romainfrancois force-pushed the ARROW-3761/CompressedStream branch from 0ba65a0 to 14a7ef1 Compare December 4, 2018 20:33
@romainfrancois romainfrancois force-pushed the ARROW-3761/CompressedStream branch 2 times, most recently from cd5e8c9 to 6c8907c Compare December 10, 2018 18:15
@romainfrancois romainfrancois force-pushed the ARROW-3761/CompressedStream branch from 6c8907c to feb948a Compare January 2, 2019 09:42
@romainfrancois romainfrancois force-pushed the ARROW-3761/CompressedStream branch from feb948a to 397baca Compare January 7, 2019 15:19

@wesm wesm left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1

@wesm wesm closed this in f576c3e Jan 29, 2019
@romainfrancois romainfrancois deleted the ARROW-3761/CompressedStream branch February 5, 2019 09:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants