Support binary compression in sas7bdat

[evanmiller]

Binary (aka Ross) compression is currently not supported. It looks like we can use the Python sas7bdat package as a template for implementing the decompression algorithm:

https://pypi.python.org/pypi/sas7bdat

(The library is MIT licensed.)

At present, compressed files fail with the error message "File has unsupported compression scheme".

[xiaodaigh]

Compression support is definitely needed! I have to stick to the parso library in Java until this is supported.

[ajdamico]

hi, not sure if this is the right place for a minimal reproducible example since tidyverse/haven#31 was closed? the latest version of haven still fails on compressed data.. thanks

# devtools::install_github('biostatmatt/sas7bdat.parso')
library(haven)
library(sas7bdat.parso)


tf1 <- tempfile()

download.file( "http://www.census.gov/housing/extract_files/data%20extracts/cpsasec14/hhld.sas7bdat" , tf1 , mode = 'wb' )

# fails
x1 <- read_sas( tf1 )

# works
y1 <- read.sas7bdat.parso( tf1 )

[hadley]

@evanmiller is this on the schedule?

[evanmiller]

No schedule for this -- note that many compression issues were misdiagnosed as binary compression rather than bugs in the character decompressor. Seems like 90%+ compressed files in the wild are character compressed.

[evanmiller]

Note that the example file provided by @ajdamico was fixed in 69d5751 and 8c0463a.

[reikoch]

If a truly binary compressed SAS dataset is needed, you may use [https://github.com/reikoch/testfiles/blob/master/binary.sas7bdat]. haven 1.0 fails with "ReadStat: Error parsing page 0, bytes 8192-16383".