change your S3 flatfiles to use parquet #984
Description
Is your feature request related to a problem? Please describe.
.gz is not as good as .parquet
Describe the solution you'd like
convert your .gz S3 flatfiles to .parquet
Describe alternatives you've considered
no... it's pretty obvious parquet is a superior format for the files over .gz
Additional context
just do it... you'd literally save money on storage space with AWS because parquet is smaller, faster, better in every way.