Skip to content

Support zstd compression #7457

@hutch3232

Description

@hutch3232

I really appreciate how fread and fwrite support gzip compression with basically no effort on the users part. I think it would be awesome if zstandard was also supported. https://github.com/facebook/zstd

Various online benchmarks have shown compression and decompression to be faster than gzip and with better compression ratios. Another big benefit is that zstd, unlike gzip, is splittable. That means you could take these CSVs and read them with Spark in parallel.

Thanks for considering!

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions