Skip to content

Parquet: Do not compress v2 data page when compress is bad quality #8256

@mapleFU

Description

@mapleFU

Is your feature request related to a problem or challenge? Please describe what you are trying to do.

Parquet DataPageV2 allow not compress: https://github.com/apache/parquet-format/blob/master/src/main/thrift/parquet.thrift#L746

We can discard compression when size it's huge

Describe the solution you'd like

Compress but not use compressed page when it's "bad quality"

Describe alternatives you've considered

I don't know

Additional context

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementAny new improvement worthy of a entry in the changelogparquetChanges to the parquet crate

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions