Skip to content

timeline compaction blocked caused by the single archived file being too large #17783

@TheR1sing3un

Description

@TheR1sing3un

Bug Description

What happened:

timeline compaction blocked caused by the single archived file being too large

What you expected:

Even if an archved file is too large, we can skip this file and start judging whether it can be compact from the next file instead of ending it directly

Steps to reproduce:
I write a ut to verify it.

Image Image

Environment

Hudi version:
Query engine: (Spark/Flink/Trino etc)
Relevant configs:

Logs and Stack Trace

No response

Metadata

Metadata

Assignees

No one assigned

    Labels

    type:bugBug reports and fixes

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions