[paimon] Fix union read paimon dv table issue

[luoyuxia]

Purpose

Linked issue: close #xxx

Brief change log

Tests

API and Format

Documentation

[Copilot]

Pull request overview

This PR fixes union read issues for Paimon deletion-vectors enabled tables by implementing a mechanism to track multiple lake snapshots and distinguish between tiered and readable snapshots. The changes enable proper handling of L0 files in Paimon DV tables, preventing data loss during union reads.

Key changes:

• Introduces readable snapshot concept separate from tiered snapshot for Paimon DV tables
• Implements snapshot retention policy with configurable earliest snapshot ID to keep
• Adds logic to compute readable offsets based on L0 file compaction status

Reviewed changes

Copilot reviewed 46 out of 46 changed files in this pull request and generated no comments.

Show a summary per file

File	Description
FlussApi.proto	Added optional fields for snapshot_id, readable flag, ignore_previous_table_offsets, and earliest_snapshot_id_to_keep
LakeCommitResult.java	New class encapsulating commit results with readable snapshot information
CommitLakeTableSnapshotsData.java	Refactored from CommitLakeTableSnapshotData to support per-table snapshot metadata
DvTableReadableSnapshotRetriever.java	New utility to compute readable snapshots and offsets for DV tables
PaimonDvTableUtils.java	New utility for tracking L0 file lifecycle in Paimon snapshots
LakeTableHelper.java	Enhanced with snapshot retention logic based on earliestSnapshotIDToKeep
PaimonLakeCommitter.java	Updated to retrieve and return readable snapshot information for DV tables
Admin.java/FlussAdmin.java	Added getLakeSnapshot(snapshotId) and getReadableLakeSnapshot() methods
Test files	Comprehensive tests for DV table readable snapshot retrieval and retention

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.