Commit a247d36
committed
feat(iceberg): add delete file support to SnapshotProducer
This enables SnapshotProducer to accept and process delete files
(position deletes and equality deletes) alongside data files.
Changes:
- Add added_delete_files field to SnapshotProducer
- Add validate_added_delete_files() for delete file validation:
- Rejects delete files in V1 format
- Validates content types (PositionDeletes, EqualityDeletes)
- Requires equality_ids for equality delete files
- Validates partition spec compatibility
- Add write_delete_manifest() to write delete manifests with
ManifestContentType::Deletes
- Update manifest_file() to include delete manifests
- Update summary() to populate delete file metrics
- Enhance validate_duplicate_files() for both data and delete files
- Add comprehensive unit tests
This lays the groundwork for operations like RowDelta that need to
atomically commit both data files and delete files.1 parent a405414 commit a247d36
2 files changed
+372
-27
lines changed| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
90 | 90 | | |
91 | 91 | | |
92 | 92 | | |
| 93 | + | |
93 | 94 | | |
94 | 95 | | |
95 | 96 | | |
| |||
0 commit comments