Skip to content

Commit 26a3e4a

Browse files
authored
Do not scan manifests with no deletes when expiring. (Netflix#46)
1 parent 0342f23 commit 26a3e4a

1 file changed

Lines changed: 5 additions & 2 deletions

File tree

core/src/main/java/com/netflix/iceberg/RemoveSnapshots.java

Lines changed: 5 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -162,8 +162,11 @@ public void commit() {
162162
.onFailure((item, exc) ->
163163
LOG.warn("Failed to get deleted files: this may cause orphaned data files", exc)
164164
).run(manifest -> {
165-
// even if the manifest is still used, it may contain files that can be deleted
166-
// TODO: eliminate manifests with no deletes without scanning
165+
if (manifest.deletedFilesCount() != null && manifest.deletedFilesCount() == 0) {
166+
return;
167+
}
168+
169+
// the manifest has deletes, scan it to find files to delete
167170
try (ManifestReader reader = ManifestReader.read(ops.io().newInputFile(manifest.path()))) {
168171
for (ManifestEntry entry : reader.entries()) {
169172
// if the snapshot ID of the DELETE entry is no longer valid, the data can be deleted

0 commit comments

Comments
 (0)