Skip to content

Fix potential corrupt extracts when canceled #267

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
bdon merged 2 commits into from
Dec 25, 2025
Merged

Fix potential corrupt extracts when canceled #267

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
da1ac73
Fix potential corrupt extracts when canceled
bdon Dec 25, 2025
44261f5
more error handling
bdon Dec 25, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 18 additions & 4 deletions pmtiles/extract.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -462,9 +462,10 @@ func Extract(ctx context.Context, logger *log.Logger, bucketURL string, key stri
return err
}

outfile.Truncate(127 + int64(len(newRootBytes)) + int64(header.MetadataLength) + int64(len(newLeavesBytes)) + int64(totalActualBytes))

_, err = outfile.Write(headerBytes)
// set the file size and write empty space for the header for now
// see comment below
outfile.Truncate(HeaderV3LenBytes + int64(len(newRootBytes)) + int64(header.MetadataLength) + int64(len(newLeavesBytes)) + int64(totalActualBytes))
_, err = outfile.Write(make([]byte, HeaderV3LenBytes))
if err != nil {
return err
}
Expand All @@ -486,7 +487,10 @@ func Extract(ctx context.Context, logger *log.Logger, bucketURL string, key stri
return err
}

outfile.Write(metadataBytes)
_, err = outfile.Write(metadataBytes)
if err != nil {
return err
}

// 10. write the leaf directories
_, err = outfile.Write(newLeavesBytes)
Expand Down Expand Up @@ -559,6 +563,16 @@ func Extract(ctx context.Context, logger *log.Logger, bucketURL string, key stri
if err != nil {
return err
}

// Write the header when finishing,
// otherwise a extract cancelled during the tile download section
// will appear valid with "pmtiles verify"
// even though the tile contents are corrupted.
outfile.Seek(0, io.SeekStart)
_, err = outfile.Write(headerBytes)
if err != nil {
return err
}
}

logger.Printf("Completed in %v with %v download threads (%v tiles/s).\n", time.Since(start), downloadThreads, float64(len(reencoded))/float64(time.Since(start).Seconds()))
Expand Down
Loading