Avoid redundant archive rewrites when content is unchanged - #23
Open
triatomic wants to merge 3 commits into
Open
Avoid redundant archive rewrites when content is unchanged#23triatomic wants to merge 3 commits into
triatomic wants to merge 3 commits into
Conversation
A tab flagged unsaved (e.g. edit-then-revert, or an encode round-trip that yields identical bytes) still called edit_file, marking the entry modified and forcing a full repack on the next save. Compare the new bytes against the stored content first and only edit when they differ.
Adding a file whose name already exists reported it as overwritten and re-wrote the entry even when the bytes matched, dirtying the archive and forcing a full repack. Compare incoming bytes against the stored content in add_file_to_archive and _merge_archives; matching content yields a new AddOutcome.UNCHANGED that leaves the archive untouched. The add summary now reports the unchanged count and lists them in the details.
Save unconditionally re-wrote the whole BIG even with nothing modified. When writing back to the same file and modified_entries is empty, skip the repack entirely. Save As still always writes to its new path.
ClementJ18
requested changes
Jun 21, 2026
| writing_in_place = self.path is not None and os.path.normcase( | ||
| os.path.abspath(path) | ||
| ) == os.path.normcase(os.path.abspath(self.path)) | ||
| if writing_in_place and not self.archive.modified_entries: |
Owner
There was a problem hiding this comment.
I have a bit of an edge case for consideration, this breaks the flow in which you want to overwrite a .big that has been modified by another process. This has happened to me in the past, not often but it has. I think in the end I would prefer to give the control to the users on what they want to do. I like the rest of the PR, I'd just like the part that takes control of saves away from the user reverted.
I think the rest of the PR is genuinely a time saver since the user can more easily review a batch add without getting drowned out in the noise of no-write edits.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A
.bigsave is a full repack of the entire archive (pyBIG has no in-placepatching), so the amount written on save is the whole archive regardless of how
little changed. These three changes remove writes that would produce a
byte-identical result — they don't (and can't) make a genuine edit cheaper.
Changes
Skip dirtying an entry when a tab save changes nothing. A tab flagged
unsaved (edit-then-revert, or an encode round-trip that yields identical
bytes) still called
edit_file, marking the entry modified and forcing afull repack on the next save. A new
GenericTab.commit_to_archivehelpercompares the new bytes against the stored content and only edits when they
differ; the text/cah/map tabs use it.
Treat identical re-adds as unchanged instead of overwriting. Adding a
file whose name already exists was classified "overwritten" purely on the
name, and re-wrote the entry even when the bytes matched — dirtying the
archive.
add_file_to_archiveand_merge_archivesnow compare incomingbytes against the stored content; matching content yields a new
AddOutcome.UNCHANGEDthat leaves the archive untouched. The add summaryreports the unchanged count (
_tallynow returns a new/overwritten/unchangedtriple) and lists them in an "Unchanged files" section.
Skip the full repack when the archive has no changes. Save
unconditionally re-wrote the whole
.bigeven with nothing modified. Whenwriting back to the same file and
modified_entriesis empty, the repack isskipped. Save As still always writes to its new path.
Notes
entry for
InDiskArchive) — far cheaper than the repack it avoids; no eagerfull-archive hashing.