Skip to content

Comments

Skip large files when building diffs#52

Merged
ahkohd merged 1 commit intomainfrom
patch/skip-large-files
Jan 4, 2026
Merged

Skip large files when building diffs#52
ahkohd merged 1 commit intomainfrom
patch/skip-large-files

Conversation

@ahkohd
Copy link
Owner

@ahkohd ahkohd commented Jan 4, 2026

This pull request introduces logic to efficiently handle large files in multi-file diffs by checking file sizes before loading their content, and refactors how file content is read from the filesystem and git objects. The main goal is to avoid loading or diffing files larger than 2MB, treating them as binary for performance and safety. Several helper functions are added for querying file sizes from both the filesystem and git history, and the code is refactored to consistently use these helpers throughout the diffing logic.

Large file handling and binary detection:

  • Introduced a 2MB size limit (MAX_FILE_BYTES) for files in MultiFileDiff; files exceeding this size are treated as binary and their content is not loaded. [1] [2]
  • Refactored all file content loading code paths (working directory, staged, committed, and ranged diffs) to check file size before reading, using new helper methods. [1] [2] [3] [4] [5] [6]

New helper functions for file size:

  • Added get_file_at_commit_size and get_staged_content_size in git.rs to efficiently query file sizes in git history and index without loading full content. [1] [2]
  • Added new helper methods in MultiFileDiff for reading file content or treating as binary based on size, for both git and filesystem sources.

Refactoring and code simplification:

  • Removed the old diff_from_bytes method in favor of in-place logic that checks for binary files and computes diffs only for files under the size limit. [1] [2]
  • Updated all usages of raw byte reading from git and filesystem to use the new helpers, improving consistency and maintainability. [1] [2] [3] [4] [5] [6]

Minor cleanups:

  • Removed an unused import in multi.rs.

@ahkohd ahkohd merged commit 15aa7fa into main Jan 4, 2026
1 check passed
@ahkohd ahkohd deleted the patch/skip-large-files branch January 4, 2026 20:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant