BUG: Memory utilisation unacceptable in repositories with a long history

[Caleb-T-Owens]

From discord:

I've been using GitButler with a large git project (~100,000 commits) and I've been hitting issues with this error message: "The paging file is too small for this operation to complete.". Looking at committed memory from gitbutler-tauri.exe I can see sometimes it reserve a lot of memory. In this screenshot it had reserved 16GB of memory for a few seconds, I suspect that sometimes it reserve much more memory. I'm on Windows and I've been noticing some system instabilities recently and I suspect processes are crashing out of memory when they try to do any allocation while GitButler is reserving large amount of memory. Have you seen that before? Is there any possible mitigations? Thanks.

[Byron]

Thanks for sharing!

I have seen memory go up during processing, but never that high. It would be good to know if this is persistent, and also, by looking at the logs, which function was currently running or was long-running.

The logs can easily be opened with the Help menu.

[frleheri-havok]

I looked at my logs today and here are all the entries that have a busy time of more than one seconds sorted by time I had today: busy for more than one second.txt.

In the screenshot I shared I was integrating upstream changes (about 20 new commits upstream) while I had 4 branches in my workspace (each with at most 5 commits). The following line in the log match this:

2026-01-16T11:14:18.761340Z INFO integrate_upstream: crates\but-api\src\legacy\virtual_branches.rs:513: close time.busy=228s time.idle=26.4µs project_id=4d729777-894a-43a4-b278-b31712c63a6b resolutions=[Resolution { stack_id: 31172adc-5045-4529-bad4-3ea8f9c4667b, approach: Rebase, delete_integrated_branches: true }, Resolution { stack_id: f8084c66-a002-46ef-af32-a8d4a8c35eeb, approach: Rebase, delete_integrated_branches: true }] base_branch_resolution=None

[Byron]

Thanks a lot! It seems most operations take an extraordinary amount of time. How many files are tracked if I may ask?

Something that would be interesting is to see the peak memory of typical but commands. If there is high-memory usage for certain operations, something like /usr/bin/time -lp but ….

Thanks again.

[frleheri-havok]

Trying to make a repo using a public large repository, I've did a test with https://github.com/llvm/llvm-project/ with lane with a local branch f-branch-1 which only add an empty file at the root. Running this script:

function time {
param(
[Parameter(Mandatory)]
[string] $exe,

    [Parameter(ValueFromRemainingArguments)]
    [string[]] $args = @()
)

$sw = [System.Diagnostics.Stopwatch]::StartNew()

$p = Start-Process $exe $args -PassThru -NoNewWindow
$peakCommit = 0

while (-not $p.HasExited) {
    try {
        $p.Refresh()
        $peakCommit = [Math]::Max($peakCommit, $p.PrivateMemorySize64)
    } catch {}
    Start-Sleep -Milliseconds 50
}

$sw.Stop()

"Running:             $exe $args"
"Peak memory:         {0:N1} MB" -f ($p.PeakWorkingSet64 / 1MB)
"Peak virtual memory: {0:N1} MB" -f ($p.PeakVirtualMemorySize64 / 1MB)
"Runtime:             {0}"       -f $sw.Elapsed

}

time git status
time but status

time but branch unapply f-branch-1 -f

Give me this output:

PS D:\git\llvm-project-fullhistory> .\memUsage.ps1
On branch gitbutler/workspace
Untracked files:
(use "git add ..." to include in what will be committed)
memUsage.ps1

nothing added to commit but untracked files present (use "git add" to track)
Running: git status
Peak memory: 7.1 MB
Peak virtual memory: 4,196.8 MB
Runtime: 00:00:00.8369972
╭┄00 [Unassigned Changes]
┊ g0 A memUsage.ps1
┊
┊╭┄br [f-branch-1]
┊● 6873357 add test file
├╯
┊
┊● d2c5892 (upstream) ⏫ 5 new commits 2026-01-19 [SelectionDAG] Add TRUNCATE_SSAT_S/U and TRUNCATE_ (checked 1 minute
ago)
├╯ 5931034 (common base) [origin/main] 2026-01-19 CoverageMappingWriter.cpp: Append yet another coma
Running: but status
Peak memory: 560.1 MB
Peak virtual memory: 14,585.4 MB
Runtime: 00:00:13.8763747
no applied branches
Unapplied stack with branches 'f-branch-1' from workspace
Running: but branch unapply f-branch-1 -f
Peak memory: 669.2 MB
Peak virtual memory: 12,551.2 MB
Runtime: 00:00:13.9455863

Which give me the impression that my issues can be reproduce with llvm-project

[Byron]

Thanks a lot!

Indeed, I can reproduce it, but also I don't see an actual issue as this won't make the system run out of memory. The peak-memory here is 'just' 700MB, way more than Git, admittedly, but but status does more and uses in-memory caches.

The 12.5GB to 14.5GB are virtual memory from memory-mapped files. These won't ever make the system run out of memory, but could cause heavy disk usage if there isn't enough memory after all and pages are removed from the in-memory cache too often.

Interestingly, when running this on macOS I don't get to see these massive peak virtual memory sizes, even though the peak memory sizes are definitely too high with over a gigabyte. This will improve, but I'd hope to figure out what really is causing your system to go out of memory with 'real' memory consumption.