Fix MacOS memory leak

[tequa]

Fix macOS memory growth caused by unbounded Metal shader cache accumulation in QtWebEngine compositor

Problem

Forum discussion from over a year ago: https://forums.ankiweb.net/t/memory-leak-over-time/50931/32

On macOS, Anki's memory footprint grows continuously while the application is idle — typically several MB per minute — and never recovers - memory could reach 20 to 30 gigs after a week or so. This behavior was observed via macOS Activity Monitor and confirmed with Instruments.

Initial investigation ruled out the obvious candidates:

• Python tracemalloc found no growing Python allocations
• leaks found no unreachable objects — the memory is reachable, not leaked in the traditional sense
• Reducing timer frequency had no effect
• The QtWebEngineProcess helper process was stable; growth was entirely in the main Python process

Root Cause (confirmed via Instruments → Allocations + Call Tree)

Every second, Qt's scene graph fires a render-sync timer, which calls into RenderWidgetHostViewQtDelegateItem — QtWebEngine's internal QML item that composites web content using OpenGL. On each sync it allocates a new QOpenGLFramebufferObject rather than reusing the existing one.

Each new FBO has a different geometry identity, so Apple's OpenGL→Metal translation layer (GLDContextRec::buildPipelineState) compiles a fresh Metal shader pipeline state for it via AGXG15XFamilyDevice::newRenderPipelineStateWithDescriptor. The compiled pipeline state is inserted into MTLCompilerFSCache::getElement — a cache that is reachable and therefore not flagged as a leak, but that grows without bound for the lifetime of the process.

Full call chain:

QTimerInfoList::activateTimers
→ QQuickWidget::event
→ QQuickRenderControl::sync
→ RenderWidgetHostViewQtDelegateItem::updatePaintNode
→ QOpenGLFramebufferObject::QOpenGLFramebufferObject   ← new FBO every 500ms
→ glDrawArrays → GLDContextRec::buildPipelineState
→ AGXG15XFamilyDevice::newRenderPipelineStateWithDescriptor
→ MTLCompilerFSCache::getElement                       ← accumulates forever

This is a Qt/Chromium interaction bug specific to the macOS OpenGL→Metal compatibility layer; it does not affect Linux or Windows.

Fix

Set --disable-gpu-compositing in QTWEBENGINE_CHROMIUM_FLAGS on macOS (in setupGL(), qt/aqt/__init__.py). This switches Chromium's page compositor from GPU mode (which creates FBOs on every render sync) to CPU/software mode (a plain bitmap uploaded as a texture). No QOpenGLFramebufferObject is created, no Metal pipeline states are compiled, and MTLCompilerFSCache never grows.

The flag is appended to any existing value of QTWEBENGINE_CHROMIUM_FLAGS rather than overwriting it, so user-set or developer flags (e.g. --remote-allow-origins) are preserved.

Confirmation of fix

Anki running for 8 hours:

Trade-off

CPU compositing means Chromium composites page layers on the CPU (Skia) rather than the GPU. Web content renders identically; CSS animations and video still work. For Anki's workload — card text, LaTeX images, simple SVG — this is indistinguishable in practice.

If visible degradation is ever reported, the fallback --use-angle=swiftshader provides a software GPU path instead.