Implement basic memory estimation in scheduler#106

Merged

p1-0tr merged 14 commits intomainfrom

ps-memory-estimation

Jul 23, 2025

Commits on Jul 22, 2025

Implement basic memory estimation in scheduler
Piotr Stankiewicz
committed
VRAM size getter for linux
Piotr Stankiewicz
committed
VRAM size getter for windows
Piotr Stankiewicz
committed
Move VRAM size getters to a separate package
Piotr Stankiewicz
committed
Use nv-gpu-info on Windows to get VRAM size
Piotr Stankiewicz
committed
gpuinfo: Release Metal device handle in VRAM size getter
Piotr Stankiewicz
committed
inference, gpuinfo: Limit allowed models to 1 on windows/arm64 for now
Piotr Stankiewicz
committed
inference: Keep track of RAM allocated by runners
Piotr Stankiewicz
committed
inference: Fix failing llama_config unit tests
Piotr Stankiewicz
committed
inference: Fix nv-gpu-info path and wrap errors
Piotr Stankiewicz
committed
gpuinfo: Use go:build instead of obsolete +build
Piotr Stankiewicz
committed
inference: Always return 1 as VRAM size on win/arm64
Piotr Stankiewicz
committed
inference: Fallback behaviour if reading RAM/VRAM size fails
Piotr Stankiewicz
committed
inference: Fix typo in log

Piotr
and
doringeman
authored