Deterministic inference mode (CUDA): RMSNorm, MatMul, Attention, KV-cache#16016

Draft

creatorrr wants to merge 9 commits intoggml-org:masterfrom

julep-ai:master

Commits on Sep 13, 2025

feat: Deterministic RMSNorm
creatorrr
committed

Commits on Sep 14, 2025

Deterministic numerics: Project 01 (RMSNorm) + Project 02 (MatMul CUDA)
Codex CLI
committed
CUDA deterministic MoE (mul_mat_id): ensure batch invariance by sequential per-token per-slot matmul in deterministic mode when src1,dst are F32. Enable optional test via TEST_MATMUL_ID=1; matmul d…
Codex CLI
committed
Deterministic MoE (mul_mat_id): support F16/BF16 by promoting input columns to F32 in deterministic path; enable MoE invariance test by default.
Codex CLI
committed
Deterministic Attention (03A): plan/docs/tests\n\n- Add Project 03 plan (CUDA forward) aligned with implemented deterministic dispatch and launch policy.\n- Clarify KV stride constraint (multiples …
Codex CLI
committed
Deterministic Attention (03B): probe + fallback in det dispatcher; quantized K/V vec support (D=128 q4_0/q8_0); F16 tile fallback; MMA gated via env; tests for toggles + quantized; docs debug contr…
Codex CLI
committed
03B follow-ups: clarify F16 tile fallback vs quantized no-tile; document special head sizes unsupported in det mode; add env flag docs; add vec/MMA probe helpers comment; enable dual-arch build run…
Codex CLI
committed
Project: Progress 03B.3
Codex CLI
committed

Commits on Sep 15, 2025

[38;5;238m───────┬────────────────────────────────────────────────────────────────────────[0m
Codex CLI
committed