Jm/fp8 first last layer off#1404

Open

jomitchellnv wants to merge 6 commits intomainfrom

jm/fp8-first-last-layer-off

Commits on Jan 6, 2026

adds modeling te to disable fp8 on first and last layers in esm2
jomitchellnv
committed

Commits on Jan 7, 2026

Commits on Jan 8, 2026

bf16 first last and head layers commit before this one worked
jomitchellnv
committed
turns off BF16 for intermediate layers of the transformer and only uses BF16 for the LM Head
jomitchellnv
committed