[WIP] Gemma3 support.#2485

Open

krammnic wants to merge 35 commits intometa-pytorch:mainfrom

krammnic:gemma3

Commits on Mar 12, 2025

add configs

Mark Obozov
authored and
Mark Obozov
committed
update recipe registry

Mark Obozov
authored and
Mark Obozov
committed
structure

Mark Obozov
authored and
Mark Obozov
committed
gemma3 component builders (without siglip)

Mark Obozov
authored and
Mark Obozov
committed
update params, include QK-norm

Mark Obozov
authored and
Mark Obozov
committed
lint + nit

Mark Obozov
authored and
Mark Obozov
committed
fix rope scalling

Mark Obozov
authored and
Mark Obozov
committed
model builders

Mark Obozov
authored and
Mark Obozov
committed
siglip component builder

Mark Obozov
authored and
Mark Obozov
committed
lint

Mark Obozov
authored and
Mark Obozov
committed
siglip component builders

Mark Obozov
authored and
Mark Obozov
committed
siglip model builder

Mark Obozov
authored and
Mark Obozov
committed
lint

Mark Obozov
authored and
Mark Obozov
committed
fixes

Mark Obozov
authored and
Mark Obozov
committed
fix context length

Mark Obozov
authored and
Mark Obozov
committed
convert weight + fixes + checkpointer

Mark Obozov
authored and
Mark Obozov
committed
fix norm

Mark Obozov
authored and
Mark Obozov
committed
head dim in qk norm

Mark Obozov
authored and
Mark Obozov
committed
rms norm

Mark Obozov
authored and
Mark Obozov
committed
rms norm

Mark Obozov
authored and
Mark Obozov
committed
fix

Mark Obozov
authored and
Mark Obozov
committed
missing file

Mark Obozov
authored and
Mark Obozov
committed
checkpoinet for 4B, 12B, 27B
krammnic
committed

Commits on Apr 16, 2025

nits + configs

Mark Obozov
authored and
Mark Obozov
committed
__init__.py

Mark Obozov
authored and
Mark Obozov
committed
docstrings

Mark Obozov
authored and
Mark Obozov
committed
update arguments in builders

Mark Obozov
authored and
Mark Obozov
committed

Commits on May 2, 2025

fixes
AsixDO
committed
fixes
AsixDO
committed

Commits on May 28, 2025

Commits on Mar 17, 2026

some fixes

Mark Obozov
authored and
Mark Obozov
committed
vision part

Mark Obozov
authored and
Mark Obozov
committed
vision part

Mark Obozov
authored and
Mark Obozov
committed
Merge branch 'main' into gemma3
krammnic
authored