feat: backport devstral to v4#977

Merged

akoumpa merged 20 commits intomainfrom

akoumparouli/feat_backport_devstral_to_v4

Jan 7, 2026

Commits on Jan 5, 2026

introduce registry for NeMoAutoTokenizer
akoumpa
committed
backport tokenization to v4
akoumpa
committed
lint
akoumpa
committed
add file
akoumpa
committed
add missing enum
akoumpa
committed
add configs
akoumpa
committed
fix
akoumpa
committed
fix
akoumpa
committed
fix import
akoumpa
committed
handle tokenizers without add_bos_token
akoumpa
committed
use transformers api
akoumpa
committed
update configs
akoumpa
committed
lint
akoumpa
committed
lint
akoumpa
committed
fix
akoumpa
committed
fix
akoumpa
committed

Commits on Jan 6, 2026

Commits on Jan 7, 2026

omit backports
akoumpa
committed