Releasing base model checkpoints

[vishaal27]

Hi, congrats on the great paper, really enjoyed reading it!
I'm curious if its possible to release the base model checkpoints as well (after stage 2 pretraining)? This would help conduct controlled studies on pretrained base checkpoints without SFT.

Thanks and looking forward!

[lkeab]

Hi @vishaal27, thanks for your interest! We will be releasing the base model checkpoints in the next two weeks after some final organizing. Stay tuned!

[harshm2601]

What about GGUF weights?, will you guys release those weights?