Where can one find gemma-3 quantized weights?

[fac2003]

I would like to run quantized gemma-3 for inference (4b,12b and 27b model variants). The project appear to support this (--quant option in run_multimodal script for instance), but the weights do not seem to be available in pytorch format on kaggle:

Could quantized weights be made available that work directly with run_multimodal --quant or I am missing some way to convert the bfloat16 weights?

[Balakrishna-Chennamsetti]

Hi @fac2003 ,

Welcome to the Google Gemma models community. Thank you for notifying us regarding this matter. We wish to clarify that the models available on Kaggle currently support various frameworks and variants. The command flag referenced in the README is a general guide and does not specifically pertain to PyTorch. Please be advised that the PyTorch variants of the quantized models are not currently available on Kaggle.
Thanks.

[fac2003]

OK, please let me know. Thanks. Fabien

[Balakrishna-Chennamsetti]

Hi, As of now the PyTorch variants of quantized models are not available in the Kaggle. However if you would like to do quantization for the any of the Gemma models you can download the original weights of the model to local on top of that you can done the quantization process for the model weights. Then you can use the quant option while running the script.

Thanks.

[sparkFeiYang]

Would you please share the quant version of Gemma-3 27b Pytorch ckpt format model? Either int8 or int4 quant version is OK. I tried a lot and failed to use other format like gguf or safetensors/bin in this Gemma Pytorch frame.