Gemma-3 generates garbage after several hundreds (maybe) of tokens

[foldl]

[foldl]

• CPU: OK
• Vulkan: an odd max_length? such as --max_length 32767
• CUDA: ??

[arch-btw]

Maybe it's the value that seems way too high:

"model_max_length": 1000000000000000019884624838656

Found in this file: https://huggingface.co/google/gemma-3-4b-it/blob/main/tokenizer_config.json

There is a thread on it here:

https://huggingface.co/google/gemma-3-4b-it/discussions/48

[foldl]

Possibly no. model_max_length in tokenizer_config.json is not used.

It is likely related to un-aligned memory access.