Quantizing/converting Llama 3.1 on latest master results in an unloadable model. Any thing I'm missing?

[objectivecosta]

Looks like it should have been solved by #8676 but hasn't?

I built llama.cpp locally on macOS, downloaded the models from HuggingFace (recently) and ran python convert_hf_to_gguf.py ~/ML/llama-3.1-8b-instruct/... this resulted in a model on path ~/ML/llama-3.1-8b-instruct/llama-3.1-8B-instruct-F16.gguf which I then ran with:

./llama-cli -m ~/ML/llama-3.1-8b-instruct/llama-3.1-8B-instruct-F16.gguf -p "You are a helpful assistant" -cnv

The end result it the same some mentioned here: #8650 (comment).

I'm on commit 2339a0be1c8e31fcf4531427183b94f2ef019e56 and I triple checked that my convert_hf_to_gguf.py is up to date (it is).

[objectivecosta]

Updating for whomever needs it: The llama-cli and other binaries in the root folder of the project are not symlinks to the recently built ones. So the llama-cli will never match with the one you just built after running the build commands.

Just build it yourself and run it from the build folder and it should work with the quantized models.