Safetensors conversion error: KeyError: 'transformer.h.0.input_layernorm.bias'

[villesundell]

Hello everyone! 👋
I am trying to convert Poro-34B safetensors-formatted models into GGUF with:

python3 convert.py ../Poro-34B/

However, I get the following error:

Loading model file ../Poro-34B/model-00001-of-00014.safetensors
Loading model file ../Poro-34B/model-00001-of-00014.safetensors
Loading model file ../Poro-34B/model-00002-of-00014.safetensors
Loading model file ../Poro-34B/model-00003-of-00014.safetensors
Loading model file ../Poro-34B/model-00004-of-00014.safetensors
Loading model file ../Poro-34B/model-00005-of-00014.safetensors
Loading model file ../Poro-34B/model-00006-of-00014.safetensors
Loading model file ../Poro-34B/model-00007-of-00014.safetensors
Loading model file ../Poro-34B/model-00008-of-00014.safetensors
Loading model file ../Poro-34B/model-00009-of-00014.safetensors
Loading model file ../Poro-34B/model-00010-of-00014.safetensors
Loading model file ../Poro-34B/model-00011-of-00014.safetensors
Loading model file ../Poro-34B/model-00012-of-00014.safetensors
Loading model file ../Poro-34B/model-00013-of-00014.safetensors
Loading model file ../Poro-34B/model-00014-of-00014.safetensors
Traceback (most recent call last):
  File "convert.py", line 1228, in <module>
    main()
  File "convert.py", line 1161, in main
    model_plus = load_some_model(args.model)
  File "convert.py", line 1078, in load_some_model
    model_plus = merge_multifile_models(models_plus)
  File "convert.py", line 593, in merge_multifile_models
    model = merge_sharded([mp.model for mp in models_plus])
  File "convert.py", line 572, in merge_sharded
    return {name: convert(name) for name in names}
  File "convert.py", line 572, in <dictcomp>
    return {name: convert(name) for name in names}
  File "convert.py", line 547, in convert
    lazy_tensors: list[LazyTensor] = [model[name] for model in models]
  File "convert.py", line 547, in <listcomp>
    lazy_tensors: list[LazyTensor] = [model[name] for model in models]
KeyError: 'transformer.h.0.input_layernorm.bias'

Am I doing something wrong, or is this particular variation of safetensors unsupported? 🤔

(Using the most recent lama.cpp HEAD 8e672efe632bb6a7333964a255c4b96f018b9a65)

[KerfuffleV2]

convert.py is just for LLaMA architecture models. From looking at that repo, it seems like Poro uses the Bloom architecture. I think you want to use convert-hf-to-gguf.py instead.

[villesundell]

Yep, this was it, thank you! 🙏 The conversion and the resulting model worked fine 👍 I assumed that "convert.py" is the main script calling the other "converts" when needed, but I should have checked the code instead of assuming things 😅