add the handle to the model during loadmodel

[HarvieKrumpet]

Please add this to your loadmodel routine

public unsafe nint LoadModel(string modelPath, LlmModelOptions? modelOptions = default, Action<float>? progressCallback = default, bool waitForMainLoop = true)
{
...
            nint handle = _model.Create(() => llama_load_model_from_file(modelPath, mparams), llama_free_model);
...
return handle;

So that I have access to the actual loaded model for running additional functions (such as load meta vars...)

[dranger003]

It's already there, use ModelNativeHandle and ContextNativeHandle.

[HarvieKrumpet]

right! I forgot I was already using it. sorry about that. and thanks for fixing that last issue so quickly. been working like charm now.

btw, I now get these warning messages from llama.cpp. everything appears to be working. but I guess I am not in the loop to keep up with what this is supposed to be telling me.

llm_load_vocab: control token: 128095 '<|reserved_special_token_90|>' is not marked as EOG
llm_load_vocab: control token: 128188 '<|reserved_special_token_183|>' is not marked as EOG
llm_load_vocab: control token: 128127 '<|reserved_special_token_122|>' is not marked as EOG
llm_load_vocab: control token: 128116 '<|reserved_special_token_111|>' is not marked as EOG
llm_load_vocab: control token: 128133 '<|reserved_special_token_128|>' is not marked as EOG
llm_load_vocab: control token: 128152 '<|reserved_special_token_147|>' is not marked as EOG
llm_load_vocab: control token: 128193 '<|reserved_special_token_188|>' is not marked as EOG
llm_load_vocab: control token: 128098 '<|reserved_special_token_93|>' is not marked as EOG

[HarvieKrumpet]

also get this...

llm_load_tensors: tensor 'token_embd.weight' (q4_K) (and 722 others) cannot be used with preferred buffer type CUDA_Host, using CPU instead

[dranger003]

I think these are just warnings and they may have been added recently but I'm not entirely sure.

[HarvieKrumpet]

I been using the meta parameters as an integral part of my setup. but I now realize there is no standardization on the meaning of the values. I cannot even rely on the model name in those parameters. They "could" be quite valuable to hint to my setup what a model can (and cannot do). Some even list the various merged models that the model was built from. Who has influence on what the key values are named?

[dranger003]

The values are set when the model is converted (i.e. they are coming from the source model). During the conversion to GGUF you can also override some of the values using the conversion script. And there are some values that can be set using gguf-py/scripts/gguf_set_metadata.py after the conversion as well.