Skip to content

Count model parameters and size using only the GGUF tensors #10285

@slaren

Description

@slaren

Discussed in #10274

The number of parameters and size of the model currently is calculated from the tensors created after the model is loaded, which in some cases may contain duplicated tensors, resulting in an inaccurate and inconsistent reporting of the model size. To address this, llama_model_n_params and llama_model_size should be modified to return the value as calculated in llama_model_loader::n_elements and n_bytes, which could be stored in llama_model while loading the model.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workinggood first issueGood for newcomers

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions