Skip to content

Conversation

@ngxson
Copy link
Member

@ngxson ngxson commented May 16, 2024

This PR add correct type definitions for GGUF split.*, and use them in ggufAllShards for sanity checks.

The sanity checks does not really serves real-life purpose, since it is very unlikely that users mess up with file name. For now, it mostly to demonstrate how to use these metadata keys.

This PR also modify output format of ggufAllShards. Because metadata is saved in the first split only, we can simply discard metadata of other splits.

For more details, please have a look on gguf-split source code:

https://github.com/ggerganov/llama.cpp/blob/24ecb58168dce81646c2ed425690a106591c8c6d/examples/gguf-split/gguf-split.cpp#L223-L225

@ngxson ngxson requested review from julien-c and mishig25 as code owners May 16, 2024 22:02
Copy link
Member

@julien-c julien-c left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe of interest to @phymbert too

@julien-c
Copy link
Member

and btw related a bit to ggml-org/ggml#826

@coyotte508
Copy link
Member

what's the status of this? (btw feel free to merge PRs in huggingface/gguf)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants