Skip to content

Conversation

@n00b001
Copy link
Contributor

@n00b001 n00b001 commented Feb 2, 2025

Make sure to read the contributing guidelines before submitting a PR

@github-actions github-actions bot added the python python script changes label Feb 2, 2025
Copy link
Collaborator

@IMbackK IMbackK left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah that seams a bit immature.

Copy link
Collaborator

@JohannesGaessler JohannesGaessler left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My understanding is that convert_hf_to_gguf_update.py is a dev script used to update the Python code in convert_hf_to_gguf.py via manual copy-pasting. If this PR were merged as-is the two files would I think become inconsistent. A review from someone who is actually knowledgeable about the Python conversion scripts would be appreciated.

Edit: I meant "actually knowledgeable" in reference to myself, not others.

@ggerganov
Copy link
Member

The convert_hf_to_gguf_update.py script needs to be executed in order to update the vocab test input/outputs in models/ggml-vocab*.gguf.inp/.out. This change won't affect the contents of convert_hf_to_gguf.py because it is only modifying the unit tests and not the tokenizer list.

@CISC CISC merged commit f7873fc into ggml-org:master May 28, 2025
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

python python script changes

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants