py : update outdated copy-paste instructions [no ci] (ggml-org#10667)

danbev · web-flow · commit d405804be84c · 2024-12-05T09:47:55.000+02:00
This commit updates the copy-paste instruction in
convert_hf_to_gguf_update.py to reflect that convert_hf_to_gguf.py
will have already been updated with the new get_vocab_base_pre()
function when this script completes.
diff --git a/convert_hf_to_gguf_update.py b/convert_hf_to_gguf_update.py
@@ -17,7 +17,7 @@
 #
 #   python3 convert_hf_to_gguf_update.py <huggingface_token>
 #
-# - Copy-paste the generated get_vocab_base_pre() function into convert_hf_to_gguf.py
+# - The convert_hf_to_gguf.py script will have had its get_vocab_base_pre() function updated
 # - Update llama.cpp with the new pre-tokenizer if necessary
 #
 # TODO: generate tokenizer tests for llama.cpp

Original file line number	Diff line number	Diff line change
`@@ -17,7 +17,7 @@`
`17`	`17`	`#`
`18`	`18`	`# python3 convert_hf_to_gguf_update.py <huggingface_token>`
`19`	`19`	`#`
`20`		`-# - Copy-paste the generated get_vocab_base_pre() function into convert_hf_to_gguf.py`
	`20`	`+# - The convert_hf_to_gguf.py script will have had its get_vocab_base_pre() function updated`
`21`	`21`	`# - Update llama.cpp with the new pre-tokenizer if necessary`
`22`	`22`	`#`
`23`	`23`	`# TODO: generate tokenizer tests for llama.cpp`