File tree Expand file tree Collapse file tree 1 file changed +10
-6
lines changed Expand file tree Collapse file tree 1 file changed +10
-6
lines changed Original file line number Diff line number Diff line change @@ -529,19 +529,22 @@ Then you'll need to use a custom chat handler to load the clip model and process
529529You can also pull the model from the Hugging Face Hub using the ` from_pretrained ` method.
530530
531531``` python
532- >> > from llama_cpp import Llama
533- >> > from llama_cpp.llama_chat_format import MoondreamChatHandler
534- >> > chat_handler = MoondreamChatHandler.from_pretrained(
532+ from llama_cpp import Llama
533+ from llama_cpp.llama_chat_format import MoondreamChatHandler
534+
535+ chat_handler = MoondreamChatHandler.from_pretrained(
535536 repo_id = " vikhyatk/moondream2" ,
536537 filename = " *mmproj*" ,
537538)
538- >> > llm = Llama.from_pretrained(
539- repo_id = " vikhyatk/moondream2"
539+
540+ llm = Llama.from_pretrained(
541+ repo_id = " vikhyatk/moondream2" ,
540542 filename = " *text-model*" ,
541543 chat_handler = chat_handler,
542544 n_ctx = 2048 , # n_ctx should be increased to accomodate the image embedding
543545)
544- >> > llm.create_chat_completion(
546+
547+ respoonse = llm.create_chat_completion(
545548 messages = [
546549 {
547550 " role" : " user" ,
@@ -553,6 +556,7 @@ You can also pull the model from the Hugging Face Hub using the `from_pretrained
553556 }
554557 ]
555558)
559+ print (response[" choices" ][0 ][" text" ])
556560```
557561
558562** Note** : Multi-modal models also support tool calling and JSON mode.
You can’t perform that action at this time.
0 commit comments