-
I saw some other workarounds for Ollama and others (replacing the model name, etc) but those didn't work out for me (vllm can also be configured to do something similar). Thanks |
Beta Was this translation helpful? Give feedback.
Replies: 1 comment
-
Solution found: (actually it was on VLLM part but leaving here for future readers) |
Beta Was this translation helpful? Give feedback.
Solution found: (actually it was on VLLM part but leaving here for future readers)
vllm-project/vllm#22386