llama.cpp support

[icoicqico]

I think llama.cpp is one of the common way to host llm locally, so it is good to have llama.cpp support. I have tried to use ollama, Lm studio settings, it doesn't work. Thanks

[ClemTB]

Hello guys,

I too have issues with lm studio, but I found out that kilo was not requesting data using streaming. Doing so lead the request to be so long that it never ends in a reasonable time with my hardware (4070 Ti, 64Gb ram).
Hope this helps or give insight to the dev team

Regards