not enough space in the buffer (needed 442368, largest block available 290848) GGML_ASSERT:

[adi1999]

Just for those facing this issue while trying to run a local LLM - revert llama-cpp-python to 0.2.0 , pip install llama-cpp-python==0.2.0

[fynnfluegge]

Hey @adi1999, good catch! Would it makes sense to add the explicit version to the installation command in llm.py? Do you know what is the cause of this issue?

[adi1999]

Hey @fynnfluegge , maybe coz the model that we're using is trained on bigger context size, while llama.cpp sets 512 by default, so either we can increase it manually, or using this specific version is a workaround.
Btw, you've done awesome work :)

[fynnfluegge]

Alright, thank you @adi1999 !