Replies: 1 comment
-
I also get this error (ollama 0.4.2) when using llama2 (with 128k token ctx). |
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
-
Using Llama CPP locally an error occurs after many questions:
The error occurs always when the generated responses exceed approximately 3500 Tokens circa 16000 Characters.
The context is probably too big. How to adjust this? I don't want to enlarge the context but rather have a context only on the last the last 1 or 2 questions.
The error can be simply reproduced by doing something like this:
PC used - Apple M1 Max, 32Gb
Beta Was this translation helpful? Give feedback.
All reactions