Skip to content

Conversation

@N0rthHXD
Copy link

fixes issue #12264

Added support for n_predict values of -2.
n_predict = -2 this parameter ensures that n_predicts is the context length, but it is not implemented.

@ngxson
Copy link
Collaborator

ngxson commented Mar 12, 2025

Maybe duplicate of #12323 ?

@N0rthHXD
Copy link
Author

Maybe duplicate of #12323 ?

I think #12323 lacking some judgment logic, and n_ctx error by @ggerganov's review. If you think it is a duplicate, I can communicate my algorithm and close the PR.

@N0rthHXD
Copy link
Author

https://github.com/ggml-org/llama.cpp/pull/12323/files#:~:text=if%20((params.,%7D
if global_params.n_predict == 16, params.n_predict = -2, this logic will return false, that's error.

@ngxson
Copy link
Collaborator

ngxson commented Mar 13, 2025

Tbh this n_predict == -2 is just too messy, as we have no notion of global value or request value is more prioritized.

I personally want to remove it since we already had --no-ctx-shift that does the same thing, cc @ggerganov too

@ggerganov
Copy link
Member

I personally want to remove it since we already had --no-ctx-shift that does the same thing, cc @ggerganov too

Makes sense. Let's remove it to make things simpler.

@N0rthHXD
Copy link
Author

n_predict == -2 is messy indeed.

@ngxson
Copy link
Collaborator

ngxson commented Mar 13, 2025

superseded by #12364

@ngxson ngxson closed this Mar 13, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants