-
Notifications
You must be signed in to change notification settings - Fork 13.7k
mtmd-cli : allow using --jinja #16718
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
| ctx.n_past = 0; | ||
| llama_memory_seq_rm(llama_get_memory(ctx.lctx), 0, 1, -1); // keep BOS | ||
| ctx.reset_chat_history(); | ||
| llama_memory_clear(llama_get_memory(ctx.lctx), true); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This will no longer keep the BOS - just making sure it is intended.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes it's expected, as the BOS token will always be added along with the first formatted message
|
Btw @ggerganov , I'm currently having 2 small models failed in the test, using Just wondering if there are any recent changes in Metal backend that could affect this? |
|
It's possible - can you pass me a command with one of the failures to look into it? |
|
I used this command to run all tests: But you can also run one test manually: The answer should contain the world "New York" Optionally, you can also let |
|
Hmm after bisect seems like the problem comes from #16206 , no idea why my tests was OK back then. I'll investigate more on this and will let you know. |
|
Merging this because the error is unrelated to the current PR |
|
The test also fails with CPU-only build, so it should not be related to the Metal backend. |
* mtmd-cli : allow using --jinja * support -sys * implement chat_history * fix clear memory * rm -sys support, added TODO
* mtmd-cli : allow using --jinja * support -sys * implement chat_history * fix clear memory * rm -sys support, added TODO
This change is extracted from #16701
In order to have jinja support, I needed to add
chat_historytomtmd_cli_contextTested and confirmed to work with Gemma 3