Merged
Conversation
1. Make the template not an optional for apply_chat_template. This ensures you don't accidentally use the chatml template. 2. Improve performance for the expected case of using get_chat_template by returning a new LlamaChatTemplate struct that internally stores the string as a CString. Unless you try to explicitly create a copy or print, there's no extra copy into a Rust string that's created. Similarly, get_chat_template -> apply_chat_template no longer copies the template string. 3. Improve documentation including documentating what the add_ass parameter does and suggestions on what values you probably want to use. Additionally, I've made get_chat_template and apply_chat_template docs refer to one another to make it easier to discover how to use this.
MarcusDunn
approved these changes
Feb 14, 2025
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This resolves #640 and #641
@MarcusDunn hope you don't mind, but instead of separate &str vs &CStr, I've instead created a new LlamaChatTemplate struct that internally stores the CString with convenience methods to easily retrieve &str/String from it or to construct it from a &str. I found that to be a bit cleaner.