Nice :) Actually a better method is to "recover" the state when you see Bob: / Alice:, as in #87

I use \n\n for now, because I replace all \n\n in ChatGPT generations by \n, so whenever you see \n\n it must be endoftext

hmm, can't really grasp what the PR is doing after reviewing, the out state doesn't get used? eg here and I don't think it's solving the same problem. Also looks to use some global state load_all_stat which is better for me not to use as I would like to run this in a production (in a very loose sense) environment

The target use case here supports arbitrary custom stopwords, this can be used in langchain to stop the LLM when it needs to. eg

Task: Find a list of cheeses
Action: search google for cheese
Observation:

in such a chain you might put Observation: as a stop word in order to stop there and inject your own observation based on the search results.

I replace all \n\n in ChatGPT generations by \n

yes! are you thinking of including an <end of turn> token or something? Is it even possible to finetune in new tokens?

I use \n\n for now, because I replace all \n\n in ChatGPT generations by \n, so whenever you see \n\n it must be endoftext

Sometimes the text generation stop even it doesn’t finish the text because it creates \n\n for example if i ask the bot to write code, it starts with the sentence “here is the python code to write X algorithm” and then “\n\n”. Actually it will write the code, but the chatrwkv will stop the generation because it sees “\n\n”

Oh actually I did notice something, based on this section of the HF article; do you use str.replace('\n\n', '\n') or re.replace('\n+', '\n')? I don't think the former is actually removing all double new lines? Might explain the behaviour @cahya-wirawan is seeing since you would might not expect to see that if the model was not trained on it?

Big GZ on the HF release by the way!