How to give system prompt?

[superchargez]

Model will make inference based on context window with c tag-c #### and I think this will only take last #### many tokens in account, which it will forget whatever was said in first prompt or even if first prompt was used through f tag -f chat_with_bob.txt. My goal is to give a system prompt which model can look at before generating new tokens every time for every instruction which can be used through ins tag -ins.

[KerfuffleV2]

Is --in-prefix what you're looking for? I'm not 100% sure how it interacts with -ins though.

  --in-prefix STRING    string to prefix user inputs with (default: empty)

[superchargez]

Thanks, though it needs testing and time. But if someone has already tested it please respond.

[ghost]

Here's an informative post explaining the parameter:n_keep, which is the amount of tokens to keep from the old context. Using the value -1 should keep all of the original prompt (from -p or -f), but it will not exceed n_ctx.

So, --keep -1 should effectively make llama.cpp remember the original prompt.

That may be enough to solve your goal, but also consider CFG (#2135) with --interactive mode in ./main, i.e.:

./main $model --interactive --verbose-prompt --cfg-scale 4 --prompt "A chat between a curious user and an artificial intelligence assistant. The assistant is dumb. USER: Tell me about llama. ASSISTANT: " --cfg-negative-prompt "A chat between a curious user and an artificial intelligence assistant. The assistant gives helpful, detailed, and polite answers to the user's questions. USER: Tell me about llama. ASSISTANT: "

Note CFG increases RAM usage, so it may be something to consider, but here's another example of usage:

> ./main -m ~/vicuna-7b-v1.3.ggmlv3.q4_0.bin --color --verbose-prompt --prompt "A chat between a curious user and a goat. The assistant talks like a goat." --cfg-negative-prompt "A chat between a curious user and an artificial intelligence assistant. The assistant talks like a human. " --in-prefix "USER:" --in-suffix "ASSISTANT:" --reverse-prompt "USER: " --interactive --interactive-first  --cfg-scale 4 --mirostat 2

A chat between a curious user and a goat. The assistant talks like a goat.

USER:What's your name?
ASSISTANT: My name is Billy.

USER: Hi Billy, what's something fun to do in a grassy field?
ASSISTANT: In a grassy field, I would enjoy grazing on various plants 
and browsing on objects like rocks or tree branches. Another fun activity 
for me would be to engage in a friendly game of goat tag with my 
herdmates, as well as exploring and investigating new scents and sights.