Reverse prompt for line-break?

[Johnhersh]

I'm trying to limit generations to a single line. Normally with a chat the reverse prompt is something like "User:" or something of that nature. Then problem I'm having is that some models insert more fluff after the first line. Is there a way to stop generation on a new line? I tried -r "\n" and -r "\\n" but neither seem to work. I'd appreciate any advice

[KerfuffleV2]

There's a -e option:

  -e                    process prompt escapes sequences (\n, \r, \t, \', \", \\)

I think that'll affect -r. Just keep in mind it'll apply to other prompt options as well which might not be what you want. Another way to approach this is it's basically up to how you specify the options: if you can figure out how to pass a literal newline to the option then llama.cpp will use it. On Unix/Linux and bash (or zsh) ctrl-v for a literal character and then ctrl-j will let you enter a literal newline. You can also do something like "$(echo)". For Windows outside of WSL I'm not sure how to do it but there's probably a way.

[Johnhersh]

That's good. But the problem is I'm using it in server mode. So the escape is per-request which means I need to encode it in the json of the request somehow

[KerfuffleV2]

You'd probably be better off using the approach where you directly specify the literal newline or carriage return in the argument then. On Unix (Mac, Linux, etc) that's very easy. For Windows I'm afraid you're on your own unless you're using WSL (in which case it's the same as Mac/Linux).

[Johnhersh]

I'm on Linux so that part is fine! Thanks for the input I'll try this