prompt syntax when using stream

[iplayfast]

I'm trying to interface https://github.com/paul-gauthier/aider.git with llama's stream (instead of gtp).

./server -t 16 --host 127.0.0.1 --port 8080 -m AllModels/TheBloke_CodeLlama-13B-oasst-sft-v10-GGUF/codellama-13b-oasst-sft-v10.Q8_0.gguf

stream is working and from a web browser it gives some options.
One of the options is a prompt template

{{prompt}}
{{history}}
{{char}}:

Suppose I wanted the prompt to be "you are a 1960's hippie that likes to use flowery language"
If I use

{{prompt}}
you are a 1960's hippie that likes to use flowery language
{{history}}
{{char}}:

It's ignored,
If I use

{{prompt}
you are a 1960's hippie that likes to use flowery language}
{{history}}
{{char}}:

Then it works, but keeps on conversing with itself.

2nd question, is that stream seems slow, compared to textgen-web-ui running it. I think the difference is the GPU memory can be set in text...ui, how do I set that in stream?

[staviq]

If you want to use curl directly, or something like that for streaming with server, the web prompt template doesn't really apply, those {{}} keywords get replaced in the web interface with appropriate parts of the conversation.

As for 2. server accepts most parameters for main, so if you run ./main -h you'll get a description of all parameters, including GPU use parameters, and you can use them for server

[iplayfast]

I'm trying to hook it up to aider which normally interfaces with the openai gpt. Thanks for your help.

[iplayfast]

Output from main -h doesn't seem to have any options for GPU use parameters. What should I be looking at?

[staviq]

-ngl or --n-gpu-layers is for offloading to GPU, it must be in the output of main -h, if it isn't, you compiled it without GPU support.