backend does not see the character and lora

[AmyTeaCode]

When I access the api and ask for the name that was configured through the character, I get something like AI ASSISTANT in response and it behaves differently, although the name was configured for Alice. And when I access the web panel, I get the name Alice and she behaves as prescribed in character context and lora. Maybe I'm doing something wrong

[LostRuins]

What frontend are you using? The API does not know the name of the character by default. You need to include that in the submitted context.

[LIDERCREW]

Hello, so we're not using any frontend, because we need to interact with an API through code. Is there any way to apply character context in this scenario. If there is then can you show it please or share links with examples?

[LostRuins]

Sure, here's a very simple example of using the KoboldCpp API to continue a conversation between the user and Gordon Ramsay, the AI will generate his next response

{"n":1,"max_context_length":2048,"max_length":150,"rep_pen":1.07,"temperature":0.7,"top_p":0.92,"top_k":100,"trim_stop":true,"prompt":"[The following is an interesting chat message log between User and Gordon Ramsay.]\n\nUser: Hi.\nGordon Ramsay: Hello.\nUser: What do you think of pineapple on a pizza?\nGordon Ramsay:","stop_sequence":["User:","\nUser ","\nGordon Ramsay: "],"use_default_badwordsids":false,"bypass_eos":false}

You can test it at http://localhost:5001/api and also view the swagger api documentation

[LIDERCREW]

Sure, here's a very simple example of using the KoboldCpp API to continue a conversation between the user and Gordon Ramsay, the AI will generate his next response

{"n":1,"max_context_length":2048,"max_length":150,"rep_pen":1.07,"temperature":0.7,"top_p":0.92,"top_k":100,"trim_stop":true,"prompt":"[The following is an interesting chat message log between User and Gordon Ramsay.]\n\nUser: Hi.\nGordon Ramsay: Hello.\nUser: What do you think of pineapple on a pizza?\nGordon Ramsay:","stop_sequence":["User:","\nUser ","\nGordon Ramsay: "],"use_default_badwordsids":false,"bypass_eos":false}

You can test it at http://localhost:5001/api and also view the swagger api documentation

Thanks for answering. We have another thing that needs to clarify. Is there any way to apply lora except for the merging adapter and base model?

[LostRuins]

Yes, LoRA can be loaded separately in KoboldCpp with --lora or --sdlora (for Stable Diffusion). This is not recommended however, due to reduced precision when combined with quantization, as well as poor GPU offload support.

You are strongly recommended to pre-merge any desired loras.

[LIDERCREW]

Yes, LoRA can be loaded separately in KoboldCpp with --lora or --sdlora (for Stable Diffusion). This is not recommended however, due to reduced precision when combined with quantization, as well as poor GPU offload support.

You are strongly recommended to pre-merge any desired loras.

thanks for everything!