Problems with "Chat Prompt Template"

[rozek]

Hi all,

I have a very simple flow with a "ChatLocalAI" LLM, a "Chat Prompt Template" and an "LLM Chain".

My problem is that, although I have entered both a "System Message" and a "Human Message", the LLM server (LM Studio in my case) only gets a (strange) user message:

  "messages": [
    {
      "role": "user",
      "content": "[{\"lc\":1,\"type\":\"constructor\",\"id\":[\"langchain\",\"schema\",\"SystemMessage\"],\"kwargs\":{\"content\":\"Perform the instructions to the best of your ability.\",\"additional_kwargs\":{}}},{\"lc\":1,\"type\":\"constructor\",\"id\":[\"langchain\",\"schema\",\"HumanMessage\"],\"kwargs\":{\"content\":\"### Instruction: who was Joseph Weizenbaum?\\n### Response:\",\"additional_kwargs\":{}}}]"
    }
  ]

Is there s.th. I'll have to configure or add in order to get a "proper" LLM request?

[rozek]

well, what I found out so far: it seems to be a problem with "ChatLocalAI" rather than "Chat Prompt Template" as everything seems to work fine when using the "ChatOpenAI" LLM with an appropriate "BasePath"

[rozek]

I meanwhile found the problem. In contrast to "ChatOpenAI" (which works fine), "ChatLocalAI" uses

import { OpenAIChat } from 'langchain/llms/openai'
import { OpenAIChatInput } from 'langchain/chat_models/openai'

and

        const model = new OpenAIChat(obj, { basePath })

After replacing these lines by

import { ChatOpenAI, OpenAIChatInput } from 'langchain/chat_models/openai'

and

        const model = new ChatOpenAI(obj, { basePath })

the prompt waas interpreted properly.

[HenryHengZJ]

oh so using const model = new ChatOpenAI(obj, { basePath }) has better response than using const model = new OpenAIChat(obj, { basePath })?

would you mind sharing your flow to test it out?

[rozek]

Sure, no problem.

Actually, I now use my own Chat Model node (called ChatLMStudio) which is part of a minimal chat flow with a "Prompt Template" and an "LLM Chain" - see attachment.

You may use it with any other OpenAI compatible server by changing "Base Path" and "Model Name" etc.

My typical question to start with is "who was Joseph Weizenbaum?"

LM Studio Test Chatflow.txt

[rozek]

Oops,

you should also replace

       this.baseClasses = [this.type, 'BaseChatModel', ...getBaseClasses(OpenAIChat)]

by

        this.baseClasses = [this.type, ...getBaseClasses(ChatOpenAI)]