Logging requests for debugging?

[moyix]

I'm using chat-ui with a mixtral vllm endpoint (using the OpenAI endpoint type) and trying to debug some errors I'm getting from the server ("Conversation roles must alternate user/assistant/user/assistant/..."; probably something to do with the chat prompt template?). Is there any way to dump the requests being made by chat-ui so I can see what the sequence of messages it's creating are?

I tried starting the server with the environment variable DEBUG=true set, but the logged requests don't show the actual content, just [Object]:

OpenAI:DEBUG:request http://isabella:8000/v1/chat/completions {
  method: 'post',
  path: '/chat/completions',
  body: {
    model: 'mistralai/Mixtral-8x7B-Instruct-v0.1',
    messages: [ [Object] ],
    stream: true,
    max_tokens: 32000,
    stop: [],
    temperature: 0.7,
    top_p: 1,
    frequency_penalty: 1
  },
  stream: true
} {

In case it's helpful, here's my .env.local:

MODELS=`[
  {
    "name": "mistralai/Mixtral-8x7B-Instruct-v0.1",
    "id": "mistralai/Mixtral-8x7B-Instruct-v0.1",
    "chatPromptTemplate": "<s> {{#each messages}}{{#ifUser}}[INST]{{#if @first}}{{#if @root.preprompt}}{{@root.preprompt}}\n{{/if}}{{/if}} {{content}} [/INST]{{/ifUser}}{{#ifAssistant}} {{content}}</s> {{/ifAssistant}}{{/each}}",
    "parameters": {
      "temperature": 0.7,
      "top_p": 1.0,
      "repetition_penalty": 1.0,
      "max_new_tokens": 32000,
      "stop": []
    },
    "endpoints": [{
      "type" : "openai",
      "baseURL": "http://isabella:8000/v1",
    }]
  }
]`

[moyix]

Just throwing a console.log into the source worked fine as a workaround here. The underlying issue was that the summarization feature was including a system prompt, and Mixtral doesn't support system prompts.

[hapatrick]

Could you tell me how you fixed the system prompt problem? I have exactly the same problem right now and I can't figure it out.