Ollama: "truncating input prompt" on edit & submit.

[sghunterfan]

When using ollama as an endpoint, "max content tokens" and "max output tokens" are being ignored, and prompt goes to "react website development" mode, idk why.

Here's how to reproduce it:

• create a new chat with a modelSpecs endpoint (configuration below)
• Send a prompt like "Give me a list of all known ceiling bearings"
• Ignore response, instead change it to "Sure thing!" and click Save & Submit

Expected output:
Sure thing! Here's the list of ...

Actual output:

Sure thing! Let me know what you need help with or if there's any specific type of content you'd like to see as an artifact. Whether it's a Mermaid diagram, React component, HTML structure, or something else entirely, I'm here to assist! 🚀

If you have a particular request in mind, please let me know the details and I'll create it for you!

Error produced in ollama logs:
time=2025-02-03T19:00:17.885Z level=WARN source=runner.go:129 msg="truncating input prompt" limit=2048 prompt=2498 keep=4 new=2048

Configuration:

Endpoints:

  custom:
    - name: 'ship'
      apiKey: "ollama"
      baseURL: "http://localhost:11434/v1"
      models:
        default:
          [
            "hf.co/bartowski/QwQ-32B-Preview-GGUF:Q6_K_L",
            "hf.co/bartowski/Qwen2.5-32B-Instruct-GGUF:Q6_K_L",
            "Qwen2.5-72B-Instruct-Q5_K_M-merged:latest",
            "Qwen2-VL-72B-Instruct-Q6_K_L-merged:latest"
          ]
        fetch: true
      titleConvo: true
      titleModel: "hf.co/bartowski/Qwen2.5-32B-Instruct-GGUF:Q6_K_L"
      summarize: false
      summaryModel: "hf.co/bartowski/Qwen2.5-32B-Instruct-GGUF:Q6_K_L"
      forcePrompt: false
      iconURL: "[truncated]" // it is set to a valid URL, and it works
      modelDisplayLabel: "SHIP"

modelSpecs:
enforce: false
prioritize: true

    - name: "qwen72b"
      label: "SHIP PRO"
      description: "[truncated]."
      iconURL: "[truncated]" 
      showIconInMenu: true
      showIconInHeader: true
      preset:
        default: false
        endpoint: "ship"
        model: "Qwen2.5-72B-Instruct-Q5_K_M-merged:latest"
        modelLabel: "SHIP"
        temperature: 0.9
        top_p: 0.9
        frequency_penalty: 1.1
        maxContextTokens: 32768
        max_tokens: 8192
        resendFiles: true
        greeting: "[truncated]"
        promptPrefix: "<|im_start|>You are a helpful assistant. You should think step-by-step."
        stop: [
          "<|im_start|>",
          "<|im_end|>"
        ]