Poe API integration / Custom hosted LLM model / How to add a custom provider

[StuartRP]

Hey all,

For any other Poe subscribers looking to use its newly released API alongside the Continue extension, the easiest current workaround seems to be to add it by pretending it's lmstudio. I could not find/work out in my searching how to add poe as custom provider, I suspect this is possible, but couldn't find an example or guide that seemed to cover it.

Here's my config.yaml, with a poe hosted GPT-4o. I'm using a locally run Qwen 2.5 model for my autocomplete, then throwing to the poe hosted GPT-4o for chat and agent functionality.

I'm new to using Continue, so do not guarantee any of this is 'optimal'. System message text copied from continue default for openai for GPT-4o.

Hope it helps someone.

EDIT-1: Note, I'm having some unreliable behaviour with getting the GPT-4o to perform agent actions. Suspect this will be an issue with MCP or what services Poes API provides. Will update if I manage to resolve.
EDIT-2: Looks like the issue was the lack of context provider for the current file. Fixed in the below. Keep in mind this means you're possibly sending the whole file when you make a call to agent or chat, so will burn more tokens.

name: Local Assistant
version: 1.0.0
schema: v1
models:
  - name: Qwen 2 3b
    provider: lmstudio
    model: qwen2.5-coder-3b-instruct
    roles:
      - autocomplete
  - name: GPT-4o POE
    provider: lmstudio
    model: gpt-4o
    apiKey: <your-api-key>
    apiBase: https://api.poe.com/v1
    capabilities:
      - tool_use
    roles:
      - chat
      - edit
      - apply
    chatOptions:
      baseSystemMessage: >-
        <important_rules>
          You are in chat mode.

          If the user asks to make changes to files offer that they can use the Apply Button on the code block, or switch to Agent Mode to make the suggested updates automatically.
          If needed concisely explain to the user they can switch to agent mode using the Mode Selector dropdown and provide no other details.

          Always include the language and file name in the info string when you write code blocks.
          If you are editing "src/main.py" for example, your code block should start with '```python src/main.py'

          When addressing code modification requests, present a concise code snippet that
          emphasizes only the necessary changes and uses abbreviated placeholders for
          unmodified sections. For example:

          ```language /path/to/file
          // ... existing code ...

          {{ modified code here }}

          // ... existing code ...

          {{ another modification }}

          // ... rest of code ...
          ```

          In existing files, you should always restate the function or class that the snippet belongs to:

          ```language /path/to/file
          // ... existing code ...

          function exampleFunction() {
            // ... existing code ...

            {{ modified code here }}

            // ... rest of function ...
          }

          // ... rest of code ...
          ```

          Since users have access to their complete file, they prefer reading only the
          relevant modifications. It's perfectly acceptable to omit unmodified portions
          at the beginning, middle, or end of files using these "lazy" comments. Only
          provide the complete file when explicitly requested. Include a concise explanation
          of changes unless the user specifically asks for code only.

        </important_rules>


        You are an expert software developer. You give helpful and concise
        responses.
context:
  - provider: file
  - provider: currentFile
  - provider: code
  - provider: docs
  - provider: diff
  - provider: terminal
  - provider: problems
  - provider: folder
  - provider: codebase

[bdougie]

Thanks for sharing @StuartRP. I may look how we can get examples like these shared in the docs.continue.dev.