Enhancement: X.ai

[Dual-0]

What features would you like to see added?

xAI launches an API
Documentation

Try it with custom endpoint. Fails because of the name of user role.

More details

1. test with /v1/chat/completions

Docs

example POST /v1/chat/completions

{
  "messages": [
    {
      "role": "system",
      "content": "You are a test assistant."
    },
    {
      "role": "user",
      "content": "Testing. Just say hi and nothing else."
    }
  ],
  "model": "grok-beta"
}

example Response

{
  "choices": [
    {
      "finish_reason": "stop",
      "index": 0,
      "message": {
        "content": "Hi",
        "role": "assistant"
      }
    }
  ],
  "created": 1728646283,
  "id": "c6f2d009-77ca-40d9-9de5-6d19716e1b4d",
  "model": "grok-beta",
  "object": "chat.completion",
  "system_fingerprint": "fp_9877325691",
  "usage": {
    "completion_tokens": 2,
    "prompt_tokens": 25,
    "total_tokens": 27
  }
}

My test config:

    - name: "X.ai"
      apiKey: "${XAI_API_KEY}"
      baseURL: "https://api.x.ai/v1"
      models:
        default: ["grok-beta"]
        fetch: true
      titleConvo: true
      titleModel: "grok-beta"
      modelDisplayLabel: "X.ai"

Failure:

2024-10-22 13:13:04 warn: [OpenAIClient.chatCompletion][stream] API error
2024-10-22 13:13:04 error: [handleAbortError] AI response error; aborting request: 400 "Only messages of role 'user' can have a name."

2. test with /v1/completions

Docs

example POST /v1/completions

{
  "prompt": "1, 2, 3, 4, ",
  "model": "grok-beta",
  "max_tokens": 3
}

example Response

{
  "choices": [
    {
      "finish_reason": "length",
      "index": 0,
      "text": "5, "
    }
  ],
  "created": 1728652460,
  "id": "3a34a6a3-82b2-46d9-874d-99dbca084813",
  "model": "grok-beta",
  "object": "text_completion",
  "system_fingerprint": "fp_8933231290",
  "usage": {
    "completion_tokens": 3,
    "prompt_tokens": 12,
    "total_tokens": 15
  }
}

My test config:

    - name: "X.ai"
      apiKey: "${XAI_API_KEY}"
      baseURL: "https://api.x.ai/v1/completions"
      models:
        default: ["grok-beta"]
        fetch: false
      titleConvo: true
      titleModel: "grok-beta"
      directEndpoint: true
      modelDisplayLabel: "X.ai"

Failure:

unfortunately the answer is incorrectly formatted... maybe because of the stop command?

2024-10-22 13:31:35 warn: [OpenAIClient.chatCompletion][create] API error
2024-10-22 13:31:35 error: [OpenAIClient.chatCompletion] Unhandled error type Error: 422 Failed to deserialize the JSON body into the target type: prompt: data did not match any variant of untagged enum SampleContent at line 14 column 1
2024-10-22 13:31:35 error: [OpenAIClient] There was an issue generating the title with the completion method Error: 422 Failed to deserialize the JSON body into the target type: prompt: data did not match any variant of untagged enum SampleContent at line 14 column 1

Which components are impacted by your request?

Endpoints

Pictures

Code of Conduct

• I agree to follow this project's Code of Conduct

[danny-avila]

will be made compatible after this change: #4632