Skip to content

fix: remove stream_options from xAI provider to fix Grok 4 API errors #6212

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
roomote wants to merge 1 commit into from
Closed

fix: remove stream_options from xAI provider to fix Grok 4 API errors #6212

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 0 additions & 1 deletion src/api/providers/__tests__/xai.spec.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -276,7 +276,6 @@ describe("XAIHandler", () => {
temperature: 0,
messages: expect.arrayContaining([{ role: "system", content: systemPrompt }]),
stream: true,
stream_options: { include_usage: true },
}),
)
})
Expand Down
13 changes: 8 additions & 5 deletions src/api/providers/xai.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -48,13 +48,13 @@ export class XAIHandler extends BaseProvider implements SingleCompletionHandler
const { id: modelId, info: modelInfo, reasoning } = this.getModel()

// Use the OpenAI-compatible API.
// Note: Grok models don't support stream_options parameter
const stream = await this.client.chat.completions.create({
model: modelId,
max_tokens: modelInfo.maxTokens,
temperature: this.options.modelTemperature ?? XAI_DEFAULT_TEMPERATURE,
messages: [{ role: "system", content: systemPrompt }, ...convertToOpenAiMessages(messages)],
stream: true,
stream_options: { include_usage: true },
...(reasoning && reasoning),
})

Expand All @@ -78,12 +78,15 @@ export class XAIHandler extends BaseProvider implements SingleCompletionHandler
if (chunk.usage) {
// Extract detailed token information if available
// First check for prompt_tokens_details structure (real API response)
const promptDetails = "prompt_tokens_details" in chunk.usage ? chunk.usage.prompt_tokens_details : null;
const cachedTokens = promptDetails && "cached_tokens" in promptDetails ? promptDetails.cached_tokens : 0;
const promptDetails = "prompt_tokens_details" in chunk.usage ? chunk.usage.prompt_tokens_details : null
const cachedTokens = promptDetails && "cached_tokens" in promptDetails ? promptDetails.cached_tokens : 0

// Fall back to direct fields in usage (used in test mocks)
const readTokens = cachedTokens || ("cache_read_input_tokens" in chunk.usage ? (chunk.usage as any).cache_read_input_tokens : 0);
const writeTokens = "cache_creation_input_tokens" in chunk.usage ? (chunk.usage as any).cache_creation_input_tokens : 0;
const readTokens =
cachedTokens ||
("cache_read_input_tokens" in chunk.usage ? (chunk.usage as any).cache_read_input_tokens : 0)
const writeTokens =
"cache_creation_input_tokens" in chunk.usage ? (chunk.usage as any).cache_creation_input_tokens : 0

yield {
type: "usage",
Expand Down