stream-text: incompatible with `response_format: { type: 'json' }`

[kingyue737]

The stream-text is not returning content in chunks as expected — instead, it returns everything all at once at the end.

No such issue in v0.2.2

Reproduction

import { streamText } from '@xsai/stream-text'
import { env } from 'node:process'

export default defineEventHandler(async (event) => {
  const { content } = await readValidatedBody(
    event,
    z.object({
      content: z.string(),
    }).parse,
  )

  const { textStream } = await streamText({
    apiKey: env.OPENAI_API_KEY!,
    baseURL: env.OPENAI_BASE_URL!,
    messages: [
      { role: 'system', content: AVeryLongContent },
      { role: 'user', content },
    ],
    model: 'deepseek-chat',
    response_format: { type: 'json_object' },
  })

  for await (const textPart of textStream) {
    console.log(textPart)
  }

  // return textStream
})

[kwaa]

I can't seem to reproduce.

const { textStream } = await streamText(...)
const date = Date.now()

for await (const textPart of textStream) {
  console.log(Date.now() - date, textPart)
}

[kwaa]

OK, you shouldn't specify response_format.

const { textStream } = await streamText({
  apiKey: env.OPENAI_API_KEY!,
  baseURL: env.OPENAI_BASE_URL!,
  messages: [
    { role: 'system', content: AVeryLongContent },
    { role: 'user', content },
  ],
  model: 'deepseek-chat',
- response_format: { type: 'json_object' },
})

[kingyue737]

Thank you for looking into this! I think this is a legitimate practice to generate formatted JSON text with the Deepseek API 🤔. See here: https://api-docs.deepseek.com/guides/json_mode

[kwaa]

Thank you for looking into this! I think this is a legitimate practice to generate formatted JSON text with the Deepseek API 🤔. See here: api-docs.deepseek.com/guides/json_mode

If you need to generate an object that conforms to a specific format, use @xsai/stream-object. If you need to generate an arbitrary object, xsAI only makes a request and does not guarantee that the output is valid.

[kwaa]

Do you have any other questions? If not, I will close this issue.

[kingyue737]

I'm using Deepseek API which does not support response_format: 'json_schema'. streamObject will result into error:

This response_format type is unavailable now

Since some commonly used large model APIs support JSON mode—like OpenAI, Qwen, and DeepSeek, I feel like this might be a bug that we can't specify response_format as json_object in streamText. Another package, openai, as well as xsai v0.2.2, don't have this issue.

[kwaa]

I'm using Deepseek API which does not support response_format: 'json_schema'. streamObject will result into error:

This response_format type is unavailable now

Since the Ollama I'm using doesn't support response_format: 'json', I'll mark this as a help wanted issue.