Feature Request: Update "assistant_stream" to support Multimodal (Images/Voice) & Leverage Free Tier Tokens

[mysubcult]

Hi @remdex,

I am writing to request an update for the assistant_stream logic, which hasn't been updated in a while.

1. The Economic Context (Why this is important now)
OpenAI has a "Data Sharing" program where many accounts (even Tier 1) receive "Complimentary Daily Tokens".
For example, we now get up to 2.5 million tokens per day for free on models like gpt-4o-mini.

This makes the Assistants API incredibly viable for production.
The issue with using the standard Response API (Chat Completions) for similar tasks is that Vector Store / File Search usage there is paid and billed separately — it is explicitly NOT included in the "Complimentary Daily Tokens".
Therefore, to maximize the benefit of the free tier (which covers the model inference), the Assistants API is the most efficient choice.

However, the current implementation in assistant_stream is lagging behind the capabilities needed to fully utilize this.

2. The Request: Multimodal Support
Currently, the assistant_stream implementation does not seem to handle Images or Voice messages, whereas the standard "Bot" implementation in LHC already supports them.

Could you please port the logic from the standard Bot integration into the assistant_stream?

• Images (Vision):
  In the standard bot (as per docs here), images are passed via file_body_embed / base64.

  • Goal: When a user uploads an image to the Assistant, it should be passed to the thread so the Assistant can analyze it (e.g., "What is in this picture?").

• Voice (Audio):
  In the standard bot (as per docs here), audio is sent to Whisper for transcription, and the text is then sent to the LLM.

  • Goal: When a user sends a voice message, the Assistant should receive the transcribed text as a user message in the thread.

Since the gpt-4o-mini model is now free for many users (up to 2.5M tokens/day), updating assistant_stream to support Images and Voice would make it a powerful, all-in-one solution for Live Helper Chat users.

Thank you for considering this!

[remdex]

Assistant api is depreciated in favor of Responses API. So nothing will be done and responses api should be used. Or you talking about something else.

[mysubcult]

Assistant api is depreciated in favor of Responses API. So nothing will be done and responses api should be used. Or you talking about something else.

Yes, and the opeai developers wrote that everything there will work right up to august 2026 (it's still a year away) and then, when the conditions of the Response Api are approximately equal to the Assist Api, then it will be disabled.

[mysubcult]

I can't find the way at least to limit this is Response Api...max_num_result not working for me there

[mysubcult]

https://platform.openai.com/docs/guides/conversation-state

My be this will help?

[remdex]

I have nothing to do with all this stuff :)

[mysubcult]

I have nothing to do with all this stuff :)

You have integration with Response Api that send every message as i single message, but there is a way to do this like conversations, line as in assistant. Can you help to do that also?

[mysubcult]

https://platform.openai.com/docs/api-reference/conversations/create

[remdex]

In my clients use cases was easier to send full conversation. So you can make other flow if you want :)

[mysubcult]

In my clients use cases was easier to send full conversation. So you can make other flow if you want :)

Even when using previous_response_id, all previous input tokens for responses in the chain are billed as input tokens in the API.
But it will be more correct and more convenient to log requests if each chat is in a separate conversation. Just like you already implemented it in your time in the AssistanT API.

[mysubcult]

So you can make other flow if you want :)

i can't, thats why iam asking :)