About using streaming while waiting for model response

[miguellobato84]

Hi, I'm wondering if makes sense to use a streaming (aka keep waiting for) the response for the model. For some models like 4o it makes sense, but for others like o3 I don't think so and we should change to a poll approach

The problem arises when the app is behind some kind of firewall that closes connections without activity after 10-30s, so models like o3 becomes unusable.

Problems will also appear when research models, which can take upto 30 mins, are added. We can´t keep an active connection without receiving data for that amount of time.

[danny-avila]

Maybe they would make sense as a scheduled task that runs in the background?

That feature is on the new roadmap for the year:
https://www.librechat.ai/blog/2025-02-20_2025_roadmap#planned-items

[miguellobato84]

sure, that makes sense :)

[Pfaufisch]

I feel like this issue became more apparent recently, with GPT-5 being super slow¹. Multiple people reported issues on Discord. Might be worth considering to prioritize work on scheduled tasks?

¹) As per Artifical Analysis: "GPT-5 (high) has a higher latency compared to average, taking 79.21s to receive the first token (TTFT)."