You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
So far, I have found that it is quite easy to realise streaming in Bentoml while yielding the outputs. However, I am wondering how I could achieve this SSE-conformant streaming, i.e. how it sends the JSON in style.
data: some_data\n\n
Simply building this as a string didn't work as it was not a valid base model. I have seen older pull requests on this topic where it was achieved with runners, but I have since found that this is already outdated.
reacted with thumbs up emoji reacted with thumbs down emoji reacted with laugh emoji reacted with hooray emoji reacted with confused emoji reacted with heart emoji reacted with rocket emoji reacted with eyes emoji
Uh oh!
There was an error while loading. Please reload this page.
-
Hello,
So far, I have found that it is quite easy to realise streaming in Bentoml while yielding the outputs. However, I am wondering how I could achieve this SSE-conformant streaming, i.e. how it sends the JSON in style.
data: some_data\n\n
Simply building this as a string didn't work as it was not a valid base model. I have seen older pull requests on this topic where it was achieved with runners, but I have since found that this is already outdated.
Thanks in advance.
Beta Was this translation helpful? Give feedback.
All reactions