[Question]: flask application sending a response back to LibreChat UI

[victorstorchan]

What is your question?

hello, I am currently facing an issue when trying to send back an answer to the librechat UI:
I have a custom endpoint from a flask app that produces a text answer wrapped with a chat_completion function (with streaming) based on a query coming from LibreChat. I am unable to send a response back to the librechat UI.

More Details

without streaming, when forcePrompt:true, I have a "blank" answer (nothing returned, this issue seems to have already reported)
however, when forcePrompt:false, I have a [OpenAIClient.chatCompletion] Unhandled error type request ended without sending any chunks error.

Can anyone share an example of generate_stream() function and json response that work?

@app.route('/v1/chat/completions', methods=['POST'])
def chat_completions():

for now, this dummy example does not work:

@app.route('/v1/chat/completions', methods=['POST'])
def chat_completions():
    def generate_stream():
        # First chunk: Metadata and initial response
        yield json.dumps({
            "id": "chatcmpl-unique-id",
            "object": "chat.completion",
            "created": 1733327883,
            "model": "llama-3.2-3b-instruct",
            "choices": [
                {
                    "index": 0,
                    "delta": {"role": "assistant", "content": "Here is the start of the response"}
                }
            ]
        }) + "\n"

        # Simulated delay for intermediate chunks
        import time
        time.sleep(0.1)

        # Second chunk: Continuation of the response
        yield json.dumps({
            "choices": [
                {
                    "index": 0,
                    "delta": {"content": " and here is more content."}
                }
            ]
        }) + "\n"

        # Final chunk: Finish reason
        yield json.dumps({
            "choices": [
                {
                    "index": 0,
                    "finish_reason": "stop"
                }
            ]
        }) + "\n"

    # Return the response as a stream
    return Response(generate_stream(), content_type="application/json; charset=utf-8")

Thanks

What is the main subject of your question?

No response

Screenshots

No response

Code of Conduct

• I agree to follow this project's Code of Conduct

[linkedlist771]

same issue, any solution yet?