stream mode is not supported

[AkisAya]

Describe the bug
when i request with a stream option like this

{
    "model": "auto",
    "messages": [
        {
            "role": "user",
            "content": "what is H20"
        }
    ],
    "stream": true
}

the response is still a blocking response

To Reproduce
add "stream": true to request body

Expected behavior
return text/event response body

Screenshots

Additional context
i found openai.ChatCompletionNewParams in openai-go sdk has no Stream field , so the parse logic in handleRequestBody lose the user input stream: true option

// parseOpenAIRequest parses the raw JSON using the OpenAI SDK types
func parseOpenAIRequest(data []byte) (*openai.ChatCompletionNewParams, error) {
	var req openai.ChatCompletionNewParams
	if err := json.Unmarshal(data, &req); err != nil {
		return nil, err
	}
	return &req, nil
}

[rootfs]

ref #203 @tao12345666333

[tao12345666333]

Yes, I'm working on this, you can check #203 for the progress. Thanks.

[tao12345666333]

Hi @AkisAya the PR has been merged #203

Could you please upgrade to the latest commit, and run the test again? Thank you!

Note: When enabling stream, make sure the request includes the text/event-stream header value (this is the standard practice for SSE). I'm not quite sure how Postman in your screenshot handles SSE requests, so you'll need to verify it yourself. Alternatively, you can use any SDK that supports streaming to validate the request.

[AkisAya]

@tao12345666333 no, the problem still exists.

case 1: specify a model

{
    "model": "Qwen3-235B-A22B",
    "messages": [
        {
            "role": "user",
            "content": "what is H20"
        }
    ],
    "stream": true
}

this will get a sse response.

case 2: use auto model

{
    "model": "auto",
    "messages": [
        {
            "role": "user",
            "content": "what is H20"
        }
    ],
    "stream": true
}

this will get a non-sse response

the problem happens because handleRequestBody function will mutate request body when model is auto but will not when model is specified.

parseOpenAIRequest will serialize a raw request body to openai-go's openai.ChatCompletionNewParams, which does not have a stream field, so the serialize request body loses the stream field and when model is auto, enovy will receive a mutated body which does not contain the stream filed

				// Set the response with body mutation and content-length removal
				response = &ext_proc.ProcessingResponse{
					Response: &ext_proc.ProcessingResponse_RequestBody{
						RequestBody: &ext_proc.BodyResponse{
							Response: &ext_proc.CommonResponse{
								Status:         ext_proc.CommonResponse_CONTINUE,
								HeaderMutation: headerMutation,
								BodyMutation:   bodyMutation,
							},
						},
					},
				}

but when model is not auto, ext_proc will not mutate the body, even though the comment says Set the response with body mutation and content-length removal

		// Set the response with body mutation and content-length removal
		response = &ext_proc.ProcessingResponse{
			Response: &ext_proc.ProcessingResponse_RequestBody{
				RequestBody: &ext_proc.BodyResponse{
					Response: &ext_proc.CommonResponse{
						Status: ext_proc.CommonResponse_CONTINUE,
						HeaderMutation: &ext_proc.HeaderMutation{
							SetHeaders: setHeaders,
						},
					},
				},
			},
		}

there should be workaround to preserve stream field when return the mutated body to ext_proc

[AkisAya]

@tao12345666333 i opened a pr as workaround, #282

[tao12345666333]

Thank you @AkisAya I did overlook it in my previous implementation