[RFC]: Improved performance of long input/long output scenarios (input 2500, output 900) in high-concurrency scenarios (concurrency 4, 10). E2E performance improved by 20%.

[amy-why-3459]

Motivation.

Improved performance of long input/long output scenarios (input 2500, output 900) in high-concurrency scenarios (concurrency 4, 10). E2E performance improved by 20%.

Proposed Change.

Please provide the detailed design document of the RFC using the template.

Feedback Period.

No response

CC List.

No response

Any Other Things.

No response

Before submitting a new issue...

• Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.

[LJH-LBJ]

正在识别优化点，代码开发中