Skip to content

并发数超过100后,大于100的部分大模型输出首字突然增由1.4s突变到40多s #15

@lairdleng

Description

@lairdleng

Image

Image
Image
如上所示,修改最大并发上限为1000,测试时设置并发数101,大模型回答的首字延迟突变到40多s,请问是程序里面是否设置了100完成之后在并发执行下一轮

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions