Skip to content

Commit 0881524

Browse files
pskiran1yinggeh
andauthored
Update protobuf/model_config.proto
Co-authored-by: Yingge He <[email protected]>
1 parent 2b8fe03 commit 0881524

File tree

1 file changed

+2
-2
lines changed

1 file changed

+2
-2
lines changed

protobuf/model_config.proto

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1662,8 +1662,8 @@ message ModelEnsembling
16621662

16631663
//@@ .. cpp:var:: uint32 max_inflight_requests
16641664
//@@
1665-
//@@ The maximum number of concurrent inflight requests at each ensemble
1666-
//@@ step.
1665+
//@@ The maximum number of concurrent inflight requests allowed at each ensemble
1666+
//@@ step per inference request.
16671667
//@@ This limit prevents unbounded memory growth when decoupled models
16681668
//@@ produce responses faster than downstream models can consume them.
16691669
//@@ Default value is 0, which indicates that no limit is enforced.

0 commit comments

Comments
 (0)