Skip to content

Commit 5f0647f

Browse files
authored
fix: turbomind backend config in cli serve (#3784)
1 parent 9098ae8 commit 5f0647f

File tree

1 file changed

+2
-0
lines changed

1 file changed

+2
-0
lines changed

lmdeploy/cli/serve.py

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -359,6 +359,8 @@ def api_server(args):
359359
cache_block_seq_len=args.cache_block_seq_len,
360360
enable_prefix_caching=args.enable_prefix_caching,
361361
max_prefill_token_num=args.max_prefill_token_num,
362+
num_tokens_per_iter=args.num_tokens_per_iter,
363+
max_prefill_iters=args.max_prefill_iters,
362364
communicator=args.communicator,
363365
hf_overrides=args.hf_overrides)
364366
chat_template_config = get_chat_template(args.chat_template)

0 commit comments

Comments
 (0)