Merge remote-tracking branch 'origin/prefill_overlap' into prefill_overlap

wanghao7 · wanghao7 · commit 94d7045868e5 · 2025-04-02T16:59:36.000+08:00
diff --git a/lightllm/server/api_tgi.py b/lightllm/server/api_tgi.py
@@ -126,6 +126,7 @@ async def tgi_generate_impl(request: Request, httpserver_manager: HttpServerMana
                 "generated_tokens": count_output_tokens_dict[sub_id],
                 "finish_reason": finish_status_dict[sub_id].get_finish_reason(),
                 "tokens": tokens_dict[sub_id],
+                "prompt_tokens": tokens_dict[sub_id][0]["prompt_tokens"],
             }
         if prompt_token_ids is not None:
             ret["prompt_token_ids"] = prompt_token_ids

Original file line number	Diff line number	Diff line change
`@@ -126,6 +126,7 @@ async def tgi_generate_impl(request: Request, httpserver_manager: HttpServerMana`
`126`	`126`	`"generated_tokens": count_output_tokens_dict[sub_id],`
`127`	`127`	`"finish_reason": finish_status_dict[sub_id].get_finish_reason(),`
`128`	`128`	`"tokens": tokens_dict[sub_id],`
	`129`	`+ "prompt_tokens": tokens_dict[sub_id][0]["prompt_tokens"],`
`129`	`130`	`}`
`130`	`131`	`if prompt_token_ids is not None:`
`131`	`132`	`ret["prompt_token_ids"] = prompt_token_ids`