Replies: 1 comment
-
Since v0.4.0, the pipeline can output logprobs and token ids. |
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
-
想集成lmdeploy做推理,用于rlhf过程,这个过程需要得到一次generate的output ids,但是调用lmdeploy的pipeline,只能得到decode后的output str,请问有什么推荐的解决办法吗?
vllm有generate接口,可以一次处理一个batch的prompts,直接得到output_ids,lmdeploy有没有类似的接口呢?
Beta Was this translation helpful? Give feedback.
All reactions