Replies: 2 comments
-
@irexyc 帮忙提供下方式,并更新下文档吧。 |
Beta Was this translation helpful? Give feedback.
0 replies
-
你是想在生成阶段返回prompt的logprobs么?还是单纯做prefill获取prompt的logprobs? |
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
vllm可以通过配置prompt_logprobs来返回,但似乎不能设置startlen
sglang可以通过return_logprob来返回,同时可以设置需要startlen来减少返回probs的个数
但目前我没找到LMdeploy有这个功能,想问是否还未支持?
Beta Was this translation helpful? Give feedback.
All reactions