Obtain proability of tokens #9689
Unanswered
ChristianHinge
asked this question in
Q&A
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
I wish to implement LLama.cpp for yes/no question answering based on images. In this context I would like to get the token probabilities (so I can use them as a measure of uncertainty).
For the server implementation, it is possible to do
n_probs=N
and obtain the probabilities of N most probable tokens. However, I do not see a similar option forllama-cli
(orllama-minicpmv-cli
, which is the one I use).AFAIK the server implementation does not currently support image-input, so I am left with the CLI's.
Have I missed something or do you have any ideas on how I may address this?
Beta Was this translation helpful? Give feedback.
All reactions