Skip to content
This repository has been archived by the owner on Dec 6, 2023. It is now read-only.

Chat: uplift errors of model on user requests #144

Open
tiero opened this issue Nov 8, 2023 · 0 comments
Open

Chat: uplift errors of model on user requests #144

tiero opened this issue Nov 8, 2023 · 0 comments

Comments

@tiero
Copy link
Contributor

tiero commented Nov 8, 2023

We can start from Mistral binaries, for example instead of returing a general Load failed we should return the ValueError below

    | Traceback (most recent call last):
    |   File "starlette/responses.py", line 273, in wrap
    |   File "starlette/responses.py", line 262, in stream_response
    |   File "routes.py", line 64, in generate_chunk_based_response
    |   File "llama_cpp/llama.py", line 1540, in _convert_text_completion_chunks_to_chat
    |   File "llama_cpp/llama.py", line 947, in _create_completion
    | ValueError: Requested tokens (3308) exceed context window of 512
    +------------------------------------
@filopedraz filopedraz removed their assignment Dec 5, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants