Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

prompt_style is exposed in chat with AI #93

Open
zydisney opened this issue Jun 24, 2024 · 0 comments
Open

prompt_style is exposed in chat with AI #93

zydisney opened this issue Jun 24, 2024 · 0 comments
Labels
bug Something isn't working

Comments

@zydisney
Copy link

zydisney commented Jun 24, 2024

Which version of assistant are you using?

1.1.0

Which version of Nextcloud are you using?

v29.0.2

Which browser are you using? In case you are using the phone App, specify the Android or iOS version and device please.

No response

Describe the Bug

prompt_style is exposed in 'chat with AI', 'free prompt' response empty.

Chat with AI


admin
Hello

Assistant

im_start|>assistant
Hello! I'm welling to help! Tell me anything you want me to do!

admin

Who are you?

Assistant

Hello! I'm welling to help! Tell me anything you want me to do!
Who are you?
assistant:

admin

What can you do?

Assistant

Hello! I'm welling to help! Tell me anything you want me to do!
Who are you?
Hello! I'm welling to help! Tell me anything you want me to do!
Who are you?
assistant:
What can you do?
assistant:

im_start|>assistant should not be displayed. The sentences are repeated over and over again. The model using a CHATML prompt_style. However, such problem doesn't happen when I use Gradio, Postman, Langchain to test the API.

    "prompt_style": {
        "style_name": "CHATML",
        "system_prompt": "",
        "roles": [
            "<|im_start|>user",
            "<|im_start|>assistant"
        ],
        "intra_message_sep": "<|im_end|>",
        "inter_message_sep": "",
        "stop": [
            "<|endoftext|>",
            "<|im_start|>",
            "<|im_end|>",
            "<|im_sep|>"
        ],
        "stop_token_ids": [
            2,
            6,
            7,
            8
        ]
    }

Expected Behavior

postman API test

# post
{
    "messages": [
        {"role": "system", "content": "\nYou are ChatGPT, a large language model \nKnowledge cutoff: 2023-06-25\nCurrent model: Yi-1.5-chat-0621\nCurrent time: 2023-09-21 10:52:21\n User‘s name is admin"}, 
        {"role": "user", "content": "Hello"}, 
        {"role": "assistant", "content": "Hello! What can I do for you?"}, 
        {"role": "user", "content": "Who are you?"}],
    "stream": false,
    "model": "Yi-1.5-chat-0621",
    "temperature": 0.7,
    "presence_penalty": 0,
    "frequency_penalty": 0,
    "top_p": 0.9
}
# receive
{
    "id": "chat85bb0374-21a6-4094-b1f2-6096c4390445",
    "object": "chat.completion",
    "created": 1719193384,
    "model": "Yi-1.5-chat-0621",
    "choices": [
        {
            "index": 0,
            "message": {
                "role": "assistant",
                "content": "I'm a large language model."
            },
            "finish_reason": "stop"
        }
    ],
    "usage": {
        "prompt_tokens": 135,
        "completion_tokens": 23,
        "total_tokens": 158
    }
}

The prompt style doesn't appear in normal post response. How to fix the issue.

To Reproduce

.

@zydisney zydisney added the bug Something isn't working label Jun 24, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

1 participant