How do I get a complete response without having to press enter all the time?

[gpayer]

So running various models with chat like behavior works nicely, however I always have to trigger llama.cpp to give more tokens by pressing enter.
I couldn't find a parameter to improve that behavior and it seems nobody else has this problem? I'm highly confused about that.

My expectation would be that the output runs without interruption until the "reverse prompt" appears and I can enter something again.

So is there a way to give control to llama until the reverse prompt appears again?

[batmanonline]

this might be related to issue #993

[mgroeber9110]

You could try using the --ignore-eos option, which stops the model from ever predicting end-of-text on its own.

[batmanonline]

@mgroeber9110 thanks, just tried it but unfortunatelly llama still stops randomly and requires Enter key to be pressed until it can finally return control to the user.

[mgroeber9110]

Does someone perhaps have an exact command line where this happens? With -n -1 and --ignore-eos, I can't see an obvious reason why the model should stop for anything but a reverse prompt, but the exact behaviour may depend on the type of instruction mode being used.

[DannyDaemonic]

@mgroeber9110 It's worth a shot though.

When reporting, include the full command line instruction and the entire output (including the seed, and the text you provided)

If you didn't specify the seed, that will also be in the output.

main: seed = 1682050141
llama.cpp: loading model from models/gpt4-alpaca-lora-30b/ggml-model-q4_0-full.bin
llama_model_load_internal: format = ggjt v1 (latest)
llama_model_load_internal: n_vocab = 32000

[DannyDaemonic]

Also, make sure you're using -n -1 as it will return control to you when you hit the -n count.