Issue with Truncated Responses in UI

[gabrielsusai]

Hi,

I'm working with the most recent code from GitHub. However, I've encountered an issue where the output response is cut off halfway through the completion. I'm linked to the Azure Search Service (OYD) and have set up the index properly. The chat does generate a response with the correct references, but it's truncated mid-way and fails to display correctly in the UI. GPT4 model is used. Below is my configuration.

Chat with data: common settings

DATASOURCE_TYPE=AzureCognitiveSearch
SEARCH_TOP_K=5
SEARCH_STRICTNESS=3
SEARCH_ENABLE_IN_DOMAIN=True

Chat with data: Azure AI Search

AZURE_SEARCH_SERVICE=service_name
AZURE_SEARCH_INDEX=index_name
AZURE_SEARCH_KEY=dummy
AZURE_SEARCH_SEMANTIC_SEARCH_CONFIG=default
AZURE_SEARCH_INDEX_IS_PRECHUNKED=False
AZURE_SEARCH_TOP_K=5
AZURE_SEARCH_ENABLE_IN_DOMAIN=False
AZURE_SEARCH_QUERY_TYPE=semantic
AZURE_SEARCH_STRICTNESS=3

Any help would be appreciated.

Thanks

[gabrielsusai]

I'm a bit stuck at the moment. Any assistance would be greatly appreciated. Let me know if I should post this in a different forum

[jhicks-nfg]

Increase Max Tokens. You can also enter "continue" into the chat and it will pick up where the response left off...
Would be nice if the interface picked up on that and let you know with a continue button or some such.