使用自定义agent问答，有时模型会给出重复的两次回答，请问是什么原因，如何解决这个问题？

[Liwan-Chen]

[dosubot[bot]]

Hey there, @Liwan-Chen! Fancy seeing you back here. Hope you're doing well! 🚀

The issue you're experiencing with your custom question-answering agent providing the same response twice to a single input could be related to the handling of the self.out flag in the on_llm_new_token method. This flag is used to control whether new tokens are added to the queue based on the presence of special tokens within the input. However, the logic for resetting the self.out flag to True (which allows for output) is only present in the on_tool_end method and not after processing each token in on_llm_new_token. This could potentially lead to scenarios where the flag's state incorrectly allows or blocks the addition of new tokens to the queue, resulting in duplicate or missing responses.

To resolve this issue, you could consider the following adjustments:

1. Reset the self.out flag appropriately: Ensure that the self.out flag is reset to True at the correct places within your code. After processing a token in on_llm_new_token and deciding whether to add it to the queue based on the presence of special tokens, you should evaluate if self.out needs to be reset for subsequent tokens.

2. Review the logic for handling special tokens: In the on_llm_new_token method, the logic splits the token on the special token and updates the cur_tool dictionary without considering if multiple special tokens could be present in a single input. Ensure that your logic accurately reflects the intended behavior when special tokens are found.

3. Ensure consistent state management: The asynchronous nature of your callback handlers means that state management is crucial. Ensure that the updates to self.cur_tool and self.out are consistently applied across all scenarios to prevent race conditions or state inconsistencies that could lead to duplicate responses.

Here's a revised snippet of the on_llm_new_token method with a suggestion to reset the self.out flag more judiciously:

async def on_llm_new_token(self, token: str, **kwargs: Any) -> None:
    special_tokens = ["Action", "<|observation|>"]
    token_processed = False
    for stoken in special_tokens:
        if stoken in token:
            before_action = token.split(stoken)[0]
            self.cur_tool.update(
                status=Status.running,
                llm_token=before_action + "\n",
            )
            self.queue.put_nowait(dumps(self.cur_tool))
            self.out = False
            token_processed = True
            break

    if not token_processed and self.out:
        self.cur_tool.update(
            status=Status.running,
            llm_token=token,
        )
        self.queue.put_nowait(dumps(self.cur_tool))
    elif not token_processed:
        # Reset self.out to True here if necessary, depending on your application's logic
        pass

This adjustment ensures that self.out is only set to False when a special token is processed, potentially reducing the chance of duplicate responses due to incorrect flag handling. However, the exact solution may need further adjustments based on the detailed behavior of your application and how tokens are expected to be processed and outputted.

You can find more details about the self.out flag and its usage in the on_llm_new_token method in the server/agent/callbacks.py file in the Langchain-Chatchat repository.

Sources

• server/agent/callbacks.py

About Dosu

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.