multi agent system with a supervisor stuck in a loop of endlessly calling the same agent over and over again

[AsmaaMHadir]

Checked other resources

• I added a very descriptive title to this question.
• I searched the LangChain documentation with the integrated search.
• I used the GitHub search to find a similar question and didn't find it.

Commit to Help

• I commit to help with one of those options 👆

Example Code

tavily_tool = TavilySearchResults(max_results=5)
google_tools = load_tools(["google-scholar", "google-finance"], llm=llm)

    
    
@tool
def scrape_webpages(urls: List[str]) -> str:
    """Use requests and bs4 to scrape the provided web pages for detailed information."""
    loader = WebBaseLoader(urls)
    docs = loader.load()
    return "\n\n".join(
        [
            f'<Document name="{doc.metadata.get("title", "")}">\n{doc.page_content}\n</Document>'
            for doc in docs
        ]
    )
           
  
tavily_tool = TavilySearchResults(max_results=5)
google_tools = load_tools(["google-scholar", "google-finance"], llm=llm)
e2b_data_analysis_tool = E2BDataAnalysisTool(
    # Pass environment variables to the sandbox
    # env_vars={"MY_SECRET": "secret_value"},
    on_stdout=lambda stdout: print("stdout:", stdout),
    on_stderr=lambda stderr: print("stderr:", stderr),
    on_artifact=save_artifact,
).as_tool()  

fa_agent_prompt = "You are an expert financial analyst who is capable of analyzing data and providing insightful answers and generating visualizations and charts "
financial_analyst_agent = create_agent(llm, [e2b_data_analysis_tool, tavily_tool, scrape_webpages]+google_tools, fa_agent_prompt)
financial_analyst_node = functools.partial(agent_node, agent=financial_analyst_agent, name="Financial Analyst")

NEO4J_URL = os.getenv("NEO4J_URL")
NEO4J_USERNAME = os.getenv("NEO4J_USERNAME")
NEO4J_PASSWORD = os.getenv("NEO4J_PASSWORD")
examples_file_path = "../examples.json" 

# Open the file and parse the JSON
with open(examples_file_path, 'r', encoding='utf-8') as file:
    data = json.load(file)

query_examples = data
graph = Neo4jGraph(
    url=NEO4J_URL, username=NEO4J_USERNAME, password=NEO4J_PASSWORD
    )


EXAMPLES_PROMPT_TEMPLATE = """   
                Input: {db_question},
                Output: {db_query}
            """
                            

example_prompt = PromptTemplate(input_variables=["db_question", "db_query"], template=EXAMPLES_PROMPT_TEMPLATE)

example_selector = MaxMarginalRelevanceExampleSelector.from_examples(
    # This is the list of examples available to select from.
    query_examples,
    # This is the embedding class used to produce embeddings which are used to measure semantic similarity.
    OpenAIEmbeddings(),
    # This is the VectorStore class that is used to store the embeddings and do a similarity search over.
    Chroma,
    # This is the number of examples to produce.
    k=2
)

prompt_cypher = FewShotPromptTemplate(
    example_selector=example_selector,
    example_prompt=example_prompt,
    suffix="The question is:\n{question}",
    prefix="""Task: Generate a Cypher query to query a graph database based on the user's question.
Instructions:
Use the provided schema for node types, relationship types, and properties in the graph database. Only incorporate these defined elements.
Avoid utilizing any other node types, relationship types, or properties not present in the provided schema. Here's the schema definition:

{schema}

if the question matches one of the sample questions in the knowledge base then just return the query used to answer it.
if the user asks to retrieve a piece of information about a document or section given their name, then use a `WHERE` statement 
and a cypher regular expression matching without case sensitivity like in the queries in your knowledge base when filtering by the name. 

Use the statement `(t:transaction)-[:CONTAINS*]->(s)` in the cypher query
with the `*` sign next to the relationship label `CONTAINS` and where s is the section node you are looking for.


Ensure the generated query captures relevant information from the graph database without reducing the retrieved data due to variations in user wording.
Note: do not include any explanations or apologies in your responses. 
Do not respond to inquiries seeking information other than the construction of a Cypher statement.
Do not include any text except the generated Cypher statement.
""",
    input_variables=["schema", "question"]
)


QA_GENERATION_TEMPLATE = """
        Task: answer the question you are given based on the context provided.
        Instructions:
        You are an assistant that helps to form nice and human understandable answers. 
        Use the context information provided to generate a well organized and comprehensve answer to the user's question. 
        When the provided information contains multiple elements, structure your answer as a bulleted or numbered list to enhance clarity and readability.
        You must use the information to construct your answer. 
        The provided information is authoritative; do not doubt it or try to use your internal knowledge to correct it. 
        Make the answer sound like a response to the question without mentioning that you based the result on the given information. 

        Here's the information:
        {context}
        

        Question: {question}
        Answer:
            """

prompt_qa = PromptTemplate(input_variables=["context", "question"], template=QA_GENERATION_TEMPLATE)
graph = Neo4jGraph(
    url=NEO4J_URL, username=NEO4J_USERNAME, password=NEO4J_PASSWORD
    )
cypher_chain =GraphCypherQAChain_custom.from_llm(
        cypher_llm=llm,
        qa_llm=llm,
        graph=graph, # define
        verbose=True,
        return_intermediate_steps=True,
        validate_cypher=True,
        cypher_prompt=prompt_cypher, # define
        qa_prompt=prompt_qa # define
    )

@tool
def graph_qa(question: Annotated[str,"user's question"]):
    """Use this to answer questions about the graph database
    """
    result = cypher_chain.invoke({"query":question})
    return result['result']
    
graph_retrieval_agent = create_agent(llm, [graph_qa], "You are an expert in answering questions about information stored in a Neo4j graph database.")
graph_retrieval_node = functools.partial(agent_node, agent=graph_retrieval_agent , name="Graph Retriever")


@tool
def get_presentation_url(bucket_name: Annotated[str,"name of the bucket in which wre uploaded the presentation"],
                         object_name: Annotated[str,"name of presentation file"]):
    """Use this to get the url of the presentation for the user to access"""
    return create_presigned_url(bucket_name,object_name)
@tool
def create_presentation():
    """Use this to generate a power point presentation and retrieve its name (key)
    """
    prs = Presentation()
    s3 = boto3.client('s3', aws_access_key_id=aws_access_key, aws_secret_access_key=aws_secret_key, region_name=aws_region)
    ppt_key = generate_unique_object_key()
    uploaded = upload_to_s3(prs,s3, bucket_name, ppt_key)
    if uploaded:
        print("Successfully generated and uploaded the presentation")
        return ppt_key


@tool
def write_to_presentation(pres:Annotated[str,"key of the ppt presentation in the s3 bucket to write content to"],presentation_content: Annotated[str,"content to be written on the presentation's slides"],max_characters):
    """Use this to write content to the given generated power point presentation's slides
    """
    bucket_name = 'qa-ppt-presentations'
    presentation = read_presentation_from_s3(bucket_name, pres)
    if len(presentation_content) > max_characters:
        # split slide_content
        slide_content_chunks = split_text_into_chunks(presentation_content,max_characters)
        
        for chunk in slide_content_chunks:
            addSlide(presentation,chunk)
        
    else:
        addSlide(presentation,presentation_content)
        
    presentation_bytes = BytesIO()
    presentation.save(presentation_bytes)
    presentation_bytes.seek(0)
    s3 = boto3.client('s3', aws_access_key_id=aws_access_key, aws_secret_access_key=aws_secret_key, region_name=aws_region)
    # save prsentation
    s3.put_object(Body=presentation_bytes, Bucket=bucket_name, Key=pres)

@tool
def read_presentation(pres):
    """Use this to access the text in each slide of the presentation
    """
    slides = {}

    for i, slide in enumerate(pres.slides):
        for shape in slide.shapes:
            if shape.has_text_frame:
                slides[f"{i + 1}"] = shape.text
    return slides

@tool
def create_outline(points:Annotated[List[str], "List of main points or sections."]):
    """Use this first to create a text outline of the presentation"""
    outline = ""
    for p in points:
        outline += p + "\n"
    return outline 

presentation_generator_prompt= "You are an expert in generating a power point presentation and crafting and writing the presentation's content in an organized manner"
presentation_generation_agent = create_agent(llm, [create_presentation, write_to_presentation,get_presentation_url,read_presentation, create_outline], presentation_generator_prompt)
presentation_generation_node = functools.partial(agent_node, agent=presentation_generation_agent, name="Presentation Generator")



members = ["Financial Analyst","Presentation Generator","Graph Retriever"]
# The agent state is the input to each node in the graph
class AgentState(TypedDict):
    # The annotation tells the graph that new messages will always
    # be added to the current states
    messages: Annotated[Sequence[BaseMessage], operator.add]
    # The 'next' field indicates where to route to next
    next: str

system_prompt=(
    "You are a supervisor tasked with managing a conversation between the"
    " following workers:  {members}. Given the following user request,"
    " respond with the worker to act next. Each worker will perform a"
    " task and respond with their results and status."
    "the Graph Retriever worker is responsible for answering questions about transactions and sections in the graph database."
    "The Financial Analyst is responsible for providing expert finanacial information analysis, generating charts, and analysing text and numeric data."
    "The Presentation Generator is responsible for generating a power point presentation about the subject requested by the user."
    "delegate tasks properly between agents in order to successfully fulfill the user's request and answer their questions."
    "When finished respond with FINISH."
)

supervisor_chain = create_team_supervisor(llm,system_prompt,members)

workflow = StateGraph(AgentState)
workflow.add_node("Graph Retriever", graph_retrieval_node)
workflow.add_node("Financial Analyst", financial_analyst_node)
workflow.add_node("Presentation Generator", presentation_generation_node)
workflow.add_node("supervisor", supervisor_chain)

for member in members:
    # We want our workers to ALWAYS "report back" to the supervisor when done
    workflow.add_edge(member, "supervisor")
# The supervisor populates the "next" field in the graph state
# which routes to a node or finishes
conditional_map = {k: k for k in members}
conditional_map["FINISH"] = END
workflow.add_conditional_edges("supervisor", lambda x: x["next"], conditional_map)
# Finally, add entrypoint
workflow.set_entry_point("supervisor")

graph = workflow.compile()

from langchain_core.messages import HumanMessage

inputs = {"messages": [HumanMessage(content="what are the most important risk factors across the transactions in graph?")]}
for output in graph.stream(inputs,  {"recursion_limit": 100}):
    if "__end__" not in output:
        print(output)
        print("----")

Description

I am trying to implement the proposed multi-agent with supervisor architecture presented here using LangGraph. The supervisor agent is supposed to take a user's question and decide which agent should go next, the selected agent then reports back with findings and then supervisor agent selects whether to finish or to call another agent and so on until the supervisor returns Finish. My problem is that when I run the graph the supervisor agent keeps on invoking the same agent over and over again and the user never gets a final response back. Here are the functions I am using to define the agents, agent nodes, and the supervisor:

def create_agent(llm: ChatOpenAI, tools: list, system_prompt: str):
    # Each worker node will be given a name and some tools.
    prompt = ChatPromptTemplate.from_messages(
        [
            (
                "system",
                system_prompt,
            ),
            MessagesPlaceholder(variable_name="messages"),
            MessagesPlaceholder(variable_name="agent_scratchpad"),
        ]
    )
    agent = create_openai_tools_agent(llm, tools, prompt)
    executor = AgentExecutor(agent=agent, tools=tools,max_iterations=2)
    return executor

def agent_node(state, agent, name): # state: message
    """Returns the node's final response which will be appended to the graph's state

    Args:
        state (string): message to pass to the agent
        agent (AgentExecutor): node's agent
        name (string): agent's name (team member's node)

    Returns:
        string: agent's final response
    """
    result = agent.invoke(state)
    return {"messages": [HumanMessage(content=result["output"], name=name)]} # make agent's response human readabale 


def create_team_supervisor(
    llm: ChatOpenAI, system_prompt, members
) -> str:
    """An LLM-based router."""
    options = ["FINISH"] + members # list of ["FINISH","TEAM 1","TEAM 2",...]
    
    function_def = { # openai function which routes to the "next" agent
        "name": "route",
        "description": "Select the next role.",
        "parameters": {
            "title": "routeSchema",
            "type": "object",
            "properties": {
                "next": {
                    "title": "Next",
                    "anyOf": [
                        {"enum": options},
                    ],
                },
            },
            "required": ["next"],
        },
    }
    prompt = ChatPromptTemplate.from_messages(
        [
            ("system", system_prompt),
            MessagesPlaceholder(variable_name="messages"),
            (
                "system",
                "Given the conversation above, who should act next?"
                " Or should we FINISH? Select one of: {options}",
            ),
        ]
    ).partial(options=str(options), members=", ".join(members))
    return (
        prompt
        | llm.bind_functions(functions=[function_def], function_call="route")
        | JsonOutputFunctionsParser()
    )

Why is this happening? what am I wrongly implementing the multi-agent with supervisor graph? what do I need to fix about the code to ensure that the graph chain takes a question , calls the right agents, and retrieves the final answer to the user?

System Info

langchain==0.1.5
langchain-community==0.0.17
langchain-core==0.1.16
langchain-experimental==0.0.50
langchain-openai==0.0.5
langchainhub==0.1.14
python==3.10.12

[dosubot[bot]]

🤖

Based on the information you've provided, it seems like the supervisor agent is stuck in a loop where it keeps invoking the same agent. This could be due to the logic in your create_team_supervisor function or the way you're using the AgentExecutor.

Firstly, I would recommend checking the logic in your route function. This function should be designed to select the next agent based on the current state of the conversation. If it's always selecting the same agent, it could be because the conditions for selecting other agents are never met.

Secondly, you might want to consider limiting the number of times an agent can be invoked consecutively. This can be done by adding a counter that increments each time the same agent is invoked and resets when a different agent is invoked. If the counter reaches a certain limit, you can force the supervisor to select a different agent.

Here's a rough example of how you could implement this:

class AgentExecutor:
    def __init__(self, agent, tools, max_iterations=2):
        self.agent = agent
        self.tools = tools
        self.max_iterations = max_iterations
        self.iterations = 0
        self.last_agent = None

    def invoke(self, state):
        if self.last_agent == self.agent and self.iterations >= self.max_iterations:
            # Force the supervisor to select a different agent
            return {"output": "FINISH"}
        result = self.agent(state)
        if self.last_agent == self.agent:
            self.iterations += 1
        else:
            self.iterations = 0
        self.last_agent = self.agent
        return result

In this code, AgentExecutor keeps track of the last agent that was invoked and the number of times it has been invoked consecutively. If the same agent is invoked more than max_iterations times, it forces the supervisor to select a different agent by returning {"output": "FINISH"}.

Please note that this is a simplified example and you might need to adjust it to fit your specific use case.

Regarding the changes in the LangChain repository after version 0.1.5, the major change that could potentially affect the behavior of the supervisor agent is the relocation of the create_xorbits_agent function from the langchain package to the langchain_experimental package. If you're using this function in your code, you'll need to update your import statements to reflect its new location. You can find more details about this change in the LangChain repository.

I hope this helps! If you have any further questions, feel free to ask.

Sources

• LangChain retrigger tools multiple times until hit agent limits
• libs/langchain/langchain/agents/agent.py
• libs/langchain/langchain/agents/agent_toolkits/xorbits/init.py

About Dosu

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.

[AsmaaMHadir]

@dosu-bot
this is my route function:

def create_team_supervisor(
    llm: ChatOpenAI, system_prompt, members
) -> str:
    """An LLM-based router."""
    options = ["FINISH"] + members # list of ["FINISH","TEAM 1","TEAM 2",...]
    
    function_def = { # openai function which routes to the "next" agent
        "name": "route",
        "description": "Select the next role.",
        "parameters": {
            "title": "routeSchema",
            "type": "object",
            "properties": {
                "next": {
                    "title": "Next",
                    "anyOf": [
                        {"enum": options},
                    ],
                },
            },
            "required": ["next"],
        },
    }
    prompt = ChatPromptTemplate.from_messages(
        [
            ("system", system_prompt),
            MessagesPlaceholder(variable_name="messages"),
            (
                "system",
                "Given the conversation above, who should act next?"
                " Or should we FINISH? Select one of: {options}",
            ),
        ]
    ).partial(options=str(options), members=", ".join(members))
    return (
        prompt
        | llm.bind_functions(functions=[function_def], function_call="route")
        | JsonOutputFunctionsParser()
    )

and this is the system prompt I provide for the supervisor to act on:

system_prompt=(
    "You are a supervisor tasked with managing a conversation between the"
    " following workers:  {members}. Given the following user request,"
    " respond with the worker to act next. Each worker will perform a"
    " task and respond with their results and status."
    "the Graph Retriever worker is responsible for answering questions about transactions and sections in the graph database."
    "The Financial Analyst is responsible for providing expert finanacial information analysis, generating charts, and analysing text and numeric data."
    "The Presentation Generator is responsible for generating a power point presentation about the subject requested by the user."
    "delegate tasks properly between agents in order to successfully fulfill the user's request and answer their questions."
    "When finished respond with FINISH."
)

am I missing something in my route function that doesn't let the llm router to return FINISH when an answer had been met?

[sreenathelloti]

AsmaaMHadir Feb 21, 2024
Author

did u resolve it?

[dosubot[bot]]

🤖

Based on the code you've shared, it seems like the issue might be related to how the route function is implemented. The route function is responsible for deciding which agent should act next or whether to finish the conversation. If it's not properly implemented, it could lead to the supervisor agent invoking the same agent repeatedly.

In your create_team_supervisor function, you're binding the route function to the language model (llm) and using it to decide the next agent. However, it's not clear how the route function is making this decision. The function should have some logic to analyze the current state of the conversation and decide the next agent based on that. If this logic is not implemented correctly, it could lead to the same agent being selected repeatedly.

Here's a simplified example of how the route function could be implemented:

def route(state):
    # Analyze the state of the conversation
    # For example, check the last message sent by the user
    last_user_message = state["messages"][-1]["content"]

    # Based on the last user message, decide the next agent
    if "graph" in last_user_message:
        return "Graph Retriever"
    elif "financial" in last_user_message:
        return "Financial Analyst"
    elif "presentation" in last_user_message:
        return "Presentation Generator"
    else:
        return "FINISH"

In this example, the route function checks the last message sent by the user and decides the next agent based on the content of the message. If the message contains the word "graph", it selects the "Graph Retriever" agent. If it contains the word "financial", it selects the "Financial Analyst" agent. If it contains the word "presentation", it selects the "Presentation Generator" agent. If none of these words are present in the message, it returns "FINISH", indicating that the conversation should be finished.

Please note that this is a very simplified example and the actual implementation of the route function should be more complex, taking into account the entire state of the conversation and possibly using some form of machine learning to make the decision.

I hope this helps! If you have any more questions, feel free to ask.

Source

Sources

• libs/langchain/tests/unit_tests/agents/test_agent_async.py
• libs/langchain/langchain/agents/openai_tools/base.py

About Dosu

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.

[jalakoo]

@AsmaaMHadir To stop endless loops, we've started adding the following to the end of our custom tool descriptions:

    Do not call this tool more than once.
    Do not call another tool if this returns results.

And has been working good for us. Example: https://github.com/neo4j-examples/rag-demo/blob/main/rag_demo/graph_cypher_tool.py