Data Interpreter Multi-Agent Pipeline #469

zyzhang1130 · 2024-10-21T14:31:22Z

name: Pull Request
about: Create a pull request

Description

An example to demonstrate the problem-solving capability of multiagent LLMs with adaptive planning on various tasks.

Checklist

Please check the following items before code is ready to be reviewed.

Code has passed all tests
Docstrings have been added/updated in Google Style
Documentation has been updated
Code is ready for review

update update update updated Update di_multiagent.py update update Create README.md Update README.md undo unnecessary changes undo unnecessary changes Update .pre-commit-config.yaml Update .pre-commit-config.yaml Update common.py remove changes in other files Update README.md update updated README and directory name remove Update di_multiagent.py

Slight modification to the format_instruction to improve the success rate of `RegexTaggedContentParser` parsing the model API output.

DavdGao

Well done! My only concern is the modification within the react agent. Please see inline comments for details.

DavdGao · 2024-12-06T02:22:54Z

src/agentscope/agents/react_agent.py

@@ -109,7 +109,7 @@ def __init__(

        # Initialize a parser object to formulate the response from the model
        self.parser = RegexTaggedContentParser(
-            format_instruction="""Respond with specific tags as outlined below:
+            format_instruction="""Respond with specific tags as outlined below in json format:


Why do we need to add the requirement "in json format" here?

For string type, the json format is

<thought>'this is a test'</thought>

while the non-json format is

<thought>this is a test</thought>

Based on my testing, without adding 'in json format', RegexTaggedContentParser cannot successfully parse boolean variables or None (will treat them as strings) and hence cannot be recognized correctly by tools like 'execute_python_code'. E.g., if the RegexTaggedContentParser returns a string None, if use_docker is None will be false.

Update: I have modified my PR to override RegexTaggedContentParser's format_instruction within my example. I would caution against using RegexTaggedContentParser when try_parse_json=True without explicit instruction of adhering the arguments of interest to JSON format in format_instruction, since LLMs will likely output contents in other formats and cause json.loads to fail.

…reinforce the output adhere to json format within this example

…o reinforce the output adhere to json format within this example.

ZiTao-Li · 2024-12-16T23:53:53Z

examples/data_interpreter_mulit-agent/README.md

+
+  This step enables the executed code by the agents to perform required operations that are otherwise restricted by default. Ensure you understand the security implications of modifying these restrictions.
+
+- Comment out the following in `src/agentscope/utils/common.py`:


Why we have specific working directory here?

The original create_tempdir creates a temporary directory and changes the current working directory to it; however, it does not persist across different runs of code execution. My solution is to use the current directory for executing code and preserving the execution results.

ZiTao-Li · 2024-12-16T23:57:15Z

examples/data_interpreter_mulit-agent/di_agents.py

+from agentscope.service.service_status import ServiceExecStatus
+
+
+def read_csv_file(file_path: str) -> ServiceResponse:


Group this read_csv_file with the one in di_multiagent.py into a util.py file?

I ended up not using read_csv_file and write_csv_file (the agent is able to use code to achieve the same result), and I observed that giving the agent too many tools can have a detrimental effect on choosing the correct tool to use, so I have removed read_csv_file and write_csv_file from both scripts.

ZiTao-Li · 2024-12-17T00:03:30Z

examples/data_interpreter_mulit-agent/di_agents.py

+                                "dependent_task_ids": list[str] = "ids of tasks prerequisite to this task",
+                                "instruction": "what you should do in this task, one short phrase or sentence",
+                                "task_type": "type of this task, should be one of Available Task Types",
+                                "task_type": "type of this task, should be one of Available Task Types",


Duplicate lines?

what are the Available Task Types?

Duplicates removed.

Since this framework is designed to be generic, there are actually no pre-defined Available Task Types. task_type is optional since its only purpose at the moment is to give the solver a high-level idea of what kind of tool to employ. I have corrected the relevant prompts.

ZiTao-Li · 2024-12-17T00:11:44Z

examples/data_interpreter_mulit-agent/di_agents.py

+                "dependent_task_ids": list[str] = "ids of tasks prerequisite to this task",
+                "instruction": "what you should do in this task, one short phrase or sentence",
+                "task_type": "type of this task, should be one of Available Task Types",
+                "task_type": "type of this task, should be one of Available Task Types",


Same questions as in the comment above

same as the above

ZiTao-Li · 2024-12-17T00:12:32Z

examples/data_interpreter_mulit-agent/di_agents.py

+                                "dependent_task_ids": list[str] = "ids of tasks prerequisite to this task",
+                                "instruction": "what you should do in this task, one short phrase or sentence",
+                                "task_type": "type of this task, should be one of Available Task Types",
+                                "task_type": "type of this task, should be one of Available Task Types",


Same questions as the comments above

same as the above

ZiTao-Li · 2024-12-17T00:14:14Z

examples/data_interpreter_mulit-agent/di_multiagent.py

+        )
+
+
+_ = load_dotenv(find_dotenv())  # read local .env file


Why here need to have load_dotenv?

This line locates the .env file and load_dotenv() reads the .env file and updates os.environ. Without it the API keys won't be loaded properly.

unused tool

unused

zyzhang1130 added 2 commits November 20, 2024 11:49

Update di_multiagent.py

83edd23

zyzhang1130 force-pushed the data_interpreter_multiagent branch from b01b066 to 83edd23 Compare November 20, 2024 03:53

Update di_multiagent.py

1c5f1ba

zyzhang1130 marked this pull request as ready for review November 20, 2024 04:03

zyzhang1130 changed the title ~~[WIP] Data interpreter multiagent pipeline~~ Data interpreter multiagent pipeline Nov 20, 2024

zyzhang1130 changed the title ~~Data interpreter multiagent pipeline~~ Data Interpreter Multi-Agent Pipeline Nov 20, 2024

Update react_agent.py

0941af3

Slight modification to the format_instruction to improve the success rate of `RegexTaggedContentParser` parsing the model API output.

DavdGao reviewed Dec 6, 2024

View reviewed changes

zyzhang1130 added 5 commits December 11, 2024 09:41

Update react_agent.py

41624dd

Update react_agent.py

b178779

Overwrite the parser attribute with the custom format_instruction to …

d22313e

…reinforce the output adhere to json format within this example

# Overwrite the parser attribute with the custom format_instruction t…

e95b6bb

…o reinforce the output adhere to json format within this example.

pre-commit run

f78ce6b

ZiTao-Li reviewed Dec 17, 2024

View reviewed changes

zyzhang1130 added 4 commits December 21, 2024 14:31

Update di_agents.py

fee024f

Update di_agents.py

f940eb9

removed write_csv_file

2f42284

unused tool

removed write_csv_file

5ffb7d2

unused

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Data Interpreter Multi-Agent Pipeline #469

Data Interpreter Multi-Agent Pipeline #469

zyzhang1130 commented Oct 21, 2024 •

edited

Loading

DavdGao left a comment •

edited

Loading

DavdGao Dec 6, 2024

zyzhang1130 Dec 6, 2024

zyzhang1130 Dec 11, 2024 •

edited

Loading

ZiTao-Li Dec 16, 2024

zyzhang1130 Dec 21, 2024

ZiTao-Li Dec 16, 2024

zyzhang1130 Dec 21, 2024 •

edited

Loading

ZiTao-Li Dec 17, 2024

zyzhang1130 Dec 21, 2024

ZiTao-Li Dec 17, 2024

zyzhang1130 Dec 21, 2024

ZiTao-Li Dec 17, 2024

zyzhang1130 Dec 21, 2024

ZiTao-Li Dec 17, 2024

zyzhang1130 Dec 21, 2024


		This step enables the executed code by the agents to perform required operations that are otherwise restricted by default. Ensure you understand the security implications of modifying these restrictions.

		- Comment out the following in `src/agentscope/utils/common.py`:

		from agentscope.service.service_status import ServiceExecStatus


		def read_csv_file(file_path: str) -> ServiceResponse:

Data Interpreter Multi-Agent Pipeline #469

Are you sure you want to change the base?

Data Interpreter Multi-Agent Pipeline #469

Conversation

zyzhang1130 commented Oct 21, 2024 • edited Loading

name: Pull Request about: Create a pull request

Description

Checklist

DavdGao left a comment • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

zyzhang1130 Dec 11, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

zyzhang1130 Dec 21, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

zyzhang1130 commented Oct 21, 2024 •

edited

Loading

name: Pull Request
about: Create a pull request

DavdGao left a comment •

edited

Loading

zyzhang1130 Dec 11, 2024 •

edited

Loading

zyzhang1130 Dec 21, 2024 •

edited

Loading