How to add RAG into Data Interpreter

[cjl0222]

Hi! Thanks for your great work. I'm wondering whether you could add an example or documentation for integrating RAG into Data Interpreter. It seems that there are only examples for RAG Retrieval and Rerank, but I have no idea how to utilize this ability into the current Data Interpreter Machine learning modeling procedure. Thanks!

[iorisa]

When the Data Interpreter object's rc.memory content is too large, you can use the RAG approach to retrieve this memory, which reduces the cost of a single interaction and avoids exceeding the maximum token limit of the LLM.

For example, using RAG to override the self.get_memories() function used in metagpt/roles/di/data_interpreter.py:

    async def _think(self) -> bool:
        """Useful in 'react' mode. Use LLM to decide whether and what to do next."""
        user_requirement = self.get_memories()[0].content
        context = self.working_memory.get()

        if not context:
            # just started the run, we need action certainly
            self.working_memory.add(self.get_memories()[0])  # add user requirement to working memory
            self._set_state(0)
            return True

        prompt = REACT_THINK_PROMPT.format(user_requirement=user_requirement, context=context)
        rsp = await self.llm.aask(prompt)
        rsp_dict = json.loads(CodeParser.parse_code(block=None, text=rsp))
        self.working_memory.add(Message(content=rsp_dict["thoughts"], role="assistant"))
        need_action = rsp_dict["state"]
        self._set_state(0) if need_action else self._set_state(-1)

        return need_action

The original definition of the get_memories function in metagpt/roles/role.py is as follows:

    def get_memories(self, k=0) -> list[Message]:
        """A wrapper to return the most recent k memories of this role, return all when k=0"""
        return self.rc.memory.get(k=k)