Merge pull request #308 from Undertone0809/v1.11.0/add-tool-schema

feat: add tool schema

Makefile

@@ -1,6 +1,7 @@
 SHELL := /usr/bin/env bash
 OS := $(shell python -c "import sys; print(sys.platform)")
 
+# all test files define here
 DEV_TEST_TOOL_FILES := ./tests/tools/test_human_feedback_tool.py ./tests/tools/test_calculator.py ./tests/tools/test_python_repl_tools.py ./tests/tools/test_sleep_tool.py
 DEV_TEST_HOOK_FILES := ./tests/hook/test_llm.py ./tests/hook/test_tool_hook.py
 DEV_TEST_LLM_FILES := ./tests/llms/test_openai.py
@@ -59,10 +60,13 @@ check-codestyle:
 .PHONY: lint
 lint: check-codestyle test
 
-#* https://github.com/Maxlinn/linn-jupyter-site-template/blob/main/.github/workflows/linn-jupyter-site-template-deploy.yml
+# https://github.com/Maxlinn/linn-jupyter-site-template/blob/main/.github/workflows/linn-jupyter-site-template-deploy.yml
+# Any notebook will be converted here.
+# If there are any notebook will be changed, then the notebook will be converted to markdown and pushed to the repo.
 .PHONY: build-docs
 build-docs:
 	jupyter nbconvert ./docs/uses_cases/chat_usage.ipynb --to markdown
+	jupyter nbconvert ./example/tools/custom_tool_usage.ipynb --to markdown --output-dir ./docs/modules/tools
 
 .PHONY: start-docs
 start-docs:

README_zh.md

@@ -121,7 +121,7 @@ promptulate框架的设计原则包括：模块化、可扩展性、互操作性
 欢迎加入群聊一起交流讨论有关LLM相关的话题，链接过期了可以issue或email提醒一下作者。
 
 <div style="width: 250px;margin: 0 auto;">
-    <img src="https://zeeland-bucket.oss-cn-beijing.aliyuncs.com/images/20231226204735.png"/>
+    <img src="https://zeeland-bucket.oss-cn-beijing.aliyuncs.com/images/20240108220626.png"/>
 </div>
 
 

docs/README.md

@@ -121,7 +121,7 @@ promptulate框架的设计原则包括：模块化、可扩展性、互操作性
 欢迎加入群聊一起交流讨论有关LLM相关的话题，链接过期了可以issue或email提醒一下作者。
 
 <div style="width: 250px;margin: 0 auto;">
-    <img src="https://zeeland-bucket.oss-cn-beijing.aliyuncs.com/images/20231226204735.png"/>
+    <img src="https://zeeland-bucket.oss-cn-beijing.aliyuncs.com/images/20240108220626.png"/>
 </div>
 
 ## 贡献

docs/_sidebar.md

@@ -11,7 +11,8 @@
     - [:closed_book: OpenAI](modules/llm/openai.md#openai)
     - [:green_book: 百度文心](modules/llm/erniebot.md#百度文心erniebot)
   - [:robot: Agent 智能代理](modules/agent.md#agent)
-  - [:toolbox: Tool 外部工具扩展](modules/tools.md#简介)
+  - [:toolbox: Tool 外部工具扩展](modules/tools/index#Tool)
+    - [🧰 Custom Tool](modules/tools/custom_tool_usage.md#custom-tool)
   - [🐠 Formatter 格式化输出](modules/formatter.md#简介)
   - [:probing_cane: Hook&Lifecycle 钩子与生命周期](modules/hook.md#what-is-hook)
   - [:department_store: Memory 数据存储](modules/memory.md#memory)

docs/modules/agent.md

@@ -8,18 +8,15 @@ Agent是`promptulate`的核心组件之一，其核心思想是使用llm、Tool
 
 ```python
 import promptulate as pne
-from promptulate.tools import (
-    DuckDuckGoTool,
-    Calculator,
-)
+from promptulate.tools import DuckDuckGoTool, Calculator
 
 
 def main():
     tools = [
         DuckDuckGoTool(),
         Calculator(),
     ]
-    agent = pne.ToolAgent(tools)
+    agent = pne.ToolAgent(tools=tools)
     prompt = """Who is Leo DiCaprio's girlfriend? What is her current age raised to the 0.43 power?"""
     agent.run(prompt)
 
@@ -50,10 +47,7 @@ Agent默认使用OpenAI的`gpt-3.5-turbo-16k`模型，如果你想要切换为
 
 ```python
 import promptulate as pne
-from promptulate.tools import (
-    DuckDuckGoTool,
-    Calculator,
-)
+from promptulate.tools import DuckDuckGoTool, Calculator
 
 
 def main():

docs/modules/agents/assistant_agent_usage.ipynb

@@ -0,0 +1,66 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "source": [
+    "# Assistant Agent\n",
+    "Assistant Agent is a robust and versatile agent, capable of utilizing various tools, memory, data sources, and more. It functions similarly to the OpenAI Assistant API but offers enhanced power and customization options.\n",
+    "\n",
+    "The following example show how to use it."
+   ],
+   "metadata": {
+    "collapsed": false
+   },
+   "id": "3a848b95bc9db337"
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 2,
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Hello! How can I assist you today?\n"
+     ]
+    }
+   ],
+   "source": [
+    "import promptulate as pne\n",
+    "\n",
+    "llm = pne.ChatOpenAI(model=\"gpt-4-1106-preview\")\n",
+    "res = llm(\"hello\")\n",
+    "print(res)"
+   ],
+   "metadata": {
+    "collapsed": true,
+    "ExecuteTime": {
+     "end_time": "2023-12-13T05:04:32.727844200Z",
+     "start_time": "2023-12-13T05:04:30.871969Z"
+    }
+   },
+   "id": "initial_id"
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 2
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython2",
+   "version": "2.7.6"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}

docs/modules/tools/custom_tool_usage.md

@@ -0,0 +1,99 @@
+# Custom Tool
+Tool is a very important part of Agent. Agent can use tool to do many things, such as get data, process data, and so on. Agent has many built-in tools, but sometimes we need to customize our own tools. Now we will introduce how to custom tool in promptulate.
+
+In promptulate, there are some methods to create tool for agent. Now we will introduce them one by one.
+
+
+We aim to create the most straightforward development approach for developers. To integrate a tool with the language learning model (LLM), you may follow these steps:
+
+In OpenAI's framework:
+1. **Implement the tool's logic**: Write the code that defines what your tool does.
+2. **Declare the tool's schema**: Specify the structure and parameters of your tool.
+
+In Langchain:
+1. **Extend the `Tool` class**: Create a new class that inherits from `Tool`.
+2. **Implement the `_run()` method**: This method should contain the operational logic of your tool.
+3. **Define `name` and `description` attributes**: These should clearly describe your tool's purpose and functionality.
+
+In Promptulate:
+1. **Provide your function**: Simply pass your function to Promptulate.
+2. **Let Promptulate handle the rest**: Promptulate automatically takes care of the necessary configurations and integrations for your tool.
+
+Promptulate simplifies the process, requiring minimal effort from the developer to make a tool functional with LLM. This is an awesome attempt we made, and we hope you will enjoy it! 
+
+## Define your tool
+
+We are going to use the following method to declare your tool. The following example show how to custom a web_search tool by function declared and use tool in ToolAgent. Here we use duckduckgo as search engine.
+
+
+```python
+import promptulate as pne
+from promptulate.tools.duckduckgo.tools import DuckDuckGoTool
+
+
+def web_search(keyword: str) -> str:
+    """search by keyword in web.
+    Args:
+        keyword: keyword to search
+
+    Returns:
+        str: search result
+    """
+    return DuckDuckGoTool().run(keyword)
+
+
+agent = pne.ToolAgent(tools=[web_search])
+resp: str = agent.run("What's the weather in NewYork tomorrow?")
+```
+
+```text
+Agent Start...
+[User] What's the weather in NewYork tomorrow?
+[Thought] I need to find out the weather in New York tomorrow.
+[Action] web_search args: {'keyword': 'weather forecast New York tomorrow'}
+[Observation] Hourly 10-Day Calendar History Wundermap access_time 6:06 PM EST on January 7, 2024 (GMT -5) | Updated 36 minutes ago --° | 33° 36 °F like 36° Cloudy N 0 Tomorrow's temperature is forecast to... News Headlines Briefing on the Winter Storm for this Weekend MY FORECAST Central Park NY Fair 32°F 0°C Today Sunny High: 38°F Tonight Increasing Clouds Low: 29°F change location New York, NY Weather Forecast Office NWS Forecast Office New York, NY Weather.gov > New York, NY Current Hazards Current Conditions Radar Forecasts Rivers and Lakes The National Weather Service issued winter storm warnings for a number of counties in the tri-state area, including Warren, Sussex, Morris, and northern Bergen counties, along with Rockland,... New York (United States of America) weather - Met Office Search Today 2° -3° Partly cloudy changing to clear by nighttime. Sunrise: 07:21 Sunset: 16:42 L UV Sat 6 Jan 2° 1° Sun 7 Jan 3° 0° Mon... New York detailed weather forecast for tomorrow hourly, Weather in New York tomorrow - accurate weather forecast. En Es . Widgets for website . Weather in United States New York . Today Tomorrow 3 Days 7 Days 10 Days Weekend . Weather in New York tomorrow. Sunday, 31 December 2023 . Day 1:00 PM
+[Agent Result] The weather in New York tomorrow will be partly cloudy changing to clear by nighttime.
+Agent End.
+```
+
+As you can see, the only you need to do is to provide a function to Promptulate. Promptulate will automatically convert it to a tool that can be used by the language learning model (LLM). The final presentation result it presents to LLM is an OpenAI type JSON schema declaration.
+
+Actually, Promptulate will analysis function name, parameters type, parameters attribution, annotations and docs when you provide the function. We strongly recommend that you use the official best practices of Template for function writing. The best implementation of a function requires adding type declarations to its parameters and providing function level annotations. Ideally, declare the meaning of each parameter within the annotations. 
+
+If you want to see json schema of your function, you can use the following method to get it.
+
+
+```python
+import json
+from promptulate.tools.base import function_to_tool_schema
+
+
+def web_search(keyword: str) -> str:
+    """search by keyword in web.
+    Args:
+        keyword: keyword to search
+
+    Returns:
+        str: search result
+    """
+    return "web search result"
+
+
+schema = function_to_tool_schema(web_search)
+print(json.dumps(schema, indent=2))
+```
+
+    {
+      "type": "object",
+      "properties": {
+        "keyword": {
+          "type": "string"
+        }
+      },
+      "required": [
+        "keyword"
+      ],
+      "description": "search by keyword in web.\n    Args:\n        keyword: keyword to search\n\n    Returns:\n        str: search result\n    ",
+      "name": "web_search"
+    }
+

docs/modules/tools.md → docs/modules/tools/index.md

@@ -26,6 +26,10 @@ tools模块为LLM提供了调用外部工具扩展的能力，可以说tools是
 
 Tool模块的主要作用就是为Agent提供tool能力支持，详情查看[Agent](modules/agent.md#agent)
 
+## Custom Tool
+
+See detail in [Custom Tool](modules/tools/custom_tool_usage.md#custom-tool)
+
 ## 工具的使用
 
 在大多数情况下，工具用于给Agent使用，而Tool也可以剥离Agent单独进行使用，下面的示例展示了如何使用一个DuckDuckGo进行外部搜索。
@@ -90,77 +94,6 @@ if __name__ == "__main__":
     example()
 ```
 
-## 自定义Tool
-
-promptulate支持自定义tool，其定义方式十分简单，并且提供了函数式和继承式两种方式进行自定义，下面将会展示两种工具定义方式。
-
-### 函数式（推荐）
-
-如果你的tool逻辑较为简单，promptulate提供了方便地函数式工具定义方式，下面的示例展示了一个模拟搜索引擎工具的定义与使用：
-
-```python
-from promptulate.tools import define_tool
-
-
-def web_search(query: str):
-    return f"answer: {query}"
-
-
-def example():
-    tool = define_tool(name="web_search", description="A web search tool", callback=web_search)
-    tool.run("Shanghai weather tomorrow.")
-
-
-if __name__ == '__main__':
-    example()
-```
-
-### 类声明式
-
-如果你的工具逻辑较为复杂，可以使用继承式的定义方式，下面的示例展示了如何自定义一个Tool类，从而构建一个shell工具。
-
-```python
-import warnings
-import sys
-
-from promptulate.tools import Tool
-from promptulate.tools.shell.api_wrapper import ShellAPIWrapper
-
-
-def _get_platform() -> str:
-    """Get platform."""
-    system = sys.platform
-    if system == "Darwin":
-        return "MacOS"
-    return system
-
-
-class ShellTool(Tool):
-    """Tool to run shell commands."""
-
-    name: str = "terminal"
-    description: str = f"Run shell commands on this {_get_platform()} machine."
-
-    def _run(self, command: str) -> str:
-        warnings.warn(
-            "The shell tool has no safeguards by default. Use at your own risk."
-        )
-        """Run commands and return final output."""
-        return ShellAPIWrapper().run(command)
-
-
-def example():
-    tool = ShellTool()
-    tool.run("echo HelloWorld")
-
-
-if __name__ == '__main__':
-    example()
-```
-
-上面的示例继承了Tool，并且需要实现name和description两个属性，用于给Agent构建system prompt的输入，此外，你还需要实现_run方法，通过_run来运行tool，对于一个复杂的Tool，你可以采用上面的方式进行定义与逻辑处理。
-
-
 ## 有LLM能力的Tool
 
 在`promptulate`中，为了构建更加智能的Agent，一些提供给Agent的Tool也是有大语言模型调用权限的，它们一般有一些简单的能力处理功能。如果你有需要，你可以直接使用这些带有LLM的Tool，下一章节会演示如何使用Tool。