deploy: c187983

_modules/agentscope/agents/rpc_agent.html

@@ -122,15 +122,15 @@ <h1>Source code for agentscope.agents.rpc_agent</h1><div class="highlight"><pre>
 <span class="kn">from</span> <span class="nn">agentscope._init</span> <span class="kn">import</span> <span class="n">_INIT_SETTINGS</span>
 <span class="kn">from</span> <span class="nn">agentscope._init</span> <span class="kn">import</span> <span class="n">init</span>
 <span class="kn">from</span> <span class="nn">agentscope.agents.agent</span> <span class="kn">import</span> <span class="n">AgentBase</span>
-<span class="kn">from</span> <span class="nn">agentscope.rpc.rpc_agent_pb2</span> <span class="kn">import</span> <span class="n">RpcMsg</span>  <span class="c1"># pylint: disable=E0611</span>
 <span class="kn">from</span> <span class="nn">agentscope.message</span> <span class="kn">import</span> <span class="n">MessageBase</span>
 <span class="kn">from</span> <span class="nn">agentscope.message</span> <span class="kn">import</span> <span class="n">Msg</span>
 <span class="kn">from</span> <span class="nn">agentscope.message</span> <span class="kn">import</span> <span class="n">PlaceholderMessage</span>
 <span class="kn">from</span> <span class="nn">agentscope.message</span> <span class="kn">import</span> <span class="n">deserialize</span>
 <span class="kn">from</span> <span class="nn">agentscope.message</span> <span class="kn">import</span> <span class="n">serialize</span>
 <span class="kn">from</span> <span class="nn">agentscope.utils.logging_utils</span> <span class="kn">import</span> <span class="n">logger</span>
-<span class="kn">from</span> <span class="nn">agentscope.rpc.rpc_agent_client</span> <span class="kn">import</span> <span class="n">RpcAgentClient</span>
-<span class="kn">from</span> <span class="nn">agentscope.rpc.rpc_agent_pb2_grpc</span> <span class="kn">import</span> <span class="p">(</span>
+<span class="kn">from</span> <span class="nn">agentscope.rpc</span> <span class="kn">import</span> <span class="p">(</span>
+    <span class="n">RpcAgentClient</span><span class="p">,</span>
+    <span class="n">RpcMsg</span><span class="p">,</span>
     <span class="n">RpcAgentServicer</span><span class="p">,</span>
     <span class="n">add_RpcAgentServicer_to_server</span><span class="p">,</span>
 <span class="p">)</span>

_modules/agentscope/message.html

@@ -97,7 +97,7 @@ <h1>Source code for agentscope.message</h1><div class="highlight"><pre>
 
 <span class="kn">from</span> <span class="nn">loguru</span> <span class="kn">import</span> <span class="n">logger</span>
 
-<span class="kn">from</span> <span class="nn">.rpc.rpc_agent_client</span> <span class="kn">import</span> <span class="n">RpcAgentClient</span>
+<span class="kn">from</span> <span class="nn">.rpc</span> <span class="kn">import</span> <span class="n">RpcAgentClient</span>
 <span class="kn">from</span> <span class="nn">.utils.tools</span> <span class="kn">import</span> <span class="n">_get_timestamp</span>
 
 

_modules/agentscope/rpc/rpc_agent_client.html

@@ -91,13 +91,19 @@ <h1>Source code for agentscope.rpc.rpc_agent_client</h1><div class="highlight"><
 <span></span><span class="c1"># -*- coding: utf-8 -*-</span>
 <span class="sd">&quot;&quot;&quot; Client of rpc agent server &quot;&quot;&quot;</span>
 
+<span class="kn">from</span> <span class="nn">typing</span> <span class="kn">import</span> <span class="n">Any</span>
+
 <span class="k">try</span><span class="p">:</span>
     <span class="kn">import</span> <span class="nn">grpc</span>
 <span class="k">except</span> <span class="ne">ImportError</span><span class="p">:</span>
     <span class="n">grpc</span> <span class="o">=</span> <span class="kc">None</span>
 
-<span class="kn">from</span> <span class="nn">agentscope.rpc.rpc_agent_pb2</span> <span class="kn">import</span> <span class="n">RpcMsg</span>  <span class="c1"># pylint: disable=E0611</span>
-<span class="kn">from</span> <span class="nn">agentscope.rpc.rpc_agent_pb2_grpc</span> <span class="kn">import</span> <span class="n">RpcAgentStub</span>
+<span class="k">try</span><span class="p">:</span>
+    <span class="kn">from</span> <span class="nn">agentscope.rpc.rpc_agent_pb2</span> <span class="kn">import</span> <span class="n">RpcMsg</span>  <span class="c1"># pylint: disable=E0611</span>
+    <span class="kn">from</span> <span class="nn">agentscope.rpc.rpc_agent_pb2_grpc</span> <span class="kn">import</span> <span class="n">RpcAgentStub</span>
+<span class="k">except</span> <span class="ne">ModuleNotFoundError</span><span class="p">:</span>
+    <span class="n">RpcMsg</span> <span class="o">=</span> <span class="n">Any</span>
+    <span class="n">RpcAgentStub</span> <span class="o">=</span> <span class="n">Any</span>
 
 
 <div class="viewcode-block" id="RpcAgentClient">

_sources/tutorial/207-monitor.md.txt

@@ -2,9 +2,9 @@
 
 # Monitor
 
-In multi-agent applications, particularly those that rely on external model APIs, it's crucial to monitor the usage and cost to prevent overutilization and ensure compliance with rate limits. The `MonitorBase` class and its implementation, `DictMonitor`, provide a way to track and regulate the usage of such APIs in your applications. In this tutorial, you'll learn how to use them to monitor API calls.
+In multi-agent applications, particularly those that rely on external model APIs, it's crucial to monitor the usage and cost to prevent overutilization and ensure compliance with rate limits. The `MonitorBase` class and its implementation, `SqliteMonitor`, provide a way to track and regulate the usage of such APIs in your applications. In this tutorial, you'll learn how to use them to monitor API calls.
 
-## Understanding the `MonitorBase` Class
+## Understanding the Monitor in AgentScope
 
 The `MonitorBase` class serves as an interface for setting up a monitoring system that tracks various metrics, especially focusing on API usage. It defines methods that enable registration, checking, updating, and management of metrics related to API calls.
 
@@ -22,36 +22,41 @@ Here are the key methods of `MonitorBase`:
 - **`set_quota`**: Adjusts the quota for a metric, if the terms of API usage change.
 - **`get_metric`**: Returns detailed information about a specific metric.
 - **`get_metrics`**: Retrieves information about all tracked metrics, with optional filtering based on metric names.
+- **`register_budget`**: Sets a budget for a certain API call, which will initialize a series of metrics used to calculate the cost.
 
-## Using the `DictMonitor` Class
+## Using the Monitor
 
-The `DictMonitor` class is a subclass of the `MonitorBase` class, which is implemented in an in-memory dictionary.
+### Get a Monitor Instance
 
-### Initializing DictMonitor
-
-Create an instance of `DictMonitor` to begin monitoring:
+Get a monitor instance from `MonitorFactory` to begin monitoring, and note that multiple calls to the `get_monitor` method return the same monitor instance.
 
 ```python
-monitor = DictMonitor()
+# make sure you have called agentscope.init(...) before
+monitor = MonitorFactory.get_monitor()
 ```
 
-### Registering API Usage Metrics
+> Currently the above code returns a `SqliteMonitor` instance, which is initialized in `agentscope.init`.
+> The `SqliteMonitor` class is the default implementation of `MonitorBase` class, which is based on Sqlite3.
+
+### Basic Usage
+
+#### Registering API Usage Metrics
 
 Register a new metric to start monitoring the number of tokens:
 
 ```python
 monitor.register("token_num", metric_unit="token", quota=1000)
 ```
 
-### Updating Metrics
+#### Updating Metrics
 
 Increment the `token_num` metric:
 
 ```python
 monitor.add("token_num", 20)
 ```
 
-### Handling Quotas
+#### Handling Quotas
 
 If the number of API calls exceeds the quota, a `QuotaExceededError` will be thrown:
 
@@ -63,15 +68,15 @@ except QuotaExceededError as e:
     print(e.message)
 ```
 
-### Retrieving Metrics
+#### Retrieving Metrics
 
 Get the current number of tokens used:
 
 ```python
 token_num_used = monitor.get_value("token_num")
 ```
 
-### Resetting and Removing Metrics
+#### Resetting and Removing Metrics
 
 Reset the number of token count at the start of a new period:
 
@@ -85,16 +90,83 @@ Remove the metric if it's no longer needed:
 monitor.remove("token_num")
 ```
 
-## Using Singleton Access
+### Advanced Usage
+
+> Features here are under development, the interface may continue to change.
 
-`MonitorFactory` provides a singleton instance of a `MonitorBase` to ensure consistent access throughout your application.
+#### Using `prefix` to Distinguish Metrics
 
-### Acquiring the Singleton Monitor Instance
+Assume you have multiple agents/models that use the same API call, but you want to calculate their token usage separately, you can add a unique `prefix` before the original metric name, and `get_full_name` provides such functionality.
 
-Get the singleton instance of the monitor:
+For example, if model_A and model_B both use the OpenAI API, you can register these metrics by the following code.
 
 ```python
-monitor = MonitorFactory.get_monitor()
+from agentscope.utils.monitor import get_full_name
+
+...
+
+# in model_A
+monitor.register(get_full_name('prompt_tokens', 'model_A'))
+monitor.register(get_full_name('completion_tokens', 'model_A'))
+
+# in model_B
+monitor.register(get_full_name('prompt_tokens', 'model_B'))
+monitor.register(get_full_name('completion_tokens', 'model_B'))
+```
+
+To update those metrics, just use the `update` method.
+
+```python
+# in model_A
+monitor.update(openai_response.usage.model_dump(), prefix='model_A')
+
+# in model_B
+monitor.update(openai_response.usage.model_dump(), prefix='model_B')
+```
+
+To get metrics of a specific model, please use the `get_metrics` method.
+
+```python
+# get metrics of model_A
+model_A_metrics = monitor.get_metrics('model_A')
+
+# get metrics of model_B
+model_B_metrics = monitor.get_metrics('model_B')
 ```
 
+#### Register a budget for an API
+
+Currently, the Monitor already supports automatically calculating the cost of API calls based on various metrics, and you can directly set a budget of a model to avoid exceeding the quota.
+
+Suppose you are using `gpt-4-turbo` and your budget is $10, you can use the following code.
+
+```python
+model_name = 'gpt-4-turbo'
+monitor.register_budget(model_name=model_name, value=10, prefix=model_name)
+```
+
+Use `prefix` to set budgets for different models that use the same API.
+
+```python
+model_name = 'gpt-4-turbo'
+# in model_A
+monitor.register_budget(model_name=model_name, value=10, prefix=f'model_A.{model_name}')
+
+# in model_B
+monitor.register_budget(model_name=model_name, value=10, prefix=f'model_B.{model_name}')
+```
+
+`register_budget` will automatically register metrics that are required to calculate the total cost, calculate the total cost when these metrics are updated, and throw a `QuotaExceededError` when the budget is exceeded.
+
+```python
+model_name = 'gpt-4-turbo'
+try:
+    monitor.update(openai_response.usage.model_dump(), prefix=model_name)
+except QuotaExceededError as e:
+    # Handle the exceeded quota
+    print(e.message)
+```
+
+> **Note:** This feature is still in the experimental stage and only supports some specified APIs, which are listed in `agentscope.utils.monitor._get_pricing`.
+
 [[Return to the top]](#monitoring-and-logging)

_sources/tutorial/301-community.md.txt

@@ -11,15 +11,15 @@ Becoming a part of the AgentScope community allows you to connect with other use
 
 ## Discord
 
-- **Join our Discord:** Collaborate with the AgentScope community in real-time. Engage in discussions, seek assistance, and share your experiences and insights on [Discord](https://discord.gg/Fwg5hZ2S).
+- **Join our Discord:** Collaborate with the AgentScope community in real-time. Engage in discussions, seek assistance, and share your experiences and insights on [Discord](https://discord.gg/eYMpfnkG8h).
 
 ## DingTalk (钉钉)
 
 - **Connect on DingTalk:** We are also available on DingTalk. Join our group to chat, and stay informed about AgentScope-related news and updates.
 
   Scan the QR code below on DingTalk to join:
 
-  <img width="150" src="https://img.alicdn.com/imgextra/i2/O1CN01tuJ5971OmAqNg9cOw_!!6000000001747-0-tps-444-460.jpg" alt="AgentScope-logo">
+  <img width="150" src="https://img.alicdn.com/imgextra/i2/O1CN01tuJ5971OmAqNg9cOw_!!6000000001747-0-tps-444-460.jpg" alt="AgentScope-dingtalk">
 
   Our DingTalk group invitation: [AgentScope DingTalk Group](https://qr.dingtalk.com/action/joingroup?code=v1,k1,20IUyRX5XZQ2vWjKDsjvI9dhcXjGZi3bq1pFfDZINCM=&_dt_no_comment=1&origin=11)