Merge pull request #150 from openchatai/enhancement/state_manager

Adding readme for state manager

llm-server/docs/3.state_manager.md

@@ -1,90 +1,31 @@
-### We should capture the initial requests required to fetch the bootstrapping data from external sources, then we should store the results in one of those boxes
+To integrate a third-party application (e.g., Trello) with the OpenCopilot's LLM server, users should create functions similar to the provided `process_state` function and store them in the specified directory. Here's a step-by-step guide on what users need to do:
 
-```json
-{
-  "appId": "XKCD",
-  "appName": "XKCD",
-  "description": "App description (Helps in alignining the bot)",
-  "entities": {
-    "users": {
-      "dataSource": "Fetch users",
-      "parseFunction": "def parse(text): \n users = json.loads(text)['users'] \n return users",
-      "filterFunction": "def filter(data): \n return [user for user in data if user['active']]",
-      "transformFunction": "def transform(data): \n for user in data: user['name'] = user['first'] + ' ' + user['last'] \n return data",
-      "data": []
-    },
-    "products": {
-      "dataSource": "Fetch products",
-      "data": [],
-      "parseFunction": "def parse(text): \n data = xmltodict.parse(text) \n products = data['products']['product'] \n return products",
-      "filterFunction": "def filter(data): \n return [p for p in data if p['in_stock'] == 'true']",
-      "transformFunction": "def transform(data): \n for p in data: p['price'] = float(p['price']) \n return data"
-    }
-  },
-  "relationships": [
-    {
-      "from": "users",
-      "to": "products",
-      "type": "purchase"
-    }
-  ]
-}
-```
+1. **Access the Directory**:
 
-- `bootstrapRequests` captures the initial API calls required
-- Each entity maps to a `dataSource` request 
-- The raw API response can be parsed into the `data` array
+   Users should navigate to the directory where the integration functions need to be stored. In this case, the directory path is:
 
+   ```
+   OpenCopilot/llm-server/integrations/
+   ```
 
+2. **Create a Python File**:
 
-### How do we actually load the state
+   Users should create a new Python file with a meaningful name that reflects the integration they are building. For example, if integrating with Trello, they can name it `trello.py`. This file will contain the integration functions.
 
-```python
-import requests
-import pymongo
+3. **Write Integration Functions**:
 
-# Load initial state template from DB 
-init_state = db.init_state.find_one({"appId": "myApp"})
+   Inside the `trello.py` file, users should define functions similar to the `process_state` function provided in the example. These functions should perform the following tasks:
 
-for request in init_state["bootstrapRequests"]:
+   - Interact with the third-party application's API to retrieve relevant data.
+   - Transform the data to meet the specific requirements or format needed for the LLM or other applications.
 
-  url = request["endpoint"]
-  method = request["method"]  
+   Users should ensure that their functions are organized, well-documented, and designed to provide context to the LLM server. The functions should return data in a structured format.
 
-  response = requests.request(method, url)
+   For example, an integration function for Trello could retrieve data about Trello cards or members and transform it as required.
 
-  entity = init_state["entities"][request["key"]]
-
-  # Execute parse function
-  data = exec(entity["parseFunction"], {"text": response.text})
-
-  # Execute filter function
-  filtered = exec(entity["filterFunction"], {"data": data})  
-
-  # Execute transform function
-  transformed = exec(entity["transformFunction"], {"data": filtered})
-
-  # Store final data
-  entity["data"] = transformed
-
-# Save updated initial state back to DB
-db.init_state.update_one({"appId": "myApp"}, {"$set": init_state})
-
-print("Initialized state for app:", init_state["appName"])
-print("Loaded entities:", list(init_state["entities"].keys()))
-```
-
-The key steps are:
-
-- Load template state for app from DB
-- Make bootstrap requests 
-- For each entity, execute parse, filter, transform functions
-- Update entity data with final result
-- Save full initialized state back to DB
-
-This allows dynamically initializing the state by executing the logic defined in the template.
-
-
----
+4. **Save the File**:
 
+   Once the integration functions are defined, users should save the `trello.py` file in the specified directory.
 
+5. **Usage in LLM Server**:
+Usage is straightforward. You can invoke the integration functions within the LLM server by specifying the integration file's name when making calls to the server. The server will automatically detect and run the necessary functions, loading the data into memory. It will also periodically refresh the data to ensure it remains up-to-date and relevant.