NB0-SanityCheck NB1 - 0x8b6-V1.0.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# NB #0:  Sanity Check NB1 - 0x8b6-V1.0\n",
    "\n",
    "The purpose of this notebook is to compare the accuracy of our model and its various ways of data gathering against actual pool history.\n",
    "\n",
    "This simulation will compare three modes:\n",
    "\n",
    "1- **Contract Call:** getting input data from user contract call and exact smart contract method( parsed from decoded LOG_CALL anonymous events that balancer uses to log all activity) and apply our model to get outputs\n",
    "\n",
    "2- **Simplified:** getting input data from pool events (LOG_JOIN, LOG_EXIT, LOG_SWAP) parsed from BigQuery and applying our model to get outputs. This method should have more deviations because it's assuming the flavour of BPool join, swap, exit, joinswap and exitswap and using after the fact output data as input.\n",
    "\n",
    "3- **Relay Output:** just adding and removing balances as seen in actual on-chain pool events"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# A. System Context"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### A1.1 System Specification\n",
    "System specification details are available in the Balancer Simulations documentation:\n",
    "- [Mathematical Specification](https://token-engineering-balancer.gitbook.io/balancer-simulations/additional-code-and-instructions/balancer-the-python-edition/balancer_math.py)\n",
    "- [Model Architecture](https://token-engineering-balancer.gitbook.io/balancer-simulations/balancer-simulations/v10nboverview)\n",
    "- [Naming Convention](https://token-engineering-balancer.gitbook.io/balancer-simulations/additional-code-and-instructions/naming-convention)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# B. cadCAD Simulations"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### B1.1 Dependencies"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd \n",
    "from cadCAD.configuration.utils import config_sim"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### B1.2 Initialize Pool\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this section you specify the Balancer Pool to be analyzed and run scripts to produce the'genesis_state', based on on-chain data. \n",
    "\n",
    "More information here: [Balancer Simulations documentation](https://token-engineering-balancer.gitbook.io/balancer-simulations/additional-code-and-instructions/onchaintransactions).  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pprint\n",
    "pp = pprint.PrettyPrinter(indent=4)\n",
    "\n",
    "from decimal import Decimal\n",
    "parameters = {\n",
    "    'spot_price_reference': ['DAI', 'DAI', 'DAI'],\n",
    "    'decoding_type': ['REPLAY_OUTPUT', 'SIMPLIFIED', 'CONTRACT_CALL']\n",
    "}\n",
    "pp.pprint(parameters)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Import Pool Transactions and Initialize Pool state\n",
    "\n",
    "Genesis state, pool transactions, and (external) USD token prices are pulled from on-chain/API data collected in a .json file.  \n",
    "**Choose *your* Balancer Pool, and produce this .json file first, then add the file path below.**\n",
    "\n",
    "Learn more how to create this file using parsing scripts in the [documentation](https://token-engineering-balancer.gitbook.io/balancer-simulations/v/master/additional-code-and-instructions/onchaintransactions).\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from model.genesis_states import generate_initial_state\n",
    "\n",
    "initial_values = generate_initial_state(initial_values_json='data/0x8b6e6e7b5b3801fed2cafd4b22b8a16c2f2db21a-initial_pool_states-prices.json', spot_price_base_currency=parameters['spot_price_reference'][0])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### State Variables and Initial Values"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print('## State Variables')\n",
    "print('# Pool')\n",
    "pool = initial_values['pool']\n",
    "pp.pprint(initial_values)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### External Token Prices"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print('# External token prices, initial state')\n",
    "token_prices = initial_values['token_prices']\n",
    "pp.pprint(token_prices)\n",
    "\n",
    "print('# Action Type')\n",
    "action_type = initial_values['action_type']\n",
    "pp.pprint(action_type)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### B1.3 State Update Functions & Policies\n",
    "\n",
    "Balancer Simulations replicate Balancer Pool Transactions in state update functions.  \n",
    "For a detailed description, please visit the [Balancer Simulations documentation](https://token-engineering-balancer.gitbook.io/balancer-simulations/additional-code-and-instructions/balancer-the-python-edition)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### B1.4 Partial State Update Blocks\n",
    "\n",
    "Partial State Update Blocks combine the following steps:  \n",
    "1. **Initialize pool**, generate genesis state\n",
    "2. Compute subsequent **actions in discrete timesteps**, store datetime and update the pool state variables\n",
    "3. Update external **USD prices in discrete timesteps**, and store datetime  \n",
    "\n",
    "For more information please visit the [Balancer Simulations documentation/Model Overview](https://token-engineering-balancer.gitbook.io/balancer-simulations/v/master/balancer-simulations/v10nboverview).  \n",
    "\n",
    "To inject historical on-chain transactions to the model, reference the **actions.json of your pool below**."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from model.partial_state_update_block import generate_partial_state_update_blocks\n",
    "\n",
    "result = generate_partial_state_update_blocks('data/0x8b6e6e7b5b3801fed2cafd4b22b8a16c2f2db21a-actions-prices.json')\n",
    "partial_state_update_blocks = result['partial_state_update_blocks']\n",
    "pp.pprint(partial_state_update_blocks)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### B1.5 Configuration\n",
    "\n",
    "[cadCAD simulations](https://github.com/cadCAD-org/cadCAD/blob/master/documentation/README.md) support Monte Carlo runs, and parameter sweeps which are not applied in this notebook and can be ignored here.  \n",
    "Timesteps 'T' are defined by your pool's .json file, no need to make edits here."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "steps_number = result['steps_number']\n",
    "print('# Steps ', steps_number)\n",
    "sim_config = config_sim(\n",
    "    {\n",
    "        'N': 1,  # number of monte carlo runs\n",
    "        'T': range(steps_number - 1),  # number of timesteps\n",
    "        'M': parameters,  # simulation parameters\n",
    "    }\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### C1.5 Execution"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "\n",
    "from model.sim_runner import *\n",
    "\n",
    "df = run(initial_values, partial_state_update_blocks, sim_config)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### B1.7 Simulation Output Preparation\n",
    "\n",
    "Post-processing (utils.py) splits up state variable dictionaries, and adds metrics to the data frame, such as \n",
    "- `token_total_value` (Total Value Locked in the pool)\n",
    "- `invariant` (Invariant V of the pool)\n",
    "- `generated_fees_(tokensymbol)` (fee collected per transaction)\n",
    "- `token_k_values`  (token value in USD)  \n",
    "based on  \n",
    "- `token_k_balances`\n",
    "- `token_k_price`  (external price feed)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "from model.parts.utils import post_processing\n",
    "\n",
    "p_df = post_processing(df, include_spot_prices=False)\n",
    "\n",
    "p_df"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# D. Simulation Outcome – Verification of Results\n",
    "\n",
    "Below we show a range of plots showing potential deviation of the simulation output vs. the actual on-chain pool balances (Relay Output).\n",
    "\n",
    "Modes for reference:\n",
    "\n",
    "1- **Contract Call:** getting input data from user contract call and exact smart contract method( parsed from decoded LOG_CALL anonymous events that balancer uses to log all activity) and apply our model to get outputs\n",
    "\n",
    "2- **Simplified:** getting input data from pool events (LOG_JOIN, LOG_EXIT, LOG_SWAP) parsed from BigQuery and applying our model to get outputs. This method should have more deviations because it's assuming the flavour of BPool join, swap, exit, joinswap and exitswap and using after the fact output data as input.\n",
    "\n",
    "3- **Relay Output:** just adding and removing balances as seen in actual on-chain pool events"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "import plotly.express as px\n",
    "import plotly.graph_objects as go\n",
    "from plotly.subplots import make_subplots"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### D1.1 Token Balances"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "k = 2 #define number of tokens in your pool\n",
    "fig = make_subplots(rows=k, cols=1)\n",
    "\n",
    "fig.add_trace(go.Scatter(x=p_df.timestep, y=p_df.token_dai_balance, line=dict(color='#4675ed'), name='token_dai_balance'), row=1, col=1)\n",
    "fig.add_trace(go.Scatter(x=p_df.timestep, y=p_df.token_weth_balance, line=dict(color='#4145ab'), name='token_weth_balance'), row=2, col=1)\n",
    "fig.update_layout(height=400, width=1000, title_text=\"<b>Token Balances in #</b>\")\n",
    "fig.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Simplified Mode = subset 0\n",
    "# Contract Call Mode = subset 1\n",
    "# Relay Output = subset 2\n",
    "\n",
    "fig = px.scatter(p_df,x=p_df['timestep'], y=p_df['token_weth_balance'], facet_row=\"subset\")\n",
    "fig.update_layout(height=800, width=1000, title_text=\"<b>WETH Balances all modes</b>\")\n",
    "fig.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "# Simplified Mode = subset 0\n",
    "# Contract Call Mode = subset 1\n",
    "# Relay Output = subset 2\n",
    "\n",
    "\n",
    "fig2 = px.scatter(p_df,x=p_df['timestep'], y=p_df['token_dai_balance'], facet_row=\"subset\")\n",
    "fig2.update_layout(height=800, width=1000, title_text=\"<b>DAI Balances all modes</b>\")\n",
    "fig2.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### D1.2 Deviations"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_simplified = p_df[p_df['subset'] == 0]\n",
    "df_simplified = df_simplified[['change_datetime','token_dai_balance', 'token_weth_balance']].copy()\n",
    "df_simplified = df_simplified.rename(columns={\"token_dai_balance\" : \"token_dai_balance_simplified\", 'token_weth_balance': 'token_weth_balance_simplified' })\n",
    "df_simplified = df_simplified.set_index('change_datetime')\n",
    "\n",
    "df_contract_call = p_df[p_df['subset'] == 1]\n",
    "df_contract_call = df_contract_call[['change_datetime','token_dai_balance', 'token_weth_balance']].copy()\n",
    "df_contract_call = df_contract_call.rename(columns={\"token_dai_balance\" : \"token_dai_balance_contract_call\", 'token_weth_balance': 'token_weth_balance_contract_call' })\n",
    "df_contract_call = df_contract_call.set_index('change_datetime')\n",
    "\n",
    "df_reference = p_df[p_df['subset'] == 2] \n",
    "df_reference = df_reference[['change_datetime', 'token_dai_balance', 'token_weth_balance']].copy()\n",
    "df_reference = df_reference.rename(columns={\"token_dai_balance\" : \"token_dai_balance_reference\", 'token_weth_balance': 'token_weth_balance_reference' })\n",
    "df_reference =df_reference.set_index('change_datetime')\n",
    "\n",
    "df_result = df_reference.join(df_contract_call).join(df_simplified)\n",
    "\n",
    "df_result['token_dai_error_simplified'] = (df_result['token_dai_balance_simplified']-df_result['token_dai_balance_reference'])/df_result['token_dai_balance_reference']\n",
    "df_result['token_weth_error_simplified'] = (df_result['token_weth_balance_simplified']-df_result['token_weth_balance_reference'])/df_result['token_weth_balance_reference']\n",
    "df_result['token_dai_error_contract_call'] = (df_result['token_dai_balance_contract_call']-df_result['token_dai_balance_reference'])/df_result['token_dai_balance_reference']\n",
    "df_result['token_weth_error_contract_call'] = (df_result['token_weth_balance_contract_call']-df_result['token_weth_balance_reference'])/df_result['token_dai_balance_reference']\n",
    "\n",
    "\n",
    "df_result = df_result.reset_index()\n",
    "\n",
    "df_result\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# DAI Deviation Simplified Mode vs. Relay Output\n",
    "\n",
    "fig_diff_simplified_dai = px.scatter(p_df,x=df_result.index, y=df_result['token_dai_error_simplified'] )\n",
    "fig_diff_simplified_dai.update_layout(height=800, width=1000, title_text=\"<b>DAI Deviation Simplified Mode vs. Relay Output</b>\")\n",
    "fig_diff_simplified_dai.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# DAI Deviation Contract_Call Mode vs. Relay Output\n",
    "\n",
    "fig_diff_contractcall_dai = px.scatter(p_df,x=df_result.index, y=df_result['token_dai_error_contract_call'])\n",
    "fig_diff_contractcall_dai.update_layout(height=800, width=1000, title_text=\"<b>DAI Deviation Contract Call Mode vs. Relay Output</b>\")\n",
    "fig_diff_contractcall_dai.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# WETH Deviation Simplified Mode vs. Relay Output\n",
    "\n",
    "\n",
    "fig_diff_simplified_weth = px.scatter(p_df,x=df_result.index, y=df_result['token_weth_error_simplified'])\n",
    "fig_diff_simplified_weth.update_layout(height=800, width=1000, title_text=\"<b>WETH Deviation Simplified Mode vs. Relay Output</b>\")\n",
    "fig_diff_simplified_weth.show()\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# WETH Deviation Contract_Call Mode vs. Relay Output\n",
    "\n",
    "\n",
    "fig_diff_contractcall_weth = px.scatter(p_df,x=df_result.index, y=df_result['token_weth_error_contract_call'])\n",
    "fig_diff_contractcall_weth.update_layout(height=800, width=1000, title_text=\"<b>WETH Deviation Contract Call Mode vs. Relay Output</b>\")\n",
    "fig_diff_contractcall_weth.show()\n"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}