predict-solubility-using-graph-convolutional-network.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Credit \n",
    "\n",
    "https://medium.com/@tejpal.abhyuday/application-of-gnn-for-calculating-the-solubility-of-molecule-graph-level-prediction-8bac5fabf600"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Predicting molecule solubility using graph neural networks\n",
    "\n",
    "The goal of using graph-based representations in the `torch_geometric` library is to enable the application of graph neural networks (GNNs) and other graph-based machine learning techniques for tasks such as predicting solubility or other chemical properties. GNNs can operate directly on these graph structures, taking into account the connectivity of atoms and the associated node and edge features to make predictions.\n",
    "\n",
    "[Source: ChatGPT]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## The ESOL dataset\n",
    "\n",
    "The ESOL dataset, which stands for \"Extended Solubility\", is a dataset commonly used in the field of cheminformatics and machine learning for predicting the solubility of chemical compounds. The dataset contains information about the solubility of various organic molecules in water.\n",
    "\n",
    "1. **Graph Representation**: Each chemical compound is represented as a graph, where atoms are nodes, and chemical bonds are edges. This graph representation captures the connectivity and structure of the molecule.\n",
    "\n",
    "2. **Node Features**: The node feature vectors, in this case, typically represent the properties of individual atoms within the molecule. These features can include:\n",
    "\n",
    "   - **Atom Type**: Each atom is assigned a specific atom type based on its element (e.g., carbon, hydrogen, oxygen, etc.). This is often one-hot encoded or represented as a categorical feature.\n",
    "\n",
    "   - **Atomic Charges**: The partial charges on each atom, which describe the distribution of electric charge within the molecule.\n",
    "\n",
    "   - **Hybridization**: Information about the hybridization state of each atom (e.g., sp3, sp2, sp).\n",
    "\n",
    "   - **Atomic Mass**: The mass of each atom.\n",
    "\n",
    "   - **Formal Charge**: The formal charge on each atom.\n",
    "\n",
    "   - **Other Atom-specific Properties**: Depending on the specific implementation, additional atom-specific properties may also be included as node features.\n",
    "\n",
    "3. **Edge Features**: In addition to node features, edge features can be included in the graph representation. These features typically describe the type of chemical bond between connected atoms (e.g., single, double, or triple bonds) and may also include bond distances or bond angles.\n",
    "\n",
    "4. **Graph Structure**: The graph structure itself is represented by adjacency matrices or edge lists, which define how atoms are connected by chemical bonds.\n",
    "\n",
    "[Source: ChatGPT]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "ESOL(1128)"
      ]
     },
     "execution_count": 1,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from torch_geometric.datasets import MoleculeNet\n",
    " \n",
    "# https://pytorch-geometric.readthedocs.io/en/latest/generated/torch_geometric.datasets.MoleculeNet.html#torch_geometric.datasets.MoleculeNet\n",
    "DATA = MoleculeNet(root=\".\", name=\"ESOL\")\n",
    "DATA"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Investigate the dataset\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Dataset type:  <class 'torch_geometric.datasets.molecule_net.MoleculeNet'>\n",
      "Number of features per graph node:  9\n",
      "Number of distinct target values (solubilities):  734\n",
      "Number of graphs:  1128\n",
      "Example graph:  Data(x=[32, 9], edge_index=[2, 68], edge_attr=[68, 3], smiles='OCC3OC(OCC2OC(OC(C#N)c1ccccc1)C(O)C(O)C2O)C(O)C(O)C3O ', y=[1, 1])\n",
      "Number of nodes in example graph:  32\n",
      "Number of edges in example graph:  68\n"
     ]
    }
   ],
   "source": [
    "print(\"Dataset type: \", type(DATA))\n",
    "print(\"Number of features per graph node: \", DATA.num_features)\n",
    "print(\"Number of distinct target values (solubilities): \", DATA.num_classes)\n",
    "print(\"Number of graphs: \", len(DATA))\n",
    "print(\"Example graph: \", DATA[0])\n",
    "print(\"Number of nodes in example graph: \", DATA[0].num_nodes)\n",
    "print(\"Number of edges in example graph: \", DATA[0].num_edges)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[8, 0, 2, 5, 1, 0, 4, 0, 0],\n",
       "        [6, 0, 4, 5, 2, 0, 4, 0, 0],\n",
       "        [6, 0, 4, 5, 1, 0, 4, 0, 1],\n",
       "        [8, 0, 2, 5, 0, 0, 4, 0, 1],\n",
       "        [6, 0, 4, 5, 1, 0, 4, 0, 1],\n",
       "        [8, 0, 2, 5, 0, 0, 4, 0, 0],\n",
       "        [6, 0, 4, 5, 2, 0, 4, 0, 0],\n",
       "        [6, 0, 4, 5, 1, 0, 4, 0, 1],\n",
       "        [8, 0, 2, 5, 0, 0, 4, 0, 1],\n",
       "        [6, 0, 4, 5, 1, 0, 4, 0, 1],\n",
       "        [8, 0, 2, 5, 0, 0, 4, 0, 0],\n",
       "        [6, 0, 4, 5, 1, 0, 4, 0, 0],\n",
       "        [6, 0, 2, 5, 0, 0, 2, 0, 0],\n",
       "        [7, 0, 1, 5, 0, 0, 2, 0, 0],\n",
       "        [6, 0, 3, 5, 0, 0, 3, 1, 1],\n",
       "        [6, 0, 3, 5, 1, 0, 3, 1, 1],\n",
       "        [6, 0, 3, 5, 1, 0, 3, 1, 1],\n",
       "        [6, 0, 3, 5, 1, 0, 3, 1, 1],\n",
       "        [6, 0, 3, 5, 1, 0, 3, 1, 1],\n",
       "        [6, 0, 3, 5, 1, 0, 3, 1, 1],\n",
       "        [6, 0, 4, 5, 1, 0, 4, 0, 1],\n",
       "        [8, 0, 2, 5, 1, 0, 4, 0, 0],\n",
       "        [6, 0, 4, 5, 1, 0, 4, 0, 1],\n",
       "        [8, 0, 2, 5, 1, 0, 4, 0, 0],\n",
       "        [6, 0, 4, 5, 1, 0, 4, 0, 1],\n",
       "        [8, 0, 2, 5, 1, 0, 4, 0, 0],\n",
       "        [6, 0, 4, 5, 1, 0, 4, 0, 1],\n",
       "        [8, 0, 2, 5, 1, 0, 4, 0, 0],\n",
       "        [6, 0, 4, 5, 1, 0, 4, 0, 1],\n",
       "        [8, 0, 2, 5, 1, 0, 4, 0, 0],\n",
       "        [6, 0, 4, 5, 1, 0, 4, 0, 1],\n",
       "        [8, 0, 2, 5, 1, 0, 4, 0, 0]])"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# nodes of example graph \n",
    "DATA[0].x # shape: [num_nodes, num_node_features]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[-0.7700]])"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# the target value of the example graph is its solubility\n",
    "DATA[0].y"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[ 0,  1],\n",
       "        [ 1,  0],\n",
       "        [ 1,  2],\n",
       "        [ 2,  1],\n",
       "        [ 2,  3],\n",
       "        [ 2, 30],\n",
       "        [ 3,  2],\n",
       "        [ 3,  4],\n",
       "        [ 4,  3],\n",
       "        [ 4,  5],\n",
       "        [ 4, 26],\n",
       "        [ 5,  4],\n",
       "        [ 5,  6],\n",
       "        [ 6,  5],\n",
       "        [ 6,  7],\n",
       "        [ 7,  6],\n",
       "        [ 7,  8],\n",
       "        [ 7, 24],\n",
       "        [ 8,  7],\n",
       "        [ 8,  9],\n",
       "        [ 9,  8],\n",
       "        [ 9, 10],\n",
       "        [ 9, 20],\n",
       "        [10,  9],\n",
       "        [10, 11],\n",
       "        [11, 10],\n",
       "        [11, 12],\n",
       "        [11, 14],\n",
       "        [12, 11],\n",
       "        [12, 13],\n",
       "        [13, 12],\n",
       "        [14, 11],\n",
       "        [14, 15],\n",
       "        [14, 19],\n",
       "        [15, 14],\n",
       "        [15, 16],\n",
       "        [16, 15],\n",
       "        [16, 17],\n",
       "        [17, 16],\n",
       "        [17, 18],\n",
       "        [18, 17],\n",
       "        [18, 19],\n",
       "        [19, 14],\n",
       "        [19, 18],\n",
       "        [20,  9],\n",
       "        [20, 21],\n",
       "        [20, 22],\n",
       "        [21, 20],\n",
       "        [22, 20],\n",
       "        [22, 23],\n",
       "        [22, 24],\n",
       "        [23, 22],\n",
       "        [24,  7],\n",
       "        [24, 22],\n",
       "        [24, 25],\n",
       "        [25, 24],\n",
       "        [26,  4],\n",
       "        [26, 27],\n",
       "        [26, 28],\n",
       "        [27, 26],\n",
       "        [28, 26],\n",
       "        [28, 29],\n",
       "        [28, 30],\n",
       "        [29, 28],\n",
       "        [30,  2],\n",
       "        [30, 28],\n",
       "        [30, 31],\n",
       "        [31, 30]])"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# the edges of the example graph are in sparse Coordinate Format (COO)\n",
    "# (also called adjacency list: https://distill.pub/2021/gnn-intro/\n",
    "DATA[0].edge_index.t() # shape [num_edges, 2]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[ 1,  0,  0],\n",
       "        [ 1,  0,  0],\n",
       "        [ 1,  0,  0],\n",
       "        [ 1,  0,  0],\n",
       "        [ 1,  0,  0],\n",
       "        [ 1,  0,  0],\n",
       "        [ 1,  0,  0],\n",
       "        [ 1,  0,  0],\n",
       "        [ 1,  0,  0],\n",
       "        [ 1,  0,  0],\n",
       "        [ 1,  0,  0],\n",
       "        [ 1,  0,  0],\n",
       "        [ 1,  0,  0],\n",
       "        [ 1,  0,  0],\n",
       "        [ 1,  0,  0],\n",
       "        [ 1,  0,  0],\n",
       "        [ 1,  0,  0],\n",
       "        [ 1,  0,  0],\n",
       "        [ 1,  0,  0],\n",
       "        [ 1,  0,  0],\n",
       "        [ 1,  0,  0],\n",
       "        [ 1,  0,  0],\n",
       "        [ 1,  0,  0],\n",
       "        [ 1,  0,  0],\n",
       "        [ 1,  0,  0],\n",
       "        [ 1,  0,  0],\n",
       "        [ 1,  0,  0],\n",
       "        [ 1,  0,  0],\n",
       "        [ 1,  0,  0],\n",
       "        [ 3,  0,  0],\n",
       "        [ 3,  0,  0],\n",
       "        [ 1,  0,  0],\n",
       "        [12,  0,  1],\n",
       "        [12,  0,  1],\n",
       "        [12,  0,  1],\n",
       "        [12,  0,  1],\n",
       "        [12,  0,  1],\n",
       "        [12,  0,  1],\n",
       "        [12,  0,  1],\n",
       "        [12,  0,  1],\n",
       "        [12,  0,  1],\n",
       "        [12,  0,  1],\n",
       "        [12,  0,  1],\n",
       "        [12,  0,  1],\n",
       "        [ 1,  0,  0],\n",
       "        [ 1,  0,  0],\n",
       "        [ 1,  0,  0],\n",
       "        [ 1,  0,  0],\n",
       "        [ 1,  0,  0],\n",
       "        [ 1,  0,  0],\n",
       "        [ 1,  0,  0],\n",
       "        [ 1,  0,  0],\n",
       "        [ 1,  0,  0],\n",
       "        [ 1,  0,  0],\n",
       "        [ 1,  0,  0],\n",
       "        [ 1,  0,  0],\n",
       "        [ 1,  0,  0],\n",
       "        [ 1,  0,  0],\n",
       "        [ 1,  0,  0],\n",
       "        [ 1,  0,  0],\n",
       "        [ 1,  0,  0],\n",
       "        [ 1,  0,  0],\n",
       "        [ 1,  0,  0],\n",
       "        [ 1,  0,  0],\n",
       "        [ 1,  0,  0],\n",
       "        [ 1,  0,  0],\n",
       "        [ 1,  0,  0],\n",
       "        [ 1,  0,  0]])"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# edge attributes of example graph\n",
    "DATA[0].edge_attr # shape [num_edges, num_edge_features]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Visualize one of the molecules in the dataset "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAcIAAACWCAIAAADCEh9HAAAABmJLR0QA/wD/AP+gvaeTAAAgAElEQVR4nO3deVyU1f4H8M8MIwyKgIoLIKi4oCZeZXHfWX6WS5LiltBPyyG1S/66rxzNCr3pvWM3u3TziqhZYgqBN0W9JmKaKUKBQgpiKJuyCKjIKsvMnN8fzzSSWwjPMw8w3/df4zCc83Xh43nOc55zJIwxEEIIaS6p2AUQQkjbRjFKCCEtQjFKCCEtQjFKCCEtQjFKCCEtQjFKiAAqK5GdDY0GAOrrUVame7+8HHV1ItZFhEAxSgjfVCosW4Y9e+DlhYwMXL6M4GDdl7ZuxblzohZH+CcTuwBC2pfcXJw/j2PHAODXX7F2LdavF7smIiyKUUJ4dfUqXF11r52dcfMmAJw8iVdfBYDLlzFpkmi1EWHQRX2bUVVVtX37dktLy969e3/22Wdil0Oewtr64UxofT1MTQHAxwf792P/fvj6ilgaEQjFaBuQnp7+1ltv2dvbr1q1qrKysqCg4C9/+cvmzZvpQd7WyMMDly4hNRX19di4EQsXil0QEZyEfhRbrfr6+piYmJ07d37//ffcX5Obm5ufn19hYeG2bdu0Wu2MGTPCw8O7du0qdqXk94qLERKCoiJ4emLJEty6hcREzJ8PAEePYuBADB4sdomETxSjrVFBQcGuXbtCQ0NLSkoAdO7cedGiRStXrrSwsKiurh4+fPjp06cXLVpUUlLi6OgYHR09atQosUsmjaSkYONGTJuGoCCxSyEGwUirodFo4uLi/Pz8ZDLdrb8hQ4aEhISUl5dz75uYmHh7e3Mfvnnz5pgxYwCYmZmFhISIWzn5na++YgBbtEjsOoiB0Nxoq1BcXLxly5YBAwZ4e3tHR0ebmJj4+fnFxcV9//33FRUVw4YN4943NTV1cHDQaDQAHBwcfvjhh6CgoLq6utWrVwcEBNTU1Ij9+yAAgOxsAOjfX+w6iKGInePGLjk5WaFQyOVy7q+jf//+KpWqpKQkOTnZ39+/Q4cO3PsDBw5UqVSlpaWPt7Bv375OnToBGDFixI0bNwz/WyCPWrKEAWzPHrHrIAZCMSoOrVbr7+8/YMAALiVlMtmcOXNiY2Pv3bsXFhY2bNgw7n0TE5OZM2fGxcVptdpntHb16tUhQ4YAsLS0PHjwoMF+F+TJxo5lADt7Vuw6iIFQjIpj3bp1XFD26tVLqVTm5uZyw1JuXAnA1tZWqVTm5eU1scGKiop58+YBkEgkQUFB9fX1gtZPnqVnTwawW7fEroMYCMWoOHx8fACMGzfuwYMHu3fvdnd359JTKpX6+PgcOnRIrVY/b5tarTYkJISbB5g8eXJRUZEQlZM/UFnJJBImlzONRuxSiIFQjIrD09MTwPHjxxljLi4uAKytrRUKRXp6egtbPnv2rK2tLQB7e/vz58/zUSx5Hr/8wgA2ZIjYdRDDoTv14sjOzgbQv39/AH/961/Dw8OLiorCwsKGDh3awpYnTZqUmprq6elZUFAwZcoUlUrFaGmwAV3Ozz89Zcr1KVPELoQYDsWoCNRq9a1bt6RSaZ8+fQDMmTPH399ff7O+5Xr06HHixIl33nlHo9GsW7fOuBbnJyZixw7da5UK164BwLZtWLAA8+cjLEzo/uMyMjx/+GHbb0ssiDGgGBVBbm6uWq12cHAwMzMTqAuZTLZ169bw8HAAycnJhYWFAnXU6lRUoKhI9zo/HzU1+O47XLmCyEh88w3i43H6tKD9c9cZTk5OgvZCWhXaKE8EQv+klZeXR0REDB482NfXVyKRmJiY9OjRQ6C+WqPUVOzZAwAZGQBw8iT8/SGRAMBrr+HkSUybJlznWVlZ+G26hhgJilERCP2Tlp6evmLFCg8Pjy+++IIxNmDAAP3TpUbBygp9+gBA584AoNVC2uiqS+CZYhqNGiG6qBeB0D9p+pg20pFRv37w9ISnJ3r3BoDJkxETo/tSTAwmT0ZVFdRqIXrWaDR5eXkSiaRfv35CtE9aJ4pREQidblz7Tk5Oxjgyksthaal7bW0NU1P4+kImw4IF8PODjQ1efBGvvQZPz4dTqPy5detWfX29nZ2dubk5742TVsuYrvVaDaHTTd9+SkqKoB21RpMmPTylY9Mm3YvNmx9+gNv9s7AQbm745htMnMhj59euXYOx/YETGo2KIicnB0KORvWLUo30ov7ZHByQmgovLxQVYdo0bNnCy2xpVlbW2rVrFy9e7OjoeO3atfT09Ja3SdoKilFDKykpqaiosLa27tKli0Bd6NPTGC/qm6J7d5w4geBgaLVYuxZz5uD+/ea1pNFoYmJipk+fPmjQoC1btpSVlVVVVZWWlo4dOzYqKorfqknrJfZjVEYnISEBgJubm0DtV1dXSyQSMzOzhoYGMzMziURSXV0tUF9t3uHDzNqaAczZOe85H8MtKipSqVR9+/blfo7MzMy4LWIrKysX/nb+kkKhqKur47/ssjK2fTvbvJklJzPG2I0bLCFB96XvvmNP2k2RCIpi1NC+/vprAPPnzxeo/StXrgBwdnbOzc0FYGdnJ1BH7URuLvPwSJ40SS6Xc+vD/lBTtoINCwszNTUF4OHhkZuby2fBtbVswgR2/Di7fJm9/DKLjWVHj7LNm3VffeMNduUKn92RJqCLekPT30YXtH26om+qPn1w9uyuAQNqa2tff/31lStX1tXVPfGDFRUVO3fudHFxcXd337dvn1ar9fLyOnLkyK+//qpUKm1sbBp/WKFQxMfH9+3bNykpyd3dPS4ujreCY2MxdSpefBEuLvjnPxEaCgANDaipQU0NNBreOiJNRjFqaI03JRGCPqbp/lJTmZvv+OKLvXv3mpubh4aGjhs3jrsHqHfx4sXAwEA7O7vAwMC0tDRuK9js7Oy4uLhZs2ZJuOejHuPu7p6UlOTj43Pnzp0XX3xxw4YNWq22paXeu4eSEtja6n5pZ4fiYgA4dgxvv42338aFCy3tgjw/ilFDM8xqJxqNPq+AgID4+HgnJ6dLly55eHicOHGitrY2Ojp6woQJ7u7uO3furKmp8fLyioqKysvLU6lUjo6Of9imjY3NiRMnuE22Nm7c+PLLL5eVlTWnuNpaREdj/HhMnIiBA6FfBpCWBmdnAPD1xa5d2LWL3/VbpIlo3aihNR4k/vTTTy4uLh07duSxfX16cveyKEabbuTIkUlJSf7+/sePH58xY4a5uXl1dTWAbt26LV26NDAwUH/oS9NJJBKlUuni4uLv73/s2LERI0YcPHjQw8Ojqd+fkYEdOxAerltLYGODvn1x5w7++lc4OODrr7FtG7KynrcqwjOxJ2eNy7179yQSiUwmU6vVmZmZVlZWQ4YMuXr1Ko9dDBo0CEBaWhq3o/6FCxd4bLwlGhrY3buMMXb6NPv2W92bO3aIWNGTabXaDz74wMrKytTU1M3NLSwsjJelDnl5eaNHjwYgl8t37tz57A9r6urYN9+wKVOYRMIABrBx49i+fay2liuRJSWx2Fh2/z5jjN29y27e1H1nZiajhRkGRzFqUNxTLgC2bduWnp7ORZ6VldWhQ4d4aV+j0egXOXHrUouLi3lpuSWKiphKxfr0YQEBjDG2aRPr10/3gz9pkrilPVl8fDyAESNG8NtsbW1tUFAQ9w/A39//ielcUFCgUqkG9O2r7tWLAaxzZ6ZQsJQUfish/KIYNTT9sUurVq26c+eOn58f+DuHLi8vD4CtrS03B2dhYcFLzc2j1bK4ODZ3LpPJdCOqESOYWs02bWL/+hebN4+x1hqj3D6tCxYsEKhxbhqn8YHYGo3m+PHjs2bNMjEx4f55nF64kP3736yiQogaCL8oRkXA3RQG4ObmlpOTExYWxi1CnDRpUmFhYUtaPn36NIAJEyYkJycDGD58OF81P5f791lYGHvhBV16mpiwmTNZXBwrKWE//8w2bWKnTrF33mFHjrTSGN2wYQOA9957T6D2U1JSuGlWS0vL8PDwkJAQ/RS2qakpt4z/2Udqk1aFYlQcFy9e5PZS69at24kTJ86dO2dnZwegR48ep06danaz9fX1N27cSEtL++abbwD4+vryWHNTcMdET5u2jwtQOzumVLKbN1lyMlMoWMeOrH9/9tFH7NQpVlHBpk5lY8cauMAmCQgIALB7927huigrK5s9ezZ3xcAFqJOTk0qlKikpEa5TIhCKUdHcuXNn+vTp+O1m7u3bt728vADIZDKVStWSwUhlZaWvry+AAG4yUnhVVVW7du1ydXXlEqFTJ5uXX244fJiVlbHQUPanP+mGpVIpmzGDrV/PuP8pIiPZoEGGKfD5jB8/HsCZM2cE7UWr1U6ZMsXExGTYsGHHjx/X0IHMbRbFqJi0Wq1KpeKmw1566aWSkpLg4GCpVApg9uzZZWVlz9vgtWvXlEpl165duRlSMzOzXbt2CVG5HvcMD9cjAGtr66CgoKysrLS0ihUrWOfOugDt2ZOtW8dyctilS+xvf3s449fi86QF0atXLwB5eXlCdzRv3jwABw4cELojIiiKUfGdPn26Z8+eABwcHBITE48cOWJtbQ1g4MCBly9fbkoLtbW1Bw4cmNho6fX48eO5sS0AhULx4MEDISo/cuSIvsdx48bt27evoqIiKirKy8urTx9Pbq2OmxsLC2NlZSwqinl56aZK+X3KnF/6vV3UarXQfY0cORJAYmKi0B0RQVGMtgq3bt0aM2YMADMzs5CQkOvXrw8fPpxbY/js/TLy8/ODg4P1J9Z17txZoVCkpqZyX927dy93U3jkyJFZWVm8l11dXe3o6KhQKFJSUrKyspRKZffu3blKrKys3n+/JC2NZWayv/yFdeumG5Z26cJWr2Ytu5EmLP3eLgboi/v/spT2ZGrjKEZbi8aLCpcsWVJaWsrd6ADw/vvvP/JhjUYTFxfn5+enP6tuyJAhISEhlZWVj3wyJSWFe2LKysrqW/2qd/48ePDg5MmTfn5++pU6XCVlZWVHjyZ7ez9cP+7hwb74og2sDT98+DA3xyJ0R3fu3OFu1gvdEREaxWjr8vXXX3fq1IkLo6tXr+7du9fCwuLkyZP6D9y+fVulUulPTNNvc/mMNsvLy1955RX9vSy+rlWLiorWrFnDrfbnBs6vvfZaYmIitxFnnz59ZDJ5r15quZz5+bHz53np0xA+/fRTAG+99ZbQHf3000/chYLQHRGhUYy2OhkZGUOHDuXGKdHR0foVMNxaIrlczgXogAEDHt/m8mm0Wm1ISAg3dJ0yZcrt27ebXZ5Wqz1z5sz8+fO5/TS5Kd1PPvnkzp07cXFxc+fO1Q+QBw8evHt32vPfJxPZqlWrAHz66adCdxQREQFg7ty5QndEhEYx2hpVVFTon26aO3fupk2buKlSAFKplNtnqBmDyh9++IG7B927d+9mPGtfXl4eFhbm4uLSuJIvv/yyqqpq69at3IOtADp06ODn53f69Ok2uoD8xRdfBBATEyN0R5s2bQKwZs0aoTsiQqMYbaW0Wu0nn3yin3AE0KtXL6VS2cKt1PPz88eNG6dfndrE7+IGwtxsA7eUSqlU6tcDcUcKA7Czs1MqlTf122S0Tfq9XYTuaNmyZQB2tMLdWchzohht1bZv3z5w4MAePXpERUU1NDTw0mZDQ4NSqeQCcdGiRVVVVU/7ZG1tLbd6qfE6qqioqMef/d+3b19MTIwBVggJrfHeLkL3NXnyZACNJ75JG0UxaqQOHTpkZWXFzWA+PvLKzMxsfDCGlZWVQqEwwABNdPq9XQzQl4ODAwAhFqIRA6MYNV7Xrl0bNmwYAAsLi8jISMaYWq2Oi4ubOXOm/mAMHjfcbBP0e7sI3VFtba1UKpXJZC3f1ouIjna/N17Ozs4JCQnLly+PjIxcuHDh559/npOTU1hYCKBTp06LFy9esWIF95iN8RD6wEG9nJwcrVbbr18//QmjpO2iGDVqFhYWERERU6dOXbVqVV5eXmFhobOz89KlS5cvX65/TN6oCH3g4CMd0REv7QMdaUegUChcXV3z8/P/8Y9/NN7cxNiUlpaeOHECwCOnJQuBzm1tTyhGCQAUFxcDmDNnjtiFiIM7QrlPnz4pKSldu3b96KOPuHNEhEOj0faEYpSgvr4+Pz/fxMSkKYcGtyeVlZWhoaHDhw/njlCur6/38fFxdHS8ffv21KlTP/vsM+G6ptFoe0IxSpCbm6vRaBwdHfXPd7Z/v/wSunatvb39ypUrr1y50qtXr/Xr12dnZ8fGxiYlJXE7D6xevdrX17e8vFyI/mk02p5QjBJj+pGuq0N0NLy9MWLEuISEyspKbkVXTk7Opk2buME493zX4cOHra2tDx8+PGrUqLS0NH6rYIzl5OTASP7MjQDFKDGOC8ysLKxZg969MX8+Tp2CldXg0aOvXr36yIYverNnz05MTBw2bFhmZqa3l9eD6Gj+CskKCgpSq9Xm5uY1NTV8NUtERAueSLsbjRYWQqmERoOGBqxfDwDr1uHkSWi1AODmhjffxKJFZp06DXlmM87OzomJiYGBgTNu3jSfPx8KBT7/HM2d99BoNMeOHQsNDeVO/TQxMamvr3d1dY2MjJw0aVLz2iSthdjr/4n4PgwMdHJw+DY6WuxCeDJ7NktJYYyxkhI2ahS7coVJJMzMjPn5sWduzPpUYWHM1JQBzN2d5eQ873dzG7D27duX+4njtog9ePCg/gTD4ODgNrobFuFQjBLGXFwYwC5eFLsOPmi1zMPj4S8XLmTXr7MDB9i9ey1qNjmZ9evHAGZjw2Jjm1rJ99/XBgR0NjfnAtTZ2fnTTz+991slarW6hScYklaCYpQwZmHBANZufowbx+iCBezGDX6avXOHTZ/OACaRMKWSPeM85Pv3WVgYGzaMOz5FNXasl5fXkSNHnjjkPHr0aJcuXQAMHDjwl19+4adUYlgUo0avqIgBrFs3sevgz8svM+5Qv9JSNno043H7Po2Gffghk0oZwE6cYIyx7GyWksJqa3Uf+PlntnQpMzfXnT/VuzfbuFFTUPDsVnNzcz08PADI5fLdu3fzVi0xFAljTJQ5WdJaxMdjwgR4eODnn8UuhScFBVi7Fmo1Ghrw/vsYMYLn9r/7DmfO4OOPsXw55HI4OuL4cezejfffR2QkAEil8PbGm29i1iw02nj7GWpra//85z/v3r0bgL+/f1hYmPlvUwGkDRA7x4nYwsMZwBYuFLsOXoWEsMhIJugedPHxTKHQvU5IYMuWsZAQZm3NFAqWnt68Jvfu3culp6ura3Z2Nm+lEoHRulGjl5UFAO1p0WhtLf7v/+DvD6mQ/7wzMh6Oc0eOREYGFAoUFSEsDEOHNq/JgICA+Ph4JyenS5cueXh4cPukkNaPYtToZWcDQLtZNAogOxuMoW/fJl5QN5ONDUpKdK9LStC9O8zN8dgy/uc1cuTIlJQUX1/fu3fvvvTSS2vXrtVoNC0tlQiMYtToFRQA7StGDTO+9vFBbCx+/hmFhVi3Dm+8wVfDlpaW//nPf1QqlVQq3bJli7e3N7f/Fmm1KEaNW1kZgoORkIAxY8QuhT9cjAr9H4O5Ob79FrGx2LoVr7+OWbN4bFsikSiVylOnTvXs2fPMmTPu7u4XLlzgsX3CL4pRIxYfj4ULkZ6O8HB8+KHY1fCHm6YwwGxvr1744ANs3YqpU4VofsqUKUlJSWPGjMnPz584ceLnn38uRC+k5ShGjVhwMMLDsWIFtm/H1avIzRW7IJ4YZjRqEA4ODufOnZs4caJUKo2IiBC7HPJkFKNG7P599Oype+3iguvXRa2GPwYbjRqETCZbtmyZWq1uP3vHtDsUo0asQwfU1upeFxY+jNS2TKvVHu/a9VcnJ/TrJ3YtvGlvW3C1OxSjRmzZMrz7LnJzEROD4mK4uGDvXty9y39HZWVQqfD22zh0iP/Gf6+goGDGhQtTampgYSF0XwZjsGOfSfNQjBqx5csxcyZ27UJ+Pg4dwtGjWLoUbm5ITuazF7Uac+Zg7FisX48ff8T27Xw2/ph2OXAz2LHPpHkoRo3b//wPNm/GqlUwN4erK8aMQV4eJkwAj6e5JSfjhRcweTJ69MDf/459+3hr+Una5U7+7fI31Z5QjJLf9O6NH3+EUom6OqxejSVLUF3dogYZQ04O7t2D/th3uRwNDS2v9Bna32i0qqqqtLRULpfb2tqKXQt5MopR0ohMBpUK+/ejUyfs3w8PD1y92px2Kiqwcyf+9Cd4eMDJCSkpuvezsmBvz2O9j2t/MaqfGJVIJGLXQp6MYpQ8ZvFiJCbC2RkZGbdXrDj0XPeFLl7EG2/A1haBgbhyRTf8HDkSy5bhk08QGIiPPkJSEqZPhzAPOLa/699u+fkbpkx5ffJksQshTyf2FlOktSovr162rH+XLhKJ5N13321oaHjGZx/U1LA9e5iHh267YomEeXuz//yH6b+roIBdusRqahhjbNQoBjAHB5aQwHvVNjY2AAoLC3lvWTQff8wA9vbbYtdBnopilDxLWFhYhw4dAEycOPGJ2ZSZmalUKrt16/bz0KEMYFZWTKFgaWnPajQ/n40fzwAmkzGViq9SMzMzX331VRMTEzMzs3Z1QtybbzKA/etfYtdBnopilPyBc+fO2dnZAejRo8epU6e4N+vr66Ojoz09PfUTdhuXLGFffcUePGhSow0NTKlkEgkD2KJFrLKy2eXV19dHRUVNmzZNX8ny5cub3Vpr5OPDAHbsmNh1kKeiGCV/rKSkRH8a8Pr16//+9787OjpymSWXy/39/S9dutScdqOiWOfODGAvvKDJyHje7y4oKFCpVA4ODo0riWveEcqtWf/+DGDP/+dDDIZilDRJQ0PDmjVrJBJJ586dudhydnZWqVR3795tUbu//spcXLRWVjOGDImMjGzKd2g0mri4OD8/P5lMxmclrZNazTp0YBJJU4f5RAwUo+Q5LF26lIuts2fP8tZoVdX+1au5QFy9enX90w9QKisrCwkJ0S9mMjU19fPzi4uLa1czoY/IztadMEpaMYpR8hxWrlwJICQkhPeWw8LCTE1NAXh4eOTm5j7y1eTkZIVCoT8s097ePjg4uLi4mPcyWp24OAawyZPFroM8i0ygdVSkXRJucbtCoXB1dfXz80tKSnJ3dz9w4IC3t3dlZWVERERoaGhqaioAqVTq5eWlUCh8fX31V/TtnJkZvL3h7i52HeRZ6Jx68hwGDRp0/fr19PT0oc09/PLZSktLFy9efOrUKRMTkzFjxly+fLmyshJAz549X3/9dYVC0adPHyH6baVychAejtpa+Ppi1CgkJUEi0UXq3r3w80PHjmKXSACKUdJ0Go2mY8eODQ0NVVVVHQX7AWaMffzxx++9956NjU1JSYmbm5tCoQgICJC3+NDNNqakBHPmIDQUlpZ46y289x4yMiCVYtkyAJg1C19++XCzAiIqilHSVLm5uf369bOzsyvgDhMVUq9evYqLi2NjY318fITuq5XasQNSKRQKAEhORlgYRo9GRQUWLACA//1fRERQjLYSxjHBRPhgsF0va2trS0tLZTLZtGnThO6r9bp7F/qZk+7dddtpx8Xh9m0AyMkRrTDyGIpR0lQG2/UjJydHq9U6OTkZy32kJxo6FCkp8PUFgEuXMGwYAPj56S7qMzLErI38nhH/MyXPyWB70NGZGQAwezb278f69bC0RFwcIiIQEyN2TeTJKEZJUxks3ejMDAAwMUF0NH79FTU1eOcddOiAuXMffnXbNnTpIl5x5HcoRklTGSzd2t/Wy80kkWDw4Ie/bJybRrXwq9WjbZtJUxksRtvf1sukfaMYJU1y7969srIyCwuL7t27C90XjUZJ20IxSprEYENRxlhOTg6Afv36Cd0XIbygGCVNYrAL7cLCwgcPHvTo0cPS0lLovgjhBcUoaRID36anK3rShlCMkiah+0uEPA3FKGkSgw0SaTRK2hyKUdIkBhsk0iNMpM2hGCV/4Nq1a0ql8t69ezKZ7N69e0J3R48wkTaHYpQ8WV1d3f79+ydOnDhkyJCPP/64qqpKrVZPmjTpiy++ELRfGo2SNodilDyqoKBgw4YNDg4OS5YsOX/+vKWlpUKhSEpKCgoKqq2tfeONNwICAmpqaoTouqqqqrS0VC6X29raCtE+IUKgZ+rJbzQa/Pe/2l27piYkXL97F8DIkSNXrFixePHiTp06AXB3d/fw8AgMDNy3b9+VK1cOHjzI+6W3figqldJ/8KTNoBglQHExvvoKO3YgN1cKhHh5Rdnbr1ixYvTo0Y98cMmSJW5ubnPnzk1NTXV1dd2zZ8/cxtsOtVhycjIA4zpwibQDop5LSoSUlKR7cfcuy8rSvb50iX33HdMfTZyczPz9WYcODGAAGzCAqVSstPTZDVdUVMybNw+ARCIJCgp6xsnyTZecnOzv7y+Tyfr27dulS5eEhISWt0mIYVCMtl9jx+pexMay4GDGGAsMZEolCw9nPj7sxx+ZQqFLT5mMvfIKO3mSabVNbFur1YaEhHTo0AHA5MmTi4qKmldjWVlZSEjI4N+2g5PJZHZ2dgDMzMy2b9/evDYJMTCK0fbrkRjNyGB+frp3CguZlxc7coTZ2jKlkuXlNa+Hs2fPcveC7O3tz58//1zfm5ycrFAouFlXALa2tkqlMi8vr6GhQalUcm+++uqrVVVVzauNEIOhGG2/7OzY4sVs8WI2bRoLDmbHjrEPP3z4VVdXptGwhoYWdlJcXMwdPCeTyVQqlfaPxrMPHjyIiooaP348F5QSicTLyysqKuqRmYEDBw5wCTt48OD09PQWFkmIoChG269HRqMXL7KlS3XvVFSwiRP56ocbP0okEgBz5sy5f//+Ez+WmZmpVCq7devGBai1tbVCoXhGRKalpXEX+5aWlilHj/JVLSG8oxhtvx6JUa2WTZ/OIiNZWhoLCGCRkfz2FhMTY21tDWDQoEFXrlzRv69Wq48cOeLl5cXlLAA3N7ewsLDq6uo/bLOysnLBggWvDBzIOnViQUGMj3tZhPCOYrT90gdlXh7jbndnKWUAAANySURBVHw/eMD27GGbN7MLF4ToMDMz08XFBYCFhUVERERBQYFKpXJwcODSUy6X+/v7X7p06bna1Gq1Vf/+t24twaRJrLBQiMoJaQkJY0zI9VTEuFRXVwcGBu7fvx+AVCrVarUAhg0b9uabb/r7+zd/J+bz57FgAQoL0b07IiLg6cljzYS0EMUo4d+GDRv++c9/1tfXz5o1S6FQeHp66q/om6+0FIsX49QpyGRYvx7BwWh5m4TwgR65I/wbNWpURUXF+PHjo6KiGs+Ktkj37vjuOyiV0GiwcSPWrOGhTUL4QDFK+MdtdjdgwACe25XJoFLhyBE4OGDpUly+DF9fLF6MOXNw5QoAvPSS7pMnTyIkhOfeCXkKeqae8E/Yze5mzoSPD0xNMX48vv0WPXuiqAjz5iE+Hvfv6z5TVwdh9qAi5HEUo4R/gm+9bGqK+/fRsSN69gQAW1vI5SgvR0EBNm4EgMxMvPCCUL0T8nsUo4R/hjhxRCaDWv3wl2o1OnRAt25YuBAAzpyB8Bv1E8KhuVHCM8ZYTk4OhN7B3sICJia4dg0Arl6FmRk6doRcDmdnODvD3l7Argn5PRqNEp7dvn27pqbGxsam+atEm+jLL/HBB6irg1yOPXsAYOhQ3Ze6dIGdnbC9E/IbWjdKeFaZmHhrzZr8vn19wsPFroUQQ6CLesKzzpmZQ8+d82k8cUlIu0YxSviWlQUAdEIyMRoUo4Rv2dkAQCckE6NBMUr4RqNRYmQoRgnfaDRKjAzdqSe8qqqCpSXMzFBdDTprnhgH+odOeJWdDcbQrx9lKDEetPye8Eoux2uv6R51J8Q40JCB8CcrC6GhkErh5gYAly/jv//VfWnvXhQUiFgaIcKh0SjhSXk5lizBvn3o3RvvvIO6OnTtitRUzJgBAPHxcHWlR91Ju0SjUcKTU6cwaxYGDIBcjuBgREQAwO3bSE1Fairu3hW7PkKEQqNRwpOKClhZ6V5bWaGiAgBu3EBcHAC6oiftGI1GCU9GjEBiou51QgJGjACACRPw7rt4910MHy5iaYQIikajhCcjR8LeHq++CkdHJCdj716kpIhdEyGGQMvvCa/u30d5ORwdIZGgrg5qNTp1AoCKCnTsCBn9t03aIYpRQghpEZobJYSQFqEYJYSQFqEYJYSQFqEYJYSQFqEYJYSQFvl/rBpeF9vHnFsAAAImelRYdHJka2l0UEtMIHJka2l0IDIwMjIuMDkuNQAAeJx7v2/tPQYg4GWAACYgVgBiJSBuYORgSADSjIxsYJqJCUIzA8VBarBziVPFDDRTA8RlYYfQzGwOIJqZhc0hA0QDFRDDgJuN7FDq8bkZGBkYmYCuAzqMgYWVgZWNgY2dgZ2DgYOTgZOLgYubgZuHgYeXg5mbj4GPP4OJXyBBQDCDSVAoQUg4g4lThEFElEFEjEFMnEFMgkFCkoFFikFKmkFKhkFGlkFGjkFOnkGYL0GCnUGOiUGEGWgjG5OcjBQLMxsHp4iYBDsbv4CgkDCfeBcj0C3QCGJQeMp1xUG9hecAiBNQfsvhSovpfhC7tXqLw1L2UDuwuHCHgzPbzn0gNqe+i8McZz57ENt27zX7OdEJe0Dsydx77WezHQDrPb96m+2EEhawmfktPfsPtU4Giy8Skj/gnnEHzLbgyj5wkdMPzE42jDvwvVEPbOaPBNYDjcoLwGxTJ6P9bGwvwewakakHopacAbOFVvQdkC91dQCx7y7eciBduAHMFuJ5eKBnazSYXVvx/ICJ6k+w+k01Bw9cPdgMZpe/1j4gFu4LdtvH2KYDEScKwezDz/fvT2JpAbNzZ97cf3vqZjD7St6TvU3Tk8FsuxWd9jNV+sHse+yBDjc3M4Ht8hRgdehItQWzVd9McZBRVQez+yNmO7CrNIPZ75/vcrj57AbYDaoBzx3WPxEAi4sBANFKkxTbkMSEAAAC7HpUWHRNT0wgcmRraXQgMjAyMi4wOS41AAB4nH1VW27bMBD89yl4AQv7JvmZxEFRFLGBNu0d+t/7o7OUQzkoUVlcSNRoOZpZrk8lj++Xb7//lHnI5XQqhf5z9t7LLyWi01vJi/L8+uXrtby8Pz1/zLzcfl7ffxSVooZ38PuMfXq/vX3McLmV2EJCcXOWjbt7RKGNxnG8KuUFwCaVwsqZtkqdvC2ACqBvYUQ1Hws10rrAGVa2TVzUaybUYK2rlR0JdaNwrYzH7l65L3CBhLxFreGUCYk1ui2AFQl5sxqdo5x5cyIhWQAbgJhtHaqkOBQ9eJWxY2kkol5rl7xiF1+SZNiTqay584BW4a6r1Tm9OSu+SAym44saWVtqyekOoFa5RhKMxjHU+gep+/rUjMwgA3hS0xXSyjUXxTerKJBNyPrKIE6HzraFt+oMpPdWl0UEvQfSvBGu4WpzqStFOU06Zx3BHUWhwCPYuUKmS+e6kQqEQk5T6b5cve9IUQUB8OwWZqsaluFSbARAFSBJuvpKJeFdz2ghVtMDFLwZraCSfuJLqvYY0FprW2cdLqFGe5egfInFXX0Ftb32Ao0hqxSSMYxaZh0+0cbMDoYggL3Osqo92TcTVbf8KjhWVXhVelLH9pQIJi8oLBORJbAhJZ7XzjBCtu4o++XaaVNWkzbQgLTMsRQUZ7aQ5uYQCcL2xks7lUdPQsk3TjfRFvDOCpgWoUAEtiiWRlvsyyb3er18aqN7Y32+XS9HY82fHO0TN0WPJskYdvRCxvCj4zFGHI2NMerRvhijHU2KMfrRijjHY8PhEfihr/AIk1yKMsLkhzahGXhSZLtjJktse8vAkyi2tGXgyRUlYxl40sV2tQw8GaMaLIM8br/BWSZnGWLiPASV+8zknE0qg0zOYveZyVlSWgSZnCWG2DIpS+qLIJOytPvMpCypMoI+1uI+MylrUkbQSRnVZePbJh3Vnc5cSu3+0mOhPZZV3n/81+P69Be+FId2nyj0xAAAAXF6VFh0U01JTEVTIHJka2l0IDIwMjIuMDkuNQAAeJxNkUuK5DAQRK8yMJsqkEX+P3jp/dQh+hp9+I5Ub0ZgLJ5DoYj0v7/P8/o8/Hlez+eR83qD/D7y354/7y/+msV/vl8X7ehQ0cW7hHzdl2wqIwMQZyoQ3ZacsaAtjuSDIsUXjhdZaZ5zVu68Lt4p3ArEmzqzB7GLy7oJ5s1tYLgHV8OMYR8to3IiGRCZcdyJNXrsFWrHhra7J/e6DflEY1QarMh3+w4jRKYtVDTHAq2SjiipyWtQSKhOAG54zTEULkbjaHNbd26Rcl2yoeyYu3B5DQjmsAHl5rEMDYt98kngGxRkgnHe2GSzALS3ohUiMDPsMTz34FOUEjbrMoxARc7IrFtiEItPmGtiYa0L9VjIzqyjYuav09QGIUlq56DMLD8ovNLRy7syDzEvjBjTLJchMzKfGiTnL+YmlRyFqUx1EFG1gEvbaO8r0DEsQUhafb2/fwCLgIFXlzYIjgAAAABJRU5ErkJggg==",
      "text/plain": [
       "<rdkit.Chem.rdchem.Mol at 0x188737290>"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from rdkit import Chem\n",
    "from rdkit.Chem.Draw import IPythonConsole\n",
    "\n",
    "Chem.MolFromSmiles(DATA[0][\"smiles\"])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Implement a Graph Convolutional Neural Network"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "GCN(\n",
      "  (initial_conv): GCNConv(9, 64)\n",
      "  (conv1): GCNConv(64, 64)\n",
      "  (conv2): GCNConv(64, 64)\n",
      "  (conv3): GCNConv(64, 64)\n",
      "  (out): Linear(in_features=128, out_features=1, bias=True)\n",
      ")\n",
      "Number of parameters:  13249\n"
     ]
    }
   ],
   "source": [
    "import torch\n",
    "from torch.nn import Linear\n",
    "import torch.nn.functional as F \n",
    "from torch_geometric.nn import GCNConv, TopKPooling, global_mean_pool\n",
    "from torch_geometric.nn import global_mean_pool as gap, global_max_pool as gmp\n",
    "\n",
    "EMBEDDING_DIM = 64\n",
    "\n",
    "class GCN(torch.nn.Module):\n",
    "    def __init__(self):\n",
    "        super(GCN, self).__init__()\n",
    "        torch.manual_seed(42)\n",
    "\n",
    "        self.initial_conv = GCNConv( # The graph convolutional operator from the “Semi-supervised Classification with Graph Convolutional Networks” paper\n",
    "          in_channels=DATA.num_features, # number of features per node of graph before transformation\n",
    "          out_channels=EMBEDDING_DIM # number of features per node of graph after transformation\n",
    "        )\n",
    "        self.conv1 = GCNConv(EMBEDDING_DIM, EMBEDDING_DIM)\n",
    "        self.conv2 = GCNConv(EMBEDDING_DIM, EMBEDDING_DIM)\n",
    "        self.conv3 = GCNConv(EMBEDDING_DIM, EMBEDDING_DIM)\n",
    "        self.out = Linear(\n",
    "          in_features=EMBEDDING_DIM*2, # we stack the different global pooling aggregations below\n",
    "          out_features=1\n",
    "        )\n",
    "\n",
    "    def forward(self, x, edge_index, batch_index):\n",
    "        # First Conv layer\n",
    "        hidden = self.initial_conv(x, edge_index)\n",
    "        hidden = F.tanh(hidden)\n",
    "\n",
    "        # Other Conv layers\n",
    "        hidden = self.conv1(hidden, edge_index)\n",
    "        hidden = F.tanh(hidden)\n",
    "        hidden = self.conv2(hidden, edge_index)\n",
    "        hidden = F.tanh(hidden)\n",
    "        hidden = self.conv3(hidden, edge_index)\n",
    "        hidden = F.tanh(hidden)\n",
    "          \n",
    "        # Global Pooling (stack different aggregations over nodes of graph)\n",
    "        hidden = torch.cat([gmp(hidden, batch_index), \n",
    "                            gap(hidden, batch_index)], dim=1)\n",
    "\n",
    "        # Apply a final (linear) classifier.\n",
    "        out = self.out(hidden)\n",
    "\n",
    "        return out, hidden\n",
    "\n",
    "MODEL = GCN()\n",
    "print(MODEL)\n",
    "print(\"Number of parameters: \", sum(p.numel() for p in MODEL.parameters()))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Train the Graph Convolutional Network "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "from torch_geometric.data import DataLoader\n",
    "import warnings\n",
    "warnings.filterwarnings(\"ignore\")\n",
    "\n",
    "NUM_GRAPHS_PER_BATCH = 64\n",
    "\n",
    "# Use GPU for training (if available)\n",
    "DEVICE = torch.device(\"cuda:0\" if torch.cuda.is_available() else \"cpu\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Starting training...\n",
      "Epoch 0 | Train Loss 11.665949821472168\n",
      "Epoch 100 | Train Loss 1.0266482830047607\n",
      "Epoch 200 | Train Loss 2.4161503314971924\n",
      "Epoch 300 | Train Loss 0.21653079986572266\n",
      "Epoch 400 | Train Loss 0.14055132865905762\n",
      "Epoch 500 | Train Loss 0.3436800539493561\n",
      "Epoch 600 | Train Loss 0.060888830572366714\n",
      "Epoch 700 | Train Loss 0.10678458213806152\n",
      "Epoch 800 | Train Loss 0.23447926342487335\n",
      "Epoch 900 | Train Loss 0.13027147948741913\n",
      "Epoch 1000 | Train Loss 0.09288126230239868\n",
      "Epoch 1100 | Train Loss 0.08742401748895645\n",
      "Epoch 1200 | Train Loss 0.1677463799715042\n",
      "Epoch 1300 | Train Loss 0.0401727594435215\n",
      "Epoch 1400 | Train Loss 0.1080005094408989\n",
      "Epoch 1500 | Train Loss 0.023777754977345467\n",
      "Epoch 1600 | Train Loss 0.06994298845529556\n",
      "Epoch 1700 | Train Loss 0.03386954218149185\n",
      "Epoch 1800 | Train Loss 0.05925782397389412\n",
      "Epoch 1900 | Train Loss 0.05499924719333649\n"
     ]
    }
   ],
   "source": [
    "def train(model, data):\n",
    "  model = model.to(DEVICE)\n",
    "\n",
    "  loss_fn = torch.nn.MSELoss()\n",
    "\n",
    "  optimizer = torch.optim.Adam(model.parameters(), lr=0.0007)\n",
    "\n",
    "  data_size = len(data)\n",
    "  train_loader = DataLoader(\n",
    "    data[:int(data_size * 0.8)], \n",
    "    batch_size=NUM_GRAPHS_PER_BATCH, \n",
    "    shuffle=True\n",
    "  )\n",
    "\n",
    "  for batch in train_loader:\n",
    "    # Use GPU\n",
    "    batch.to(DEVICE)  \n",
    "    # Reset gradients\n",
    "    optimizer.zero_grad() \n",
    "    # Passing the node features and the edge info\n",
    "    pred, embedding = model(batch.x.float(), batch.edge_index, batch.batch) \n",
    "    # Calculating the loss and gradients\n",
    "    loss = loss_fn(pred, batch.y)     \n",
    "    loss.backward()  \n",
    "    # Update using the gradients\n",
    "    optimizer.step()   \n",
    "  return loss, embedding\n",
    "\n",
    "def train_wrapper():\n",
    "  print(\"Starting training...\")\n",
    "  losses = []\n",
    "  for epoch in range(2000): \n",
    "      loss, h = train(MODEL, DATA)\n",
    "      losses.append(loss)\n",
    "      if epoch % 100 == 0: \n",
    "        print(f\"Epoch {epoch} | Train Loss {loss}\")\n",
    "  return losses \n",
    "\n",
    "LOSSES = train_wrapper()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Visualize training loss\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAjIAAAGwCAYAAACzXI8XAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8pXeV/AAAACXBIWXMAAA9hAAAPYQGoP6dpAABPSElEQVR4nO3deVxU5eIG8GcAQVAWURFQ3E3NLZckskXLm5q3NOuWZTdbbmZpy20zK0tt0bLMX+bVFrfKJe2mds00N1xRXEDFBQURUAQEZNhkWOb9/YEzzmEWZg4znDnwfD8fPjpnzjnzHgbmPLyrRgghQERERKRCHkoXgIiIiEguBhkiIiJSLQYZIiIiUi0GGSIiIlItBhkiIiJSLQYZIiIiUi0GGSIiIlItL6UL4Gp6vR4ZGRnw9/eHRqNRujhERERkByEECgsLER4eDg8P6/Uu9T7IZGRkICIiQuliEBERkQzp6elo06aN1efrfZDx9/cHUPWNCAgIULg0REREZI+CggJEREQY7+PW1PsgY2hOCggIYJAhIiJSmZq6hbCzLxEREamWokFm9+7deOCBBxAeHg6NRoP169cbnysvL8eUKVPQq1cvNGnSBOHh4XjqqaeQkZGhXIGJiIjIrSgaZIqLi9GnTx8sWLDA7LmSkhIcPXoU06ZNw9GjR/Hbb78hMTERDz74oAIlJSIiInekEUIIpQsBVLWBrVu3DqNHj7a6z6FDhzBw4ECkpqaibdu2FvfR6XTQ6XTGx4bOQlqtln1kiIiIVKKgoACBgYE13r9V1UdGq9VCo9EgKCjI6j6zZs1CYGCg8YtDr4mIiOov1QSZ0tJSTJkyBY8//rjNZDZ16lRotVrjV3p6eh2WkoiIiOqSKoZfl5eX49FHH4UQAgsXLrS5r4+PD3x8fOqoZERERKQktw8yhhCTmpqKHTt2sJ8LERERGbl1kDGEmHPnzmHnzp1o3ry50kUiIiIiN6JokCkqKkJSUpLxcUpKCuLj4xEcHIywsDA88sgjOHr0KDZu3IjKykpkZmYCAIKDg+Ht7a1UsYmIiMhNKDr8Ojo6GkOGDDHbPn78eEyfPh0dOnSweNzOnTsxePBgu17D3uFbRERE5D7svX8rWiMzePBg2MpRbjLFDREREbkpt+4j487yS8pQpKuAf+NGCPRtpHRxiIiIGiTVzCPjbj7bnIg7PtuJ5fsvKF0UIiKiBotBRibDquJs/SIiIlIOg4xMHoYgAyYZIiIipTDIyKRBVZLRM8cQEREphkFGJkONDNuWiIiIlMMgI5NGwxoZIiIipTHI1BL7yBARESmHQUYmj+s1MmxZIiIiUg6DjEyG4ddsWiIiIlIOg4xMxr6+bFoiIiJSDIOMTB43JpIhIiIihTDIyGSokdGzkwwREZFiGGTk4hIFREREimOQkck4aknhchARETVkDDIysWmJiIhIeQwyMnEeGSIiIuUxyMikMfaRYZIhIiJSCoOMTDfmkSEiIiKlMMjIpGHTEhERkeIYZGS6sUQBkwwREZFSGGRk0oDDr4mIiJTGICOTByfEIyIiUhyDjEwctURERKQ8BhmZ2NmXiIhIeQwyMrGzLxERkfIYZGRiZ18iIiLlMcjIxM6+REREymOQkYmdfYmIiJTHICMTm5aIiIiUxyAjE2tkiIiIlMcgI5Nh+LWeOYaIiEgxDDIyGTv7KlsMIiKiBo1BRqbrOYbzyBARESmIQUYmjYZVMkREREpjkJHpRtMSkwwREZFSGGTkMnT21StcDiIiogaMQUYmQx8Z1sgQEREph0FGJg+ufk1ERKQ4BhmZbqx+rWw5iIiIGjIGGZkMnX05bImIiEg5DDIyGdZaYo0MERGRchhk5OJaS0RERIpjkJHJg2stERERKY5BRiZNzbsQERGRizHIEBERkWopGmR2796NBx54AOHh4dBoNFi/fr3keSEEPvjgA4SFhcHX1xdDhw7FuXPnlCksERERuR1Fg0xxcTH69OmDBQsWWHz+888/x9dff41Fixbh4MGDaNKkCYYNG4bS0tI6Lql17CJDRESkHC8lX3zEiBEYMWKExeeEEJg3bx7ef/99jBo1CgDw448/olWrVli/fj3Gjh1bl0U1o2EnGSIiIsW5bR+ZlJQUZGZmYujQocZtgYGBiIyMRExMjNXjdDodCgoKJF9ERERUP7ltkMnMzAQAtGrVSrK9VatWxucsmTVrFgIDA41fERERLi0nERERKcdtg4xcU6dOhVarNX6lp6e79PU4IR4REZFy3DbIhIaGAgCysrIk27OysozPWeLj44OAgADJlyuwjwwREZHy3DbIdOjQAaGhodi+fbtxW0FBAQ4ePIioqCgFS0ZERETuQtFRS0VFRUhKSjI+TklJQXx8PIKDg9G2bVu89tpr+Pjjj9GlSxd06NAB06ZNQ3h4OEaPHq1coYmIiMhtKBpkDh8+jCFDhhgfv/766wCA8ePHY9myZXj77bdRXFyMCRMmID8/H3fccQc2b96Mxo0bK1VkIiIiciOKBpnBgwfb7Cyr0Wgwc+ZMzJw5sw5LZR9NtdWWrhTqsOZwOv4xoA1C/Bm0iIiI6oLb9pFRmwk/HcacLYl4dtkhpYtCRETUYDDIOElcWj4AIOESJ+AjIiKqKwwytcRpZIiIiJTDICMT55EhIiJSHoMMERERqRaDDBEREakWg0wtCbCTDBERkVIYZIiIiEi1GGSIiIhItRhkaonDr4mIiJTDICOThuOviYiIFMcgQ0RERKrFIENERESqxSBTS+wjQ0REpBwGGZnYQ4aIiEh5DDJERESkWgwyREREpFoMMrXEJQqIiIiUwyAjE6eRISIiUh6DDBEREakWgwwRERGpFoNMLXEeGSIiIuUwyMik4UwyREREimOQISIiItVikCEiIiLVYpCpJXaRISIiUg6DjEycR4aIiEh5DDJERESkWgwyREREpFoMMrXFTjJERESKYZCRiV1kiIiIlMcgQ0RERKrFIENERESqxSBTS4KdZIiIiBTDICMT55EhIiJSHoMMERERqRaDTC0JtiwREREphkFGNrYtERERKY1BhoiIiFSLQYaIiIhUi0GmlthFhoiISDkMMjJx+DUREZHyGGSIiIhItRhkiIiISLUYZGpJCAHByWSIiIgUwSAjk2kXmd7T/1KsHERERA0Zg0wtpeWVoFBXoXQxiIiIGiS3DjKVlZWYNm0aOnToAF9fX3Tq1AkfffSRWzXl5BSVKV0EIiKiBstL6QLY8tlnn2HhwoVYvnw5evTogcOHD+OZZ55BYGAgXnnlFaWLR0RERApz6yCzf/9+jBo1CiNHjgQAtG/fHqtWrUJsbKzCJQM0nEiGiIhIcW7dtHT77bdj+/btOHv2LADg2LFj2Lt3L0aMGGH1GJ1Oh4KCAskXERER1U9uXSPzzjvvoKCgAN26dYOnpycqKyvxySefYNy4cVaPmTVrFmbMmFGHpSQiIiKluHWNzJo1a7BixQqsXLkSR48exfLly/HFF19g+fLlVo+ZOnUqtFqt8Ss9Pb0OS0xERER1ya1rZN566y288847GDt2LACgV69eSE1NxaxZszB+/HiLx/j4+MDHx8flZWMPGSIiIuW5dY1MSUkJPDykRfT09IRer1eoRERERORO3LpG5oEHHsAnn3yCtm3bokePHoiLi8PcuXPx7LPPKl00IiIicgNuHWTmz5+PadOm4aWXXkJ2djbCw8Pxwgsv4IMPPlC6aEREROQG3DrI+Pv7Y968eZg3b57SRTHDaWSIiIiU59Z9ZIiIiIhsYZAhIiIi1WKQkYlNS0RERMpjkCEiIiLVYpAhIiIi1WKQISIiItVikJFJw0UKiIiIFMcgQ0RERKrFIENERESqxSBDREREqsUgIxe7yBARESmOQYaIiIhUi0GGiIiIVItBhoiIiFSLQUYmdpEhIiJSHoMMERERqRaDDBEREakWgwwRERGpFoOMTBoNe8kQEREpjUGGiIiIVItBhoiIiFSLQYaIiIhUi0FGJvaQISIiUh6DDBEREakWgwwRERGpFoMMERERqRaDjEycRoaIiEh5DDJERESkWgwyREREpFoMMjJpOACbiIhIcQwyREREpFoMMkRERKRaDDJERESkWgwyMnH4NRERkfIYZIiIiEi1GGSIiIhItRhkiIiISLUYZGRiFxkiIiLlMcgQERGRajHIEBERkWoxyBAREZFqMcjIxU4yREREimOQISIiItVikCEiIiLVkhVk0tPTcfHiRePj2NhYvPbaa/juu++cVjAiIiKimsgKMk888QR27twJAMjMzMTf/vY3xMbG4r333sPMmTOdWkB3pWEnGSIiIsXJCjIJCQkYOHAgAGDNmjXo2bMn9u/fjxUrVmDZsmXOLB8RERGRVbKCTHl5OXx8fAAA27Ztw4MPPggA6NatGy5fvuy80gG4dOkSnnzySTRv3hy+vr7o1asXDh8+7NTXICIiInWSFWR69OiBRYsWYc+ePdi6dSuGDx8OAMjIyEDz5s2dVrirV69i0KBBaNSoEf7880+cOnUKX375JZo1a+a01yAiIiL18pJz0GeffYaHHnoIc+bMwfjx49GnTx8AwO+//25scnKGzz77DBEREVi6dKlxW4cOHWweo9PpoNPpjI8LCgqcVh5TGnaRISIiUpysGpnBgwcjJycHOTk5WLJkiXH7hAkTsGjRIqcV7vfff8eAAQPwj3/8AyEhIejbty++//57m8fMmjULgYGBxq+IiAinlYeIiIjci6wgc+3aNeh0OmMTT2pqKubNm4fExESEhIQ4rXDnz5/HwoUL0aVLF2zZsgUvvvgiXnnlFSxfvtzqMVOnToVWqzV+paenO608RERE5F5kNS2NGjUKY8aMwcSJE5Gfn4/IyEg0atQIOTk5mDt3Ll588UWnFE6v12PAgAH49NNPAQB9+/ZFQkICFi1ahPHjx1s8xsfHx9gR2ZXYskRERKQ8WTUyR48exZ133gkA+PXXX9GqVSukpqbixx9/xNdff+20woWFheHmm2+WbOvevTvS0tKc9hpERESkXrKCTElJCfz9/QEAf/31F8aMGQMPDw/cdtttSE1NdVrhBg0ahMTERMm2s2fPol27dk57DSIiIlIvWUGmc+fOWL9+PdLT07Flyxbcd999AIDs7GwEBAQ4rXD//ve/ceDAAXz66adISkrCypUr8d1332HSpElOew0iIiJSL1lB5oMPPsCbb76J9u3bY+DAgYiKigJQVTvTt29fpxXu1ltvxbp167Bq1Sr07NkTH330EebNm4dx48Y57TXk0nD8NRERkeI0Qggh58DMzExcvnwZffr0gYdHVR6KjY1FQEAAunXr5tRC1kZBQQECAwOh1WqdWlsUm5KHR7+NsfjchdkjnfY6REREDZG9929Zo5YAIDQ0FKGhocZVsNu0aePUyfCIiIiIaiKraUmv12PmzJkIDAxEu3bt0K5dOwQFBeGjjz6CXq93dhmJiIiILJJVI/Pee+9h8eLFmD17NgYNGgQA2Lt3L6ZPn47S0lJ88sknTi2kO2IXGSIiIuXJCjLLly/HDz/8YFz1GgB69+6N1q1b46WXXmoQQYaIiIiUJ6tpKS8vz2KH3m7duiEvL6/WhSIiIiKyh6wg06dPH3zzzTdm27/55hv07t271oUiIiIisoespqXPP/8cI0eOxLZt24xzyMTExCA9PR2bNm1yagHdFbvIEBERKU9Wjczdd9+Ns2fP4qGHHkJ+fj7y8/MxZswYnDx5Ej/99JOzy6g6uopKHEm9ikq9rCl6iIiIyE6yJ8Sz5NixY+jXrx8qKyuddcpac9WEeIcv5OGRRZYnxLu/Vyg2ncjE5CGd8eawrk57TSIioobC3vu3rBoZsm3TiUwAwPd7zhu3/XUyE8Pn7UZiZqFSxSIiIqp3GGRkcnQemQk/HcGZzEJMXnnUNQUiIiJqgBhk6lhhaYXSRSAiIqo3HBq1NGbMGJvP5+fn16YsRERERA5xKMgEBgbW+PxTTz1VqwIRERER2cuhILN06VJXlUOFOJMMERGR0thHxoW4sCQREZFrMcjUE8cv5uOB+XuxPzlH6aIQERHVGQYZmdyttuXJHw7ixCUtnvj+oNJFISIiqjMMMnVMwDXLFhRwWDcRETVADDJERESkWgwyREREpFoMMjK5WRcZIiKiBolBhoiIiFSLQYaIiIhUi0GmjgnXDFoiIiJqkBhkZNK420QyDjibVYgH5u/FjjNZSheFiIioVhhkGqBJK47ixCUtnl12WOmiEBER1QqDTAN0taRc6SIQERE5BYOMC2k4SJuIiMilGGRkkhtR2NeXiIjIeRhkiIiISLUYZIiIiEi1GGSIiIhItRhkZFLxNDJERET1BoNMHXNkZt//HrmIp5bEoqDU2cOl2eWYiIjqBwYZN/bG2mPYffYK/rMzWemiEBERuSUGGRcyND/9a/mhWp1He40T2BEREVnCICOTI5PdbTud7cKSEBERNVwMMi4muNw1ERGRyzDIuBhzDBERkeswyLiYO+YYhisiIqovGGRksnceGfOmJaYIIiIiZ2GQcTE9cwsREZHLeCldgPpOuLgGRgjBsERERA0Wg4wLaeD6/ihjFu5HdoHOtS9CRETkphhkVC4uLd/hY1iBQ0RE9YWq+sjMnj0bGo0Gr732mtJFsZueQ4SIiIhcRjVB5tChQ/j222/Ru3dvpYvikOo5hrmGiIjIeVQRZIqKijBu3Dh8//33aNasmdLFAeDA8GvXFoOIiKhBU0WQmTRpEkaOHImhQ4fWuK9Op0NBQYHkS0lcooCIiMh13L6z7+rVq3H06FEcOmTfCtKzZs3CjBkzXFwq+zkaY7Ql5Xjsuxj8vXeYS8oDMFwREVH94dY1Munp6Xj11VexYsUKNG7c2K5jpk6dCq1Wa/xKT093cSltE3rH9l+8LwVnMgvxxV9nXVMgIiKiesStg8yRI0eQnZ2Nfv36wcvLC15eXti1axe+/vpreHl5obKy0uwYHx8fBAQESL5cQYOaO8loNBqzCfFqqgspq7A/+TirZiUttwQjv96DDfGXnHI+IiKiuuLWTUv33nsvTpw4Idn2zDPPoFu3bpgyZQo8PT0VKpn91NCK8+66EziZUYBXV8dj1C2tlS4OERGR3dw6yPj7+6Nnz56SbU2aNEHz5s3NtrsrR3OMI0saOCskFeoqnHMiIiKiOubWTUv1gSsnxFuwM8ll5yYiIlIDt66RsSQ6OlrpIgBwYB4ZCzlm+u8ncSqjACufj4SXp7wseS6rEF9uZYdgIiJq2Fgj42JmnX2FwLL9FxB7IQ97zuVYOsAu2mvlTigdERGRujHIuJqNYFKpN3/S3oao2jRYqaD/MRERkV0YZFyselZhiCAiInIeBhmZ7F9rSRpd8ktsNwnZOzeMGoZ1ExERuRqDjIu5KnBwmQEiIiIGGZdj3CAiInIdBhkXc7TmxN7da9XZl+mKiIjqCQYZmexZawmwHRos9bNhxiAiIrIfg4wLaWA7yNSmZsSZtSp29lsmIiJyOwwyLubI2kkAm32IiIgcwSAjU22WKHD0HERERGQZg4xM9tacuGrRSEdreiTHVisTK4GIiEitGGRksjdIOBoSLJ2XNTdERESWMcjIZPcwaQeTjBJ9ZJiTiIhIrRhkZLI/cLhqal/XnJaIiEhNGGRksrdpycIC10REROQkDDIyuappye7Xd9NzERER1SUGGZnsX0qAMYGIiMhVGGRksiegFOoqZHT2rfvgw86+RESkVgwyMrls1JKTX782r0FEROTuGGRksjcMuOOEeERERPUFg4xMSjQBERERkRSDjEyuagKqk9FQzGBERFRPMMjIZG+NjK0mIMPSA3q9QGxKHkrKKpxRNCIiogbDS+kCqJX9i0bWvM/ivSn4ZNNp3Nq+GbqG+tv3+va9PBERUb3GGhmZ7G9asr7ntTI9AGBVbBoA4NCFq4qstURERKRWrJGRyf4J8aybtPIoYs63dUp5iIiIGiLWyMhkdx+ZGnb7+UCarBnpOGqKiIiIQUY2exeDZOAgIiJyHQYZmVy1+nXylSI7X1++6sdqZNQIrT2cjmPp+bUoBRERUe2xj4xcdiaJSgeTTG5RmYzC1I6jlUb7knLw1q/HAQAXZo90QYmIiIjswxoZmVy1RIGjwUcJ57IKlS4CERERAAYZ2ewNKI4Gk0pnDIciIiJqIBhkXMzhIGPn/kouGqmR06mGiIjIBRhkZLK34sSeYGIaC/QKNC05mks4EouIiNwFg4xM9jYtOdxHxknz09g+lkGEiIjqBwYZmRp52vetsyfImO6RVaCTWSIiIqKGh8OvZbqtY3O79quwo6no/JVih19fTqVKdkEpFu5KRnFZpeMHExERuSEGGZk8PezrWOJOw6lfXhWHgyl5tT4PO/sSEZG7YNOSiznaR8aV4jkTLxER1TMMMi5WXumaICPnrLoKvdPLQUREpCQGGReb+tsJpYtARERUbzHIuJg79ZEhIiKqbxhkVIpzwRARETHIEBERkYoxyKhUQ6yPySsu48rbREQkwSBDqtHvo63421e7kZRdpHRRiIjITbh1kJk1axZuvfVW+Pv7IyQkBKNHj0ZiYqLSxXILSnaRUXo+vEMXaj+pHxER1Q9uHWR27dqFSZMm4cCBA9i6dSvKy8tx3333objY8Sn965+G2LhUhf2ciYjIwK2XKNi8ebPk8bJlyxASEoIjR47grrvusniMTqeDTndj4cWCggKXllEpDXlUt2jAIY6IiKTcukamOq1WCwAIDg62us+sWbMQGBho/IqIiKir4tWJuLSr+GHPec5PQ0REBBUFGb1ej9deew2DBg1Cz549re43depUaLVa41d6enodltL1HvrPfnz8x2msj7ukdFFs+k90En4+kKp0MYiIqJ5z66YlU5MmTUJCQgL27t1rcz8fHx/4+PjUUamUczbbfYchp+eV4PPNVZ2yn7ytncKlcY6LV0sQHugLDztXPSciorqhihqZyZMnY+PGjdi5cyfatGmjdHEUcfhCHk5c1Bof6524/qOzb82FpRXG/7tiBuK67uz765GLuOOznXjr1+N1+8JERFQjtw4yQghMnjwZ69atw44dO9ChQweli6SIgmvleGRRDB745kZtlFr6yNSHEUbztp0FAPz36EWFS0JERNW5ddPSpEmTsHLlSmzYsAH+/v7IzMwEAAQGBsLX11fh0tWd/JJys216lSQEvRDwcHKdjzqunIiI6oJb18gsXLgQWq0WgwcPRlhYmPHrl19+UbpodcrSBHTODDLODgam5XVJxZFKQhwREbmeW9fIuPsKz9MfuBnT/3fK5a+jqzDvEKNk05Ij9StqqTkiIiJ1cusaGXf39KAO6NMm0OWvE5tiPiW/M3OMszv7mtbIMMcQEZErMcjUkqdCw3H1DiaZuqzdMn2p+jALr9JrSxERkXUMMrWkWJCpIZh8uysZE348jIrKqmYppWpGXNECpv5oREREzsIgU0seCv25XmkjmeSXlGHWn2fw16ksbD5ZNdKrLm/+0s6+jB1EROQ6DDK1pFzTkvXnbpm51fj/0nJDjYwygUI4ceI+4zmZjYiI6DoGmVpy16YlA0Pp6rRGxqT7MGtkiIjIlRhkakmpIGOracmUx/V3WKk8wRhDRESuxCBTS54K9ZGxN5jM2nQGpeWVdTp6yPS1Pt54Cp9vPuPc89dhKkvNLUZ63jWnnvPNtcfw+i/xTj0nEVFD5dYT4qmBu6+GnF2ow/e7z6Os0pmrTNq+ZtOc8VvcJQDAK/d2QeNGns4rQx25e060U8+nLSnHr0eq1mx6b2R3NG9a/1dqJyJyJQaZWhrUqTm2nspSuhg2fbn1rMvOLYSAplqwsdQvxp2XVKhLpt8Hlaz7SUTk1ti0VEtP3tZO6SLUyk8HUnFZWyr7eEv5hP17G4bXVsdh1Dd7jXMVEREpgUGmlrw81f0tnLY+weEgU1RaYfN5VwcZBiXnKNJV4LejF6G9Zr66uj3Wx2fg2EUtjqRedXLJiIjsp+67MCniM5POu5YyhaVmJIYP9/PmmmN4fc0xTF55tFbnqU9vbWl5Jf46mYkine2wTkTug0GGasXSCKJ11zv4muJ8MlXc6btgmPV5z7mcWp2nPr21M/53EhN+OlLrcEdEdYdBhmrF0j1s2f4LZttq27FVzpDrkrIK/PfIReQVl9XuxanBWBWbDgCITryicEmIyF4MMlQr1fOF1VW5ax1kHD/VhxtO4o21x/D00tjavbgTKbVUBBFRfcUgQ7VSfaI9a01ItW1aMj3e3jDw+7EMAMDxi9pavbYz6SWBjKFGLT7aeApDvohGYam8jtFE5DoMMmRTpV5gz7krVke2VM8U1pZOcOSWfeB8LradysK09QnILih1+Hh3JgkvKr6ohlaztHhvClJyirH28EWli0JE1XBCPLJp6b4UfPzHaXRt5Y8t/76rxv2trcptb43MiYtajP3ugPHxhdxi/PRcZL3pUGp6HbXtN7Q6Ng1rDqfj+6cG1PkMwfXl/XAUO60TuR/WyJBN6+OrRiAlZhVafN7eGhl7bwAHU3Ilj09lFFS9jozqC4WWwbJJ2GhacrTZ4p3fTuBoWj7munDmZmt4Oycid8EgQ05lNbCYbD5wPhf/Wn4IF6+WmO2mq5BW6Wg0QMIlrdsvA2Eva0sUzNt2Fr2m/4U/jl92+JzFCsx5IumzxFhDRApikCGL0vNK8NSSWCRcKrC5n1lnXyvtJXHp+cgt0gEAxn53ANtOZ+ONNcfM9istr6y2RYO/z9+LySvj7C+8GzP97pj2M5m37RwA4P31Jxw+Z/W1ruoCW1jIHgt2JmH4vN3QlthX27jlZCb2nOPQd3IMgwxJGNbNeevXY9h9tuYPFLOmJStB5oWfjmDQZzsk2wxLI+QU6fBjzAUUlJajrFqNjKXFxa3dREvLK/HZ5jM4fCEPAKCB+cHaa+UoV3BtINOgZ3GdKhnnVKLjLfuKkD3mbEnEmcxCLN57vsZ9swtL8cJPR/DPxbENrjM51Q6DDEn87avd0OsFMu1cf2nKf49LFg201YG1tNy82QgAnl4aiw82nMTba4+b3citVTZkF5Ri1p+nkZZ7o3lqyb4ULIxOxiOLYiweM2/bWfSZ8RdG/N8e64WsQ1xwkxoKnR1/PJhOXMnfA3IEgwxJpOQU42qJ/TPhbjx+GevjM4yPHflL3ZBRDM1XhinzpfuYJxkBgckr4/DtrvN47LsboSU5u1h6bLVDDc03SdlFdpfR2aR9ZCytSWX7+yeEwLPLDuGJ7w/Y3M/VWCNDrmTp5+uHPefxzNJY6CqqNz9TQ8cgQ2Y8LbXnAJi0wvL6M/kmwScuzf6VkD00GkltDgCz2GKpRkYIIPZ685GjK3crTTohnrma4kFZpR47zmRjf/KN0V0CwKYTlzH1t+N11mxW26HjRLZY+vn6+I/T2Jl4BeuOmq/lRg0bgwyZ0QvLHUj/OHEZV22sW6SrqMTEn+1fbE+jAYbX0MzjYaEc1m6idd3n9X/HMvDfI45NkCZqqJGpKclYm6fnpRVHsSo2Hb8cSneoPHK5sg9Dam4xHvxmr6wRXK5muGwhBNLzSiTfh+zCUuRc79BOUpZqVm2xVeNXXMYaGZJikCEzFdbulrD8AWMIPZeuXnPodTQaTY3NPFcs3BisfciZ1eY4VBrHlJZX4uVVcXhj7THEJOciPj3fruMkNTJ25pjySr2xE7W1eXoMrhTWzY1UEiadnGne+e8JHL+oxSQ3XoF69p9ncOfnO/Ht7qpOrKXllRj4yXYM+Hib1Q7vZJtp2LEVZNxweihSGIMMmfnrZBZScopr3vG6WZtOQwiBwlLH5jPJyDcPPr/FSauNq49iAiwP8f7rZCbWOlg7Uhs6k47Lj39/AKMX7EN6nnReHL1e4Ejq1WpDyh1bM6qiUo/Bc6Ix8us9EELUeJPU27GPU7jwJawth1EX7K3VMwSY2X+eAQDkmwwvLlJgXp/6wHQqB2ZBcgSDDJl5f32C1edKLFTrVugFEi4VoMCOmWlNb96WzmVPjUKlEGY3nAk/HanxOGfSVZqXfV9SjuTxot3JeHjhfkxeGYeM/GvYeDwDFabDry2ct3q4uZBbgkv513AmsxDllcL66uLXzd+RhKFzd7k8zLiys68a72Fenjd+IHXV5kIq0lUY51Ai+7AzOTmCQYYccufnOy1uLymrsKtGxhkjhvR6IWmWuSajzdwQCOLSruL8FfMyXSurxLxt1qf+15Wb1xRVVAsPS/amAAC2nc7C4C+iMXllHFYdTDM+b0/TkmnH61WxaRY/4BMzpctHpOQUGwOhEFW1Qrb6NslR040mLbfErmBriRrnEMktuvH9rR7Qe364Bf0/3ib7+1FSVoGES1pVfl8cIWlaYpUMOYBBxsnaBvspXQRFLNqVjBOXtDXu54zPp+rnuGY2GzDw+pp4yePjF/Mljyuur+r90H/2454vd5kdP3dronG4tiXmMxBXTf41eeVR4w3HtJyGJrI9527U2lgefi19bFrx9OHvJ3E2yzx0ncm0vA4WAOw+l4OHF+7HkC+jre4jh60uMmm5Jbhrzk4M+GibU1/TwJU3uZpaliwtx5CpLcWwebuNj4vLLAf6ZJkh/h+LYvD3+Xux6YT59ARq4WhHfHve4vVxl/DcskN48oeD+DHmgqxyUf3AIONEIf4+WPNClMtf584uLVz+Go7amXgFC6OTa9wvq6D2w6WrBwBL/Wh+O3pJMvLqhz0pkuf/OJGBfy6OtfoaMedzrT4HmE/uB1T17dh4/DIW7kpGsa7CclAx/b/FGhnpxuq1PJb6FVliuPRt19eoyrdzinh72ZoP58D1712Zk4aCJ2YWIuF6SC7WVeDOz3fi9V/inXLu6uREpF1nsyWPTWsInVGLcvL6wqlrj9TNiDSlSPvIWP++GZ557Zd4bD+Tjb1JOfhgw0kXl672rpVVqrZW7f31JzB83m6Lf8C5AwYZJ3rjvptqdfxLgzvVuI+HBmjm512r11HSU0ushwd7/adaYLJUI1Pd78cyJI///Yt0nafqzUs1NZPZmpTr882JuO+r3XZ1zK2u+qbq88LUNGqp+rlNpwS6WlzmvCYmG6OvnDkMvrxSj2HzduPv8/eiSFeBPxMycSn/mlmn8LpizzDiMjtnunaUK1tbKvUCn246je2nlVuc1XSwZH3rI5OWW4LuH2zG5FXOWzNuXdxFHLNztKS93l13AhN/OmIWuH4+kIYzmYX4y00X72WQcbJGno5/is99tA8WPdkPrw29CR+P7mlzX08PDW5q1VRu8eolOX1kqnvwm30AbjQZpeaar8wNAMlXijB4zk58vjnR5vku5V+zsgTBjY1peSVmfYaqH1JRKd1i7190huNMa6X6frQVfT/aarEGy1GmN1XTm05ukQ7ZJh22L+Vfw9rD6bIn6jMNqQV1MJpJTgarHm5MA6wzO13X1KRWUanHiYtaWU1v6+Iu4bvd5/Hc8sMOlynhktYpEzFKavlsnM7Zw6+vlVXinf8eR3Rids07y7T8etOXs+ZGOnQhD//+5RhGLdjnlPMBVZ8tKw+mYfPJTKt9Gd21RolBxsmaN/Wxe9/PH+mNp6LaYfQtrTG8Zxi8vTzw5G3t0LdtkNVjNBoNglRcI+MK1mpkHPnAK9JVYOPxDNz8wWYs3Zdidb/PN5/BhdwS48zCtli6iV0wCUgvrTiKoXN3YazJMgtlFXpEzdqOZdfLUL15xt77RfUmKVPOGN4sJMPIDf8K9P94G+ZsuRHyhn65C2/9ehyL91r/ntpSbhK6vKzMOF0bSdmFeHPtMaTm2j/dgJlqxTL93ttTsxCXdhU/H0it8SZRUyh6b10CHvhmL+Ztt963yxpH54Ay+GZnEv4+fy+m/Hpc1vGmhJVw7GoLdyVj9aF0PL30kF37u8PN3BXLrEgCuBtcoyMYZJyoU8uqmpLIDsF27f/ogAjMHNUTHtU+oGeN6SV5bNonxkMD+HjxbTNVfbirXJNXxkEvgBn/O2V1H0d+v+39MDhwXhqKLmtLMf1/p5BfUma2hEOlrT9VLexnqZln4/GMGmtldiZmY0O89eYbSzUyK0xGZBkYQmb1oelAVWfNb3bYvumWmpSzUgjZf40bagyulVXi4tUbYfKxbw/g1yMX8Yy9NzELvWiql6nSpBbNVqA0eOg/+/H++gRsP227RqCmn6dfDlf1oZlfw/fUkq9sjNCz5ZsdSQDM53+So6Z1yFzlYp7l2ldL/v1LPIbP2+Pwek/uPolfSk4xfjNZ+mHuX/J+HpTCO6IT/D55EP5v7C0Y0N6+AFOTbqEBksfzH+9r/L+HRgOfRp5OeZ36wtAZsrpCF0xM5shkZ7UdXXPLzK1mI2DsPeVX10dcrbNwg5nxv1NmHbMLqw0NfmbpIby6Oh6XTDoXa0vKUakXEELgsEmNlOGeY2v+IUvrd732Szy++OusWTu/6T3MNKRWVJrPH2SPmf87hV7TtyAttwRD5+7CHZ/txLmsqpFeudf7DJ2/PgGkpaU5avoLvHp/qgorTUsCwNmsQqs1K+dqmuXazlmbLS3roTR7SmQaXmx9yx35rar+cy3nfOl5JcYgvi7uEhKzCrE/yfZgAFdz9js85ItovP3fG7Vq1vrCWPr9cAcMMk7Qu00QRt3S2qnn/GhUDwQ38camV+6UNCV5aDRozBoZCWeMhLKX6WKNNbHnr/GapOdJq/zt/Uv1j+OXodcLq6OV/kyQttW/vz4BX28/h9OXCyQ3WkPn4NTcYvSZ+Rfu/789GPJFNF5dHe9QmQzNQhuPZ+CeL6NxJvNG+IxLu4qfYi4Y+yeZ1nroTGpkHPl+CiFQpKuAXi+wZF8KSsv1+E90kjGYbXWgU2tNLztzo7QGz/T7ZxpmF0Un476vdmP675ZH2NT0fbQ22/aXfyVKarZc0AJXJ0y/z87oW7Q54TJ6Tf8L82toaqspqN75+U6M++EgjposiGupZs7mazi0t3sx/f6464+Wl9IFqO+G9wjF5pOOz//wz6j2ePK2dmYJWKMBa2Sq2Wuh2aK+qN7/p3rnX1tsjTCoftPcEF81qmvu1rM489Fw4/bS8kr8cfwyDqdW1cAkZpnPWaMXwMkM23MIGWpkJq+sGrXx6qp443PTrzflZRaU4q1h3STHmQ73rNTrJTUylXphdaX22X+eMS4jYGB6rJeHxqyD6pVCncUbqKPNHHnFN2pOTJuDDO/HTwdS8ZGFTv1yavCyCkox/3rzjoGz/2our9QjPa8EHVveGGSw9VRWjQtkmr539hTJVtOSnH4p7/x2AgDw5dazePneLjZe177zxaflm5TH4eKolr3ryCmJf9q72KJ/9pd9rKUPJEdqZHq3CZT92mpia0I4tcup1pzgyNwsE3+2vmyDpYn1LL3GzI2nMGnlUSzdd8Hq/lcKSzHy6702y5KWd03yF21eifkwcNPJAg1s1cjYWty0eoipztPDA79WW5sr8lPLE/g5GmSmmcxp4kjNgpx7o6VlPsoq9A6NIqopJDz/42Hc8+UubDpxWbJt6m8nbP48/nwg1eQ1qv4t1lXg1dVx2GLhjzu9pJN01b/5JWWIT8936ZIb9p65NiWoKcdZm5+lolKP/9t2DkdSax5c4CoP/We/8f9u2rLEIOMKtn7gDRPmPTOovaxzhwU2RqBfI7v2Xf/SILvPe/dNLWWVh1yr+urfOicMna7Jmcs3guHxizXP1jzNjsnITl8uwBiTD0RLH9w3Rj/d2GZ6vbpyvWSosyO1UwCw0WToq6cG+L5a2LF2rzQtz6ebzjg0hNaRG7CcDq7Wzr/VSm3clpOZ2BB/CdGJ2XhjzTEU6SpqLGN04hUAMI7ms/eaci3MWfT9nvPYEJ+BFyysjSZdGb7qwdC5uzF6wT58/Mdpu15z77kcPLJwP85lFdYYHpKyi3Ams8Du77urZpT+5I9T6DZts8VazVWxafhq21k8vDDGwpFVagqi205lOW1+INPfvzOZBdjvJrXhDDIuYAgFTX3MW+4GdgjGqZnD8OEDPRw658rnI3F7p+ZYMK4fgnzNh18PNOlo/NHonjg5Y5jZaCgAGNO3NTZMMg84T0S2RfSbgx0qE7nehWrz2ThjDpiaPPqt9Q9NZ7EUZE5c0pqNlDLdz7DStIGhhiY9rwSpucV4c23VvBrWhqaadsr19PQwdvC15UxmAf72lXQJi0krj9Z43PbTWRj3wwGzFdFtyS8px+nLljuuW2Pt58HSqJrySj1e+OkIXl0dj6eXHsJ/j17ENzuSUG5nIDTcxOwdsWP66WP4Sz6rwHpzlOkN2dAkZ2i+Wrb/gl2v+eTigzicehV/+2q3JOj+98hF/LDnRnCt1AsMnbsLw+ftsXt+IsmCr8Kx5i5be35/fdZxSyOF7KlttpWvinQV+NePh/Hc8sNOmW/LtEZm+Lw9eOKHg7WbusBJ2EfGBSbc1RHhQY1xW8fmFp/383b82357pxa4vVPVMGxLNwEPk0j6t+6t0MRCiAKAxt6e6BMRZHz8+MAIjItshx7hAdBoNEiZdT8W7TqPedvOYliPULMZcZU2pGtL7Lz+F2JDUH1Ez6JdNS8DoQbWbp6mnYgBSP5yjzmfixKTn/3jF/ORlleC99ZJR0tVX2fLkmk2RlgZrDiYanZuexkmlssptH8m5WX7L2DZ/gtY99Lt6Nu2mcV93v71GN69v7txAECplVBhaQbiIguzVZ/M0KLcziH9BpaW5wCqhrbHnM/B7Z1aoHEjT8lN79td5+Hp4WHWNHGlUIfmTbzh4aGRDumXMSHe00uls4abjlp8Y23VTN5DuoWgU8umKDEZDZiptW+wgOnUB4dTr+KNtcfw/sju+MeACLuOd0RecdXUC/ZUAlXo9fD0sNxv0jS87DiTjZG9w2pVrpdWHMVPzw3EnV1u1OAnXylCu+ZNanXe2mKQcYFGnh54qG8b4+P37u+OTzadxvsjuzvl/I0beeK9+7ujpKzSOP+D6Q+86ezCH4/uKRkW61uto7CPlyd6tr7Rl0aj0eDFwZ3w4vXlEhwJMh1bNsH5K65N52FBvujZOgAJlxz7y5XqB9NgZ22tLHuaw+whN8SYuqy1PNHcY9/G4Od/RVqs8t92Ogt92zbDrrNXsKlaM9aawxfh6aHBrDG9AVjvW2EYVbPlZCYOnM/F07e3x0ULk97tOZeDzzdLa7qKdRU4lp6P99cnSGutrn+sWHvN7h9sBgCMvTUCsx/uLXmuQi/w9fZzaGEyYejB87l47LsDAIA7OrfAuMi2xueSrxTh5nDpNBQGn20+g8cHtjXbHm3HHzhXCnW4dPWapBbB1hInpsO3TWtkDH9QvPXrcZzLLsKl/GuYP7avWS34xuMZdi8LklOkg15fNcVAv4+2AoBdwcMwJcKxi1p0CWkq+SPWtPZs0sqjGND+XrQKaCy5PluL41ZfugWo+r1L/vR+4+O6qCWuCYNMHXj+ro4Y06+1Q7P+2nNO4MZEVqbNWE0b3/j/k7e1kwSZgMbS/jXOnHhq48t34OYPtgAAuoQ0rXFeDIN7uoVgxxn7pgfvFurv8Poijw5ogzWHL1p8rm/bIMSZjEaw5G83t7La54DIlgIra3YdTMnDtPUJWH3IfCHIBTuT8f3uFKsdaVfFpuOFuzqhfYsmNhfxS8kpNtZo2eqs/fMB6USGPT7cYnG/2JQ8bD2VhY4tbf/1vfpQulmQMTAd6WQIMUDVyEPT6315VZzViUV1FXqMWmC7c7k16XkleKvaLMSmNTIJl7ToHhaAL/5KxLmsImwzCZp5VgLJd9f7Wk28qxN6tQnEhvhLCG7ijcTMwhr79pi+f8cuatHx3U2SNfdMA1dMci5aNPU2TrxqUKEX+OPEZUxeGYe+bYOw7nrfyMTMQknNEwBEfrodG1++w/jH65d/nbXabPfrkYt4c+0xi8+ZhpeJPx/FhdkjbV6nq6kiyCxYsABz5sxBZmYm+vTpg/nz52PgwIFKF8shzgwxlvh5e+J/k++oGp7tZX149jN3tJc8dmRujnfv74ZPN52x+NySpwfAz9sLgzo3x76kXDwV1Q4FpRXG6erH9G2NxKxCi5PX9WkTJAkym165E2l5xejXrhkGfrJdsu/YW9tidaxjqwC/fE8Xq0Fmwp0d8eIK630ePnu4Fx4dEIF3153AKgdf15k6tGhidR4RUidLIcagptFpg7+Itvl89UVRneX5Hw/XGGQA4M7Pd5jNgVST2BTpyJzX11i/Bluj7mypHmIA6Wfg3+fvxYwHe5hNGAkAP8akmm0zVVpRiaTsQrPm0equlVXise9irNYcmi6KW6K7EXQe/74q+H3w95sloaqismqNJACIS8tHprYU205n4f31CQg1qX0x+Pv8vZj7aB808/NGnI0/Cq2FGADG6RjchdsHmV9++QWvv/46Fi1ahMjISMybNw/Dhg1DYmIiQkJClC6e22jR1Ae97BhuXb1Gxt4Oa1/8ow/G9G2NVbHpKLhWjvt7heGn68Mro98cjPYtqj7cFo+/FWezCtGrdSB0FXrsOnsFBdfK8d7I7ghu4o0OUzcBAAa0a4bDqVXDcT000r4vN4cH4ObwAItVst5eHphwV0e89ku83TUlEcF+ksdjb40w3kQCfW98P4L8GplNIPfYrW2vl9G+cYeTh3TGNzuTat7RQfd0C5G9XlGfiCCnr5JLDZc9zceOhhhLlJof6kMrExbW5B+L7Oskb2iCs4elDunVJ2A0NEMZ3Dbrxh9/mVYmC7UVEu1RvVk3v6RM0TUA3X7U0ty5c/H888/jmWeewc0334xFixbBz88PS5YsUbpobuGbJ/rinm4h+PfQm6zuY2tF7urLIVS3ePwATLy7E8b0bQ0PDw02vXIn9kwZgvf/3h3PDuqAlc9HGkMMUNV/p3ebIGg0GjRu5Ik1L0Rh82t3oXlTH8m8OA/1a42notqhpb8Pxt3WDnP+0QfPDuqArf++y7hP9Wavv19vLx7dtzW2v3E3Fo7rhzbNfG2W37C8Q+y79xq3zX64N9687yaMviVc0iG7eoiZPKSz8f+mQ073v3MPxt4agVfu7YI1L0QZ177qFuqPN4d1xbv335jUzf96k1/19bNMffaw9eeAqr5Hr//N+vsLAL9MuM3qc6P6hBv/7+0p/ZVf+Xwkvnmib/VDXMpd56IgInlu/WQbcmuYINGVNMIdlvK0oqysDH5+fvj1118xevRo4/bx48cjPz8fGzZsMDtGp9NBp7vxDS0oKEBERAS0Wi0CAmzftOurwxfy8NavxzH9wR7GoeEnM7TYl5SDZwZ1QCPPusuz568U4fCFq3ikf5uqkQp6YXGYOFDV6dDQXn/4/aEI9vM221cIgZWxadh47DJKyirwywtRuFZWieQrRWjfoomkc+GFnGL4NPJAWKA0/Ly37gRWHEzDW8O6YsvJTDT18cKKf0VKgteB87kY+90BdAv1x+bX7pIcb5hfQqOp6ix9tbgMy2Mu4OF+bdC4kScSMwsxqHNzfLf7PFJyinE6sxAD2jUz1rAY2peTsgsxdO5u3HdzKwxo3wyNG3li/o4kLH36VvRsHQghBJ5aEos953LQ0t8H+6bcg/u+2oVmTbyx7qVB+CnmgsU5XRaO64ebQv0RnXgF93QLwZAvotEnIghrXrjN2AzZ/p0/jPsH+TXCT89GYuPxDGivleNMZiFevqezcSTO/429BZ4eGmRqS/HcHR0wf0cSfj6Qikf6t8G429rh6SWxOJddBC8PjaTavnEjD+x+ewhC/Bsj4ZIWf59vuZ/Dk7e1xYqDaZh4dyeUlldCrxdYXkO1PlDVvNnI0wMdWzZFypViPLn4oPF8Px9Ig5+3J0L8fYxD2u/s0gLJ2UXIuN5H4uvH++KVVXE1vo4lvdsEWmwqiAj2tat2okVTH5sz5d7RuYVTaig6tWyCZCs1KqNvCcf6eGVHKQ7tHoJt1RbP/L+xt9TYXEPKe/meznjjvq5OPWdBQQECAwNrvH+7dZDJyMhA69atsX//fkRFRRm3v/3229i1axcOHjxodsz06dMxY8YMs+0NOcio2c7EbGgADO7qumbEsgo9TmZo0btNEDw01qd4P3+lCOFBvmjspCUiNsRfQtdQf5u1YkIISXmuFOrw7a5kPB7ZFp1aNkVFpR4ajUYyVf+Ji1r4N/bC2axCxKXn4637ukoCYGFpOfy8vSTHHL+YjyOpVzE+qj0qhbAYbpOyi3AsPR8PXa+dsya3SIctJ7PwSP82KCmrgK+3J8orBZp4e0qu5VxWIZbtv4Bn7+iATi2bIruwFD5engj0bWR23QmXtGji44WIZr7w9NBUTRWQU4xP/jiFFwd3Qp82QfCqVub9yVU3/qiOzXE+pxgRzfzgbWFW7NTcYvg28kRIQGMkXynC2cxCXMgtQZGuHI08PfDM7R1wMkOL5k19cFl7DZ1DmmLbqSzcFOqP8EBfY6fbfUk5iOrUHL6NPLHiYBraNffDnV1aQq8XyC0uQ3ATb1To9fD29MCVQh2yC3XYeioLj94agaY+Xpjxv5O4p1sIYlPy8GNMKh4d0Aav3NsFHhoNwoOqwndZhR5L9qUgLLAxbmrlj1l/nsFzd3RATHIuOrTww9J9F5BZUIrdbw9BypUbwb1xo6rr9vLwwOw/T+P7PSl4dEAbaK+VI6pjcwzuGoL2LZrgaNpVdAlpCk8PDdYcSof2WgUCfb2My0hMGtIJFZUCI3qFobS8EgfP5yHA1wvtWzTB7Z2aY93RS/gxJhVDu4egf/tg3NYxGKcvF+JCTjFSc0ug0QAD2jdDZIfmKLhWjm2nsxCdeAURwX6YeHdHBPl5I6dIh58PpEIDDQZ1bo4B7YNxKf8ahBDw9vLA7E1nsPVUFsr1ehyd9jd4aDSY/ecZxKVdxbnsInw8uifaNPNDp5ZN0MjLA6+tjkdpeSXevb87uoX6w8vTAwfP56K8UuDA+VzcEhGEsko9Fu1KNoZR30ae+N/LgxAS0BhFpRV4ccVRnL9ShAf7hKORpweiE7NxS0RQ1aivR3ojJjkXP+xNwe2dmmNMvzboHuYPHy9P5BbpsOnEZYzq2xo5hTo09fHCoQtX8dW2s4jsEIyf/xWJ1NxizPjfKePM1k28PdEtLABHrje939YxGAPbByMltwQv39MZOYU65JWUoai0AjeHB2DRrmRkF+jg6+1pPEe3UH+se2kQ/jqVibbBfth8MhPnsorQv10z+DbyRE6RDl1D/fF7fAbu7xWGJj6e8PX2wm9HL+KebiFo6uOFDG2pcZqCrq38UVBajvJKPQa0C8ZD/Vpj8Z4UjB0YgTWH03Euqwi/vBCFziHSjsi11WCDDGtkiIiI1M/eIOPWnX1btGgBT09PZGVJO3RmZWUhNDTU4jE+Pj7w8XHtCCEiIiJyD27d2dfb2xv9+/fH9u03emHr9Xps375dUkNDREREDZNb18gAwOuvv47x48djwIABGDhwIObNm4fi4mI888wzSheNiIiIFOb2Qeaxxx7DlStX8MEHHyAzMxO33HILNm/ejFatWildNCIiIlKYW3f2dQZ7OwsRERGR+7D3/u3WfWSIiIiIbGGQISIiItVikCEiIiLVYpAhIiIi1WKQISIiItVikCEiIiLVYpAhIiIi1WKQISIiItVikCEiIiLVcvslCmrLMHFxQUGBwiUhIiIiexnu2zUtQFDvg0xhYSEAICIiQuGSEBERkaMKCwsRGBho9fl6v9aSXq9HRkYG/P39odFonHbegoICREREID09vd6u4VTfr7G+Xx9Q/6+R16d+9f0a6/v1Aa67RiEECgsLER4eDg8P6z1h6n2NjIeHB9q0aeOy8wcEBNTbH06D+n6N9f36gPp/jbw+9avv11jfrw9wzTXaqokxYGdfIiIiUi0GGSIiIlItBhmZfHx88OGHH8LHx0fporhMfb/G+n59QP2/Rl6f+tX3a6zv1wcof431vrMvERER1V+skSEiIiLVYpAhIiIi1WKQISIiItVikCEiIiLVYpCRacGCBWjfvj0aN26MyMhIxMbGKl0ku8yaNQu33nor/P39ERISgtGjRyMxMVGyz+DBg6HRaCRfEydOlOyTlpaGkSNHws/PDyEhIXjrrbdQUVFRl5di0fTp083K3q1bN+PzpaWlmDRpEpo3b46mTZvi4YcfRlZWluQc7nptBu3btze7Ro1Gg0mTJgFQ3/u3e/duPPDAAwgPD4dGo8H69eslzwsh8MEHHyAsLAy+vr4YOnQozp07J9knLy8P48aNQ0BAAIKCgvDcc8+hqKhIss/x48dx5513onHjxoiIiMDnn3/u6ksDYPv6ysvLMWXKFPTq1QtNmjRBeHg4nnrqKWRkZEjOYek9nz17tmQfpa4PqPk9fPrpp83KP3z4cMk+an0PAVj8fdRoNJgzZ45xH3d+D+25LzjrszM6Ohr9+vWDj48POnfujGXLltX+AgQ5bPXq1cLb21ssWbJEnDx5Ujz//PMiKChIZGVlKV20Gg0bNkwsXbpUJCQkiPj4eHH//feLtm3biqKiIuM+d999t3j++efF5cuXjV9ardb4fEVFhejZs6cYOnSoiIuLE5s2bRItWrQQU6dOVeKSJD788EPRo0cPSdmvXLlifH7ixIkiIiJCbN++XRw+fFjcdttt4vbbbzc+787XZpCdnS25vq1btwoAYufOnUII9b1/mzZtEu+995747bffBACxbt06yfOzZ88WgYGBYv369eLYsWPiwQcfFB06dBDXrl0z7jN8+HDRp08fceDAAbFnzx7RuXNn8fjjjxuf12q1olWrVmLcuHEiISFBrFq1Svj6+opvv/1W0evLz88XQ4cOFb/88os4c+aMiImJEQMHDhT9+/eXnKNdu3Zi5syZkvfU9HdWyeur6RqFEGL8+PFi+PDhkvLn5eVJ9lHreyiEkFzX5cuXxZIlS4RGoxHJycnGfdz5PbTnvuCMz87z588LPz8/8frrr4tTp06J+fPnC09PT7F58+ZalZ9BRoaBAweKSZMmGR9XVlaK8PBwMWvWLAVLJU92drYAIHbt2mXcdvfdd4tXX33V6jGbNm0SHh4eIjMz07ht4cKFIiAgQOh0OlcWt0Yffvih6NOnj8Xn8vPzRaNGjcTatWuN206fPi0AiJiYGCGEe1+bNa+++qro1KmT0Ov1Qgh1v3/VbxJ6vV6EhoaKOXPmGLfl5+cLHx8fsWrVKiGEEKdOnRIAxKFDh4z7/Pnnn0Kj0YhLly4JIYT4z3/+I5o1aya5vilTpoiuXbu6+IqkLN0Eq4uNjRUARGpqqnFbu3btxFdffWX1GHe5PiEsX+P48ePFqFGjrB5T397DUaNGiXvuuUeyTU3vYfX7grM+O99++23Ro0cPyWs99thjYtiwYbUqL5uWHFRWVoYjR45g6NChxm0eHh4YOnQoYmJiFCyZPFqtFgAQHBws2b5ixQq0aNECPXv2xNSpU1FSUmJ8LiYmBr169UKrVq2M24YNG4aCggKcPHmybgpuw7lz5xAeHo6OHTti3LhxSEtLAwAcOXIE5eXlkveuW7duaNu2rfG9c/drq66srAw///wznn32WcmiqGp+/0ylpKQgMzNT8p4FBgYiMjJS8p4FBQVhwIABxn2GDh0KDw8PHDx40LjPXXfdBW9vb+M+w4YNQ2JiIq5evVpHV2MfrVYLjUaDoKAgyfbZs2ejefPm6Nu3L+bMmSOpslfD9UVHRyMkJARdu3bFiy++iNzcXONz9ek9zMrKwh9//IHnnnvO7Dm1vIfV7wvO+uyMiYmRnMOwT23vnfV+0Uhny8nJQWVlpeTNAoBWrVrhzJkzCpVKHr1ej9deew2DBg1Cz549jdufeOIJtGvXDuHh4Th+/DimTJmCxMRE/PbbbwCAzMxMi9dveE5JkZGRWLZsGbp27YrLly9jxowZuPPOO5GQkIDMzEx4e3ub3SBatWplLLc7X5sl69evR35+Pp5++mnjNjW/f9UZymOpvKbvWUhIiOR5Ly8vBAcHS/bp0KGD2TkMzzVr1swl5XdUaWkppkyZgscff1yy+N4rr7yCfv36ITg4GPv378fUqVNx+fJlzJ07F4D7X9/w4cMxZswYdOjQAcnJyXj33XcxYsQIxMTEwNPTs169h8uXL4e/vz/GjBkj2a6W99DSfcFZn53W9ikoKMC1a9fg6+srq8wMMg3YpEmTkJCQgL1790q2T5gwwfj/Xr16ISwsDPfeey+Sk5PRqVOnui6mQ0aMGGH8f+/evREZGYl27dphzZo1sn9J3NnixYsxYsQIhIeHG7ep+f1ryMrLy/Hoo49CCIGFCxdKnnv99deN/+/duze8vb3xwgsvYNasWaqY+n7s2LHG//fq1Qu9e/dGp06dEB0djXvvvVfBkjnfkiVLMG7cODRu3FiyXS3vobX7gjtj05KDWrRoAU9PT7Pe2llZWQgNDVWoVI6bPHkyNm7ciJ07d6JNmzY2942MjAQAJCUlAQBCQ0MtXr/hOXcSFBSEm266CUlJSQgNDUVZWRny8/Ml+5i+d2q6ttTUVGzbtg3/+te/bO6n5vfPUB5bv2+hoaHIzs6WPF9RUYG8vDzVvK+GEJOamoqtW7dKamMsiYyMREVFBS5cuADA/a+vuo4dO6JFixaSn0m1v4cAsGfPHiQmJtb4Owm453to7b7grM9Oa/sEBATU6g9NBhkHeXt7o3///ti+fbtxm16vx/bt2xEVFaVgyewjhMDkyZOxbt067Nixw6wq05L4+HgAQFhYGAAgKioKJ06ckHzwGD58b775ZpeUW66ioiIkJycjLCwM/fv3R6NGjSTvXWJiItLS0ozvnZqubenSpQgJCcHIkSNt7qfm969Dhw4IDQ2VvGcFBQU4ePCg5D3Lz8/HkSNHjPvs2LEDer3eGOKioqKwe/dulJeXG/fZunUrunbtqniThCHEnDt3Dtu2bUPz5s1rPCY+Ph4eHh7G5hh3vj5LLl68iNzcXMnPpJrfQ4PFixejf//+6NOnT437utN7WNN9wVmfnVFRUZJzGPap9b2zVl2FG6jVq1cLHx8fsWzZMnHq1CkxYcIEERQUJOmt7a5efPFFERgYKKKjoyXDAEtKSoQQQiQlJYmZM2eKw4cPi5SUFLFhwwbRsWNHcddddxnPYRhmd99994n4+HixefNm0bJlS7cYovzGG2+I6OhokZKSIvbt2yeGDh0qWrRoIbKzs4UQVUMI27ZtK3bs2CEOHz4soqKiRFRUlPF4d742U5WVlaJt27ZiypQpku1qfP8KCwtFXFyciIuLEwDE3LlzRVxcnHHUzuzZs0VQUJDYsGGDOH78uBg1apTF4dd9+/YVBw8eFHv37hVdunSRDN3Nz88XrVq1Ev/85z9FQkKCWL16tfDz86uToa22rq+srEw8+OCDok2bNiI+Pl7yO2kY6bF//37x1Vdfifj4eJGcnCx+/vln0bJlS/HUU0+5xfXVdI2FhYXizTffFDExMSIlJUVs27ZN9OvXT3Tp0kWUlpYaz6HW99BAq9UKPz8/sXDhQrPj3f09rOm+IIRzPjsNw6/feustcfr0abFgwQIOv1bS/PnzRdu2bYW3t7cYOHCgOHDggNJFsgsAi19Lly4VQgiRlpYm7rrrLhEcHCx8fHxE586dxVtvvSWZh0QIIS5cuCBGjBghfH19RYsWLcQbb7whysvLFbgiqccee0yEhYUJb29v0bp1a/HYY4+JpKQk4/PXrl0TL730kmjWrJnw8/MTDz30kLh8+bLkHO56baa2bNkiAIjExETJdjW+fzt37rT4Mzl+/HghRNUQ7GnTpolWrVoJHx8fce+995pdd25urnj88cdF06ZNRUBAgHjmmWdEYWGhZJ9jx46JO+64Q/j4+IjWrVuL2bNnK359KSkpVn8nDfMCHTlyRERGRorAwEDRuHFj0b17d/Hpp59KQoCS11fTNZaUlIj77rtPtGzZUjRq1Ei0a9dOPP/882Z/+Kn1PTT49ttvha+vr8jPzzc73t3fw5ruC0I477Nz586d4pZbbhHe3t6iY8eOkteQS3P9IoiIiIhUh31kiIiISLUYZIiIiEi1GGSIiIhItRhkiIiISLUYZIiIiEi1GGSIiIhItRhkiIiISLUYZIiIiEi1GGSIqMHRaDRYv3690sUgIidgkCGiOvX0009Do9GYfQ0fPlzpohGRCnkpXQAianiGDx+OpUuXSrb5+PgoVBoiUjPWyBBRnfPx8UFoaKjkq1mzZgCqmn0WLlyIESNGwNfXFx07dsSvv/4qOf7EiRO455574Ovri+bNm2PChAkoKiqS7LNkyRL06NEDPj4+CAsLw+TJkyXP5+Tk4KGHHoKfnx+6dOmC33//3bUXTUQuwSBDRG5n2rRpePjhh3Hs2DGMGzcOY8eOxenTpwEAxcXFGDZsGJo1a4ZDhw5h7dq12LZtmySoLFy4EJMmTcKECRNw4sQJ/P777+jcubPkNWbMmIFHH30Ux48fx/33349x48YhLy+vTq+TiJyg1utnExE5YPz48cLT01M0adJE8vXJJ58IIYQAICZOnCg5JjIyUrz44otCCCG+++470axZM1FUVGR8/o8//hAeHh4iMzNTCCFEeHi4eO+996yWAYB4//33jY+LiooEAPHnn3867TqJqG6wjwwR1bkhQ4Zg4cKFkm3BwcHG/0dFRUmei4qKQnx8PADg9OnT6NOnD5o0aWJ8ftCgQdDr9UhMTIRGo0FGRgbuvfdem2Xo3bu38f9NmjRBQEAAsrOz5V4SESmEQYaI6lyTJk3MmnqcxdfX1679GjVqJHms0Wig1+tdUSQiciH2kSEit3PgwAGzx927dwcAdO/eHceOHUNxcbHx+X379sHDwwNdu3aFv78/2rdvj+3bt9dpmYlIGayRIaI6p9PpkJmZKdnm5eWFFi1aAADWrl2LAQMG4I477sCKFSsQGxuLxYsXAwDGjRuHDz/8EOPHj8f06dNx5coVvPzyy/jnP/+JVq1aAQCmT5+OiRMnIiQkBCNGjEBhYSH27duHl19+uW4vlIhcjkGGiOrc5s2bERYWJtnWtWtXnDlzBkDViKLVq1fjpZdeQlhYGFatWoWbb74ZAODn54ctW7bg1Vdfxa233go/Pz88/PDDmDt3rvFc48ePR2lpKb766iu8+eabaNGiBR555JG6u0AiqjMaIYRQuhBERAYajQbr1q3D6NGjlS4KEakA+8gQERGRajHIEBERkWqxjwwRuRW2dhORI1gjQ0RERKrFIENERESqxSBDREREqsUgQ0RERKrFIENERESqxSBDREREqsUgQ0RERKrFIENERESq9f++eovw32NgdwAAAABJRU5ErkJggg==",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "import seaborn as sns\n",
    "\n",
    "def plot_train_loss(): \n",
    "  losses_float = [float(loss.cpu().detach().numpy()) for loss in LOSSES] \n",
    "  loss_indices = range(len(losses_float))\n",
    "  ax = sns.lineplot(x=loss_indices, y=losses_float)\n",
    "  ax.set(xlabel='Epoch', ylabel='Loss')\n",
    "  \n",
    "plot_train_loss()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Predict solubility on test data and compare with true solubilities"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAjUAAAGwCAYAAABRgJRuAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8pXeV/AAAACXBIWXMAAA9hAAAPYQGoP6dpAAA7eUlEQVR4nO3de3SU1b3/8c8EciVXmUSIhBAgRakmIBwQwYDIUeqVkoWKVgEpP62oVLxFrYJFhShaFbRSLQg9p4g2HrVqaTVQuYhFJZpiuSQIQgkIQZgkhGRymd8fNCMhyeSZZK7PvF9rZa3meSaTbzLY+WTv797b4nA4HAIAAAhyYf4uAAAAwBMINQAAwBQINQAAwBQINQAAwBQINQAAwBQINQAAwBQINQAAwBS6+rsAX2psbFRZWZni4uJksVj8XQ4AADDA4XCosrJSqampCgtrezwmpEJNWVmZ0tLS/F0GAADogH379qlXr15t3g+pUBMXFyfp5C8lPj7ez9UAAAAjKioqlJaW5nwfb0tIhZqmKaf4+HhCDQAAQaa91hEahQEAgCkQagAAgCkQagAAgCkQagAAgCkQagAAgCkQagAAgCkQagAAgCkQagAAgCkQagAAgCkQagAAgCmE1DEJAADA82zVdpVX2VVRU6f46HBZu0UoISbC53UQagAAQIeVHTuhBwqKtb6k3HktJ9OqBblZSk2M9mktTD8BAIAOsVXbWwQaSVpXUq68gmLZqu0+rYdQAwAAOqS8yt4i0DRZV1Ku8ipCDQAACAIVNXUu71e2c9/TCDUAAKBD4qPCXd6Pa+e+pxFqAABAh1hjI5STaW31Xk6mVdZY366AItQAAIAOSYiJ0ILcrBbBJifTqvzcLJ8v62ZJNwAA6LDUxGgtmjxY5VV2VdbUKS4qXNZY9qkBAABBKCHGPyHmdEw/AQAAUyDUAAAAUyDUAAAAUyDUAAAAUyDUAAAAUyDUAAAAUyDUAAAAU2CfGgAA4BZbtV3lVXZV1NQpPjpc1m6BsU8NoQYAABhWduyEHigo1vqScue1nEyrFuRmKTUx2o+VBeH004svvqg+ffooKipKw4cP1+bNm/1dEgAAIcFWbW8RaCRpXUm58gqKZau2+6myk4Iq1KxatUqzZ8/WnDlztGXLFmVnZ+uyyy7ToUOH/F0aAACmV15lbxFomqwrKVd5FaHGsGeffVYzZszQtGnTNHDgQL388suKiYnR0qVL/V0aAACmV1FT5/J+ZTv3vS1oQo3dbtcXX3yhcePGOa+FhYVp3Lhx2rRpU6tfU1tbq4qKimYfAACgY+Kjwl3ej2vnvrcFTagpLy9XQ0ODzjzzzGbXzzzzTB08eLDVr5k/f74SEhKcH2lpab4oFQAAU7LGRign09rqvZxMq6yx/l0BFTShpiMefPBB2Ww258e+ffv8XRIAAEErISZCC3KzWgSbnEyr8nOz/L6sO2iWdFutVnXp0kXfffdds+vfffedevTo0erXREZGKjIy0hflAQAQElITo7Vo8mCVV9lVWVOnuKhwWWMDY5+aoBmpiYiI0JAhQ1RYWOi81tjYqMLCQo0YMcKPlQEAEFoSYiLULyVWg3onqV9KbEAEGimIRmokafbs2ZoyZYqGDh2qYcOG6bnnntPx48c1bdo0f5cGAAD8LKhCzXXXXafDhw/r0Ucf1cGDBzVo0CCtXr26RfMwAAAIPRaHw+HwdxG+UlFRoYSEBNlsNsXHx/u7HAAAYIDR9++g6akBAABwhVADAABMgVADAABMgVADAABMIahWPwEAQout2q7yKrsqauoUHx0ua7fA2OQNgYlQAwAISGXHTuiBgmKtLyl3XsvJtGpBbpZSE6P9WBkCFdNPAICAY6u2twg0krSupFx5BcWyVdv9VBkCGaEGABBwyqvsLQJNk3Ul5SqvItSgJaafAABe0Zl+mIqaOpf3K9u5j9BEqAEAeFxn+2Hio8Jd3o9r576n0bAcHAg1AACPaq8fZtHkwe0GAmtshHIyrVrXyhRUTqZV1ljfBQoaloMHPTUAAI/yRD9MQkyEFuRmKSfT2ux6TqZV+blZPhsloWE5uDBSAwDwGFu1Xd+380ZvtB8mNTFaiyYPVnmVXZU1dYqLCpc11rPTPu1NKxkJaExDBQ5CDQDAI5qmaaZe2Mfl49zph0mI8V7vipFpJRqWgwvTTwCATjt1mqZo3zGN7N+91cf5uh+mLUanlQKtYRmuEWoAAJ126jTN0g27NW1kRotg4+t+GFeM9v00NSy3JlACGn7A9BMAoNNOnaaptjforpVFumVUhm4ZmaHa+kb16R6jsxKjAyLQSManlZoalvMKiputxAqkgIYfEGoAAJ12+jRNtb1Bi9eUOj8vnD06oAKAO9NKvmhYhmcw/QQA6LRgm6Zxt96EmAj1S4nVoN5J6pcSS6AJUIQaAECnBcq+MkYFW70wxuJwOBz+LsJXKioqlJCQIJvNpvj4eH+XAwCm07TvS7BM0wRbvaHK6Ps3PTUAAI/x5r4y3hBs9cI1pp8AAIApEGoAAIApEGoAAIApEGoAAIApEGoAAIApEGoAAIApEGoAAIApsE8NAASQps3gKmrqFB8dLms39lEBjCLUAECAKDt2Qg8UFGv9aadBL8jNUmpitB8rA4ID008AEABs1fYWgUaS1pWUK6+gWLZqu58qA4IHIzUAEADKq+wtAk2TdSXlKq+yB/w0FFNn8DdCDQAEgIqaOpf3K9u5729MnSEQMP0EAAEgPirc5f24du77E1NnCBSEGgAIANbYCOVkWlu9l5NplTU2cKdxjEydAb5AqAGAAJAQE6EFuVktgk1OplX5uVkB3ZsS7FNnMA96agAgQKQmRmvR5MEqr7KrsqZOcVHhssYGfrOtO1NnNBPDmwg1ABBAEmKC702+aepsXStTUKdOndFMDG9j+gkA0ClGps5oJoYvMFIDAEEsUKZz2ps6M8M+PAh8hBoACFKBNp3jauqMZmL4AtNPABCEgm06J5j34UHwINQAQBAKtr1hgnkfHgQPQg0ABKFgm84J5n14EDzoqQGAIBSM0znBug8PggehBgCCkNG9YQJNMO7Dg+DB9BMABCGmc4CWGKkBgCDFdA7QHKEGAIIY0znAD5h+AgAApkCoAQAApsD0EwDA6wLljCqYG6EGAOBVgXZGFcyL6ScAgNcE2xlVCG6EGgCA1wTbGVUIboQaAIDXBNsZVQhuhBoAgNcE4xlVCF6EGgAIMbZqu3YdqlLR3qPadbjKq30tTWdUtSaQz6hCcGL1EwCEEF+vRGo6oyqvoLjZ4ZucUQVvsDgcDoe/i/CViooKJSQkyGazKT4+3t/lAIBP2artumNlUauNuzmZVi2aPNhrIaNpnxrOqEJHGH3/ZqQGAEKEkZVI3goanFEFXyDUAIAf+XKnXVYiwewINQDgJ231t+TnZikmoovHww4rkWB2hBoA8IO2dtr9/Nuj+vb7ar24plTrSz3bzNu0EmldGz01rERCsGNJNwD4QVv9LbeMytCiNSXNAo3kmWMFmlYinb7EmpVIMAtGagDAD9rqbxmclqjFa0pbveeJZt7UxGgtmjyYlUgwJUINAPhBW/0ttfWNLr/OSDNve83HrESCWRFqAMAP2upviezquivAVTOvrdquo9V1euTtf2p96RHndW9urgcEEnpqAMAP2upvOVRZq4s6cKxA2bET+mDrQT18WqCRPNOPAwSDoAg1e/bs0fTp05WRkaHo6Gj169dPc+bMkd3Of6AAgldTf0vh7NF6+/YLVTh7tC4/t4fy3WzmbVpJlRIXqY2nBZomTf04gJkFxfTT9u3b1djYqCVLlqh///7aunWrZsyYoePHj2vhwoX+Lg8AOqy1/paEGLnVzNu0kmrysN4uvxeb68HsgiLUjB8/XuPHj3d+3rdvX+3YsUO//e1vXYaa2tpa1dbWOj+vqKjwap0A4CnuNPM2raTqTD8OYAZBMf3UGpvNpjPOOMPlY+bPn6+EhATnR1pamo+qAwDfaVpJVbTvmEb2797qY9hcD6EgKENNaWmpFi1apFtvvdXl4x588EHZbDbnx759+3xUIQD4TtNKqqUbdmvayIwWwYbN9RAqLA6Hw+Gvb56Xl6f8/HyXj9m2bZvOPvts5+f79+/X6NGjNWbMGL366qtufT+jR5cDQLApO3ZCeQXF+vzbo7plVIYGpyVKknolRatHfBSBBkHN6Pu3X0PN4cOHdeRI6536Tfr27auIiJP/MZaVlWnMmDG64IIL9NprrykszL2BJkINADNr2nSPnYJhNkbfv91uFF67dq0uvvjiThXXJDk5WcnJyYYeu3//fl188cUaMmSIli1b5nagAQCzY6dghDq3k8H48ePVr18/Pf744z7rUdm/f7/GjBmj3r17a+HChTp8+LAOHjyogwcP+uT7A4Cn2Krt2nWoSkV7j2rX4So2xAM8yO2Rmv379+sPf/iDli9frscee0xjx47V9OnTNWHCBOc0kad9+OGHKi0tVWlpqXr16tXsnh9nzwDALWXHTuiBguJmp3NzhAHgOZ3qqdmyZYuWLVumlStXSpJuuOEGTZ8+XdnZ2R4r0JPoqQHgL7Zqu+5YWdQs0DTJybRq0eTBTB0BbTD6/t2pxpTzzz9fDz74oO644w5VVVVp6dKlGjJkiC666CJ9/fXXnXlqADCVpl1/W8MRBoBndCjU1NXV6U9/+pMuv/xypaen669//asWL16s7777TqWlpUpPT9ekSZM8XSsABK2Kdo4o4AgDoPPc7qm58847tXLlSjkcDt1000166qmndO655zrvd+vWTQsXLlRqaqpHCwUQepqWKFfU1Ck+OlzWbsG7uie+nSMKOMIA6Dy3Q82//vUvLVq0SBMnTlRkZGSrj7FarVq7dm2niwMQuszWVNu06++6Nnpq3DnCwExhD/AktxuF161bpwsvvFBduzbPQ/X19frkk0+Uk5Pj0QI9iUZhIDiYtam2adffdacFtfzcLPU0GNTMFvYAI7y2o3CXLl104MABpaSkNLt+5MgRpaSkqKGhoWMV+wChBggOuw5V6ZJnP27zfuHs0eqXEuvDijynM7v+mjXsAe3x2o7CDodDFoulxfUjR46oW7du7j4dALRg5qbazuz6a2QFFaEGocxwqJk4caIkyWKxaOrUqc36aRoaGlRcXKwLL7zQ8xUCCDk01bbOzGEP8ATDoSYhIUHSyZGauLg4RUf/MHcbERGhCy64QDNmzPB8hQBCjiebaiXzNNZ2JOyZ5WcHjDAcapYtWyZJ6tOnj+69916mmgB4TUJMhBbkZrXZVOvOm7KZGmvdDXtm+tkBIzp1TEKwoVE48PBXJFwx0lTr6t+QGRtrja6gMuPPjtDl0Ubh888/X4WFhUpKStLgwYNbbRRusmXLFverRUjir0i0p72m2vb+DZmxsTY1MVqLJg9uN+yZ8WcH2mMo1FxzzTXOxuAJEyZ4sx6ECFu1vcWbkXTy/2zzCor5KxLtMvJvyFONtYE2omhkBRVNxQhFhkLNnDlzWv3fQEfxVyQ6y8i/IU+sourIiGIghCBWkCEUub1PDeAJ/BWJzjLybyjD2q1Tq6g6MqLYWgj673NSNPfqH6umrtFnQcfTK8iAYGAo1CQlJbnsoznV999/36mCEBr4KxKdZeTfUGdXUbk7othaCIqJ6KLrhvXW/QXF2lh6pFkN3uwf8+QKMiBYGAo1zz33nJfLQKjhr0h0ltF/Q0Yba1vj7ohiayHollEZWrZxd7NAI/mmf6wzPzsQjAyFmilTpni7DoQY/opEZ7nzb6ijRxO4O6LYWgganJaoxWtKW/16X/SPdeZYBiDYGAo1FRUVznXhFRUVLh/L/i8wir8i0VlG/w11tHHX3RHF1kJQbX2jy+9B/xjgOYZ7appO5k5MTGy1v6bpoMtAPqUbgYe/ItFZnd3Lpr3ndmdEsbUQFNk1zOX3oH8M8BxDoWbNmjU644wzJElr1671akEAAmNJsD956uf3xH5I7owothaCivYd06j+3bXhtJ4aif4xwNM4JgEIMKG+07Inf/5dh6p0ybMft3m/cPZo9UuJ7XCtbTn1eIf46HBFdAnTQ//3z3aPNgg0oR6uETg8ekzC6Y4eParf//732rZtmyRp4MCBmjZtmnM0B0DHeHun5UB/k/L0z++v/ZBamxILtv6xUA/XCE5uh5p169bpqquuUkJCgoYOHSpJeuGFF/TrX/9af/7zn5WTk+PxIoFQ4c2dloPhTcrTP38g7YcUTP1jHGOCYOW6g60VM2fO1HXXXafdu3frrbfe0ltvvaVvvvlG119/vWbOnOmNGoGQ4a2RhfbepGzV9g49r6d5+udvatxtDf0sbTMSLoFA5HaoKS0t1T333KMuXbo4r3Xp0kWzZ89WaWnrezEAMMZbIwvB8iblzs9vq7Zr16EqFe09ql2Hq1oNZk2Nu6cHm5xMq57KzZKkdp/DKCP1BAuOMUGwcnv66fzzz9e2bds0YMCAZte3bdum7OxsjxUGhCJv7bQcLG9SRn9+d6bS2lq9dNzeoDtWFnlkOi4YpvbcEUjTdoA7DI3UFBcXOz/uuusuzZo1SwsXLtSGDRu0YcMGLVy4UHfffbfuvvtub9cLmJqrkYXO7LTckTcpf4w8GPn5OzKVlhAToX4psRrUO8m52slT03EdqSfQR3WYtkOwMrSkOywsTBaLRe09NNA332NJN4LFqUuC46LCFRvVVcdr62U70bFVS7Zqu+5cWdTmCMjpjZ/+Hnk4/ec/daWQJ5Zpe3Kpt7vP5e/frVFlx060uelgIC9Dhzl5dEn37t27PVYYgPadulKm7NgJ3fvmV516E3RnZ9xAWPniaqWQJ6bSPDkd585zufrdPlBQrEeuHKguYZaAWGrPMSYtBfqWCDAYatLT071dB4BWeDJgGH2T8uayck/wRL+HJ3tG3HkuV7/b9SXl2vd9taYv/zxgRm6CaRm6twXLCFuoc7tReMWKFS7v33zzzR0uBkBzng4YRt6kAr2p2BPN1J5syHbnudr73TYdfsl+MIElEEYvYYzboWbWrFnNPq+rq1N1dbUiIiIUExNDqAE8yB8BI9BXvrh7yKS3nqMjz9Xe7/bUwy8DYVQMJwX66CV+4HaoOXr0aItrJSUl+sUvfqH77rvPI0UBOMkfAcNby8o9yRP9Hp7sGTH6XK5+tyP7d1fRvmPNrvl7VAwnBfroJX7g9uZ7rcnMzNSCBQtajOIA6Bx/LK311rJyTzt9mXZH6vLEc7jzXG39bkf2765pIzO0dEPzRRn+HhXDSYE+eokfdOhAy1afqGtXlZWVeerpAMiz0yTuYOWL95z6u7WdqFNNXYM++eaI7lpZpGr7D1tiBMqoGIJj9BInuR1q3n333WafOxwOHThwQIsXL9bIkSM9VhiAk/wVMFj54j2nL9l/+eNdLQJNII2KhTp//XEB9xnafO9UYWHNZ6wsFouSk5M1duxYPfPMM+rZs6dHC/QkNt8DEIhcbTaIwMHr5D8e3XzvVI2NjZ0qDADQHKNiwYHXKfB5rKcGAIIBu8IC5mUo1MyePdvwEz777LMdLgYAvCnQdoUlYAGeZSjUFBUVGXoyi8XSqWIAwNOagkODw6F5f/5a60uPNLvvr11hAy1gAWZgKNSsXbvW23UAgMedGhyWTv2vFoGmia93hWXbfcA7OrX53r///W/9+9//9lQtAOAxpwaHmIguignv4vLxvtwV1si2+wDc53aoaWxs1K9//WslJCQoPT1d6enpSkxM1Lx581gZBcDnbNV27TpUpaK9R7XrcJVs1ScDwanB4ZZRGWpoZ/cKX+4Ky7b7gHe4vfrp4Ycf1u9//3stWLDAudnehg0bNHfuXNXU1OiJJ57weJEA0BpXfSlVtT8Eg8Fpidr0zRGN7N9dG1uZgrrIx7vCsu0+4B1uj9QsX75cr776qn7xi18oKytLWVlZuv322/XKK6/otdde80KJANBSe30pidE/hJTa+kYt3bBb00ZmaGT/7s0eP7J/dz129Y992sPijzO9gFDg9kjN999/r7PPPrvF9bPPPlvff/+9R4oCgPa015dib2h0ntcT2TVM1fYG3bWySLeMytAtIzNUW9+oyK5hLU7G9gW23Qe8w+1Qk52drcWLF+uFF15odn3x4sXKzs72WGEA4Ep7fSnHa+udwaFo3zHn1NPiNaXNHpeTadWMURneLLVVHBoKeJ7boeapp57SFVdcoY8++kgjRoyQJG3atEn79u3TBx984PECAaA1RvpSmoLDkeN2/XTwWZr77tct+m/8OTLCtvuAZ7kdakaPHq2dO3fqxRdf1Pbt2yVJEydO1O23367U1FSPFwgArWnqS1nXyhTUqX0ppwaHxYyMAKbm9indwYxTugFzKTt2os2+lJ7syguYhsdP6S4vL9fx48eVnp7uvPb1119r4cKFOn78uCZMmKAbbrihc1UDgBtCtS+FM6OA1hkONXfeeadSU1P1zDPPSJIOHTqkiy66SKmpqerXr5+mTp2qhoYG3XTTTV4rFgBOF2p9KZwZBbTN8D41n376qa6++mrn5ytWrNAZZ5yhL7/8Uu+8846efPJJvfjii14pEgDQ/t48TbspA6HKcKg5ePCg+vTp4/x8zZo1mjhxorp2PTnYc/XVV6ukpMTjBQIATuLMKMA1w6EmPj5ex44dc36+efNmDR8+3Pm5xWJRbW2tR4sD4DttnaGEwMGZUYBrhntqLrjgAr3wwgt65ZVX9NZbb6myslJjx4513t+5c6fS0tK8UiQA76JPIzhwZhTgmuGRmnnz5undd99VdHS0rrvuOt1///1KSkpy3n/99dc1evRorxQJwHt83afBiFDHcWYU4JrhkZqsrCxt27ZNGzduVI8ePZpNPUnS9ddfr4EDB3q8QADeZaRPw1OrixgR6hzOjAJcY/M9IMQV7T2qn770SZv33779Qg3qndTmfaNs1XbdsbKo1QCVk2nVosmDeVM2qGmfmlDamwehzeOb7wEwJ1/1afhyRMjsQm1vHsAowz01AMzJV30arNwB4G2EGiDENfVpnB5sPN2nwcodAN7G9BMAn5yhZPRUbQDoKEOhpqKiwvAT0oALBCdv92mwcgeAtxkKNYmJibJYLIaesKGhoVMFAQhcRk+HbutxoXqqNgDfMBRq1q5d6/zfe/bsUV5enqZOnaoRI0ZIkjZt2qTly5dr/vz53qkSgN8Z3WOmvcexcgeAt7i9T80ll1yin//855o8eXKz63/84x/1u9/9Tn//+989WZ9HsU8N0DFG95hhLxoA3mD0/dvt1U+bNm3S0KFDW1wfOnSoNm/e7O7Tua22tlaDBg2SxWLRl19+6fXvB8D46dCcIg3An9wONWlpaXrllVdaXH/11Vd9cqDl/fffr9TUVK9/HwA/MLrHDHvRAPAnt5d0/+Y3v1Fubq7+8pe/OM9/2rx5s0pKSlRQUODxAk/1l7/8RX/7299UUFCgv/zlL179XgB+cOoeMzERXXTLqAwNTktUbX2josK7KOk/U0rsRQPAn9wONZdffrl27typ3/72t9q+fbsk6aqrrtJtt93m1ZGa7777TjNmzNDbb7+tmJgYQ19TW1ur2tpa5+fuLE0H8IOmPWY+//aoXpg8WMs27tbiNaXO+02NwOxFA8CfguJAS4fDocsvv1wjR47Ur371K+3Zs0cZGRkqKirSoEGD2vy6uXPn6rHHHmtxnUZhwH1lx07o452H9V5xmTaWHmlxv6kR+Li9oc29aHpyEjeADjDaKNyhULN+/XotWbJE33zzjd58802dddZZ+sMf/qCMjAyNGjXK8PPk5eUpPz/f5WO2bdumv/3tb3rjjTf08ccfq0uXLoZDTWsjNWlpaYQaoINKvqvUf/9mXZv3C2ePVr+UWE6RBuBRXjulu6CgQDfddJNuvPFGbdmyxRkabDabnnzySX3wwQeGn+uee+7R1KlTXT6mb9++WrNmjTZt2qTIyMhm94YOHaobb7xRy5cvb/VrIyMjW3wNgI6rqq13eb+pEZi9aAD4g9uh5vHHH9fLL7+sm2++Wa+//rrz+siRI/X444+79VzJyclKTk5u93EvvPBCs+cuKyvTZZddplWrVjmblQF4H43AAAKZ26Fmx44dysnJaXE9ISFBx44d80RNLfTu3bvZ57GxsZKkfv36qVevXl75ngBaohEYQCBze5+aHj16qLS0tMX1DRs2qG/fvh4pCkBgajqUMifT2uw6h1ICCARuj9TMmDFDs2bN0tKlS2WxWFRWVqZNmzbp3nvv1SOPPOKNGlvo06ePgmDRFmBKHEoJIFC5HWry8vLU2NioSy65RNXV1crJyVFkZKTuvfde3Xnnnd6oEUCAoREYQCDq8D41drtdpaWlqqqq0sCBA519LoGMAy0BAAg+XjvQ8pZbblFlZaUiIiI0cOBADRs2TLGxsTp+/LhuueWWThUNAADQUW6HmuXLl+vEiRMtrp84cUIrVqzwSFEAAADuMtxTU1FRIYfDIYfDocrKSkVFRTnvNTQ06IMPPlBKSopXigQAAGiP4VCTmJgoi8Uii8WiH/3oRy3uWyyWVs9ZAgAA8AXDoWbt2rVyOBwaO3asCgoKdMYZZzjvRUREKD09XampqV4pEgAAoD2GQ83o0aMlSbt371bv3r1lsVi8VhQAwD1Nh4hW1NQpPjpc1m4su0focXufmjVr1ig2NlaTJk1qdv3NN99UdXW1pkyZ4rHiAHgXb4TmUHbshB4oKNb6U46vyMm0akFullITo/1YGeBbbq9+mj9/vqxWa4vrKSkpevLJJz1SFBDKbNV27TpUpaK9R7XrcJVs1XavfJ+yYyd0x8oiXfLsx/rpS5/okmc+1p0ri1R2rOXqRgQuW7W9RaCRpHUl5corKPbavx8gELk9UrN3715lZGS0uJ6enq69e/d6pCggVPnqL+723ggXTR7MiE2QKK+yt3gdm6wrKVd5lZ3XEiHD7ZGalJQUFRcXt7j+1VdfqXv37h4pCghFnfmL293RHSNvhAgOFTV1Lu9XtnMfMBO3R2omT56su+66S3FxccrJyZEkffzxx5o1a5auv/56jxcIhIqO/sXdkdEd3gjNIz4q3OX9uHbuA2bi9kjNvHnzNHz4cF1yySWKjo5WdHS0Lr30Uo0dO5aeGqATOhI0Ojq6wxuheVhjI5ST2bLPUToZbq2xTD0hdLgdaiIiIrRq1Spt375d//u//6u33npLu3bt0tKlSxURwX88QEd1JGh0dBqJN0LzSIiJ0ILcrBavZ06mVfm5WfTTIKS4Pf3U5Ec/+lGrOwsD6JimoLGulZDSVtDo6DRS0xthXkFxs+/HG2FwSk2M1qLJg1VeZVdlTZ3iosJljWV5PkKPoVAze/ZszZs3T926ddPs2bNdPvbZZ5/1SGFAqOlI0OjMNBJvhOaSEMNrBxgKNUVFRaqrq3P+77awyzDQOe4GjY6M7pyKN0IAZmJxOBwOfxfhKxUVFUpISJDNZlN8fLy/ywE8ouzYiTZHd3qymywAEzD6/t3hnhoAgYFpJAA4yVComThxouEnfOuttzpcDICOYRoJAAwu6U5ISHB+xMfHq7CwUJ9//rnz/hdffKHCwkIlJCR4rVAAAABXDI3ULFu2zPm/H3jgAV177bV6+eWX1aVLF0lSQ0ODbr/9dvpUAACA37jdKJycnKwNGzZowIABza7v2LFDF154oY4cOeLRAj2JRmEAAIKP0fdvt3cUrq+v1/bt21tc3759uxobG919OgAAAI9we/XTtGnTNH36dO3atUvDhg2TJP3jH//QggULNG3aNI8XCAAAYITboWbhwoXq0aOHnnnmGR04cECS1LNnT91333265557PF4gEGps1XaVV9lVUVOn+OhwWbuxsgkAjOjU5nsVFRWSFDT9KfTUINCVHTvR4tTtnEyrFuRmKZWN9NpFIATMyaub79XX1+vvf/+7du3apRtuuEGSVFZWpvj4eMXGxnasYiDE2artLQKNdPK07byCYi2aPJg3aBcIhADcbhT+9ttvdd555+maa67RzJkzdfjwYUlSfn6+7r33Xo8XCISK8ip7i0DTZF1Jucqr7D6uKHi0Fwht1fzugFDgdqiZNWuWhg4dqqNHjyo6+oe/fn7605+qsLDQo8UBoaSips7l/cp27ocyAiEAqQPTT+vXr9cnn3yiiIjmw+B9+vTR/v37PVYYEGrio8Jd3o9r534oIxACkDowUtPY2KiGhoYW1//9738rLi7OI0UBocgaG6GcTGur93IyrbLG0k/TFgIhAKkDoebSSy/Vc8895/zcYrGoqqpKc+bM0eWXX+7J2oCQkhAToQW5WS2CTU6mVfm5WTQJu0AgBCB1YEn3vn37NH78eDkcDpWUlGjo0KEqKSmR1WrVunXrlJKS4q1aO40l3QgGTcuSK2vqFBcVLmssy5KNKDt2QnkFxVp32uqn/Nws9WT1ExDUjL5/d2ifmvr6eq1atUpfffWVqqqqdP755+vGG29s1jgciAg1gLkRCAFz8kqoqaur09lnn6333ntP55xzjkcK9SVCDQAAwccrB1qGh4erpqam08UBAAB4mtuNwjNnzlR+fr7q6+u9UQ+ATrJV27XrUJWK9h7VrsNVbDwHIGS4vU/NZ599psLCQv3tb3/Teeedp27dujW7/9Zbb3msOADu4agAAKHM7VCTmJio3Nxcb9QCoBM4OwpAqHM71CxbtswbdQDoJCNHBRBqAJiZ4Z6axsZG5efna+TIkfqv//ov5eXl6cSJE96sDQhK/upp4agAAKHO8EjNE088oblz52rcuHGKjo7W888/r0OHDmnp0qXerA8IKt7qaWnaf6Wipk7x0eGydmu5/wpHBQAIdYZDzYoVK/TSSy/p1ltvlSR99NFHuuKKK/Tqq68qLMztRVSA6Xirp8VoUGo6KmBdK1NQHBUAIBQYTiN79+5tdrbTuHHjZLFYVFZW5pXCgGBjpKfFXe0FpVOntjg7CkCoMzxSU19fr6ioqGbXwsPDVVfHPD0geaenxd3m39TEaC2aPJijAgCEJMOhxuFwaOrUqYqMjHReq6mp0W233dZsrxr2qUGo8kZPS0eCUkIMIQZAaDIcaqZMmdLi2s9+9jOPFgMEM2/0tND8CwDGGQ417E8DuNbU05JXUNws2HSmp4XmXwAwzq1TuoMdp3TDF5qWX3uqp6Xs2Ik2g1JPjj4AEAKMvn+7vaMwANc83dNC8y8AGEOoAYIAzb8A0D52zQMAAKZAqAEAAKZAqAEAAKZAqAEAAKZAqAEAAKZAqAEAAKZAqAEAAKZAqAEAAKZAqAEAAKZAqAEAAKZAqAEAAKZAqAEAAKZAqAEAAKZAqAEAAKZAqAEAAKZAqAEAAKZAqAEAAKYQVKHm/fff1/DhwxUdHa2kpCRNmDDB3yUBAIAA0dXfBRhVUFCgGTNm6Mknn9TYsWNVX1+vrVu3+rssAAAQIIIi1NTX12vWrFl6+umnNX36dOf1gQMHuvy62tpa1dbWOj+vqKjwWo0AAMC/gmL6acuWLdq/f7/CwsI0ePBg9ezZUz/5yU/aHamZP3++EhISnB9paWk+qhgAAPhaUISab775RpI0d+5c/epXv9J7772npKQkjRkzRt9//32bX/fggw/KZrM5P/bt2+erknEaW7Vduw5VqWjvUe06XCVbtd3fJQEATMav0095eXnKz893+Zht27apsbFRkvTwww8rNzdXkrRs2TL16tVLb775pm699dZWvzYyMlKRkZGeLRpuKzt2Qg8UFGt9SbnzWk6mVQtys5SaGO3HygAAZuLXUHPPPfdo6tSpLh/Tt29fHThwQFLzHprIyEj17dtXe/fu9WaJ6CRbtb1FoJGkdSXlyiso1qLJg5UQE+Gn6gAAZuLXUJOcnKzk5OR2HzdkyBBFRkZqx44dGjVqlCSprq5Oe/bsUXp6urfLRCeUV9lbBJom60rKVV5lJ9QAADwiKFY/xcfH67bbbtOcOXOUlpam9PR0Pf3005KkSZMm+bk6uFJRU+fyfmU79wEAMCooQo0kPf300+ratatuuukmnThxQsOHD9eaNWuUlJTk79LgQnxUuMv7ce3cBwDAqKAJNeHh4Vq4cKEWLlzo71LgBmtshHIyrVrXyhRUTqZV1ljfTz3Zqu0qr7KroqZO8dHhsnaLYAoMAEwgaEINglNCTIQW5GYpr6C4WbDJybQqPzfL52GClVgAYF4Wh8Ph8HcRvlJRUaGEhATZbDbFx8f7u5yQ0jQ6UllTp7iocFljfT86Yqu2646VRa02LudkWlmJBQAByuj7NyM18ImEGP9P8bASCwDMLSh2FAY8gZVYAGBuhBqEDFZiAYC5EWoQMppWYrXGXyuxAACeQ6hByGhaiXV6sPHXSiwAgGfRKIyQkpoYrUWTB/t9JRYAwPMINQg5gbASCwDgeUw/AQAAUyDUAAAAUyDUAAAAUyDUAAAAUyDUAAAAUyDUAAAAUyDUAAAAUyDUAAAAUyDUAAAAUyDUAAAAUyDUAAAAUyDUAAAAUyDUAAAAUyDUAAAAUyDUAAAAUyDUAAAAUyDUAAAAUyDUAAAAUyDUAAAAUyDUAAAAUyDUAAAAUyDUAAAAUyDUAAAAUyDUAAAAUyDUAAAAUyDUAAAAUyDUAAAAUyDUAAAAUyDUAAAAUyDUAAAAUyDUAAAAUyDUAAAAUyDUAAAAUyDUAAAAUyDUAAAAUyDUAAAAUyDUAAAAUyDUAAAAUyDUAAAAUyDUAAAAUyDUAAAAUyDUAAAAUyDUAAAAUyDUAAAAUyDUAAAAUyDUAAAAUyDUAAAAUyDUAAAAU+jq7wIQHGzVdpVX2VVRU6f46HBZu0UoISbC32UBAOBEqEG7yo6d0AMFxVpfUu68lpNp1YLcLKUmRvuxMgAAfsD0E1yyVdtbBBpJWldSrryCYtmq7X6qDACA5gg1cKm8yt4i0DRZV1Ku8ipCDQAgMBBq4FJFTZ3L+5Xt3AcAwFcINXApPirc5f24du4DAOArhBq4ZI2NUE6mtdV7OZlWWWNZAQUACAyEmk6yVdu161CVivYe1a7DVaZrnE2IidCC3KwWwSYn06r83CyWdQMAAgZLujshVJY6pyZGa9HkwSqvsquypk5xUeGyxrJPDQAgsDBS00GhttQ5ISZC/VJiNah3kvqlxBJoAAABh1DTQSx1BgAgsBBqOoilzgAABJagCTU7d+7UNddcI6vVqvj4eI0aNUpr1671Wz0sdQYAILAETai58sorVV9frzVr1uiLL75Qdna2rrzySh08eNAv9bDUGQCAwBIUoaa8vFwlJSXKy8tTVlaWMjMztWDBAlVXV2vr1q1+qYmlzgAABJagWNLdvXt3DRgwQCtWrND555+vyMhILVmyRCkpKRoyZEibX1dbW6va2lrn5xUVFR6ti6XOAAAEjqAINRaLRR999JEmTJiguLg4hYWFKSUlRatXr1ZSUlKbXzd//nw99thjXq0tIYYQAwBAIPDr9FNeXp4sFovLj+3bt8vhcGjmzJlKSUnR+vXrtXnzZk2YMEFXXXWVDhw40ObzP/jgg7LZbM6Pffv2+fCnAwAAvmRxOBwOf33zw4cP68iRIy4f07dvX61fv16XXnqpjh49qvj4eOe9zMxMTZ8+XXl5eYa+X0VFhRISEmSz2Zo9DwAACFxG37/9Ov2UnJys5OTkdh9XXV0tSQoLaz6wFBYWpsbGRq/UBgAAgktQrH4aMWKEkpKSNGXKFH311VfauXOn7rvvPu3evVtXXHGFv8sDAAABIChCjdVq1erVq1VVVaWxY8dq6NCh2rBhg9555x1lZ2f7uzwAABAA/NpT42v01AAAEHyMvn8HxUgNAABAewg1AADAFAg1AADAFAg1AADAFILimARPaeqJ9vQZUAAAwHua3rfbW9sUUqGmsrJSkpSWlubnSgAAgLsqKyuVkJDQ5v2QWtLd2NiosrIyxcXFyWKxtPv4iooKpaWlad++fSwBDyC8LoGL1yYw8boEJl4X4xwOhyorK5WamtridIFThdRITVhYmHr16uX218XHx/MPLgDxugQuXpvAxOsSmHhdjHE1QtOERmEAAGAKhBoAAGAKhBoXIiMjNWfOHEVGRvq7FJyC1yVw8doEJl6XwMTr4nkh1SgMAADMi5EaAABgCoQaAABgCoQaAABgCoQaAABgCoQag3bu3KlrrrlGVqtV8fHxGjVqlNauXevvsvAf77//voYPH67o6GglJSVpwoQJ/i4J/1FbW6tBgwbJYrHoyy+/9Hc5IW3Pnj2aPn26MjIyFB0drX79+mnOnDmy2+3+Li0kvfjii+rTp4+ioqI0fPhwbd682d8lBT1CjUFXXnml6uvrtWbNGn3xxRfKzs7WlVdeqYMHD/q7tJBXUFCgm266SdOmTdNXX32ljRs36oYbbvB3WfiP+++/X6mpqf4uA5K2b9+uxsZGLVmyRF9//bV+85vf6OWXX9ZDDz3k79JCzqpVqzR79mzNmTNHW7ZsUXZ2ti677DIdOnTI36UFNwfadfjwYYckx7p165zXKioqHJIcH374oR8rQ11dneOss85yvPrqq/4uBa344IMPHGeffbbj66+/dkhyFBUV+bsknOapp55yZGRk+LuMkDNs2DDHzJkznZ83NDQ4UlNTHfPnz/djVcGPkRoDunfvrgEDBmjFihU6fvy46uvrtWTJEqWkpGjIkCH+Li+kbdmyRfv371dYWJgGDx6snj176ic/+Ym2bt3q79JC3nfffacZM2boD3/4g2JiYvxdDtpgs9l0xhln+LuMkGK32/XFF19o3LhxzmthYWEaN26cNm3a5MfKgh+hxgCLxaKPPvpIRUVFiouLU1RUlJ599lmtXr1aSUlJ/i4vpH3zzTeSpLlz5+pXv/qV3nvvPSUlJWnMmDH6/vvv/Vxd6HI4HJo6dapuu+02DR061N/loA2lpaVatGiRbr31Vn+XElLKy8vV0NCgM888s9n1M888k5aGTgrpUJOXlyeLxeLyY/v27XI4HJo5c6ZSUlK0fv16bd68WRMmTNBVV12lAwcO+PvHMCWjr01jY6Mk6eGHH1Zubq6GDBmiZcuWyWKx6M033/TzT2E+Rl+XRYsWqbKyUg8++KC/Sw4JRl+XU+3fv1/jx4/XpEmTNGPGDD9VDnhWSB+TcPjwYR05csTlY/r27av169fr0ksv1dGjR5sdD5+Zmanp06crLy/P26WGHKOvzcaNGzV27FitX79eo0aNct4bPny4xo0bpyeeeMLbpYYUo6/Ltddeqz//+c+yWCzO6w0NDerSpYtuvPFGLV++3NulhhSjr0tERIQkqaysTGPGjNEFF1yg1157TWFhIf33rc/Z7XbFxMToT3/6U7OVmlOmTNGxY8f0zjvv+K+4INfV3wX4U3JyspKTk9t9XHV1tSS1+A8/LCzMOVIAzzL62gwZMkSRkZHasWOHM9TU1dVpz549Sk9P93aZIcfo6/LCCy/o8ccfd35eVlamyy67TKtWrdLw4cO9WWJIMvq6SCdHaC6++GLnqCaBxvciIiI0ZMgQFRYWOkNNY2OjCgsLdccdd/i3uCAX0qHGqBEjRigpKUlTpkzRo48+qujoaL3yyivavXu3rrjiCn+XF9Li4+N12223ac6cOUpLS1N6erqefvppSdKkSZP8XF3o6t27d7PPY2NjJUn9+vVTr169/FESdDLQjBkzRunp6Vq4cKEOHz7svNejRw8/VhZ6Zs+erSlTpmjo0KEaNmyYnnvuOR0/flzTpk3zd2lBjVBjgNVq1erVq/Xwww9r7Nixqqur049//GO98847ys7O9nd5Ie/pp59W165dddNNN+nEiRMaPny41qxZQxM3cJoPP/xQpaWlKi0tbREuQ7gTwS+uu+46HT58WI8++qgOHjyoQYMGafXq1S2ah+GekO6pAQAA5sFkKgAAMAVCDQAAMAVCDQAAMAVCDQAAMAVCDQAAMAVCDQAAMAVCDQAAMAVCDQAAMAVCDQCfmzp1arOD/Lxhz549slgs+vLLLw1/zdy5czVo0CCXjzm99jFjxuiXv/yl8/M+ffroueeec6tWAJ5BqAHgNHXqVFksFlksFoWHhysjI0P333+/ampqfF7LK6+8ouzsbMXGxioxMVGDBw/W/PnzfV7H6Z5//nm99tprbd7/7LPP9P/+3/9zfm6xWPT22297vzAAnP0EoLnx48dr2bJlqqur0xdffKEpU6bIYrEoPz/fZzUsXbpUv/zlL/XCCy9o9OjRqq2tVXFxsbZu3eqzGtqSkJDg8r7R07IBeB4jNQCaiYyMVI8ePZSWlqYJEyZo3Lhx+vDDD533GxsbNX/+fGVkZCg6OlrZ2dn605/+5Lzf0NCg6dOnO+8PGDBAzz//vFs1vPvuu7r22ms1ffp09e/fXz/+8Y81efJkPfHEE83q+PWvf61evXopMjLSeSBgW1577TUlJiY2u/b222/LYrG0eOySJUuUlpammJgYXXvttbLZbM577U2dnTr91KdPH0nST3/6U1ksFvXp00d79uxRWFiYPv/882Zf99xzzyk9PV2NjY1tPjcA1wg1ANq0detWffLJJ4qIiHBemz9/vlasWKGXX35ZX3/9te6++2797Gc/08cffyzpZNjo1auX3nzzTf3rX//So48+qoceekhvvPGG4e/bo0cPffrpp/r222/bfMzzzz+vZ555RgsXLlRxcbEuu+wyXX311SopKen4DyyptLRUb7zxhv785z9r9erVKioq0u23396h5/rss88kScuWLdOBAwf02WefqU+fPho3bpyWLVvW7LHLli3T1KlTFRbG/y0DHcV/PQCaee+99xQbG6uoqCidd955OnTokO677z5JUm1trZ588kktXbpUl112mfr27aupU6fqZz/7mZYsWSJJCg8P12OPPaahQ4cqIyNDN954o6ZNm+ZWqJkzZ44SExPVp08fDRgwQFOnTtUbb7zRbBRj4cKFeuCBB3T99ddrwIABys/P16BBgzrdpFtTU6MVK1Zo0KBBysnJ0aJFi/T666/r4MGDbj9X01RUYmKievTo4fz85z//uVauXKna2lpJ0pYtW/TPf/5T06ZN61TtQKgj1ABo5uKLL9aXX36pf/zjH5oyZYqmTZum3NxcSSdHMaqrq/Xf//3fio2NdX6sWLFCu3btcj7Hiy++qCFDhig5OVmxsbH63e9+p7179xquoWfPntq0aZP++c9/atasWaqvr9eUKVM0fvx4NTY2qqKiQmVlZRo5cmSzrxs5cqS2bdvWqZ+/d+/eOuuss5yfjxgxQo2NjdqxY0ennvdUEyZMUJcuXfR///d/kk5OjV188cXO6SoAHUOjMIBmunXrpv79+0s62bCbnZ2t3//+95o+fbqqqqokSe+//36zN37pZC+OJL3++uu699579cwzz2jEiBGKi4vT008/rX/84x9u13Luuefq3HPP1e23367bbrtNF110kT7++GMNGTLE7ecKCwuTw+Fodq2urs7t5/GEiIgI3XzzzVq2bJkmTpyoP/7xj273HQFoiVADoE1hYWF66KGHNHv2bN1www0aOHCgIiMjtXfvXo0ePbrVr9m4caMuvPDCZn0op47idNTAgQMlScePH1d8fLxSU1O1cePGZnVs3LhRw4YNa/Xrk5OTVVlZqePHj6tbt26S1OoeNnv37lVZWZlSU1MlSZ9++qnCwsI0YMCADtUdHh6uhoaGFtd//vOf69xzz9VLL72k+vp6TZw4sUPPD+AHTD8BcGnSpEnq0qWLXnzxRcXFxenee+/V3XffreXLl2vXrl3asmWLFi1apOXLl0uSMjMz9fnnn+uvf/2rdu7cqUceecTZMGvUL37xC82bN08bN27Ut99+q08//VQ333yzkpOTNWLECEnSfffdp/z8fK1atUo7duxQXl6evvzyS82aNavV5xw+fLhiYmL00EMPadeuXfrjH//Y6n4zUVFRmjJlir766iutX79ed911l6699lr16NHDvV/cf/Tp00eFhYU6ePCgjh496rx+zjnn6IILLtADDzygyZMnKzo6ukPPD+AHhBoALnXt2lV33HGHnnrqKR0/flzz5s3TI488ovnz5+ucc87R+PHj9f777ysjI0OSdOutt2rixIm67rrrNHz4cB05csTt1UPjxo3Tp59+qkmTJulHP/qRcnNzFRUVpcLCQnXv3l2SdNddd2n27Nm65557dN5552n16tV69913lZmZ2epznnHGGfqf//kfffDBBzrvvPO0cuVKzZ07t8Xj+vfvr4kTJ+ryyy/XpZdeqqysLL300kvu/dJO8cwzz+jDDz9UWlqaBg8e3Oze9OnTZbfbdcstt3T4+QH8wOI4fZIZAOAT8+bN05tvvqni4mJ/lwKYAiM1AOBjVVVV2rp1qxYvXqw777zT3+UApkGoAQAfu+OOOzRkyBCNGTOGqSfAg5h+AgAApsBIDQAAMAVCDQAAMAVCDQAAMAVCDQAAMAVCDQAAMAVCDQAAMAVCDQAAMAVCDQAAMIX/D3RkRJEHFMIrAAAAAElFTkSuQmCC",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "import pandas as pd \n",
    "\n",
    "def predict(model, data): \n",
    "  data_size = len(data)\n",
    "  test_loader = DataLoader(\n",
    "    data[int(data_size * 0.8):], \n",
    "    batch_size=NUM_GRAPHS_PER_BATCH, \n",
    "    shuffle=True\n",
    "  )\n",
    "\n",
    "  # Analyze the results for one batch\n",
    "  test_batch = next(iter(test_loader))\n",
    "\n",
    "  with torch.no_grad():\n",
    "    test_batch.to(DEVICE)\n",
    "    pred, embed = model(test_batch.x.float(), test_batch.edge_index, test_batch.batch) \n",
    "    df = pd.DataFrame()\n",
    "    df[\"y_real\"] = test_batch.y.tolist()\n",
    "    df[\"y_pred\"] = pred.tolist()\n",
    "\n",
    "  df[\"y_real\"] = df[\"y_real\"].apply(lambda row: row[0])\n",
    "  df[\"y_pred\"] = df[\"y_pred\"].apply(lambda row: row[0])\n",
    "\n",
    "  axes = sns.scatterplot(data=df, x=\"y_real\", y=\"y_pred\")\n",
    "  axes.set_xlabel(\"Real Solubility\")\n",
    "  axes.set_ylabel(\"Predicted Solubility\")\n",
    "\n",
    "predict(MODEL, DATA)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "gnn",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.4"
  },
  "orig_nbformat": 4
 },
 "nbformat": 4,
 "nbformat_minor": 2
}