notebook.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "fc82a8e4",
   "metadata": {},
   "source": [
    "# How to use our wav2vec 2.0 model for speech emotion recognition"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "23f9dd5c",
   "metadata": {},
   "source": [
    "In the following we present a hands-on to our model for dimensional emotional emotion recognition based on wav2vec 2.0. The model is publicly available for non-commercial usage from https://doi.org/10.5281/zenodo.6221127."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "313f2e2f",
   "metadata": {
    "hide_input": false
   },
   "source": [
    "## Load model"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "af91aa2a",
   "metadata": {},
   "source": [
    "We start by downloading and unpacking the model. This will get us two files, a binary ONNX file containing the model weights and a YAML file with meta information about the model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "5fa18d66",
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "import os\n",
    "\n",
    "import audeer\n",
    "\n",
    "\n",
    "model_root = 'model'\n",
    "cache_root = 'cache'\n",
    "\n",
    "\n",
    "audeer.mkdir(cache_root)\n",
    "def cache_path(file):\n",
    "    return os.path.join(cache_root, file)\n",
    "\n",
    "\n",
    "url = 'https://zenodo.org/record/6221127/files/w2v2-L-robust-12.6bc4a7fd-1.1.0.zip'\n",
    "dst_path = cache_path('model.zip')\n",
    "\n",
    "if not os.path.exists(dst_path):\n",
    "    audeer.download_url(\n",
    "        url, \n",
    "        dst_path, \n",
    "        verbose=True,\n",
    "    )\n",
    "    \n",
    "if not os.path.exists(model_root):\n",
    "    audeer.extract_archive(\n",
    "        dst_path, \n",
    "        model_root, \n",
    "        verbose=True,\n",
    "    )"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4b5c7928",
   "metadata": {},
   "source": [
    "The package [audonnx](https://github.com/audeering/audonnx) helps us to load the model. Printing the model lists the input and output nodes. Since the model operates on the raw audio stream, we have a single input node called 'signal', which expects a mono signal with a sampling rate of 16000Hz. We also see that the model has two output nodes: 'hidden_states', which gives us access to the pooled states of the last transformer layer and 'logits', which provides scores for arousal, dominance and valence in a range of approximately 0...1."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "f0b1a074",
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Input:\n",
       "  signal:\n",
       "    shape: [1, -1]\n",
       "    dtype: tensor(float)\n",
       "    transform: None\n",
       "Output:\n",
       "  hidden_states:\n",
       "    shape: [1, 1024]\n",
       "    dtype: tensor(float)\n",
       "    labels: [hidden_states-0, hidden_states-1, hidden_states-2, (...), hidden_states-1021,\n",
       "      hidden_states-1022, hidden_states-1023]\n",
       "  logits:\n",
       "    shape: [1, 3]\n",
       "    dtype: tensor(float)\n",
       "    labels: [arousal, dominance, valence]"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import audonnx\n",
    "\n",
    "\n",
    "model = audonnx.load(model_root)\n",
    "model"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "573cf22f",
   "metadata": {},
   "source": [
    "As a test, we call the model with some white noise. Note that we have to force the data type of the signal to 32-bit floating point precision. As result we get a dictionary with predictions for every output node."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "9ef10804",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'hidden_states': array([[-0.00723587,  0.00645568, -0.00865646, ...,  0.00670633,\n",
       "          0.00948439,  0.0028789 ]], dtype=float32),\n",
       " 'logits': array([[0.6563389, 0.630771 , 0.4762332]], dtype=float32)}"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import numpy as np\n",
    "\n",
    "\n",
    "np.random.seed(0)\n",
    "\n",
    "sampling_rate = 16000\n",
    "signal = np.random.normal(\n",
    "    size=sampling_rate,\n",
    ").astype(np.float32)\n",
    "\n",
    "model(signal, sampling_rate)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c016311c",
   "metadata": {},
   "source": [
    "## Predict arousal, dominance and valence"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f17622eb",
   "metadata": {},
   "source": [
    "A more advanced way of interfacing the model is offered by [audinterface](https://github.com/audeering/audinterface). Especially, the class [Feature](https://audeering.github.io/audinterface/api.html#feature) comes in handy, as it has the option to assign names to the output dimensions. To create the interface, we simply pass our callable model object as processing function. Since we are only interested in the scores for arousal, dominance and valence, we pass 'logits' as an additional key word argument to `output_names`. And we enable automatic resampling in case the expected sampling rate of the model is not matched."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "877be883",
   "metadata": {},
   "outputs": [],
   "source": [
    "import audinterface\n",
    "\n",
    "\n",
    "interface = audinterface.Feature(\n",
    "    model.labels('logits'),\n",
    "    process_func=model,\n",
    "    process_func_args={\n",
    "        'outputs': 'logits',\n",
    "    },\n",
    "    sampling_rate=sampling_rate,\n",
    "    resample=True,    \n",
    "    verbose=True,\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9ead873c",
   "metadata": {},
   "source": [
    "When we pass the signal to the interface, we get as result a table with proper column labels."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "6f001a87",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th>arousal</th>\n",
       "      <th>dominance</th>\n",
       "      <th>valence</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>start</th>\n",
       "      <th>end</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0 days</th>\n",
       "      <th>0 days 00:00:01</th>\n",
       "      <td>0.656339</td>\n",
       "      <td>0.630771</td>\n",
       "      <td>0.476233</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                         arousal  dominance   valence\n",
       "start  end                                           \n",
       "0 days 0 days 00:00:01  0.656339   0.630771  0.476233"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "interface.process_signal(signal, sampling_rate)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d21df09f",
   "metadata": {},
   "source": [
    "## Predict dimensional scores on synthetic files "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "dd18b4c0",
   "metadata": {},
   "source": [
    "In the [paper](https://arxiv.org/abs/2203.07378), we made the observation that transformer-based models - even though not trained on textual transcriptions - can implicitly learn linguistic information. To prove this for the model at hand, we repeat an experiment, where we predict neutral sounding files generated with a text-to-speech engine. First, we let [gTTS](https://pypi.org/project/gTTS/) (Google Text-to-Speech) create some sentences for us. In two of the sentences we include words with a strong positive and negative sentiment ('wonderful' / 'stupid')."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "da964c53",
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "                                                                                                    \r"
     ]
    },
    {
     "data": {
      "text/html": [
       "\n",
       "                <audio  controls=\"controls\" >\n",
       "                    <source src=\"data:audio/mpeg;base64,//NExAAQ8DlUAUMQAHPiIIRwAQTgYPz4gGg+IwsCDn/wQd2FDhcH3n8PiM+CDsHxwIFDn4YKHFg/1h8ufKAg4MF3ggUOfE4Plz6MPwQDBRxN/8bZiVZu3f/hw2I05Sr///NExA8V+ypMAZpoAO4BBSd7Y2kws/xkLJcm/+F3CrgXgwDf/l9NAlx2FD//HuPcpstND//+gTByHjMvpGH///6DJm7qQZNN//////c0W607oIF8vm4EAYiDWZmOQypT//NExAoUMSaEAZtgACVOBxOEAqHhiYgHRoBC3Li6Xb0IBGts0qyqoPyGhO1osnYsbasrd82ggWoxcatsuFxnpnZ+n788u3Z7If5rKN4sfTKBl9hCXdVja1TC1M7oLMGA//NExAwTkM58AdtIANMFYpewzMEbZNJ0yECh2fiFqWP98tqatbnVZ/vimwVZ1CxMOMplVGeJTAHSFooMW0Rqe7lPsKnQdMnUaf////12eqq5D7OGeAg+GwVwP+8KqjvR//NExBAVua6IANGElNleUNBaZiU2o4+IzLu0M8b27GZ21JNYojEKqq6vIxnPYhFCMghnJ5PydE/usgjJnFMqct7y6p/vBAu/ECz/tlwscbU4P0vR5+d94grGCs4eookh//NExAwVeSaoAMzScMzVfADWQ4ql4zN06tSHNOhsED4ZJhAxc1EBQSWjnMRiRf4u3v85vTqCAUGGw8BA0TnNA3nw2UKFx3Z4r47/Z3Leo21VjlUDDiw+ov/HrMRUpvb9//NExAkUGba4AMoSlNdEvCDeAVg+R6NQRn+zEi9bNKVyLHmPfKlHJeqpCOnGTKI6sKGNtdAm3KMDQoPo0yiajr/3J/+/Wf/LUqc76k5z1sVP///9def/9pSwVMy3nMSl//NExAsRubLAAMCSlIQWXREXcKAH+FP8otqGMHfKUqzELQkLYQC7cWVWXo9lIiTUdJWJQ2KI7KWfJ3/92qrbhe/K120pIMmUVv/94U7KQeY4lBny+miCsQb5wLiHzT/o//NExBcSmba8AMnWlC1+o6WJtwk85DZUHcB4il91vzxw09InIFab7Njazja5mLr/ax/y65jlqJ14ZbFv/////RU/Vc6UgP9xsOOc5OlgCNCZL60h9vq0VKbf2Cv9/Zp7//NExB8RoT60AJyMcLfld+FYUDBXinicR096OSJbC8ARInd91pMOhQTT3//6+nubTcZ4sUWk26aBeAQzhWsHDAiKe9phr/7r/p332R4N3cj3DsFVRJongyJTJ0l+x18H//NExCsSAPq8AGIEcPbiPsZl9YmKAQEGFFvOKGH6hkTsGlAfiBbz5dYg+5BSUM/TyeX///4QqIwhDLMPzJf+nf7ap0///818+aYpro5neykzHNcWcuxpnyKNMKgrCTi7//NExDYR8x68ABBKvADCZDhYmRWEBALhwUEQYUILnbf////////npf////v/v/2X+i7M2iMZJau1GI6zImdayxdWdRZTmMRLILiRBceQUFQ8LA5BcaGjA8HTODCwtAx4//NExEEQWxq0AAhKvar/+///2WvJH//rp/+r/9dbKf/7OzVa6Mk5pUO6IjbnOjqNep0GiZDu6kUUIIFHkjQ7HiwBiYkKCwdFRETHnDQcWYXH1f+1//8v///W7tb6Lq/L//NExFIQyxa0AABKuX+3RK/n7d2t/+3VkT3Raqro5mVd3MNdbMeVLog9GCAxFBY8XmCcua5rFJUsQcbnjBzCWTDhqQAlmZv5UA4aLyYYLSemivMy////+T6ZMRkX8h7U//NExGERYxq4AAhOvdf7fab/au6t29W+j1bVpZVVqMjlIaYyo6lRNPMpSNQxjKzo7BgJ5jIBCnKVgzzK1nS8iRUABXg5qgMKaOwQ5qEF3Zw7l9HZ5rte5vL/3lj6+Xjb//NExG4SWxq4AGBEvZrTr5flq9kwQ0id/EWcR9G2KSOPDNv++WFoRFf///beO/99t6111rlWURYWu/b6S131aAMCvBQGRI0UicPNqY43WpD+Nrz2vDZN1vmuv8kJ1RWR//NExHcSWS6kAMZMcKTKCPR1kUaNp0MJ13mDCW19OcJNEn///+f//Qp5O3dTaZBsAs3SLwp3WCjkCGUd4xMjZwba/E3NUVZON9GYiwPIqU4uEc2afsJmOsql6SkIwKPL//NExIASGTakAMUScFMWESQSlYTmyh9Dww4rqIbxp51b6ef/0gPP/nLBTPx1y73TNbqylNazInH//+fCAwQNQIGwuwYeMjQ+uGwfvZf+M+cUOvzC19sQDw7NIcsfwIH9//NExIodQdKYANtelPoqX/aXxfRC31pDtH99QwUQcSlsG4Eabc5E1LV7OHGEI3dTig2JN5y9FV/s2ppg+Zrvt/6f//6frXorIp6NHEoae5jOfV0oyn1JlIeqjefLtQR9//NExGgYUwKkANLOuCNDg2Pr3QcAWdLdfoHAJRdtdLa8T1/DFNFwoxjRGE0qa2zXKtPBTTqzipXdaz/vu6uKuOv+7vddXe9e/h/TLln0/P1NnH7XtqE0tFBNZ6EppLXT//NExFkgcyKgAMIWvOnvk2o0cpom59RBUpJtOLHLrG9pEah/eOtIjHwTD5IA+HoAgcGYJhImakzgEiB4mD1mpJFJFX3/v//ir+cj+vdfmU9R/xPz////f9/zXDf/2vpE//NExCoV2yKsAGhQvFXFxH2sXPvYvcpJLT0a4zYdyQoocI4w4pDyxxgoFYEQsaKlh8aQFVGWYHIlH3V+gMb9f//////P1/PIj+XT+0/0f9Lf9JP0Q750ZFdDurIyMNd2//NExCURWxq0ABBKvXOaLmGiapPVXMaOGGFRgdHiyMPD7KLCB1ZHEw+MCYs9atbxq6YEwKM80LfSBwB0NsmtuTU6SC6EV0ECKTU6NKEwHHibmTr1O/yrejf/tTtVv1qz//NExDIVexawAHlEuF/+j/8vM5ptal5W0e16bmVDOtUKQygJSiSFE0MpDOUokqkAjZbZ3rOOGHfMqeOG4pFSRQcCjBDi8lghA4+/+1nHjP/YGWmPjQy863EineF4tHrS//NExC8ZCcqkANPQlIN5uHyBNH+IYjM8IHgRA1r0ke3+n/yP6/LHT1uLUsR0ZpvXs//72SQHFTCAEOMhlIsID2aNqsu4brPCCTuGfs2rwJAiufSzHk+ywS4dtJbqWJ0M//NExB0XobKkANxWlPzIxqVSGcHtU3Yog3NK2zkknor3HjnviSCOP0DYByHhj3xSHN/D//af/7KT9yqqPJRQ1Z3T3//96Y3iXKtfnpFpqsf3ljDxtAItWLkTfQL3hTSC//NExBETcc6kAMwWlM4YqOBrQ5z6kRpFbrUT3xrDaepzAM6JJ9xyNM1/JC9/JWW/ewCqzmN0L/9l/fKnXTVV7am+hHOsuODvNEqRWct3EPRMsgGazVlLCgpPN4fTMVu2//NExBYSibKgANMKlLAqjz5xgBr25OGFZmZ4rH1+e0lhTEVlEx6E7ipekpcrhIAhZSPEhb5utStlQrdFFZUNB3XVgi1KnpTSMLAQ5oDgQBX8ikNqbmO9xdoN+zElpgI9//NExB4SIOqEAOaecAN/M5WvaphasxNpUZ/9KMMX4YzrFOkmkpRfYL6hq+dxaJwMdpO0Ev//0JXlFem1gDBJY8IVhqO3lh3JP6cX/Ju4KUF2oXezrTr9fy5MwtJGHdbh//NExCgSePaIAN6ecLPG/pKuhcTpZWBkQzfy8TiMRuMyv54rgQEx6FwVSz//X9TDCUP4qmdc4dy67K28EJ5rvBC1L6k3BGPeTb/yq1nqkmoBjO8MC6r5PJQxw/BeHZA4//NExDERYTqgAM4QcPQrKPZfSoq//hE4QUMSDgY/E495OrfLuI46aJ4YSxVSlKk1LDsdFkRUAMLDkg7cinHfFSol3I3OKKL0qUPvFAkv3Hp1mbjy2mjCckCMOl0CBZRb//NExD4hqb6gAM4elMLuXxPsaHxGDbG7cI9bQWTFZ12q9PnBjHGkIbxwVj3fxEzT6w4QPSIyXiZjqiHZtnY42MR4ccY4yUQfd//+SR3HLpAcg4MU0CXH++h3k/K1Vngc//NExAoUQTq8AMYecNnL8RQ5g6zjZ3VD2XUe9Ppb5j2zCsu4RDa+vhWLeoW1spmWFHT7y2fJF1mrHi25Y80K7yZygz0ixoOlBtVDWDVhsoLvHITqpvVwxSnms3ACcpbY//NExAwSyS64AH4Yca8DcJqdnLazLePdOdll+L9X9ekm1JHg+ej0JwCo1w4Bytrq9Cd7X3JlhxS7xwWD5U1jDz1Aa7f8N+kEQ74a/arn8uCBFNquoYnhKaZRsesj7MWq//NExBMSOSa0AMYScBdmd1lVd6e/CUS7v1LMekfIWfNgmeuIwsAITBRDGMJIbjrGtNo0S0STeki2DTFvtmnKCJJrE7P/sAiyHPBYIW4tzC+TmPaElsUX9DQvnkV99Jn3//NExB0R6YqwAMpWlCIZfWR2XTB4fYkdwSO2gUlh+kR5SRTJwJ5yzY2b9V/1/H8x7rO7HK0JU1K9huuJIOvMUqN43LjL1FDPeh4ViTfxCEf5Gfs98iE/sXdOAkLukNiU//NExCgSAYaoAMoSlVcrQgxe2RE74kADEk6ZYva8P8z//P7y91liGpLEcnuUr25WoYLOaEp0kqqVdhedIgf4ASlWWzW+jVxCfWr7W/9t/Waus95oiEyJVQNCptXZStYi//NExDMSCRacAMPScQ0uS9UUpItfJFtL//w6e/DhsUFl/VUWcsOTJLAUEUUNtjrg4qLAhkICayJANPyh+rW8Mvyz5l+m5/TeXbUSiVy5IDAMJHoEhKaJ1hJSUnYcFUcl//NExD0RcS5gAVgwADlMloKqKvqfrTOI1rBOQMw6AR5LiqYOfEWhEwLdiQdbdiBSU6RwxAFAW4wjU6CbmYKgjhzAWCjN11MijJcTAbFEoYpKTWkgjus4mXyXatkrqQS0//NExEohmyJMAZhoAY6aIHloEA3MDT1MzLSXtdd2QZNaRoWLVUy2bdTMpa1opUabmnLTU3N7q1XXa/fqZal1aT3/opGhcM0zcSrTwCzN4KwBkKdRSRu2VFQ9S8TRVpG6//NExBYYKu6wAYkoAEYx0ZkmI5kodyIKaxjGyqXnkY9QsXCRS7KwiZ4udzvODgMYOh8TDguVxGiqxjGaf0awkOOHUEiLMe8z6tb/1ciE5FYh1PIYqIJclQc4VouUEl7C//NExAgRkX5sAc8YAMukkwxX2Lf5xFxu2/WtWb/jTP1VQoCKPX2DMqlqpL7Mzev//8VVpepKqhVBUFT3vWCp2ViU6sFXCIl65YGqlQFEBFMmo5Rin8pzROZVRWKN0CFw//NExBQQ8D5AAU8YACoSJOEUFTpUFYih0qGpVKg6elRpCHVhpC3CUgp+/6fWGpWu5bkUaita3EqAr/nWKbwX0EOMWvG6YQxR6tjEOE9MVaAHEClmopupLGBBxVTpMPoI//NExCMZ0yIcAY9AAfPS6rVrCWSLiRBz3HzC9zcXUxvpz1z9TXN3rdO6Sku9lrpSJMz1Kxw/Pl3D2z2KQmPx7fP9/xz768TWvOsf3cNVRHljFUMZBXjULa9zMXgXsSBH//NExA4WCcaYAY9AAOiE/CtFYwJwaHs3+hoj27KT/54okUNgtG/v2Vh9pCIMOj//gbCI5/M6rf1/+klyPGVmQLOAIb/KGRYKsDzmbzjf6HnzKSauiOULKsfvFgEwwH6i//NExAgSWRakAdlgACzEWXLfOWXxC6AHMT6gudkfzz7xrC7t+7GpS0i+Mc0nxgmSHoQ4Kdssv62Rfkw5NGL0mz6ylFkic3Zf1ShMHxAD9csbiUpQyzpTtPDKlaIaKrhz//NExBEQMQ6sAMYWcIDQncwlD28x6+mXPgnf6sHmwKaqLCQBpR7iGvi64bfr3sVbyO9Eo5rf6J94mYJnpdc7GSuFjsYGm2sYsH+nsKcGTn9z7Jr2sYfnf1AEn/UGG62E//NExCMSARqoAMYacHkspY5gJ9JBAZTZbz/QfKnUmPpams45b77rrZ1IYhun6P///+vP81sDzbG2jMPxtPCCsRXHAQyw+WqUyP9s9n/02LD+x7v7eGMY3o606zUmnU56//NExC4SAZKoAMYOlIs5FqDxhyDxA1YTmuNj9U1Tv/StOURAbpoow1in88dz76EUs+YEKb9+qQzq8lyikK5qq9+/gqrrtaL0/7b25y6wWevffnPjzGShBFlkx1mPCIYM//NExDkSOZaoAMYOlHHmPqnb+3dkc5kcddALW9K9+3SJnZ53V6wLTwAIiKjjl6cSyp5a7Tnc+5JO7oH27zl10dBKGZ62BYCSnm1qKaWv013pazugiRTAtdkEiK/39h1A//NExEMRIRqkAMYYcDa6c1MwrlOGKQZcATQAFBwLk4oNJHtR6RSge1yJX6nbN7LuOX4+yzWMxEk8IxdAyldZOXazZ75taZn6rWDlaesHzT7431nY2oyMPQ3VBUiCaHZQ//NExFESiRaIAH4YcHAj3tf+BW4pDKrHOwhOdBvVG4diNHLana+Oqa7vWfN+kfJoJJMcbFYOSCLs9zud983MTtNbROgqAtOL9o10S1HuVVSqBrjqpK4KiAxlNoGrO1GY//NExFkSSQpYAMYWcAVAWW2ccebx5cprX42efvP7bLkaLAKq8uyXbWrc/k2XSoxFncWHncsDS2/z2VSRihVP+zET/yz3Prd11b/qtSLOED5IjDtrBUlJ3W8kkIFeyDOC//NExGISiPIwAVgwAMyinRXNGmgLWMkYced9Nbm5ubjwJAXyMS9flxBBakDYeJJlMkyXZvum5cRTplAzdhwFj31tppoIW55M3N2JNMlzbtrp61uu7PQQ2mB9JE4mcRMD//NExGofSyosAZxoAL+r6+00azdvm5YaprrTMUjVJZmqUs+1mZ68LnUfKLF3VfCDwCGqFxIJhcid5PXBxAEk1ODa5rlzQ4Otdg6R2igAMRDpAlw2B9PzZoaHACxFeO1U//NExD8gIwKcAY9YAYwsWMGDrPn3732xirzzH1Tp28d/3FPlBmeN11lmyeeb2fU3///+nF76529xTOq+6j+v//Z/8/u+WVVT7qVsGwafqmMRk9PKlptRLEcD7IJ1O8u6//NExBEUMRKsAYlIAPXyrvxrfDPcEwcd7TiUuKfnI05JRYyz3QxpYGyRw2ZNsY0epOFhCWiBAlFj0seABAchClbXTtVCDlqsJ6GW/64yEfCn/WgoLC2/hJC/zMX4Fogx//NExBMWOqp4AY9QABJzm8ZjQqKyt+RDEoDaAGnIcQ/H4Dw/C5Aqj81jpy/i8ocPmLvb/8emGD8eGj0sPf//5hpCxUgI1qd///+aVdC6mUdXHpsnGQqhXPOSi3jMv9ah//NExA0U4Xn8AckYAHetuxbsEJiqFcLeizFijCQG4USTDJMWtZkgY2DMBBmOiowYV3VunqrTcLSIoyInkdcmSYdA+PU8e7ctc8nWeU/RKyy9TFD1VQoRSFSVnKVZ9LoC//NExAwUKQngAEjMcCWOgICqkGX4GFPJpIK2TQUAhKNORImkZ/7zJpGfjkUQVAR4kRlTpESlQVYJQVDQhEp2n8sDR4sDUqd+DR6Cz9f/nvUeWkxBTUUzLjEwTEFNRTMu//NExA4AAANIAAAAADEwMKqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqLecwDoXk//NExGEAAANIAAAAAIWW8hZ7EsT5ll0FwVpOzjbGS8dgdK+eA8pI814cdsYmkAEDwugRRAggen4aN5iDnptCfaIw9NiabEEHPuG/hE9GWd+5P/iRG0IHczLAGBkd+fgG//NExKwAAANIAAAAAGefhm//d3+ljP1+b5JPz47/1TRFeNMcMBtXjKUrekDiFIVFhUiTXUPimDVbFabrjVO0g1ommJUbbG56//9PMtvbf+6J4BEQmAiD1B404i7fqIuI//NExKwAAANIAAAAAIqGJIdPI6pLZ1UVgI79b/1qAMHyAnYbZREJVJNJNJMsMjJAjc+MlVkpw3NyUY1cJ5K4ZW5KNXVTx/7s5TBQwIORUVAYIEcj6FBAlIuioit//KYE//NExP8aqUHoAHvMcSQ0BQq3/YkyEhcM0hIVxUUqTEFNRTMuMTAwqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq//NExOcUWRoIAHpMcKqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq//NExOgUkdmwAEpEmKqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq\" type=\"audio/mpeg\" />\n",
       "                    Your browser does not support the audio element.\n",
       "                </audio>\n",
       "              "
      ],
      "text/plain": [
       "<IPython.lib.display.Audio object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "\n",
       "                <audio  controls=\"controls\" >\n",
       "                    <source src=\"data:audio/mpeg;base64,//NExAARWEFUAUMAAHPiBBCOACCcDFuoPiAaD5cLAgs//BBxzlHLB/rD4jPgg7B8cCBQ59QYKHC4PvD+Hy58oCEMF3ggUd0ROD5c+IHcQHCjhOrRTAxKcDasdA8Bpw4H//NExA0UswJQAZRoAQEm8BoYXe2Og7Cz/ICyXMf/DlhcwKAMA3/6aaBTJA9//j3HuU2Wmaf//0CYPQ8Zl9zD///9CtN0LJp0P////+5ovR6QDD4CAy+qT4A1IqmBaBg8//NExA0VuSKEAZpgAGhokEBwY5QsBF33gdj7tF9FJw0qnVOCMnQnaysu30DbVlbslqCBlWLjVtlEXFtJnZ+l5+eXbs9kL9Jy4I31l74IG6lHf//////TU29s7FTC0U+Q//NExAkUeM54AdtgADTCAEv6g6qoaCERZNJ2x0KcuXvpanIlyM1MrW753Ts7WFxDd9asbGzrJdWOyJTAHAwWigwJtEYs93Ke+IToOmTqNOv////rs9R8JW5DVAo7F6Jf//NExAoT8a6EANGElIdNpKaDjRm7QNaDpmJS6jj4jMu7jPG9uUY/iztp2YiqVXV5GM62IRQjIIZyen5Oif3WQRkzlm7rVP94ILfkFn/y7znk96LWO5pTIagvSL14fRcN//NExA0UqR6kAMNMcEoBGDnJQ+YMbs+pnrLhpdVk93/fH3mGNmlpiBqFwYq2m9+zrAgwTmj5IWsKa10NMhgPLb2J4r3ASjwJ/eo22tQR0DDhcP2c/W5gLmGiG+eGKyhJ//NExA0V6ba0AMmSlGPw5MCEUaiCf9qNTdu6BN7w4QBk1ktlIYtZsCcgJAOKmhICckKslVkcXtuPnDwUTRHm929ne///P/7r5XvcqDFyYnX3///W/9T/UkexNA04ihoc//NExAgReULAAJTScSyRQBLgNsQ39LSJozfonjRujV/7cHwz1Gl5z8tQixO3EhZJmLuKFRHTCjSx8kXRxZ4/Tqf7x1HL5lwOtT/75//3EVcj3TZxu/Kk0QgQY7aAzd0G//NExBUSiba8AMFQlL+YWD/qIjG5I49OSgeADB6DdrgOjWMtFUPSIEXYPg+JhFrnZPVYn+l6lvhxyQVM/////700Kj+zsPQA2oiZmhYmQAjJF21BVdjqjlG2Md/NN1TG//NExB0Rgb60AKHQlNVOdRACx9I2MajB7uVKnKl6XcTf13/r39R883bHYmhT//9fXRdpTM+Uq49QFRIrSpoSCwBxx7zlT32/T9+jw7u7PHDsFVRJongyVMnSX1nnOfB///NExCoRkPq8AFnEcGskewhl9YmKAQMMKLecUMP1OicgNKAeIFvPl4gq/BCuZ9jeZ9P///wpWcylKWmrfVG+ydO/2qqdP/0/0f1eZhIpro5p7KTMc1xY4rsaZ8ijTCoY//NExDYSox68ABCKvIJOdgGEyHCxMisICA1A4KFBhRg+er/////////0v////39O//ZW/RdmbRGMktbUYjrMiZ1rLF1Z1FlOYxEsguJDBceQUFQ8Hg0guNDRgDB0zgws//NExD4Qcxq0AAhKvR6BjxaX////3//+y15Iy//rp/+r/9dbK//2HlarXIxEOaUYJuMRG3OdHUa9TkGiZDu6kUcKCBR5I0Ox4sAYmJCgsHRUREx5w0HFhwuP/9f//L////NExE8SQxawAAhKuf8G7tb2i6vy3+1aJX8/ZN2t/9G6sie6LUqp5RzMrPcw11sx46XQoPRggMRQWPDaCska5rFDx8sQLjc8WnMJZMMCSQAq3oQCMNjQmfUvX/////34//NExFkSYxq0AAhOvZMjIvre81/trlp1/XeV1t6si23q17VQqqVyNcrGUM6kdSobR+COVmdjGMrPMUMBDnQyAQpwrFDCx1W5j+N9KYJILTO+ptBrTQMynBQcQ08ksplt//NExGISCxq4ADhEvRVrda+t/8iov9b+YFB1HHCGA8IzW6cU0qKp0eBeT4m//nAQ/r1//8szXlGaZVGLKuWuZSlpR3p52bVOqI5lB6KdJNCpf2DHoOfFAt9w0879jaNY//NExGwSCTqoAM0QcIEYqRSotKSQTT/kivD2cdUVkiZYqa1msbX976v/neL/0vaDfeq6Y3Vv3///oJvrSy+ihRsY5zjVpN83MIugVBKb1+AVbDM5Aw5Lxw8wFSa9Bu/u//NExHYWqbKkAMnelNc1xULE1UHB7/U7Wf5lqGK5vcMTNswlnEkaOVLVJAxYnqccdatVw3/+3TZ/+Gff37OUT21FdMFigaP5D//8ZY2U3vGMFFk0tJ0qosu6rvocY7Al//NExG4ZGbqkANHelFr0zomZvk2mRFpoVAIbwMM9SDcoDt/FwF3f4eo730+LMCKn7eFGXJiU9oylImPf7jiuuouvhuj//OI197/rG1/4KE3vrG21wpKhjb//5CWOgc1F//NExFwZob6gANHelAm6kXbFRPCwwDk6u93li+5wxWFzcw+4EliR2xn/ZDGvvSn/Y2vjMY6vYYW1xyAKkIfPxJKHU+PpJ9PaIwBRyHoo6R+vz/srVnHmO6rf6N29f/+n//NExEgXAxKkANLOuf56zV19aZrUMbH9Tz1c8+l1QaMOy3cs1iokFph2tzkbLKurez55QHTvMt7p/bmHMSlmGx6zq5ru6syoZeW/Y64iI/5j3v7qJj+4u7+ZiL9/mnq6//NExD8dOyKkAMHQvEREeLRPdNxvljw7PmMacrY4XGbDmLIEwgSCsiyBQsXFA4CQCBANBojqD4bNDgCCMHIeB6H5baUcIRQNn5x//////P87wf/Rfo+n/3a6fs3//eh0//NExB0VIxasADhOueR9zzjUPdC7mlkVGKnHGlzx5RecJx9Tlc1SrjYbRYUGxYUDUeHkFhIRyJMSihNTTIrEsoEYlNi1+f/////+ZLl//5////////////3M8c/9U73H//NExBsTExK4AAhQuVvfCZiMYQee572oxb4tTEU5BRfIMLGETTj6Ueth4Vg+LoIimBy6Z40PxHdVCUPzBCzVgN/PmSX//0Z8HmKrWfPftX0/7f///f//fjPXftM9vuTi//NExCEVExq0AAiMvV6LfJypS6OoDdnHRJWbVzLUajE5VfmzNa+EtlsmQWEpOLRc1EoBL/05NHZamrts5RQVsVW3WjtMtkzjaVCLZQUAgAAwvD/a8k5HIARzY2HFAKAT//NExB8RkKagAMJGTBVyWWrOs2pEJYQKWzrOjAZakRDQVCbj2Bw0VhoCijzv9+r/r7OW5FWA6SNS4lATJIkz0FVUlzBRgBBi2b0vNTNagqHpPY1TVIEoIxU1bpKC5PiQ//NExCsSmSaEAN4KcLBgiysZysc7NezVbuiLspTLEkxZAV7SwOrLa7amUf0qa8/MbcyHjCiMHiaXKtqtqsBycHSMRGSp6m6hUtiEprTBiCpM7GBLDPI7R9EbC0jIqvFW//NExDMRaJKAAN5YTCED4qambmKa6Ixa5zGf////v136SYiK7jTdDjgFM5cnYYMFBjyNApMEwBhC+gYBjDQKYJArIlNAuqdocD424aXRR2/p6eL2BoOGYOV/+sMM+kcH//NExEAQ2JKIANcyTNQLvWqp+OSTpz10wnXGkJxjPxpM4KSK+d5PcrRh0mONwRsMckOMlS+YjDgNGmwbDQnCYgVOd2qftZrDiWv1SYXr3/GV2i2/u7a8vY10yaaaB4DH//NExE8acZKQANaMlJj/WjcPJrIHthRADcHJpuDYPrT7f///p0J/pr38pn4GwLQG7WYOMzKVWc6HW5DoE7wFYtHCIPNKdciH5M7YNbrWZ06LKkbDYEIpNwqFrHkPcZW1//NExDgbOZqcANaSlLs9zqQxtX4T/+IJfwRuuFkaTcpEAVSvUBhq8npm82xWOkMOXIql6ZDygzLy4Xijv////+3Tvfq6k4z+5K+s8NB3tv8cME9S55m2QswJCo+07PUN//NExB4UCZKwAMYGlalu1EtBQqSNmbgwQIhepiGHEtf2vLueiH/F/R/iLboCY4QQ5uaqWCBqbugpjMQJJj3TqlvTCTA5kzMiQQUDyJNZfCbi8ofxrkBksotn2IojRevV//NExCAVWaqsAJYKlHgV9A3cGwgkysNSfbojO/OerzZrO4wb0FW4/ocqGUSNoBw6OMUW6HLqIrQguUqGKY2J7pfDNZd+7A6QFlb93HgBkIBCmuWE3RnQdq93YjIhdR5Y//NExB0TwUagANYgcdLGudxbIzbL3DHQ7I9FMgIZ0A1EkkjJ17HCtzjvo/S6zVHRdGo6anRX8jPx7kBDVjvylkmlbDDG0c5ccULVUBQsbK9kW+GCNG+oWQsA5cIirD2D//NExCESWPaIAN4ecXc8nYjst5QLEosa0ZWEphth7hmEsh6bmPM+29V18zUdcL6kie8B/eyKcWfnHWMmBwNPRlsoMHw8tMzAodydkxlaDrGJtkgb9PpKt41lbJLO/MOv//NExCoSYP6IAN4ecWkhxwYJDX5xnYjrXZ5o2WBGM6ihv6R9YiXKyJWU4re6hMAOGh0AjqEYMo1iiEKlI/KxGwVxkBIvqF8rod3JrZ2Gd53azCGOwdUm3qgX9M6wvtQK//NExDMSQPKAAOYecZMRiUzLDtdoUbfV6zLMW0pCML7L7CKFum19mBr5ucYAJnlgHJDY7C4LgKwpHgoZjg7L/zrwp5TeXIYe6XZuMpOHML1KNX/T0aYAgmnOJa5ENpe8//NExD0R0PZ4AN4McGupklxdQlF///oqVExNioJITJ/k7QtQkhQQMRQDbX5LcwQHQAAoIFCEMCX+jb6LxnZVRv9HbD7LQ5AKWTpU17O9baWjn71vW/ufts/3evoqRX/M//NExEgRaPJwAVtgAILiM6eaoGtXK7zpGEpI1TTlOi61pJ5vhehMCU/TTGCHiXir6DMkaD3WJuJ8IH+aOgaLhaRoE/JpqMH/bbjeHMGHcTAzHH/999xGx5jlAN8bhPwQ//NExFUgqyqQAZhoALLwy////yocgXMKmFzNAmgbgb50eYWz/////ARAnBICYFAvkmbjCD1DnhOCUOpqzABsDCmB8LYI0IkYMXQAlhZoV7W/00I7Kc9VOxyEb//t0Tr///NExCUScgq8Ac0QAP/+jqUsvLtbejoYzoZ8wEDp0Qgq4GnlpYDQalRKDRHr8jvVWGqr33qALcuu0BUuVTrS4hbMzBNk+8HHyKDmUqNRzx7R8iHX28ExlKCiXmmLgCRg//NExC4UOaqkAMrOlMHBuqIOk3njRlKiMQZgIGmypTs///RtTNSDzVdT/////p8bs6vIzmbNP3a4GGL1Tj/lU0/GW0n5yBsvxeqF4ap4F5rVLe+zhN5pGr/ISYyKRQBq//NExDASaSKgAM5ScFmxyVbnhk3Z4IVLomiGv/sQZf////1er+hCzpaYLFPS293tkMvswSAVRakso2RSTR722Unyw/50KjiSOukrytk724LiyjdKXLDkmcsOAdChOuhF//NExDkSwSKcAMNGcAyt7hO31l9CNqaplQnV0K7y6h8YgggQia/R3XOhw/4TWxCmAUQYQ20aCG3KrBVmutKWJq9uxu7Ld7uh+3q9KG0zc1LylK9Ss1KJXd9u73oRCul7//NExEESKxKcAFBEud2tdFTbSM61UaHCEJW////PnLXwoqNTbJmTLMJNiTj9uunXvSyHy7vZGv2Rd/VlstHKrJq8ir2X2pu6veyK7IlURmVyrMRUeo3YbIowUgk3//9P//NExEsQ2waUADhEuazcqO66/c6V0ke/uD5FLnmx3uX8z4k86fYRI61u+1/NPP7YGLOvIa+5nJCg/DzIQzLlzp/enmZK2YIrkU+4RcFVYBdZJ89rSb/rfpDQhuXRa1aL//NExFoRyxKMABCGuOC2e98YR9lIXwHfbkLSKyC9iMVfUbCsOPaR/vfu75QxDfn/9k3P6+Nm/t8qAbX6syR/hfSWyqzRAFaVzTWRGAQ8NiEbEAjMJThl6eiOE1H26KCK//NExGUReA6UABhGAZV6nG+WNrZ1ON842w7yxzy3f5KCAYgbta5UHFBi6zt5CnMf/O0f//6P/Ure8FgwW63Yniy8XwdMy7Hm1oKC0GC0lZYCS02UdaRL60OI2NtTvuA1//NExHISgKKcAMYeTMcxoXx9Hk6jA10EeroP27VpmzKUgeBAMi5qilKTDH33fRX/rEhU1JnTY0wqnIZB8H9yh4BbiGoaVNFL+abdntafXBajrklY+Vqrnb5ld7+7//Xt//NExHsSWSakAMYacNffV2oc0wHRop16dX0vUekoo1P////o13JQ0fxRWoZGuX4y8RxrbsWxOFq7AJIWM6vrxc/V5saMO5pbY8q/Vm4xjr6b/+9/06dffVuG4g2GbR9H//NExIQSAaKkAMYKlNX0F8Mqj8720f//9G1a1jmmQNkyyu0RDjuXPCIKJBkQwCbh2GBSGyKOx8dAh9Q5IKIPIiMROS2RHC2Xel0bT/9eF4ZuMiI+j6k4LhVgvRRRltBH//NExI8SaZacAMYElMsKgoHj1l2wFihyUUwEzyNyGZmte1+N5khjFsOMKIk7FxRonQmY04K3L2tOV++TX/++fto3br0fvoxpApnhHAgqZGGQCBYce3b///qf9CsOvWFH//NExJgQeZaYAMzElIqTehWQZRUAkB0ouqKHE3TXBJBeGVAhB4Ao4flTIQyiY23OdEJZXdSka0JPmvwySDBxWkHZlj/wJji3Gx39Xv5je/P15m0bMfm0FqGDwmzKOOgk//NExKkWUVKIANZElC8oqHaXkYIogio5d65aS//9CaRcuRUmHBhd6RKUGyA0yoxNvAOqzrzInccA4IMSEqNWAycjOiDH3g4LhAcVwiNL6MRBl+bgEEfKJyVToKuI/0tu//NExKIb6ap4AN6KlKtdtZV5T14dh1JKV39tUi4CmAooXNNDZpJ5pLTZ///lkwF2uMHnwIhqAuKBk+magSBy8I+ECgMxdP8xogygkWwxMQVoLiYFabKQEDSaAswQMnDE//NExIUYAKJ4ANbwTDPCjETb1iIMJ7bp6PNErG1PIJo//vCZhY4wDJdyTwkYoc+xRD///zTWlB72LcMNNSSYsoFKkksdkqkB07vDzqrBDo8cwIuyATHPz1JRoHI+Q7lG//NExHgWYKZwAObeTCJPqyiH5EvMDJrWHQrRLC3eKtNijuj//e2MPSWmHUFXCtv///+zXeasxVVyYCbkBSJwB5EfLiMSCBy9l2uspEEgNIm2maGJuy+kBuNjRXKITpPB//NExHESOI5wAN6wTNSecny5iQZAMR2hz//NXd6GCJLqiyQZsOAdGFYnE3sGFqDqRiJAiThyl2QRKVWQVOFiN0+i3K8yTdPxLqGFaZKIig8hSaVyV7eLTCQFApZDHNf0//NExHsOuJZoANYeTDIUArIV5YXM1VhxvClDYUzbMX1VWXlVVqqGBoceK56IQVBUFTuCp1QdiIGioKurBUNCUFanw5qBoNCU7/UDR5Z0tV8GquWe2nllTEFNTEFNRTMu//NExJMSCMowAMPScDEwMFVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVTEFNRTMu//NExJ0QIJXkABjGTDEwMFVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVTEFNRTMu//NExKwAAANIAAAAADEwMFVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVb0gdpzGi//NExKwAAANIAAAAAE6ExDKA8A1Aiw0Bxmgr2RLoxGI9QMbVBtW08S+M4elK3NzVLUGHkDzCzkVrVubKQ0eHGJobKQ0ePGEwTFetBsyZMmTRqg2NoNGTPxYyaNUGzP1m//NExKwAAANIAAAAAI2AwiKmjNQbMGnzRl0NhBUbgmBVjHeF/VSydZ3qo0Dy4XDaNtfcnOois3OC8hQwhOiAwPDiwhFymhZJwtCMksEV3ND4gcXC6cT1n7xAADm5Z9xO//NExKwAAANIAAAAAHyjt1Z+T+tGycPid3L+8mXVXQcQYTO4rk0UNLaosroAYAwjH2s/XprzUZz86cu4YzoAwBCxjFYSZSyl0/NNsYPB1oKjAKCoLB2IhKo8d4iBoGjs//NExP8aES1sAHvMcKgrrBU79YKxL8r6gaEVmJX87lQVTEFNRTMuMTAwVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVTEFNRTMu//NExOkU0T4EAHpGcDEwMFVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVV//NExOgVgRH0AHsKcFVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVV//NExKwAAANIAAAAAFVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVV\" type=\"audio/mpeg\" />\n",
       "                    Your browser does not support the audio element.\n",
       "                </audio>\n",
       "              "
      ],
      "text/plain": [
       "<IPython.lib.display.Audio object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "\n",
       "                <audio  controls=\"controls\" >\n",
       "                    <source src=\"data:audio/mpeg;base64,//NExAAQ0DFQAUMYAH+IiE4AIIHwfUHxANB8RnwQOe9Z+CBwEPwfeH+XiBywwJz4IXfKBgocE4Pw/1vUCA0MFz5BQYg+fD+Hy5+GOoMFDgnV1uBj4IHfq6DAbcWB2Gm8//NExA8WKypEAZRoAAsQJz8TAYQeH+HMMSXMf/C3hOwkAcw9/+S6i4UxwFD//HOPcpsmmaf//0C4PQ0L5fcw////QQTN00KDpu3////+ty507oGhfL6aLapyptmkwOSZ//NExAkU4a6EAZpYAF5BAseCiIsIwgZ5MqNr32DRAvwmumyR3mZwB8VlB1sGkq0oqk4zWycgg1cTEKmqTB4Na+e2X30bTPUK3//bP62T8f87/v9jF32CGmJTsKZIYaen//NExAgTAMZ8AdtIAIQCBhVQFaqYhm4I3yezjjING5HBF+iiXJbU1a3Osz/3FNgqzqFiYRZTKoDolMAcsLWOGtEanu5T8qdB0ydRp//////qiUraw9BVRn0BpIQGz1cj//NExA8UWa6EANIElDqKyvjcgSPIOWRxRaJVO7pWl8cETo5Vs7MRVKrq8jO62IRQjIIuT0/J0T+6uQRkwxZuW+XVP+IFvyCz/tl3nG8nvRX955RYmq+730sbTKN2AGSC//NExBAWmS6gAMYMcGgt0F63vu6/f/e6SxzPTn3H9g6f+WTSQ2eeA0dm3qH1ZkG7/8Jk3AgtYwo+tdEysHFhY4UUwm1EV7gJR0ASz1vFg+J1LnNAw4XD6r3/ugEYAHN7//NExAgT8TK4AMTScPDFnxpQB2SCgycTmJiT5/ScuHqSS1IeLkLSDMk9lvElpoCNMlWcaMEYrbuE7J0c5IZjBQ/aXPl2vYmQMsJi5pZkMFEf////psf/eF9GxZ9i5H5I//NExAsRKa7AAMFSlRdQf5ig9tA0njTv6kJo5T8zlR7apCgFAjJ5JrH2EbHitMo6FLGjJPTMclVzv/3t783J7We6+v4AWEVuswDIQA8xtl8zOJg3RBGgw8xKr+tBP+NL//NExBkQEULAAJzQcX/lWS/mmMrKFgBg/FBR9iVZEyhYsvokQQamWR+rD/KyYmizCiWdDpW/+/3hfMap5rcVvXdoNAipnOB0WtoULmdXKE/Zzn/jLWxbocE4iQ3Cmh6P//NExCsR+bq0AMnQlHeGJJMi6WJR7+ov/nv6/+eTrCcd////d9SLy+cQpvHxlECRoyiuoKixnfwrPPRv//pUG72HjnhT0VtT4oJnQdW76kUy5+sZUf1K59UXYRBAIXB8//NExDYRYSa8AHiEcEEgI9FYWMNDIXUcJl59oQr8QAmhPXyf////CsRmEMs186//3+3T////f1zTF3RzT2VsxzXMcuxpnyKpijok52DwmQ405FYQEBsOChQYUYPnt8////NExEMQAx7AAAhKvP//////pf/9//7+nf/srfouzNojGSWrtRiOsyJnWNli6s6iynMYiWQXEkFx5BQVDwsGkFxoaMDwdM4MLB6BjxaX////3//+y15Iy//rp/+r/9db//NExFYQsxq0AAhKvSv/9h5Wq1yMRDmlGCbjERt3OjqNepyDRMh3dSKOFBARHkjQ7HiwBiYkKAwdFQiJjzhoOLDhcfXX9f//L///1u7W+i6vy3/rRK/n7d2t/9G6sie6//NExGYSWxawAAhKuS1Kq5RzMrPLmGus1jx0uhQejBAYigseG0FZI1zWKHj5YgXG54tOYSyYYEkgBPnMVznDizV/////9+JMjkfre81+1rny06/92K+vq09t6jXZqkKq//NExG8SOxq0AAhOvZUZo5SGUzyOpSPUh00FszIUxnOM7GBhgIc4Q0CAnBgAkMBjqssN91VCzA4LEmvQy3YDCAJIOWcRqLh57qU37/I77O5NTnDC5Q4cFoRiKSQQQHD6//NExHkSUxq4ABBEvb/+5VH+V9410S/SWYoUK47vgrrLLrPdauqyGVjDFHXwaUZfg+hJ2xQBVvFfzC+qoGW8CaKdvMFB3xJHhChf/O7ncyUxXpJ7V5Enhn84U1msuv/f//NExIIQ8SKwAMSEcM3/+4FP9fxPnPy/v///9JJr7rAo9YYVUbABhyDVpL+PzDIzjnG/osp5gpn74ZZgYiXOA4J38CDdShfNVADzu4tGH76ZTdd4hMoDE2RoDNexiWpG//NExJEWqbqkAMnelGOYUbVrWIAf9aV38uFfj0d/4/bmT++LJWuIGKKSN8apmJlP//xRzG1LF3xh4sLk3i5NqZzL/zkxFauZ2XwEssCSoYFygzQWmGZ/L55DpH+uOz4z//NExIkaQdKgANHelBY/MQuCjNReHaAtTFS0jY8J9Sc2UaDAJN3TNhXSsastTEz8o76jGt8gRoplJlIn3d7GPR//9htpcDIYT0j2qEZODgALk6u9/jEjokeVr9yTmIIk//NExHMZQdagANtimNrkF/H1QDK/yllz+l/6RZ10VCBXa6nE8DB8vMDYdxOOu9IdzeaSDIN7UfDTve8xmS9/oZuijPYgYUT0//////UuqVJ1meVtmWHXO6ueQyurkYWy//NExGEYKxqgANLEvO7yxuDIRNFfL5uSIgCUquv8OA5XpIvu3/4kLIRRYIh2LVPZzuWRTfcJ//98RX/383/UTfHfdv8v9/Pz6aJSI8PFaTFOvjTyz+ObrlHa9LeBQgwX//NExFMZaxaoAMFQuGSCJLGChQeIKK6DRHHEAsRByGV2qTDZHQ4nRQJItb/n8///yfMQ8AvzXiJ/+P46//5i62i/jlUlf/3+OOeG2h/3aom4HuSOhYoac3dlYjsJS5aX//NExEAXAxasAGhQuVmbYqRQo40gRRZqE1iGa4REDzjUUQQ/GA+HxwsvgxP//////6L//z//////////f8f1//f/P3E0l01vD7rC3w6olNmLulSMIkzH0rDrEQpwfPHG//NExDcRcxq0ABBQvM0IKpB40Rz3UkQXExR8ra////////+vX/8///7f+n37fStVrqYjOjWuzKqc809zkNQvYny0mQc2OFiqOY5MwuKSSjQgNRWLFJiOIo8EYOhSXFr8//NExEQQqx64AAhOvb////9K//2H+X9q1/0yUvVP+7tsW9NnZbKZFcUQtDMUokUSFHYXM6EAdUQUExo8IoJkmCJxEXRSDECAuPGjEFosBRL/lbL//5ln//Ly3BN5+Tf+//NExFQRAx68AABKvNZtrpp2UmpGbW2vv7aeZVJuNSQrldc51MYioR1jHiwrK84mHCBZxUKZEOJxRQsUFHA5xcTAUMZih/uq//////8/GQ/v+/bfW9v9lSuqm9v203Zj//NExGMR8xa8AAhKuVm0vZc6zO1R5jTraMQYYRUrvEjj0q4iVRwgHxCJmK4uKBMgoJDgBD2X/FF/////8v/mob+/lp/+b6G7exkpRH/65nMUokqoLWdEVXsahyKOFpod//NExG4Qux7AAAhKvEQ4q4FAUcrXGkKZhERQwqJHGCYiwdFTAKYVDwqqUpAX/////////6fl////+Vsxn9DGMYpSoYxn0NUpZqCnM5WVDARlyohvMY0qGDCgIUqlMoCJ//NExH4Ryx64AAhKvIUoYUFEoagEAgKGQCVArwOuiQPtN1EnEx6Kia/WfhzLG52zIrJePHHCwoekGdlCzUFiRctZW6WLz3FCLpSyagZctPBFJQ/z5OR3tr+5XquKQ51O//NExIkROx6QAUIQAIJQVgVY6ZVbJE4R0LuVQmD0IpcVUkSCQ80ZFOVwKWQc9tBkVatLtRyqLXIs6PLxfmQhVt5Mj5o6bSOTNdhXP5MIz5BBWzerTrW2hXTUmtP9vpdN//NExJcf4rpMAZlAAWtab032Wm5cOKNjhTQbTpvQL5fJcsHqO9IhFwkyCnS600GJYlxtKRieHgZFAJ2MslxWDliwHOCxuaJu2m+RTccZIJrOpLHpNA2x6EMoHVkIwNCE//NExGohYyqwAYZoAFAgKQ/r/MDV0KB6mbzcw50fTIvSSNTJA+cdMyV08YgHoDYKCzQnjDlNN6b////////4+v9+D95mJVSpQ4ka9TaWv8pJFjLVho5Cj7sOQ0SMgXHA//NExDcagq64Ac1AAdg+D2RUVgXEQkeyj6FjxxJioJ6vxCFqi0xzrRKod/NVrMRZNM600jUGhRVDxNb4pvouFcssri5z5NUtgpyn1C8IYNFu5EABAbKylzezdX6nt7m5//NExCAZKdakAMqSmO5uikp2ZGjMyojA2RRlBAqikoJEM2UZG+eXfhvegYbiXRrsLk4uLhZpExmU1CquCKZESiUbwnhotkhFh1ISWqmr5FFkcsfuygdyIBUL8Sl4TFZq//NExA4VwbakAMNYlFY+PgXYydInt0Dbr9Zr01JaCOZzrWvc7gXecEQKwbB+waD+tYOBIHp9vFpbPaRHbS053Vbh0tbQlryp8zJkS4/Pzx9b1Ygbrajmf3RWESRvRKQA//NExAoT+a6oAMqYlNBDNdf1sIKjaGP0fRujHZitmG+f6Z7mvbOy8WisSAJgUCoyKhqWrPOFdCMjpiJVs1bpaCkKFV3UamyEsKpViUOMsfGbOyX8sJWFnV/2KeGwZ7Cb//NExA0SGZqoAMlYlN8BruDcQfnbV/8hunk+Zvb5udsRtmYaEu60ll5XbCQXT7+cvFN/atu5853LrU56sr6vcwCi9v/////9CrP1XiMY2UlOTaAA3tiW5+n/LWSERbmc//NExBcRqaacANHalKv08l1LcfbMzJOYDwFseYxAmpGMkTMXk3N0x4pKpt1f6ut13TLyC1HD7Rf///+sva1+uqmRq1CX/bgIGB8q4sJhoWQDyHs0QL4xSjpl3Yxtdf/y//NExCMPOQqQANNMcGM3FbGbvs2WOgGHbhRKt5qLSrqP5ZbnnWpWIjbjQACQJxgTLWHg4SYlMK2IrgIrisyc93a12NZWsjvGoUj6pSTDCU8DIlERUBCIqVO3lSyndp5Q//NExDkRcKJsANYGTDTyoaMHezU8NVA0elYXCRTnDazFOQoE67drYJkkhlJlUy1xLSka3S9jts5QQxndhhlEBVYkjxqmnrYaS0kxRZzTJCTpIycd2WsiPWguvV2XLrqo//NExEYSeLZIAMvETOzfcpFlCGBb9sQjJKb0YfS/9uVZ4YzVW9Wr5477l4BNJMmLSQ5yRF6HeQKljhqPQlADJmTxQwOFDduzKJ3HZowsvtxQp2OV34v8zvLIUspxCkMW//NExE8SQNZEAVgYAFkF/97EAg5cwMR/wvBI9+fyYPYJ4gQ8gnf5qHsnkthOLQQ//HXEEgmDgljaVk3/9djx0DsOAiFZgGgOh9//2vh77okkAbniSiU///9Esd5e03t5//NExFkiKyqIAZhYAPJ48vVYaG////+99PfT/Zmh1U0Nzo+HmHig/P////+z32xmxmxlegedKrtS3HnqwqIVz5xBuTi944CzPlUE5xkv0sVSqKSOomhDMwkoWjB+ouHR//NExCMSGaKgAckoAHFkUNKooRzuMYTQoVTRbdfRvrlIrug4oKUKpGoDl7SSjSLEdX11pq3odzuNlFV91h1NJS/4X4C2OvasUkdgXjkpBvm9ca1bFp3SZO2NzybahQeD//NExC0R4R6gAMPMcLZyG8mvLzZZcCFIEghS7q+kUe7/////6/vchatWgJFTcMgmVKPjLSnqRWY5EgdU7ZOQDsEDZgCEHLFpAHjfMA73WoDLGtHL+4YjRTijTD7N5XQ2//NExDgeOaqUAMvelAG+eMBgPtUxcKekjAsOn7mb6KrEPxMw90j1xTPzfePu+P90374zrc2bw41BAQNmqH///xaiobaSYgFkCk4I1Dwwyx64I97LLcYJnZ+9KgWnB95M//NExBIU+aKgAMyalBGKNMb57Iea4/FbKT2KJW49PMBwHlrCZExFQl6Dx8SZh+LinRHmZRjPdyG9Sb6urov0850Td4NuVTV///vX9X9SVecrpKD4b22NIS7sadE7WnbZ//NExBEU+bKgAMQalOAnIVCeU1RqpoMPzuxw/ROkgzrJxlRz1MsNs0NC0WJATYiNPFRrc6SyTwsj7rJS9Dv7d671zyKSzSkxyuv////6/9LP9lUDY9ZRMONLM44F/yqN//NExBASWaqkAMNUlKIDpN1pjsKSKiaW5Ub8tTVLt5nsan1wmjc8kIQ4ZFx90FsxY1lYhR11tp1bt07dGShZL////6f//VXn4OEJvkfJKPIleMaBnSSFmmZFA+9RKH8///NExBkRESqkAMZScNzV79Sqx/xb2lke+KTusTGMsQBdeUyT0iR/9C/ojYoULCYmYBVz/+PQ06aV1euJtitKtuIGeSQWMOAY1AFQhF2LwuJ3URh+tAqqyyro38u+HsmV//NExCcSgS6YAMUWcMyPqEoDTHwTqtpJZLzEyfpEOTKeUrNETv9YiFTD///////pi8L2uQ6B9XEyosZKCNC4bIgJgopvg77fT6i1X609j+XdZ6y7+VL1cfKrqqQuCxM0//NExDAROSaIANaScKspP9LPjkq/ss9XGlnlv2W2ipGaJgMFNdWuYCkHgtAsFMKd8iAU0loo9EgKEAr4SeIuzL537NnmPLU7G7PTuUY1wIUgpQcPC0RKPAI8IXciqyWQ//NExD4RWM5gAVsYALOsOq9v/31VeZPCGG5oyCRMuw3Gy9j2f8xLEY2S8oF9wWkSwLn6kS+UwKMCEhdgqX0LOgO8lC6JoJ/+aLd8cJfMiGXy//bdnTJMkECWJ44Bz/9X//NExEshKyp8AZhoAPhV2J5sZjkLC+Mv/+zbe7hNBykE6JmJ+G2DvFoMOOwEk////2bxOEBOw5A8AbAV4b4lA7w54sxOx6JI5qVCAwL/////StP6vS6WT9W+nTtQ89lH//NExBkWOp6wAcE4ARzhqco46sUc1yrljDpYkOHRSeGiClgeBGUEcuXUXjc4RB0kxM0m4ikicfMaOCkuQnTWc0ogtCLma9T0FhNrxhVUd/muHK9//0///Z6P/tyzEla+//NExBMQgpKwAUIQAL/9aloZJTIKQUzEVZj1moqs88olbHIPuVis7GOFcqgMiiSFLUpDKoo0tPdUr10LC4IahdQxCcopo4C8C5TJY4R+H2yuSJqTjX/HsM5CFqAS1a6///NExCQa4qpoAZFoAJiJMMEOET4r+r8T0TEKiXh9BWTH//ECDiEaMgEsZE8OSPtf1/+E9A9SSH0TQkBgjM6OUakfUp///pCenTUT44UkTEqHkkXjFQYkAqOdnhkmSlpO//NExAsS0SH4AcYYANeNKS+y0sOGtAfClSpVswqr0o0NVUlYM3VCiSK1uiVQd0QafbloKgqdBUJgqdW7DfyrgafESpZKZ3OnVgq6V0rVBQImgtSREGFCzHR0NQoYHLAY//NExBIQsN2IADDMcCDuTKynGgRVq3Kk40sy83NmjhIfFkObFhcM4SFf1irdYo3WLf//8Ub+LdQsIzJkJE1MQU1FMy4xMDBVVVVVVVVVVVVVVVVVVVVVVVVVTEFNRTMu//NExCIAAANIAAAAADEwMFVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVN88w4A5G//NExHUAAANIAAAAAIT6jOc61Awi3h/MsEgdF9iQYCQiJZPjvM6vvqByCJGbEAkTR7lznsEELhCFrzqHhi++9gxtIHTeJxccTWXiABh8LBEPggAKonCIfIQyUwwQINJl//NExKwAAANIAAAAAN5MME/Qf6NX5MgTIKI8ax+mSrZm1ecDBMQDURaWFoxklL1j4kxYfWZyGnwcKjUDkJg2HockHnm3TNKrxK016603xaw1ipkqQFgKRAREsVMkAZQF//NExKwAAANIAAAAAEVNPKlSBI8hikTyZI1yJ1B4VzvoPQLT0HkKG8ZqcV7yG3Ik7juSakY4EM4dDweRsPZaVWTThPNz+MZXV1FYqVKIxDs5f5ioip0VEVn+i//zFBCp//NExP8ZATXsAHsScM///xTULM1s4szFRRtQsK1MQU1FMy4xMDBVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVV//NExO4YWTIEAHsQcFVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVV//NExN8SKY2wAHpElFVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVV\" type=\"audio/mpeg\" />\n",
       "                    Your browser does not support the audio element.\n",
       "                </audio>\n",
       "              "
      ],
      "text/plain": [
       "<IPython.lib.display.Audio object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "import audiofile\n",
    "import gtts\n",
    "import IPython\n",
    "\n",
    "\n",
    "sentences = [\n",
    "    'This is a notebook',\n",
    "    'This is wonderful',\n",
    "    'This is stupid',\n",
    "]\n",
    "\n",
    "tts_files = []\n",
    "for sentence in audeer.progress_bar(\n",
    "    sentences,\n",
    "    desc='Create files',\n",
    "):\n",
    "    tts_path = cache_path(f'{sentence}.mp3')\n",
    "    tts_files.append(tts_path)\n",
    "    if not os.path.exists(tts_path):\n",
    "        tts = gtts.gTTS(sentence)\n",
    "        tts.save(tts_path)\n",
    "\n",
    "for tts_file in tts_files:    \n",
    "    audio = IPython.display.Audio(tts_file)\n",
    "    IPython.display.display(audio)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a45599af",
   "metadata": {},
   "source": [
    "With help of the interface we can run the model directly on the list with file names. The interesting bit here is the valence score, which - despite a neutral sounding voice - is high for the sentence including the word 'wonderful' and zero for the sentence including the word 'stupid'. This proves the sensibility of the model to sentiment."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "d7b884f6",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "                                                                                                    \r"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th>arousal</th>\n",
       "      <th>dominance</th>\n",
       "      <th>valence</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>file</th>\n",
       "      <th>start</th>\n",
       "      <th>end</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>cache/This is a notebook.mp3</th>\n",
       "      <th>0 days</th>\n",
       "      <th>0 days 00:00:01.584000</th>\n",
       "      <td>0.50</td>\n",
       "      <td>0.54</td>\n",
       "      <td>0.53</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>cache/This is wonderful.mp3</th>\n",
       "      <th>0 days</th>\n",
       "      <th>0 days 00:00:01.632000</th>\n",
       "      <td>0.55</td>\n",
       "      <td>0.65</td>\n",
       "      <td>0.90</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>cache/This is stupid.mp3</th>\n",
       "      <th>0 days</th>\n",
       "      <th>0 days 00:00:01.560000</th>\n",
       "      <td>0.55</td>\n",
       "      <td>0.66</td>\n",
       "      <td>-0.00</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                                            arousal  \\\n",
       "file                         start  end                               \n",
       "cache/This is a notebook.mp3 0 days 0 days 00:00:01.584000     0.50   \n",
       "cache/This is wonderful.mp3  0 days 0 days 00:00:01.632000     0.55   \n",
       "cache/This is stupid.mp3     0 days 0 days 00:00:01.560000     0.55   \n",
       "\n",
       "                                                            dominance  valence  \n",
       "file                         start  end                                         \n",
       "cache/This is a notebook.mp3 0 days 0 days 00:00:01.584000       0.54     0.53  \n",
       "cache/This is wonderful.mp3  0 days 0 days 00:00:01.632000       0.65     0.90  \n",
       "cache/This is stupid.mp3     0 days 0 days 00:00:01.560000       0.66    -0.00  "
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "interface.process_files(tts_files).round(2)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1d7a51f4",
   "metadata": {},
   "source": [
    "## Use embeddings to train categorical emotion model"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "68188d94",
   "metadata": {},
   "source": [
    "The Berlin Database of Emotional Speech ([Emo-DB](emodb.bilderbar.info)) is a well known speech databases with emotional utterances by different actors. Since it is in German and contains non-emotional sentences, it will allow us to investigate if a) the model works on languages other than English b) it can also predict valence from paralinguistics only. To get the database we use [audb](https://github.com/audeering/audb), a package to manage annotated media files. When we load the data, audb takes care of caching and converting the files to the desired format. Annotations are organized as tables in [audformat](https://github.com/audeering/audformat). In the following experiment we use columns with speaker information and emotional labels."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "4ef6e326",
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Get:   emodb v1.1.1\n",
      "Cache: /media/jwagner/Data/Git/how-to/w2v2-how-to/cache/emodb/1.1.1/fe182b91\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "                                                                                                    \r"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>emotion</th>\n",
       "      <th>speaker</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>file</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>wav/03a01Fa.wav</th>\n",
       "      <td>happiness</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>wav/03a01Nc.wav</th>\n",
       "      <td>neutral</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>wav/03a01Wa.wav</th>\n",
       "      <td>anger</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>wav/03a02Fc.wav</th>\n",
       "      <td>happiness</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>wav/03a02Nc.wav</th>\n",
       "      <td>neutral</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>wav/16b10Lb.wav</th>\n",
       "      <td>boredom</td>\n",
       "      <td>16</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>wav/16b10Tb.wav</th>\n",
       "      <td>sadness</td>\n",
       "      <td>16</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>wav/16b10Td.wav</th>\n",
       "      <td>sadness</td>\n",
       "      <td>16</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>wav/16b10Wa.wav</th>\n",
       "      <td>anger</td>\n",
       "      <td>16</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>wav/16b10Wb.wav</th>\n",
       "      <td>anger</td>\n",
       "      <td>16</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>535 rows × 2 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                   emotion speaker\n",
       "file                              \n",
       "wav/03a01Fa.wav  happiness       3\n",
       "wav/03a01Nc.wav    neutral       3\n",
       "wav/03a01Wa.wav      anger       3\n",
       "wav/03a02Fc.wav  happiness       3\n",
       "wav/03a02Nc.wav    neutral       3\n",
       "...                    ...     ...\n",
       "wav/16b10Lb.wav    boredom      16\n",
       "wav/16b10Tb.wav    sadness      16\n",
       "wav/16b10Td.wav    sadness      16\n",
       "wav/16b10Wa.wav      anger      16\n",
       "wav/16b10Wb.wav      anger      16\n",
       "\n",
       "[535 rows x 2 columns]"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import audb\n",
    "import audformat\n",
    "\n",
    "\n",
    "db = audb.load(\n",
    "    'emodb',\n",
    "    version='1.1.1',\n",
    "    format='wav',\n",
    "    mixdown=True,\n",
    "    sampling_rate=16000,\n",
    "    full_path=False,    \n",
    "    cache_root=cache_root,\n",
    "    verbose=True,\n",
    ")\n",
    "speaker = db['files']['speaker'].get()\n",
    "emotion = db['emotion']['emotion'].get()\n",
    "\n",
    "audformat.utils.concat([emotion, speaker])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "04a2c177",
   "metadata": {},
   "source": [
    "Since the model was fine-tuned on dimensional scores and not categerical labels, a fine-tuning to the new targets is required. Therefore we use the output of 'hidden_states' to access to the latent space of the model, also called embeddings. We assume that they provide a compact representation of the emotional content embedded in a speech signal. The interface we created earlier offers us a convenient way to run the model directly on the index of a table. Note that we cache the feature once extracted to avoid re-calculation."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "7187692b",
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "                                                                                                    \r"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th>hidden_states-0</th>\n",
       "      <th>hidden_states-1</th>\n",
       "      <th>hidden_states-2</th>\n",
       "      <th>hidden_states-3</th>\n",
       "      <th>hidden_states-4</th>\n",
       "      <th>hidden_states-5</th>\n",
       "      <th>hidden_states-6</th>\n",
       "      <th>hidden_states-7</th>\n",
       "      <th>hidden_states-8</th>\n",
       "      <th>hidden_states-9</th>\n",
       "      <th>...</th>\n",
       "      <th>hidden_states-1014</th>\n",
       "      <th>hidden_states-1015</th>\n",
       "      <th>hidden_states-1016</th>\n",
       "      <th>hidden_states-1017</th>\n",
       "      <th>hidden_states-1018</th>\n",
       "      <th>hidden_states-1019</th>\n",
       "      <th>hidden_states-1020</th>\n",
       "      <th>hidden_states-1021</th>\n",
       "      <th>hidden_states-1022</th>\n",
       "      <th>hidden_states-1023</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>file</th>\n",
       "      <th>start</th>\n",
       "      <th>end</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>wav/03a01Fa.wav</th>\n",
       "      <th>0 days</th>\n",
       "      <th>0 days 00:00:01.898250</th>\n",
       "      <td>-0.007891</td>\n",
       "      <td>0.006736</td>\n",
       "      <td>-0.005898</td>\n",
       "      <td>-0.006566</td>\n",
       "      <td>-0.004559</td>\n",
       "      <td>-0.008148</td>\n",
       "      <td>0.168698</td>\n",
       "      <td>0.006812</td>\n",
       "      <td>0.113164</td>\n",
       "      <td>-0.001617</td>\n",
       "      <td>...</td>\n",
       "      <td>0.003587</td>\n",
       "      <td>0.011148</td>\n",
       "      <td>-0.050405</td>\n",
       "      <td>0.008359</td>\n",
       "      <td>0.034025</td>\n",
       "      <td>-0.064220</td>\n",
       "      <td>0.139168</td>\n",
       "      <td>0.007794</td>\n",
       "      <td>0.007637</td>\n",
       "      <td>0.006209</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>wav/03a01Nc.wav</th>\n",
       "      <th>0 days</th>\n",
       "      <th>0 days 00:00:01.611250</th>\n",
       "      <td>-0.007629</td>\n",
       "      <td>0.006001</td>\n",
       "      <td>-0.007628</td>\n",
       "      <td>-0.006588</td>\n",
       "      <td>-0.005104</td>\n",
       "      <td>-0.008472</td>\n",
       "      <td>0.199000</td>\n",
       "      <td>0.006391</td>\n",
       "      <td>0.071222</td>\n",
       "      <td>-0.001760</td>\n",
       "      <td>...</td>\n",
       "      <td>0.003145</td>\n",
       "      <td>0.011300</td>\n",
       "      <td>-0.018726</td>\n",
       "      <td>0.008156</td>\n",
       "      <td>0.015245</td>\n",
       "      <td>-0.056928</td>\n",
       "      <td>0.083291</td>\n",
       "      <td>0.007600</td>\n",
       "      <td>0.007721</td>\n",
       "      <td>0.005684</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>wav/03a01Wa.wav</th>\n",
       "      <th>0 days</th>\n",
       "      <th>0 days 00:00:01.877812500</th>\n",
       "      <td>-0.007874</td>\n",
       "      <td>0.006600</td>\n",
       "      <td>-0.008576</td>\n",
       "      <td>-0.007047</td>\n",
       "      <td>-0.004646</td>\n",
       "      <td>-0.008129</td>\n",
       "      <td>0.237001</td>\n",
       "      <td>0.007115</td>\n",
       "      <td>0.188141</td>\n",
       "      <td>-0.001287</td>\n",
       "      <td>...</td>\n",
       "      <td>0.003982</td>\n",
       "      <td>0.011190</td>\n",
       "      <td>-0.036640</td>\n",
       "      <td>0.008163</td>\n",
       "      <td>0.062221</td>\n",
       "      <td>-0.112500</td>\n",
       "      <td>0.302140</td>\n",
       "      <td>0.007813</td>\n",
       "      <td>0.007812</td>\n",
       "      <td>0.007133</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>wav/03a02Fc.wav</th>\n",
       "      <th>0 days</th>\n",
       "      <th>0 days 00:00:02.006250</th>\n",
       "      <td>-0.007407</td>\n",
       "      <td>0.006326</td>\n",
       "      <td>-0.010519</td>\n",
       "      <td>-0.005899</td>\n",
       "      <td>-0.004216</td>\n",
       "      <td>-0.007912</td>\n",
       "      <td>0.109786</td>\n",
       "      <td>0.006558</td>\n",
       "      <td>0.088926</td>\n",
       "      <td>-0.001507</td>\n",
       "      <td>...</td>\n",
       "      <td>0.003656</td>\n",
       "      <td>0.011320</td>\n",
       "      <td>-0.041450</td>\n",
       "      <td>0.008284</td>\n",
       "      <td>-0.010972</td>\n",
       "      <td>-0.139131</td>\n",
       "      <td>0.366497</td>\n",
       "      <td>0.007560</td>\n",
       "      <td>0.008487</td>\n",
       "      <td>0.004726</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>wav/03a02Nc.wav</th>\n",
       "      <th>0 days</th>\n",
       "      <th>0 days 00:00:01.439812500</th>\n",
       "      <td>-0.007313</td>\n",
       "      <td>0.006525</td>\n",
       "      <td>-0.010693</td>\n",
       "      <td>-0.004869</td>\n",
       "      <td>-0.004605</td>\n",
       "      <td>-0.008149</td>\n",
       "      <td>0.122609</td>\n",
       "      <td>0.006291</td>\n",
       "      <td>0.032912</td>\n",
       "      <td>-0.001508</td>\n",
       "      <td>...</td>\n",
       "      <td>0.003650</td>\n",
       "      <td>0.010983</td>\n",
       "      <td>-0.058476</td>\n",
       "      <td>0.008078</td>\n",
       "      <td>-0.008553</td>\n",
       "      <td>-0.000632</td>\n",
       "      <td>0.296581</td>\n",
       "      <td>0.007574</td>\n",
       "      <td>0.008161</td>\n",
       "      <td>0.005591</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <th>...</th>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>wav/16b10Lb.wav</th>\n",
       "      <th>0 days</th>\n",
       "      <th>0 days 00:00:03.442687500</th>\n",
       "      <td>-0.007212</td>\n",
       "      <td>0.006251</td>\n",
       "      <td>-0.006754</td>\n",
       "      <td>-0.006353</td>\n",
       "      <td>-0.004147</td>\n",
       "      <td>-0.007847</td>\n",
       "      <td>0.023312</td>\n",
       "      <td>0.007121</td>\n",
       "      <td>0.040396</td>\n",
       "      <td>-0.001996</td>\n",
       "      <td>...</td>\n",
       "      <td>0.003988</td>\n",
       "      <td>0.011588</td>\n",
       "      <td>-0.043439</td>\n",
       "      <td>0.008200</td>\n",
       "      <td>-0.027195</td>\n",
       "      <td>-0.063882</td>\n",
       "      <td>-0.051351</td>\n",
       "      <td>0.007314</td>\n",
       "      <td>0.007467</td>\n",
       "      <td>0.005633</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>wav/16b10Tb.wav</th>\n",
       "      <th>0 days</th>\n",
       "      <th>0 days 00:00:03.500625</th>\n",
       "      <td>-0.007274</td>\n",
       "      <td>0.006054</td>\n",
       "      <td>-0.006579</td>\n",
       "      <td>-0.006229</td>\n",
       "      <td>-0.004043</td>\n",
       "      <td>-0.005949</td>\n",
       "      <td>-0.005269</td>\n",
       "      <td>0.006985</td>\n",
       "      <td>-0.119985</td>\n",
       "      <td>-0.001811</td>\n",
       "      <td>...</td>\n",
       "      <td>0.003704</td>\n",
       "      <td>0.011652</td>\n",
       "      <td>-0.047649</td>\n",
       "      <td>0.008086</td>\n",
       "      <td>-0.013412</td>\n",
       "      <td>0.171493</td>\n",
       "      <td>-0.094538</td>\n",
       "      <td>0.007363</td>\n",
       "      <td>0.006186</td>\n",
       "      <td>0.005715</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>wav/16b10Td.wav</th>\n",
       "      <th>0 days</th>\n",
       "      <th>0 days 00:00:03.934187500</th>\n",
       "      <td>-0.007491</td>\n",
       "      <td>0.006536</td>\n",
       "      <td>-0.006867</td>\n",
       "      <td>-0.007232</td>\n",
       "      <td>-0.004012</td>\n",
       "      <td>-0.005708</td>\n",
       "      <td>-0.095532</td>\n",
       "      <td>0.006726</td>\n",
       "      <td>-0.191503</td>\n",
       "      <td>-0.002289</td>\n",
       "      <td>...</td>\n",
       "      <td>0.004344</td>\n",
       "      <td>0.011328</td>\n",
       "      <td>-0.064138</td>\n",
       "      <td>0.008109</td>\n",
       "      <td>-0.004146</td>\n",
       "      <td>0.111839</td>\n",
       "      <td>-0.037554</td>\n",
       "      <td>0.007447</td>\n",
       "      <td>0.006693</td>\n",
       "      <td>0.005184</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>wav/16b10Wa.wav</th>\n",
       "      <th>0 days</th>\n",
       "      <th>0 days 00:00:02.414125</th>\n",
       "      <td>-0.007130</td>\n",
       "      <td>0.005565</td>\n",
       "      <td>-0.006171</td>\n",
       "      <td>-0.005184</td>\n",
       "      <td>-0.005346</td>\n",
       "      <td>-0.009037</td>\n",
       "      <td>0.145889</td>\n",
       "      <td>0.006982</td>\n",
       "      <td>0.221980</td>\n",
       "      <td>-0.001282</td>\n",
       "      <td>...</td>\n",
       "      <td>0.004867</td>\n",
       "      <td>0.011788</td>\n",
       "      <td>-0.078933</td>\n",
       "      <td>0.008273</td>\n",
       "      <td>0.015033</td>\n",
       "      <td>0.033136</td>\n",
       "      <td>0.169886</td>\n",
       "      <td>0.007620</td>\n",
       "      <td>0.008510</td>\n",
       "      <td>0.006658</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>wav/16b10Wb.wav</th>\n",
       "      <th>0 days</th>\n",
       "      <th>0 days 00:00:02.522499999</th>\n",
       "      <td>-0.007292</td>\n",
       "      <td>0.005096</td>\n",
       "      <td>-0.007644</td>\n",
       "      <td>-0.006620</td>\n",
       "      <td>-0.004644</td>\n",
       "      <td>-0.007908</td>\n",
       "      <td>0.063259</td>\n",
       "      <td>0.007021</td>\n",
       "      <td>0.192563</td>\n",
       "      <td>-0.001140</td>\n",
       "      <td>...</td>\n",
       "      <td>0.004809</td>\n",
       "      <td>0.011697</td>\n",
       "      <td>-0.079433</td>\n",
       "      <td>0.008337</td>\n",
       "      <td>0.015448</td>\n",
       "      <td>0.015552</td>\n",
       "      <td>0.018748</td>\n",
       "      <td>0.007442</td>\n",
       "      <td>0.008904</td>\n",
       "      <td>0.007718</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>535 rows × 1024 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                                                  hidden_states-0  \\\n",
       "file            start  end                                          \n",
       "wav/03a01Fa.wav 0 days 0 days 00:00:01.898250           -0.007891   \n",
       "wav/03a01Nc.wav 0 days 0 days 00:00:01.611250           -0.007629   \n",
       "wav/03a01Wa.wav 0 days 0 days 00:00:01.877812500        -0.007874   \n",
       "wav/03a02Fc.wav 0 days 0 days 00:00:02.006250           -0.007407   \n",
       "wav/03a02Nc.wav 0 days 0 days 00:00:01.439812500        -0.007313   \n",
       "...                                                           ...   \n",
       "wav/16b10Lb.wav 0 days 0 days 00:00:03.442687500        -0.007212   \n",
       "wav/16b10Tb.wav 0 days 0 days 00:00:03.500625           -0.007274   \n",
       "wav/16b10Td.wav 0 days 0 days 00:00:03.934187500        -0.007491   \n",
       "wav/16b10Wa.wav 0 days 0 days 00:00:02.414125           -0.007130   \n",
       "wav/16b10Wb.wav 0 days 0 days 00:00:02.522499999        -0.007292   \n",
       "\n",
       "                                                  hidden_states-1  \\\n",
       "file            start  end                                          \n",
       "wav/03a01Fa.wav 0 days 0 days 00:00:01.898250            0.006736   \n",
       "wav/03a01Nc.wav 0 days 0 days 00:00:01.611250            0.006001   \n",
       "wav/03a01Wa.wav 0 days 0 days 00:00:01.877812500         0.006600   \n",
       "wav/03a02Fc.wav 0 days 0 days 00:00:02.006250            0.006326   \n",
       "wav/03a02Nc.wav 0 days 0 days 00:00:01.439812500         0.006525   \n",
       "...                                                           ...   \n",
       "wav/16b10Lb.wav 0 days 0 days 00:00:03.442687500         0.006251   \n",
       "wav/16b10Tb.wav 0 days 0 days 00:00:03.500625            0.006054   \n",
       "wav/16b10Td.wav 0 days 0 days 00:00:03.934187500         0.006536   \n",
       "wav/16b10Wa.wav 0 days 0 days 00:00:02.414125            0.005565   \n",
       "wav/16b10Wb.wav 0 days 0 days 00:00:02.522499999         0.005096   \n",
       "\n",
       "                                                  hidden_states-2  \\\n",
       "file            start  end                                          \n",
       "wav/03a01Fa.wav 0 days 0 days 00:00:01.898250           -0.005898   \n",
       "wav/03a01Nc.wav 0 days 0 days 00:00:01.611250           -0.007628   \n",
       "wav/03a01Wa.wav 0 days 0 days 00:00:01.877812500        -0.008576   \n",
       "wav/03a02Fc.wav 0 days 0 days 00:00:02.006250           -0.010519   \n",
       "wav/03a02Nc.wav 0 days 0 days 00:00:01.439812500        -0.010693   \n",
       "...                                                           ...   \n",
       "wav/16b10Lb.wav 0 days 0 days 00:00:03.442687500        -0.006754   \n",
       "wav/16b10Tb.wav 0 days 0 days 00:00:03.500625           -0.006579   \n",
       "wav/16b10Td.wav 0 days 0 days 00:00:03.934187500        -0.006867   \n",
       "wav/16b10Wa.wav 0 days 0 days 00:00:02.414125           -0.006171   \n",
       "wav/16b10Wb.wav 0 days 0 days 00:00:02.522499999        -0.007644   \n",
       "\n",
       "                                                  hidden_states-3  \\\n",
       "file            start  end                                          \n",
       "wav/03a01Fa.wav 0 days 0 days 00:00:01.898250           -0.006566   \n",
       "wav/03a01Nc.wav 0 days 0 days 00:00:01.611250           -0.006588   \n",
       "wav/03a01Wa.wav 0 days 0 days 00:00:01.877812500        -0.007047   \n",
       "wav/03a02Fc.wav 0 days 0 days 00:00:02.006250           -0.005899   \n",
       "wav/03a02Nc.wav 0 days 0 days 00:00:01.439812500        -0.004869   \n",
       "...                                                           ...   \n",
       "wav/16b10Lb.wav 0 days 0 days 00:00:03.442687500        -0.006353   \n",
       "wav/16b10Tb.wav 0 days 0 days 00:00:03.500625           -0.006229   \n",
       "wav/16b10Td.wav 0 days 0 days 00:00:03.934187500        -0.007232   \n",
       "wav/16b10Wa.wav 0 days 0 days 00:00:02.414125           -0.005184   \n",
       "wav/16b10Wb.wav 0 days 0 days 00:00:02.522499999        -0.006620   \n",
       "\n",
       "                                                  hidden_states-4  \\\n",
       "file            start  end                                          \n",
       "wav/03a01Fa.wav 0 days 0 days 00:00:01.898250           -0.004559   \n",
       "wav/03a01Nc.wav 0 days 0 days 00:00:01.611250           -0.005104   \n",
       "wav/03a01Wa.wav 0 days 0 days 00:00:01.877812500        -0.004646   \n",
       "wav/03a02Fc.wav 0 days 0 days 00:00:02.006250           -0.004216   \n",
       "wav/03a02Nc.wav 0 days 0 days 00:00:01.439812500        -0.004605   \n",
       "...                                                           ...   \n",
       "wav/16b10Lb.wav 0 days 0 days 00:00:03.442687500        -0.004147   \n",
       "wav/16b10Tb.wav 0 days 0 days 00:00:03.500625           -0.004043   \n",
       "wav/16b10Td.wav 0 days 0 days 00:00:03.934187500        -0.004012   \n",
       "wav/16b10Wa.wav 0 days 0 days 00:00:02.414125           -0.005346   \n",
       "wav/16b10Wb.wav 0 days 0 days 00:00:02.522499999        -0.004644   \n",
       "\n",
       "                                                  hidden_states-5  \\\n",
       "file            start  end                                          \n",
       "wav/03a01Fa.wav 0 days 0 days 00:00:01.898250           -0.008148   \n",
       "wav/03a01Nc.wav 0 days 0 days 00:00:01.611250           -0.008472   \n",
       "wav/03a01Wa.wav 0 days 0 days 00:00:01.877812500        -0.008129   \n",
       "wav/03a02Fc.wav 0 days 0 days 00:00:02.006250           -0.007912   \n",
       "wav/03a02Nc.wav 0 days 0 days 00:00:01.439812500        -0.008149   \n",
       "...                                                           ...   \n",
       "wav/16b10Lb.wav 0 days 0 days 00:00:03.442687500        -0.007847   \n",
       "wav/16b10Tb.wav 0 days 0 days 00:00:03.500625           -0.005949   \n",
       "wav/16b10Td.wav 0 days 0 days 00:00:03.934187500        -0.005708   \n",
       "wav/16b10Wa.wav 0 days 0 days 00:00:02.414125           -0.009037   \n",
       "wav/16b10Wb.wav 0 days 0 days 00:00:02.522499999        -0.007908   \n",
       "\n",
       "                                                  hidden_states-6  \\\n",
       "file            start  end                                          \n",
       "wav/03a01Fa.wav 0 days 0 days 00:00:01.898250            0.168698   \n",
       "wav/03a01Nc.wav 0 days 0 days 00:00:01.611250            0.199000   \n",
       "wav/03a01Wa.wav 0 days 0 days 00:00:01.877812500         0.237001   \n",
       "wav/03a02Fc.wav 0 days 0 days 00:00:02.006250            0.109786   \n",
       "wav/03a02Nc.wav 0 days 0 days 00:00:01.439812500         0.122609   \n",
       "...                                                           ...   \n",
       "wav/16b10Lb.wav 0 days 0 days 00:00:03.442687500         0.023312   \n",
       "wav/16b10Tb.wav 0 days 0 days 00:00:03.500625           -0.005269   \n",
       "wav/16b10Td.wav 0 days 0 days 00:00:03.934187500        -0.095532   \n",
       "wav/16b10Wa.wav 0 days 0 days 00:00:02.414125            0.145889   \n",
       "wav/16b10Wb.wav 0 days 0 days 00:00:02.522499999         0.063259   \n",
       "\n",
       "                                                  hidden_states-7  \\\n",
       "file            start  end                                          \n",
       "wav/03a01Fa.wav 0 days 0 days 00:00:01.898250            0.006812   \n",
       "wav/03a01Nc.wav 0 days 0 days 00:00:01.611250            0.006391   \n",
       "wav/03a01Wa.wav 0 days 0 days 00:00:01.877812500         0.007115   \n",
       "wav/03a02Fc.wav 0 days 0 days 00:00:02.006250            0.006558   \n",
       "wav/03a02Nc.wav 0 days 0 days 00:00:01.439812500         0.006291   \n",
       "...                                                           ...   \n",
       "wav/16b10Lb.wav 0 days 0 days 00:00:03.442687500         0.007121   \n",
       "wav/16b10Tb.wav 0 days 0 days 00:00:03.500625            0.006985   \n",
       "wav/16b10Td.wav 0 days 0 days 00:00:03.934187500         0.006726   \n",
       "wav/16b10Wa.wav 0 days 0 days 00:00:02.414125            0.006982   \n",
       "wav/16b10Wb.wav 0 days 0 days 00:00:02.522499999         0.007021   \n",
       "\n",
       "                                                  hidden_states-8  \\\n",
       "file            start  end                                          \n",
       "wav/03a01Fa.wav 0 days 0 days 00:00:01.898250            0.113164   \n",
       "wav/03a01Nc.wav 0 days 0 days 00:00:01.611250            0.071222   \n",
       "wav/03a01Wa.wav 0 days 0 days 00:00:01.877812500         0.188141   \n",
       "wav/03a02Fc.wav 0 days 0 days 00:00:02.006250            0.088926   \n",
       "wav/03a02Nc.wav 0 days 0 days 00:00:01.439812500         0.032912   \n",
       "...                                                           ...   \n",
       "wav/16b10Lb.wav 0 days 0 days 00:00:03.442687500         0.040396   \n",
       "wav/16b10Tb.wav 0 days 0 days 00:00:03.500625           -0.119985   \n",
       "wav/16b10Td.wav 0 days 0 days 00:00:03.934187500        -0.191503   \n",
       "wav/16b10Wa.wav 0 days 0 days 00:00:02.414125            0.221980   \n",
       "wav/16b10Wb.wav 0 days 0 days 00:00:02.522499999         0.192563   \n",
       "\n",
       "                                                  hidden_states-9  ...  \\\n",
       "file            start  end                                         ...   \n",
       "wav/03a01Fa.wav 0 days 0 days 00:00:01.898250           -0.001617  ...   \n",
       "wav/03a01Nc.wav 0 days 0 days 00:00:01.611250           -0.001760  ...   \n",
       "wav/03a01Wa.wav 0 days 0 days 00:00:01.877812500        -0.001287  ...   \n",
       "wav/03a02Fc.wav 0 days 0 days 00:00:02.006250           -0.001507  ...   \n",
       "wav/03a02Nc.wav 0 days 0 days 00:00:01.439812500        -0.001508  ...   \n",
       "...                                                           ...  ...   \n",
       "wav/16b10Lb.wav 0 days 0 days 00:00:03.442687500        -0.001996  ...   \n",
       "wav/16b10Tb.wav 0 days 0 days 00:00:03.500625           -0.001811  ...   \n",
       "wav/16b10Td.wav 0 days 0 days 00:00:03.934187500        -0.002289  ...   \n",
       "wav/16b10Wa.wav 0 days 0 days 00:00:02.414125           -0.001282  ...   \n",
       "wav/16b10Wb.wav 0 days 0 days 00:00:02.522499999        -0.001140  ...   \n",
       "\n",
       "                                                  hidden_states-1014  \\\n",
       "file            start  end                                             \n",
       "wav/03a01Fa.wav 0 days 0 days 00:00:01.898250               0.003587   \n",
       "wav/03a01Nc.wav 0 days 0 days 00:00:01.611250               0.003145   \n",
       "wav/03a01Wa.wav 0 days 0 days 00:00:01.877812500            0.003982   \n",
       "wav/03a02Fc.wav 0 days 0 days 00:00:02.006250               0.003656   \n",
       "wav/03a02Nc.wav 0 days 0 days 00:00:01.439812500            0.003650   \n",
       "...                                                              ...   \n",
       "wav/16b10Lb.wav 0 days 0 days 00:00:03.442687500            0.003988   \n",
       "wav/16b10Tb.wav 0 days 0 days 00:00:03.500625               0.003704   \n",
       "wav/16b10Td.wav 0 days 0 days 00:00:03.934187500            0.004344   \n",
       "wav/16b10Wa.wav 0 days 0 days 00:00:02.414125               0.004867   \n",
       "wav/16b10Wb.wav 0 days 0 days 00:00:02.522499999            0.004809   \n",
       "\n",
       "                                                  hidden_states-1015  \\\n",
       "file            start  end                                             \n",
       "wav/03a01Fa.wav 0 days 0 days 00:00:01.898250               0.011148   \n",
       "wav/03a01Nc.wav 0 days 0 days 00:00:01.611250               0.011300   \n",
       "wav/03a01Wa.wav 0 days 0 days 00:00:01.877812500            0.011190   \n",
       "wav/03a02Fc.wav 0 days 0 days 00:00:02.006250               0.011320   \n",
       "wav/03a02Nc.wav 0 days 0 days 00:00:01.439812500            0.010983   \n",
       "...                                                              ...   \n",
       "wav/16b10Lb.wav 0 days 0 days 00:00:03.442687500            0.011588   \n",
       "wav/16b10Tb.wav 0 days 0 days 00:00:03.500625               0.011652   \n",
       "wav/16b10Td.wav 0 days 0 days 00:00:03.934187500            0.011328   \n",
       "wav/16b10Wa.wav 0 days 0 days 00:00:02.414125               0.011788   \n",
       "wav/16b10Wb.wav 0 days 0 days 00:00:02.522499999            0.011697   \n",
       "\n",
       "                                                  hidden_states-1016  \\\n",
       "file            start  end                                             \n",
       "wav/03a01Fa.wav 0 days 0 days 00:00:01.898250              -0.050405   \n",
       "wav/03a01Nc.wav 0 days 0 days 00:00:01.611250              -0.018726   \n",
       "wav/03a01Wa.wav 0 days 0 days 00:00:01.877812500           -0.036640   \n",
       "wav/03a02Fc.wav 0 days 0 days 00:00:02.006250              -0.041450   \n",
       "wav/03a02Nc.wav 0 days 0 days 00:00:01.439812500           -0.058476   \n",
       "...                                                              ...   \n",
       "wav/16b10Lb.wav 0 days 0 days 00:00:03.442687500           -0.043439   \n",
       "wav/16b10Tb.wav 0 days 0 days 00:00:03.500625              -0.047649   \n",
       "wav/16b10Td.wav 0 days 0 days 00:00:03.934187500           -0.064138   \n",
       "wav/16b10Wa.wav 0 days 0 days 00:00:02.414125              -0.078933   \n",
       "wav/16b10Wb.wav 0 days 0 days 00:00:02.522499999           -0.079433   \n",
       "\n",
       "                                                  hidden_states-1017  \\\n",
       "file            start  end                                             \n",
       "wav/03a01Fa.wav 0 days 0 days 00:00:01.898250               0.008359   \n",
       "wav/03a01Nc.wav 0 days 0 days 00:00:01.611250               0.008156   \n",
       "wav/03a01Wa.wav 0 days 0 days 00:00:01.877812500            0.008163   \n",
       "wav/03a02Fc.wav 0 days 0 days 00:00:02.006250               0.008284   \n",
       "wav/03a02Nc.wav 0 days 0 days 00:00:01.439812500            0.008078   \n",
       "...                                                              ...   \n",
       "wav/16b10Lb.wav 0 days 0 days 00:00:03.442687500            0.008200   \n",
       "wav/16b10Tb.wav 0 days 0 days 00:00:03.500625               0.008086   \n",
       "wav/16b10Td.wav 0 days 0 days 00:00:03.934187500            0.008109   \n",
       "wav/16b10Wa.wav 0 days 0 days 00:00:02.414125               0.008273   \n",
       "wav/16b10Wb.wav 0 days 0 days 00:00:02.522499999            0.008337   \n",
       "\n",
       "                                                  hidden_states-1018  \\\n",
       "file            start  end                                             \n",
       "wav/03a01Fa.wav 0 days 0 days 00:00:01.898250               0.034025   \n",
       "wav/03a01Nc.wav 0 days 0 days 00:00:01.611250               0.015245   \n",
       "wav/03a01Wa.wav 0 days 0 days 00:00:01.877812500            0.062221   \n",
       "wav/03a02Fc.wav 0 days 0 days 00:00:02.006250              -0.010972   \n",
       "wav/03a02Nc.wav 0 days 0 days 00:00:01.439812500           -0.008553   \n",
       "...                                                              ...   \n",
       "wav/16b10Lb.wav 0 days 0 days 00:00:03.442687500           -0.027195   \n",
       "wav/16b10Tb.wav 0 days 0 days 00:00:03.500625              -0.013412   \n",
       "wav/16b10Td.wav 0 days 0 days 00:00:03.934187500           -0.004146   \n",
       "wav/16b10Wa.wav 0 days 0 days 00:00:02.414125               0.015033   \n",
       "wav/16b10Wb.wav 0 days 0 days 00:00:02.522499999            0.015448   \n",
       "\n",
       "                                                  hidden_states-1019  \\\n",
       "file            start  end                                             \n",
       "wav/03a01Fa.wav 0 days 0 days 00:00:01.898250              -0.064220   \n",
       "wav/03a01Nc.wav 0 days 0 days 00:00:01.611250              -0.056928   \n",
       "wav/03a01Wa.wav 0 days 0 days 00:00:01.877812500           -0.112500   \n",
       "wav/03a02Fc.wav 0 days 0 days 00:00:02.006250              -0.139131   \n",
       "wav/03a02Nc.wav 0 days 0 days 00:00:01.439812500           -0.000632   \n",
       "...                                                              ...   \n",
       "wav/16b10Lb.wav 0 days 0 days 00:00:03.442687500           -0.063882   \n",
       "wav/16b10Tb.wav 0 days 0 days 00:00:03.500625               0.171493   \n",
       "wav/16b10Td.wav 0 days 0 days 00:00:03.934187500            0.111839   \n",
       "wav/16b10Wa.wav 0 days 0 days 00:00:02.414125               0.033136   \n",
       "wav/16b10Wb.wav 0 days 0 days 00:00:02.522499999            0.015552   \n",
       "\n",
       "                                                  hidden_states-1020  \\\n",
       "file            start  end                                             \n",
       "wav/03a01Fa.wav 0 days 0 days 00:00:01.898250               0.139168   \n",
       "wav/03a01Nc.wav 0 days 0 days 00:00:01.611250               0.083291   \n",
       "wav/03a01Wa.wav 0 days 0 days 00:00:01.877812500            0.302140   \n",
       "wav/03a02Fc.wav 0 days 0 days 00:00:02.006250               0.366497   \n",
       "wav/03a02Nc.wav 0 days 0 days 00:00:01.439812500            0.296581   \n",
       "...                                                              ...   \n",
       "wav/16b10Lb.wav 0 days 0 days 00:00:03.442687500           -0.051351   \n",
       "wav/16b10Tb.wav 0 days 0 days 00:00:03.500625              -0.094538   \n",
       "wav/16b10Td.wav 0 days 0 days 00:00:03.934187500           -0.037554   \n",
       "wav/16b10Wa.wav 0 days 0 days 00:00:02.414125               0.169886   \n",
       "wav/16b10Wb.wav 0 days 0 days 00:00:02.522499999            0.018748   \n",
       "\n",
       "                                                  hidden_states-1021  \\\n",
       "file            start  end                                             \n",
       "wav/03a01Fa.wav 0 days 0 days 00:00:01.898250               0.007794   \n",
       "wav/03a01Nc.wav 0 days 0 days 00:00:01.611250               0.007600   \n",
       "wav/03a01Wa.wav 0 days 0 days 00:00:01.877812500            0.007813   \n",
       "wav/03a02Fc.wav 0 days 0 days 00:00:02.006250               0.007560   \n",
       "wav/03a02Nc.wav 0 days 0 days 00:00:01.439812500            0.007574   \n",
       "...                                                              ...   \n",
       "wav/16b10Lb.wav 0 days 0 days 00:00:03.442687500            0.007314   \n",
       "wav/16b10Tb.wav 0 days 0 days 00:00:03.500625               0.007363   \n",
       "wav/16b10Td.wav 0 days 0 days 00:00:03.934187500            0.007447   \n",
       "wav/16b10Wa.wav 0 days 0 days 00:00:02.414125               0.007620   \n",
       "wav/16b10Wb.wav 0 days 0 days 00:00:02.522499999            0.007442   \n",
       "\n",
       "                                                  hidden_states-1022  \\\n",
       "file            start  end                                             \n",
       "wav/03a01Fa.wav 0 days 0 days 00:00:01.898250               0.007637   \n",
       "wav/03a01Nc.wav 0 days 0 days 00:00:01.611250               0.007721   \n",
       "wav/03a01Wa.wav 0 days 0 days 00:00:01.877812500            0.007812   \n",
       "wav/03a02Fc.wav 0 days 0 days 00:00:02.006250               0.008487   \n",
       "wav/03a02Nc.wav 0 days 0 days 00:00:01.439812500            0.008161   \n",
       "...                                                              ...   \n",
       "wav/16b10Lb.wav 0 days 0 days 00:00:03.442687500            0.007467   \n",
       "wav/16b10Tb.wav 0 days 0 days 00:00:03.500625               0.006186   \n",
       "wav/16b10Td.wav 0 days 0 days 00:00:03.934187500            0.006693   \n",
       "wav/16b10Wa.wav 0 days 0 days 00:00:02.414125               0.008510   \n",
       "wav/16b10Wb.wav 0 days 0 days 00:00:02.522499999            0.008904   \n",
       "\n",
       "                                                  hidden_states-1023  \n",
       "file            start  end                                            \n",
       "wav/03a01Fa.wav 0 days 0 days 00:00:01.898250               0.006209  \n",
       "wav/03a01Nc.wav 0 days 0 days 00:00:01.611250               0.005684  \n",
       "wav/03a01Wa.wav 0 days 0 days 00:00:01.877812500            0.007133  \n",
       "wav/03a02Fc.wav 0 days 0 days 00:00:02.006250               0.004726  \n",
       "wav/03a02Nc.wav 0 days 0 days 00:00:01.439812500            0.005591  \n",
       "...                                                              ...  \n",
       "wav/16b10Lb.wav 0 days 0 days 00:00:03.442687500            0.005633  \n",
       "wav/16b10Tb.wav 0 days 0 days 00:00:03.500625               0.005715  \n",
       "wav/16b10Td.wav 0 days 0 days 00:00:03.934187500            0.005184  \n",
       "wav/16b10Wa.wav 0 days 0 days 00:00:02.414125               0.006658  \n",
       "wav/16b10Wb.wav 0 days 0 days 00:00:02.522499999            0.007718  \n",
       "\n",
       "[535 rows x 1024 columns]"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import pandas as pd\n",
    "\n",
    "\n",
    "hidden_states = audinterface.Feature(\n",
    "    model.labels('hidden_states'),\n",
    "    process_func=model,\n",
    "    process_func_args={\n",
    "        'outputs': 'hidden_states',\n",
    "    },\n",
    "    sampling_rate=16000,    \n",
    "    resample=True,    \n",
    "    num_workers=5,\n",
    "    verbose=True,\n",
    ")\n",
    "\n",
    "features_w2v2 = hidden_states.process_index(\n",
    "    emotion.index,\n",
    "    root=db.root,\n",
    "    cache_root=audeer.path(cache_root, 'w2v2'),\n",
    ")\n",
    "features_w2v2"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5711f15e",
   "metadata": {},
   "source": [
    "As classifier, we use a Support Vector Classification from [scikit-learn](https://scikit-learn.org/). Since Emo-DB does not define an official train and test set, we apply leave-one-speaker-out, i.e. we predict each speaker individually after training on the other speakers. For this purpose, we define the following utility function."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "3e673547",
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.model_selection import LeaveOneGroupOut\n",
    "from sklearn.pipeline import make_pipeline\n",
    "from sklearn.preprocessing import StandardScaler\n",
    "from sklearn.svm import SVC\n",
    "\n",
    "\n",
    "# create classifier and grouping object\n",
    "clf = make_pipeline(\n",
    "    StandardScaler(), \n",
    "    SVC(gamma='auto'),\n",
    ")\n",
    "logo = LeaveOneGroupOut()\n",
    "\n",
    "def experiment(\n",
    "    features,\n",
    "    targets,\n",
    "    groups,\n",
    "):        \n",
    "    truths = []\n",
    "    preds = []\n",
    "    \n",
    "    # leave-one-speaker loop    \n",
    "    pbar = audeer.progress_bar(\n",
    "        total=len(groups.unique()),\n",
    "        desc='Run experiment',\n",
    "    )\n",
    "    for train_index, test_index in logo.split(\n",
    "        features, \n",
    "        targets, \n",
    "        groups=groups,\n",
    "    ):\n",
    "        train_x = features.iloc[train_index]\n",
    "        train_y = targets[train_index]\n",
    "        clf.fit(train_x, train_y)\n",
    "        \n",
    "        truth_x = features.iloc[test_index]\n",
    "        truth_y = targets[test_index]\n",
    "        predict_y = clf.predict(truth_x)\n",
    "        \n",
    "        truths.append(truth_y)\n",
    "        preds.append(predict_y)\n",
    "        \n",
    "        pbar.update()\n",
    "        \n",
    "    # combine speaker folds\n",
    "    truth = pd.concat(truths)\n",
    "    truth.name = 'truth'\n",
    "    pred = pd.Series(\n",
    "        np.concatenate(preds),\n",
    "        index=truth.index,\n",
    "        name='prediction',\n",
    "    )\n",
    "    \n",
    "    return truth, pred"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7e881ebc",
   "metadata": {},
   "source": [
    "After passing the features to the utility function, we get ground truth and predictions for every sentence in the database."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "f9d7e56e",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "                                                                                                    \r"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>truth</th>\n",
       "      <th>prediction</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>file</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>wav/03a01Fa.wav</th>\n",
       "      <td>happiness</td>\n",
       "      <td>happiness</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>wav/03a01Nc.wav</th>\n",
       "      <td>neutral</td>\n",
       "      <td>neutral</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>wav/03a01Wa.wav</th>\n",
       "      <td>anger</td>\n",
       "      <td>anger</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>wav/03a02Fc.wav</th>\n",
       "      <td>happiness</td>\n",
       "      <td>happiness</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>wav/03a02Nc.wav</th>\n",
       "      <td>neutral</td>\n",
       "      <td>neutral</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>wav/16b10Lb.wav</th>\n",
       "      <td>boredom</td>\n",
       "      <td>boredom</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>wav/16b10Tb.wav</th>\n",
       "      <td>sadness</td>\n",
       "      <td>sadness</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>wav/16b10Td.wav</th>\n",
       "      <td>sadness</td>\n",
       "      <td>sadness</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>wav/16b10Wa.wav</th>\n",
       "      <td>anger</td>\n",
       "      <td>anger</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>wav/16b10Wb.wav</th>\n",
       "      <td>anger</td>\n",
       "      <td>anger</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>535 rows × 2 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                     truth prediction\n",
       "file                                 \n",
       "wav/03a01Fa.wav  happiness  happiness\n",
       "wav/03a01Nc.wav    neutral    neutral\n",
       "wav/03a01Wa.wav      anger      anger\n",
       "wav/03a02Fc.wav  happiness  happiness\n",
       "wav/03a02Nc.wav    neutral    neutral\n",
       "...                    ...        ...\n",
       "wav/16b10Lb.wav    boredom    boredom\n",
       "wav/16b10Tb.wav    sadness    sadness\n",
       "wav/16b10Td.wav    sadness    sadness\n",
       "wav/16b10Wa.wav      anger      anger\n",
       "wav/16b10Wb.wav      anger      anger\n",
       "\n",
       "[535 rows x 2 columns]"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "truth_w2v2, pred_w2v2 = experiment(\n",
    "    features_w2v2,\n",
    "    emotion,\n",
    "    speaker,\n",
    ")\n",
    "audformat.utils.concat([truth_w2v2, pred_w2v2])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e0781912",
   "metadata": {},
   "source": [
    "We measure performance by means of Unweighted Average Recall (UAR), which we calculate with [audmetric](https://github.com/audeering/audmetric)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "45640e3b",
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.9338131704157702"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import audmetric\n",
    "\n",
    "\n",
    "audmetric.unweighted_average_recall(truth_w2v2, pred_w2v2)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8bdafe79",
   "metadata": {},
   "source": [
    "## Benchmark against hand-crafted features"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2297acb8",
   "metadata": {},
   "source": [
    "A UAR of more than 93% looks encouraging. But are we actually better than a classic approach on hand-crafted feature? To find out, we repeat the previous experiment and use as input the [ComParE](http://www.tangsoo.de/documents/Publications/Schuller16-TI2.pdf) feature set comprised of more than 6k functionals. To extract the feature set we use [opensmile](https://github.com/audeering/opensmile-python)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "decad96d",
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "                                                                                                    \r"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th>audspec_lengthL1norm_sma_range</th>\n",
       "      <th>audspec_lengthL1norm_sma_maxPos</th>\n",
       "      <th>audspec_lengthL1norm_sma_minPos</th>\n",
       "      <th>audspec_lengthL1norm_sma_quartile1</th>\n",
       "      <th>audspec_lengthL1norm_sma_quartile2</th>\n",
       "      <th>audspec_lengthL1norm_sma_quartile3</th>\n",
       "      <th>audspec_lengthL1norm_sma_iqr1-2</th>\n",
       "      <th>audspec_lengthL1norm_sma_iqr2-3</th>\n",
       "      <th>audspec_lengthL1norm_sma_iqr1-3</th>\n",
       "      <th>audspec_lengthL1norm_sma_percentile1.0</th>\n",
       "      <th>...</th>\n",
       "      <th>mfcc_sma_de[14]_peakRangeAbs</th>\n",
       "      <th>mfcc_sma_de[14]_peakRangeRel</th>\n",
       "      <th>mfcc_sma_de[14]_peakMeanAbs</th>\n",
       "      <th>mfcc_sma_de[14]_peakMeanMeanDist</th>\n",
       "      <th>mfcc_sma_de[14]_peakMeanRel</th>\n",
       "      <th>mfcc_sma_de[14]_minRangeRel</th>\n",
       "      <th>mfcc_sma_de[14]_meanRisingSlope</th>\n",
       "      <th>mfcc_sma_de[14]_stddevRisingSlope</th>\n",
       "      <th>mfcc_sma_de[14]_meanFallingSlope</th>\n",
       "      <th>mfcc_sma_de[14]_stddevFallingSlope</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>file</th>\n",
       "      <th>start</th>\n",
       "      <th>end</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>wav/03a01Fa.wav</th>\n",
       "      <th>0 days</th>\n",
       "      <th>0 days 00:00:01.898250</th>\n",
       "      <td>2.935072</td>\n",
       "      <td>0.203297</td>\n",
       "      <td>0.010989</td>\n",
       "      <td>0.295957</td>\n",
       "      <td>0.521095</td>\n",
       "      <td>1.027880</td>\n",
       "      <td>0.225138</td>\n",
       "      <td>0.506785</td>\n",
       "      <td>0.731923</td>\n",
       "      <td>0.036379</td>\n",
       "      <td>...</td>\n",
       "      <td>4.534245</td>\n",
       "      <td>0.369439</td>\n",
       "      <td>2.608829</td>\n",
       "      <td>2.575876</td>\n",
       "      <td>19.005924</td>\n",
       "      <td>0.681891</td>\n",
       "      <td>117.093933</td>\n",
       "      <td>56.662125</td>\n",
       "      <td>120.517212</td>\n",
       "      <td>53.837585</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>wav/03a01Nc.wav</th>\n",
       "      <th>0 days</th>\n",
       "      <th>0 days 00:00:01.611250</th>\n",
       "      <td>3.087914</td>\n",
       "      <td>0.155844</td>\n",
       "      <td>0.889610</td>\n",
       "      <td>0.230797</td>\n",
       "      <td>0.747830</td>\n",
       "      <td>1.160158</td>\n",
       "      <td>0.517033</td>\n",
       "      <td>0.412327</td>\n",
       "      <td>0.929361</td>\n",
       "      <td>0.039586</td>\n",
       "      <td>...</td>\n",
       "      <td>5.695178</td>\n",
       "      <td>0.373871</td>\n",
       "      <td>3.123539</td>\n",
       "      <td>3.173356</td>\n",
       "      <td>-18.526028</td>\n",
       "      <td>0.684225</td>\n",
       "      <td>126.975647</td>\n",
       "      <td>63.870255</td>\n",
       "      <td>157.944702</td>\n",
       "      <td>60.625206</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>wav/03a01Wa.wav</th>\n",
       "      <th>0 days</th>\n",
       "      <th>0 days 00:00:01.877812500</th>\n",
       "      <td>3.673083</td>\n",
       "      <td>0.677778</td>\n",
       "      <td>0.011111</td>\n",
       "      <td>0.379012</td>\n",
       "      <td>0.901301</td>\n",
       "      <td>1.801537</td>\n",
       "      <td>0.522289</td>\n",
       "      <td>0.900236</td>\n",
       "      <td>1.422525</td>\n",
       "      <td>0.027108</td>\n",
       "      <td>...</td>\n",
       "      <td>8.072537</td>\n",
       "      <td>0.486622</td>\n",
       "      <td>3.802572</td>\n",
       "      <td>3.807790</td>\n",
       "      <td>-19.999348</td>\n",
       "      <td>0.385978</td>\n",
       "      <td>177.820740</td>\n",
       "      <td>89.425529</td>\n",
       "      <td>153.756653</td>\n",
       "      <td>92.122215</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>wav/03a02Fc.wav</th>\n",
       "      <th>0 days</th>\n",
       "      <th>0 days 00:00:02.006250</th>\n",
       "      <td>3.149658</td>\n",
       "      <td>0.549223</td>\n",
       "      <td>0.720207</td>\n",
       "      <td>0.252871</td>\n",
       "      <td>0.691515</td>\n",
       "      <td>1.456156</td>\n",
       "      <td>0.438644</td>\n",
       "      <td>0.764641</td>\n",
       "      <td>1.203285</td>\n",
       "      <td>0.044348</td>\n",
       "      <td>...</td>\n",
       "      <td>8.049533</td>\n",
       "      <td>0.537332</td>\n",
       "      <td>3.237765</td>\n",
       "      <td>3.310604</td>\n",
       "      <td>-17.577114</td>\n",
       "      <td>0.442254</td>\n",
       "      <td>161.931686</td>\n",
       "      <td>82.129478</td>\n",
       "      <td>134.208984</td>\n",
       "      <td>73.309341</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>wav/03a02Nc.wav</th>\n",
       "      <th>0 days</th>\n",
       "      <th>0 days 00:00:01.439812500</th>\n",
       "      <td>2.437313</td>\n",
       "      <td>0.632353</td>\n",
       "      <td>0.691176</td>\n",
       "      <td>0.386229</td>\n",
       "      <td>0.809192</td>\n",
       "      <td>1.433426</td>\n",
       "      <td>0.422963</td>\n",
       "      <td>0.624234</td>\n",
       "      <td>1.047197</td>\n",
       "      <td>0.047513</td>\n",
       "      <td>...</td>\n",
       "      <td>6.743878</td>\n",
       "      <td>0.511177</td>\n",
       "      <td>2.217380</td>\n",
       "      <td>2.281038</td>\n",
       "      <td>-16.731733</td>\n",
       "      <td>0.492748</td>\n",
       "      <td>106.720039</td>\n",
       "      <td>57.315079</td>\n",
       "      <td>97.520103</td>\n",
       "      <td>41.168545</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <th>...</th>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>wav/16b10Lb.wav</th>\n",
       "      <th>0 days</th>\n",
       "      <th>0 days 00:00:03.442687500</th>\n",
       "      <td>3.308008</td>\n",
       "      <td>0.100890</td>\n",
       "      <td>0.014837</td>\n",
       "      <td>0.450570</td>\n",
       "      <td>0.805156</td>\n",
       "      <td>1.323949</td>\n",
       "      <td>0.354586</td>\n",
       "      <td>0.518794</td>\n",
       "      <td>0.873379</td>\n",
       "      <td>0.059215</td>\n",
       "      <td>...</td>\n",
       "      <td>8.917046</td>\n",
       "      <td>0.681655</td>\n",
       "      <td>2.385576</td>\n",
       "      <td>2.390011</td>\n",
       "      <td>-19.996981</td>\n",
       "      <td>0.537082</td>\n",
       "      <td>118.114395</td>\n",
       "      <td>65.305817</td>\n",
       "      <td>102.050064</td>\n",
       "      <td>44.002724</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>wav/16b10Tb.wav</th>\n",
       "      <th>0 days</th>\n",
       "      <th>0 days 00:00:03.500625</th>\n",
       "      <td>2.817487</td>\n",
       "      <td>0.104956</td>\n",
       "      <td>0.472303</td>\n",
       "      <td>0.298808</td>\n",
       "      <td>0.898184</td>\n",
       "      <td>1.372837</td>\n",
       "      <td>0.599376</td>\n",
       "      <td>0.474654</td>\n",
       "      <td>1.074030</td>\n",
       "      <td>0.111207</td>\n",
       "      <td>...</td>\n",
       "      <td>8.454210</td>\n",
       "      <td>0.531393</td>\n",
       "      <td>3.011052</td>\n",
       "      <td>3.035630</td>\n",
       "      <td>-19.587982</td>\n",
       "      <td>0.486605</td>\n",
       "      <td>133.106750</td>\n",
       "      <td>83.190590</td>\n",
       "      <td>112.855782</td>\n",
       "      <td>72.329666</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>wav/16b10Td.wav</th>\n",
       "      <th>0 days</th>\n",
       "      <th>0 days 00:00:03.934187500</th>\n",
       "      <td>1.854920</td>\n",
       "      <td>0.119171</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.208670</td>\n",
       "      <td>0.607462</td>\n",
       "      <td>1.031825</td>\n",
       "      <td>0.398792</td>\n",
       "      <td>0.424362</td>\n",
       "      <td>0.823154</td>\n",
       "      <td>0.102642</td>\n",
       "      <td>...</td>\n",
       "      <td>6.234570</td>\n",
       "      <td>0.470659</td>\n",
       "      <td>2.219766</td>\n",
       "      <td>2.233577</td>\n",
       "      <td>-19.786058</td>\n",
       "      <td>0.620368</td>\n",
       "      <td>110.055237</td>\n",
       "      <td>51.608723</td>\n",
       "      <td>83.898209</td>\n",
       "      <td>39.287560</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>wav/16b10Wa.wav</th>\n",
       "      <th>0 days</th>\n",
       "      <th>0 days 00:00:02.414125</th>\n",
       "      <td>3.405270</td>\n",
       "      <td>0.414530</td>\n",
       "      <td>0.034188</td>\n",
       "      <td>0.232148</td>\n",
       "      <td>0.552492</td>\n",
       "      <td>0.895527</td>\n",
       "      <td>0.320345</td>\n",
       "      <td>0.343034</td>\n",
       "      <td>0.663379</td>\n",
       "      <td>0.026728</td>\n",
       "      <td>...</td>\n",
       "      <td>8.002909</td>\n",
       "      <td>0.490650</td>\n",
       "      <td>4.130598</td>\n",
       "      <td>4.145967</td>\n",
       "      <td>-19.952806</td>\n",
       "      <td>0.459358</td>\n",
       "      <td>178.499817</td>\n",
       "      <td>77.860161</td>\n",
       "      <td>144.251648</td>\n",
       "      <td>78.650452</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>wav/16b10Wb.wav</th>\n",
       "      <th>0 days</th>\n",
       "      <th>0 days 00:00:02.522499999</th>\n",
       "      <td>3.555349</td>\n",
       "      <td>0.436735</td>\n",
       "      <td>0.004082</td>\n",
       "      <td>0.460390</td>\n",
       "      <td>0.973487</td>\n",
       "      <td>1.774622</td>\n",
       "      <td>0.513097</td>\n",
       "      <td>0.801135</td>\n",
       "      <td>1.314232</td>\n",
       "      <td>0.038774</td>\n",
       "      <td>...</td>\n",
       "      <td>12.656188</td>\n",
       "      <td>0.651535</td>\n",
       "      <td>3.835181</td>\n",
       "      <td>3.883579</td>\n",
       "      <td>-19.007538</td>\n",
       "      <td>0.405047</td>\n",
       "      <td>170.342270</td>\n",
       "      <td>85.037521</td>\n",
       "      <td>176.378418</td>\n",
       "      <td>109.506447</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>535 rows × 6373 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                                                  audspec_lengthL1norm_sma_range  \\\n",
       "file            start  end                                                         \n",
       "wav/03a01Fa.wav 0 days 0 days 00:00:01.898250                           2.935072   \n",
       "wav/03a01Nc.wav 0 days 0 days 00:00:01.611250                           3.087914   \n",
       "wav/03a01Wa.wav 0 days 0 days 00:00:01.877812500                        3.673083   \n",
       "wav/03a02Fc.wav 0 days 0 days 00:00:02.006250                           3.149658   \n",
       "wav/03a02Nc.wav 0 days 0 days 00:00:01.439812500                        2.437313   \n",
       "...                                                                          ...   \n",
       "wav/16b10Lb.wav 0 days 0 days 00:00:03.442687500                        3.308008   \n",
       "wav/16b10Tb.wav 0 days 0 days 00:00:03.500625                           2.817487   \n",
       "wav/16b10Td.wav 0 days 0 days 00:00:03.934187500                        1.854920   \n",
       "wav/16b10Wa.wav 0 days 0 days 00:00:02.414125                           3.405270   \n",
       "wav/16b10Wb.wav 0 days 0 days 00:00:02.522499999                        3.555349   \n",
       "\n",
       "                                                  audspec_lengthL1norm_sma_maxPos  \\\n",
       "file            start  end                                                          \n",
       "wav/03a01Fa.wav 0 days 0 days 00:00:01.898250                            0.203297   \n",
       "wav/03a01Nc.wav 0 days 0 days 00:00:01.611250                            0.155844   \n",
       "wav/03a01Wa.wav 0 days 0 days 00:00:01.877812500                         0.677778   \n",
       "wav/03a02Fc.wav 0 days 0 days 00:00:02.006250                            0.549223   \n",
       "wav/03a02Nc.wav 0 days 0 days 00:00:01.439812500                         0.632353   \n",
       "...                                                                           ...   \n",
       "wav/16b10Lb.wav 0 days 0 days 00:00:03.442687500                         0.100890   \n",
       "wav/16b10Tb.wav 0 days 0 days 00:00:03.500625                            0.104956   \n",
       "wav/16b10Td.wav 0 days 0 days 00:00:03.934187500                         0.119171   \n",
       "wav/16b10Wa.wav 0 days 0 days 00:00:02.414125                            0.414530   \n",
       "wav/16b10Wb.wav 0 days 0 days 00:00:02.522499999                         0.436735   \n",
       "\n",
       "                                                  audspec_lengthL1norm_sma_minPos  \\\n",
       "file            start  end                                                          \n",
       "wav/03a01Fa.wav 0 days 0 days 00:00:01.898250                            0.010989   \n",
       "wav/03a01Nc.wav 0 days 0 days 00:00:01.611250                            0.889610   \n",
       "wav/03a01Wa.wav 0 days 0 days 00:00:01.877812500                         0.011111   \n",
       "wav/03a02Fc.wav 0 days 0 days 00:00:02.006250                            0.720207   \n",
       "wav/03a02Nc.wav 0 days 0 days 00:00:01.439812500                         0.691176   \n",
       "...                                                                           ...   \n",
       "wav/16b10Lb.wav 0 days 0 days 00:00:03.442687500                         0.014837   \n",
       "wav/16b10Tb.wav 0 days 0 days 00:00:03.500625                            0.472303   \n",
       "wav/16b10Td.wav 0 days 0 days 00:00:03.934187500                         0.000000   \n",
       "wav/16b10Wa.wav 0 days 0 days 00:00:02.414125                            0.034188   \n",
       "wav/16b10Wb.wav 0 days 0 days 00:00:02.522499999                         0.004082   \n",
       "\n",
       "                                                  audspec_lengthL1norm_sma_quartile1  \\\n",
       "file            start  end                                                             \n",
       "wav/03a01Fa.wav 0 days 0 days 00:00:01.898250                               0.295957   \n",
       "wav/03a01Nc.wav 0 days 0 days 00:00:01.611250                               0.230797   \n",
       "wav/03a01Wa.wav 0 days 0 days 00:00:01.877812500                            0.379012   \n",
       "wav/03a02Fc.wav 0 days 0 days 00:00:02.006250                               0.252871   \n",
       "wav/03a02Nc.wav 0 days 0 days 00:00:01.439812500                            0.386229   \n",
       "...                                                                              ...   \n",
       "wav/16b10Lb.wav 0 days 0 days 00:00:03.442687500                            0.450570   \n",
       "wav/16b10Tb.wav 0 days 0 days 00:00:03.500625                               0.298808   \n",
       "wav/16b10Td.wav 0 days 0 days 00:00:03.934187500                            0.208670   \n",
       "wav/16b10Wa.wav 0 days 0 days 00:00:02.414125                               0.232148   \n",
       "wav/16b10Wb.wav 0 days 0 days 00:00:02.522499999                            0.460390   \n",
       "\n",
       "                                                  audspec_lengthL1norm_sma_quartile2  \\\n",
       "file            start  end                                                             \n",
       "wav/03a01Fa.wav 0 days 0 days 00:00:01.898250                               0.521095   \n",
       "wav/03a01Nc.wav 0 days 0 days 00:00:01.611250                               0.747830   \n",
       "wav/03a01Wa.wav 0 days 0 days 00:00:01.877812500                            0.901301   \n",
       "wav/03a02Fc.wav 0 days 0 days 00:00:02.006250                               0.691515   \n",
       "wav/03a02Nc.wav 0 days 0 days 00:00:01.439812500                            0.809192   \n",
       "...                                                                              ...   \n",
       "wav/16b10Lb.wav 0 days 0 days 00:00:03.442687500                            0.805156   \n",
       "wav/16b10Tb.wav 0 days 0 days 00:00:03.500625                               0.898184   \n",
       "wav/16b10Td.wav 0 days 0 days 00:00:03.934187500                            0.607462   \n",
       "wav/16b10Wa.wav 0 days 0 days 00:00:02.414125                               0.552492   \n",
       "wav/16b10Wb.wav 0 days 0 days 00:00:02.522499999                            0.973487   \n",
       "\n",
       "                                                  audspec_lengthL1norm_sma_quartile3  \\\n",
       "file            start  end                                                             \n",
       "wav/03a01Fa.wav 0 days 0 days 00:00:01.898250                               1.027880   \n",
       "wav/03a01Nc.wav 0 days 0 days 00:00:01.611250                               1.160158   \n",
       "wav/03a01Wa.wav 0 days 0 days 00:00:01.877812500                            1.801537   \n",
       "wav/03a02Fc.wav 0 days 0 days 00:00:02.006250                               1.456156   \n",
       "wav/03a02Nc.wav 0 days 0 days 00:00:01.439812500                            1.433426   \n",
       "...                                                                              ...   \n",
       "wav/16b10Lb.wav 0 days 0 days 00:00:03.442687500                            1.323949   \n",
       "wav/16b10Tb.wav 0 days 0 days 00:00:03.500625                               1.372837   \n",
       "wav/16b10Td.wav 0 days 0 days 00:00:03.934187500                            1.031825   \n",
       "wav/16b10Wa.wav 0 days 0 days 00:00:02.414125                               0.895527   \n",
       "wav/16b10Wb.wav 0 days 0 days 00:00:02.522499999                            1.774622   \n",
       "\n",
       "                                                  audspec_lengthL1norm_sma_iqr1-2  \\\n",
       "file            start  end                                                          \n",
       "wav/03a01Fa.wav 0 days 0 days 00:00:01.898250                            0.225138   \n",
       "wav/03a01Nc.wav 0 days 0 days 00:00:01.611250                            0.517033   \n",
       "wav/03a01Wa.wav 0 days 0 days 00:00:01.877812500                         0.522289   \n",
       "wav/03a02Fc.wav 0 days 0 days 00:00:02.006250                            0.438644   \n",
       "wav/03a02Nc.wav 0 days 0 days 00:00:01.439812500                         0.422963   \n",
       "...                                                                           ...   \n",
       "wav/16b10Lb.wav 0 days 0 days 00:00:03.442687500                         0.354586   \n",
       "wav/16b10Tb.wav 0 days 0 days 00:00:03.500625                            0.599376   \n",
       "wav/16b10Td.wav 0 days 0 days 00:00:03.934187500                         0.398792   \n",
       "wav/16b10Wa.wav 0 days 0 days 00:00:02.414125                            0.320345   \n",
       "wav/16b10Wb.wav 0 days 0 days 00:00:02.522499999                         0.513097   \n",
       "\n",
       "                                                  audspec_lengthL1norm_sma_iqr2-3  \\\n",
       "file            start  end                                                          \n",
       "wav/03a01Fa.wav 0 days 0 days 00:00:01.898250                            0.506785   \n",
       "wav/03a01Nc.wav 0 days 0 days 00:00:01.611250                            0.412327   \n",
       "wav/03a01Wa.wav 0 days 0 days 00:00:01.877812500                         0.900236   \n",
       "wav/03a02Fc.wav 0 days 0 days 00:00:02.006250                            0.764641   \n",
       "wav/03a02Nc.wav 0 days 0 days 00:00:01.439812500                         0.624234   \n",
       "...                                                                           ...   \n",
       "wav/16b10Lb.wav 0 days 0 days 00:00:03.442687500                         0.518794   \n",
       "wav/16b10Tb.wav 0 days 0 days 00:00:03.500625                            0.474654   \n",
       "wav/16b10Td.wav 0 days 0 days 00:00:03.934187500                         0.424362   \n",
       "wav/16b10Wa.wav 0 days 0 days 00:00:02.414125                            0.343034   \n",
       "wav/16b10Wb.wav 0 days 0 days 00:00:02.522499999                         0.801135   \n",
       "\n",
       "                                                  audspec_lengthL1norm_sma_iqr1-3  \\\n",
       "file            start  end                                                          \n",
       "wav/03a01Fa.wav 0 days 0 days 00:00:01.898250                            0.731923   \n",
       "wav/03a01Nc.wav 0 days 0 days 00:00:01.611250                            0.929361   \n",
       "wav/03a01Wa.wav 0 days 0 days 00:00:01.877812500                         1.422525   \n",
       "wav/03a02Fc.wav 0 days 0 days 00:00:02.006250                            1.203285   \n",
       "wav/03a02Nc.wav 0 days 0 days 00:00:01.439812500                         1.047197   \n",
       "...                                                                           ...   \n",
       "wav/16b10Lb.wav 0 days 0 days 00:00:03.442687500                         0.873379   \n",
       "wav/16b10Tb.wav 0 days 0 days 00:00:03.500625                            1.074030   \n",
       "wav/16b10Td.wav 0 days 0 days 00:00:03.934187500                         0.823154   \n",
       "wav/16b10Wa.wav 0 days 0 days 00:00:02.414125                            0.663379   \n",
       "wav/16b10Wb.wav 0 days 0 days 00:00:02.522499999                         1.314232   \n",
       "\n",
       "                                                  audspec_lengthL1norm_sma_percentile1.0  \\\n",
       "file            start  end                                                                 \n",
       "wav/03a01Fa.wav 0 days 0 days 00:00:01.898250                                   0.036379   \n",
       "wav/03a01Nc.wav 0 days 0 days 00:00:01.611250                                   0.039586   \n",
       "wav/03a01Wa.wav 0 days 0 days 00:00:01.877812500                                0.027108   \n",
       "wav/03a02Fc.wav 0 days 0 days 00:00:02.006250                                   0.044348   \n",
       "wav/03a02Nc.wav 0 days 0 days 00:00:01.439812500                                0.047513   \n",
       "...                                                                                  ...   \n",
       "wav/16b10Lb.wav 0 days 0 days 00:00:03.442687500                                0.059215   \n",
       "wav/16b10Tb.wav 0 days 0 days 00:00:03.500625                                   0.111207   \n",
       "wav/16b10Td.wav 0 days 0 days 00:00:03.934187500                                0.102642   \n",
       "wav/16b10Wa.wav 0 days 0 days 00:00:02.414125                                   0.026728   \n",
       "wav/16b10Wb.wav 0 days 0 days 00:00:02.522499999                                0.038774   \n",
       "\n",
       "                                                  ...  \\\n",
       "file            start  end                        ...   \n",
       "wav/03a01Fa.wav 0 days 0 days 00:00:01.898250     ...   \n",
       "wav/03a01Nc.wav 0 days 0 days 00:00:01.611250     ...   \n",
       "wav/03a01Wa.wav 0 days 0 days 00:00:01.877812500  ...   \n",
       "wav/03a02Fc.wav 0 days 0 days 00:00:02.006250     ...   \n",
       "wav/03a02Nc.wav 0 days 0 days 00:00:01.439812500  ...   \n",
       "...                                               ...   \n",
       "wav/16b10Lb.wav 0 days 0 days 00:00:03.442687500  ...   \n",
       "wav/16b10Tb.wav 0 days 0 days 00:00:03.500625     ...   \n",
       "wav/16b10Td.wav 0 days 0 days 00:00:03.934187500  ...   \n",
       "wav/16b10Wa.wav 0 days 0 days 00:00:02.414125     ...   \n",
       "wav/16b10Wb.wav 0 days 0 days 00:00:02.522499999  ...   \n",
       "\n",
       "                                                  mfcc_sma_de[14]_peakRangeAbs  \\\n",
       "file            start  end                                                       \n",
       "wav/03a01Fa.wav 0 days 0 days 00:00:01.898250                         4.534245   \n",
       "wav/03a01Nc.wav 0 days 0 days 00:00:01.611250                         5.695178   \n",
       "wav/03a01Wa.wav 0 days 0 days 00:00:01.877812500                      8.072537   \n",
       "wav/03a02Fc.wav 0 days 0 days 00:00:02.006250                         8.049533   \n",
       "wav/03a02Nc.wav 0 days 0 days 00:00:01.439812500                      6.743878   \n",
       "...                                                                        ...   \n",
       "wav/16b10Lb.wav 0 days 0 days 00:00:03.442687500                      8.917046   \n",
       "wav/16b10Tb.wav 0 days 0 days 00:00:03.500625                         8.454210   \n",
       "wav/16b10Td.wav 0 days 0 days 00:00:03.934187500                      6.234570   \n",
       "wav/16b10Wa.wav 0 days 0 days 00:00:02.414125                         8.002909   \n",
       "wav/16b10Wb.wav 0 days 0 days 00:00:02.522499999                     12.656188   \n",
       "\n",
       "                                                  mfcc_sma_de[14]_peakRangeRel  \\\n",
       "file            start  end                                                       \n",
       "wav/03a01Fa.wav 0 days 0 days 00:00:01.898250                         0.369439   \n",
       "wav/03a01Nc.wav 0 days 0 days 00:00:01.611250                         0.373871   \n",
       "wav/03a01Wa.wav 0 days 0 days 00:00:01.877812500                      0.486622   \n",
       "wav/03a02Fc.wav 0 days 0 days 00:00:02.006250                         0.537332   \n",
       "wav/03a02Nc.wav 0 days 0 days 00:00:01.439812500                      0.511177   \n",
       "...                                                                        ...   \n",
       "wav/16b10Lb.wav 0 days 0 days 00:00:03.442687500                      0.681655   \n",
       "wav/16b10Tb.wav 0 days 0 days 00:00:03.500625                         0.531393   \n",
       "wav/16b10Td.wav 0 days 0 days 00:00:03.934187500                      0.470659   \n",
       "wav/16b10Wa.wav 0 days 0 days 00:00:02.414125                         0.490650   \n",
       "wav/16b10Wb.wav 0 days 0 days 00:00:02.522499999                      0.651535   \n",
       "\n",
       "                                                  mfcc_sma_de[14]_peakMeanAbs  \\\n",
       "file            start  end                                                      \n",
       "wav/03a01Fa.wav 0 days 0 days 00:00:01.898250                        2.608829   \n",
       "wav/03a01Nc.wav 0 days 0 days 00:00:01.611250                        3.123539   \n",
       "wav/03a01Wa.wav 0 days 0 days 00:00:01.877812500                     3.802572   \n",
       "wav/03a02Fc.wav 0 days 0 days 00:00:02.006250                        3.237765   \n",
       "wav/03a02Nc.wav 0 days 0 days 00:00:01.439812500                     2.217380   \n",
       "...                                                                       ...   \n",
       "wav/16b10Lb.wav 0 days 0 days 00:00:03.442687500                     2.385576   \n",
       "wav/16b10Tb.wav 0 days 0 days 00:00:03.500625                        3.011052   \n",
       "wav/16b10Td.wav 0 days 0 days 00:00:03.934187500                     2.219766   \n",
       "wav/16b10Wa.wav 0 days 0 days 00:00:02.414125                        4.130598   \n",
       "wav/16b10Wb.wav 0 days 0 days 00:00:02.522499999                     3.835181   \n",
       "\n",
       "                                                  mfcc_sma_de[14]_peakMeanMeanDist  \\\n",
       "file            start  end                                                           \n",
       "wav/03a01Fa.wav 0 days 0 days 00:00:01.898250                             2.575876   \n",
       "wav/03a01Nc.wav 0 days 0 days 00:00:01.611250                             3.173356   \n",
       "wav/03a01Wa.wav 0 days 0 days 00:00:01.877812500                          3.807790   \n",
       "wav/03a02Fc.wav 0 days 0 days 00:00:02.006250                             3.310604   \n",
       "wav/03a02Nc.wav 0 days 0 days 00:00:01.439812500                          2.281038   \n",
       "...                                                                            ...   \n",
       "wav/16b10Lb.wav 0 days 0 days 00:00:03.442687500                          2.390011   \n",
       "wav/16b10Tb.wav 0 days 0 days 00:00:03.500625                             3.035630   \n",
       "wav/16b10Td.wav 0 days 0 days 00:00:03.934187500                          2.233577   \n",
       "wav/16b10Wa.wav 0 days 0 days 00:00:02.414125                             4.145967   \n",
       "wav/16b10Wb.wav 0 days 0 days 00:00:02.522499999                          3.883579   \n",
       "\n",
       "                                                  mfcc_sma_de[14]_peakMeanRel  \\\n",
       "file            start  end                                                      \n",
       "wav/03a01Fa.wav 0 days 0 days 00:00:01.898250                       19.005924   \n",
       "wav/03a01Nc.wav 0 days 0 days 00:00:01.611250                      -18.526028   \n",
       "wav/03a01Wa.wav 0 days 0 days 00:00:01.877812500                   -19.999348   \n",
       "wav/03a02Fc.wav 0 days 0 days 00:00:02.006250                      -17.577114   \n",
       "wav/03a02Nc.wav 0 days 0 days 00:00:01.439812500                   -16.731733   \n",
       "...                                                                       ...   \n",
       "wav/16b10Lb.wav 0 days 0 days 00:00:03.442687500                   -19.996981   \n",
       "wav/16b10Tb.wav 0 days 0 days 00:00:03.500625                      -19.587982   \n",
       "wav/16b10Td.wav 0 days 0 days 00:00:03.934187500                   -19.786058   \n",
       "wav/16b10Wa.wav 0 days 0 days 00:00:02.414125                      -19.952806   \n",
       "wav/16b10Wb.wav 0 days 0 days 00:00:02.522499999                   -19.007538   \n",
       "\n",
       "                                                  mfcc_sma_de[14]_minRangeRel  \\\n",
       "file            start  end                                                      \n",
       "wav/03a01Fa.wav 0 days 0 days 00:00:01.898250                        0.681891   \n",
       "wav/03a01Nc.wav 0 days 0 days 00:00:01.611250                        0.684225   \n",
       "wav/03a01Wa.wav 0 days 0 days 00:00:01.877812500                     0.385978   \n",
       "wav/03a02Fc.wav 0 days 0 days 00:00:02.006250                        0.442254   \n",
       "wav/03a02Nc.wav 0 days 0 days 00:00:01.439812500                     0.492748   \n",
       "...                                                                       ...   \n",
       "wav/16b10Lb.wav 0 days 0 days 00:00:03.442687500                     0.537082   \n",
       "wav/16b10Tb.wav 0 days 0 days 00:00:03.500625                        0.486605   \n",
       "wav/16b10Td.wav 0 days 0 days 00:00:03.934187500                     0.620368   \n",
       "wav/16b10Wa.wav 0 days 0 days 00:00:02.414125                        0.459358   \n",
       "wav/16b10Wb.wav 0 days 0 days 00:00:02.522499999                     0.405047   \n",
       "\n",
       "                                                  mfcc_sma_de[14]_meanRisingSlope  \\\n",
       "file            start  end                                                          \n",
       "wav/03a01Fa.wav 0 days 0 days 00:00:01.898250                          117.093933   \n",
       "wav/03a01Nc.wav 0 days 0 days 00:00:01.611250                          126.975647   \n",
       "wav/03a01Wa.wav 0 days 0 days 00:00:01.877812500                       177.820740   \n",
       "wav/03a02Fc.wav 0 days 0 days 00:00:02.006250                          161.931686   \n",
       "wav/03a02Nc.wav 0 days 0 days 00:00:01.439812500                       106.720039   \n",
       "...                                                                           ...   \n",
       "wav/16b10Lb.wav 0 days 0 days 00:00:03.442687500                       118.114395   \n",
       "wav/16b10Tb.wav 0 days 0 days 00:00:03.500625                          133.106750   \n",
       "wav/16b10Td.wav 0 days 0 days 00:00:03.934187500                       110.055237   \n",
       "wav/16b10Wa.wav 0 days 0 days 00:00:02.414125                          178.499817   \n",
       "wav/16b10Wb.wav 0 days 0 days 00:00:02.522499999                       170.342270   \n",
       "\n",
       "                                                  mfcc_sma_de[14]_stddevRisingSlope  \\\n",
       "file            start  end                                                            \n",
       "wav/03a01Fa.wav 0 days 0 days 00:00:01.898250                             56.662125   \n",
       "wav/03a01Nc.wav 0 days 0 days 00:00:01.611250                             63.870255   \n",
       "wav/03a01Wa.wav 0 days 0 days 00:00:01.877812500                          89.425529   \n",
       "wav/03a02Fc.wav 0 days 0 days 00:00:02.006250                             82.129478   \n",
       "wav/03a02Nc.wav 0 days 0 days 00:00:01.439812500                          57.315079   \n",
       "...                                                                             ...   \n",
       "wav/16b10Lb.wav 0 days 0 days 00:00:03.442687500                          65.305817   \n",
       "wav/16b10Tb.wav 0 days 0 days 00:00:03.500625                             83.190590   \n",
       "wav/16b10Td.wav 0 days 0 days 00:00:03.934187500                          51.608723   \n",
       "wav/16b10Wa.wav 0 days 0 days 00:00:02.414125                             77.860161   \n",
       "wav/16b10Wb.wav 0 days 0 days 00:00:02.522499999                          85.037521   \n",
       "\n",
       "                                                  mfcc_sma_de[14]_meanFallingSlope  \\\n",
       "file            start  end                                                           \n",
       "wav/03a01Fa.wav 0 days 0 days 00:00:01.898250                           120.517212   \n",
       "wav/03a01Nc.wav 0 days 0 days 00:00:01.611250                           157.944702   \n",
       "wav/03a01Wa.wav 0 days 0 days 00:00:01.877812500                        153.756653   \n",
       "wav/03a02Fc.wav 0 days 0 days 00:00:02.006250                           134.208984   \n",
       "wav/03a02Nc.wav 0 days 0 days 00:00:01.439812500                         97.520103   \n",
       "...                                                                            ...   \n",
       "wav/16b10Lb.wav 0 days 0 days 00:00:03.442687500                        102.050064   \n",
       "wav/16b10Tb.wav 0 days 0 days 00:00:03.500625                           112.855782   \n",
       "wav/16b10Td.wav 0 days 0 days 00:00:03.934187500                         83.898209   \n",
       "wav/16b10Wa.wav 0 days 0 days 00:00:02.414125                           144.251648   \n",
       "wav/16b10Wb.wav 0 days 0 days 00:00:02.522499999                        176.378418   \n",
       "\n",
       "                                                  mfcc_sma_de[14]_stddevFallingSlope  \n",
       "file            start  end                                                            \n",
       "wav/03a01Fa.wav 0 days 0 days 00:00:01.898250                              53.837585  \n",
       "wav/03a01Nc.wav 0 days 0 days 00:00:01.611250                              60.625206  \n",
       "wav/03a01Wa.wav 0 days 0 days 00:00:01.877812500                           92.122215  \n",
       "wav/03a02Fc.wav 0 days 0 days 00:00:02.006250                              73.309341  \n",
       "wav/03a02Nc.wav 0 days 0 days 00:00:01.439812500                           41.168545  \n",
       "...                                                                              ...  \n",
       "wav/16b10Lb.wav 0 days 0 days 00:00:03.442687500                           44.002724  \n",
       "wav/16b10Tb.wav 0 days 0 days 00:00:03.500625                              72.329666  \n",
       "wav/16b10Td.wav 0 days 0 days 00:00:03.934187500                           39.287560  \n",
       "wav/16b10Wa.wav 0 days 0 days 00:00:02.414125                              78.650452  \n",
       "wav/16b10Wb.wav 0 days 0 days 00:00:02.522499999                          109.506447  \n",
       "\n",
       "[535 rows x 6373 columns]"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import opensmile\n",
    "\n",
    "\n",
    "smile = opensmile.Smile(\n",
    "    opensmile.FeatureSet.ComParE_2016,\n",
    "    opensmile.FeatureLevel.Functionals,\n",
    "    sampling_rate=16000,    \n",
    "    resample=True,    \n",
    "    num_workers=5,\n",
    "    verbose=True,\n",
    ")\n",
    "\n",
    "features_smile = smile.process_index(\n",
    "    emotion.index,\n",
    "    root=db.root,\n",
    "    cache_root=audeer.path(cache_root, 'smile'),    \n",
    ")\n",
    "features_smile"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6cb5ff8f",
   "metadata": {},
   "source": [
    "With the hand-crafted features we achieve an UAR of only 77%:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "a9f1d174",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "                                                                                                    \r"
     ]
    },
    {
     "data": {
      "text/plain": [
       "0.7707603332902749"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "truth_smile, pred_smile = experiment(\n",
    "    features_smile,\n",
    "    emotion,\n",
    "    speaker,\n",
    ")\n",
    "audmetric.unweighted_average_recall(truth_smile, pred_smile)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fb61ee1a",
   "metadata": {},
   "source": [
    "To understand, on which classes we lose performance, we plot confusion matrices for both approaches with [audplot](https://github.com/audeering/audplot). It is obvious that the handcrafted features mainly struggle with disgust and fear, but especially happiness (44%). The wav2vec 2.0 model performs better on all categories with an almost doubled recall for happiness - a huge step. We still have to note that 8% of the samples are confused with anger, a gap sentiment might close if the sentences were in English."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "1cdc77a1",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAA5sAAAFNCAYAAACOk3sLAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/YYfK9AAAACXBIWXMAAAsTAAALEwEAmpwYAAB2gElEQVR4nO3dd3gU5dfG8e9JgYQqSLGhIBAQ7KKCdGyotNCkWABRAYHXiiCK2PmJvSCCggjSFaQoKEUEFAVUerMGlKZSgmRDNvu8f+wSI9LZZLPj/bmuXNmdnZ09Z5+ZOTmzsxNzziEiIiIiIiISTjGRDkBERERERES8R82miIiIiIiIhJ2aTREREREREQk7NZsiIiIiIiISdmo2RUREREREJOzUbIqIiIiIiEjYqdkUkeNmZh+b2a2h2x3MbEGkYxIRERGRvEHNpogcN+fcdc65EZGOQ0REJLeYWSkzG2Nmv5nZLjNbaGaXH+Vzq5vZp2b2p5ltN7MJZnZqTscsEilqNkVEREREjl4hYDFwCVAcGAFMN7NCR/HcYsAQoCxwFpAKDM+ZMEUiT82myH+UmT1oZr+aWaqZrTOzK82sf+go66jQ9BVmlmRmfcxsm5ltNLNrsi3jMzPrfIjlV8529HadmbXOvexERESOnZl1NLOp2e5vMLMJ2e5vBIo4515wzm12zmU654YA+YBKZpbfzHaa2bnZnlPSzNLMrJRz7mPn3ATn3G7n3F7gNaBmLqYokqvUbIr8B5lZJaA7cKlzrjBwLfBz6OHGwEiCR1+/BWYS3FecDjwOvHkUyy8IfAqMBkoBbYBBZlYlrImIiIiE1zygtpnFmNlpBJvIGgBmdjbBTzWXZ3+CmV0Ymu9751w68AHQNtssrYF5zrltB3m9OsCqcCchkleo2RT5b8oE8gNVzCzeOfezc+6H0GPznXMznXN+YAJQEhjgnMsAxgJlzeykIyy/EfCzc264c87vnPsWeB9olSPZiIiIhIFz7keCp7ZeSLARnAn8ZmaVgboEa2Rg//xmVoTgAdrHnHO7QpNHEzzIul+70LR/MLPzgX7AA+HPRCRviIt0ACKS+5xz35vZ3UB/oKqZzQTuDT28NdusacDvzrnMbPcheGR352Fe4izgcjPLPk8cwYIsIiKSl80D6gEVQrd3Emw0a4TuA2BmicBUYJFz7plsz58LFAhdNGgrwcZ1UvYXMLMKwMfA/znn5udQHiIRp082Rf6jnHOjnXO1CDaGDvhfGBe/keApQydl+ynknOsaxtcQERHJCfubzdqh2/MINpt1Q7cxs/zAZGATcGf2J4cO0I4neCptW2Cacy51/+NmdhYwC3jCOaeDsOJpajZF/oPMrJKZNQgVSx/BTywDR3jasZgGJJnZzWYWH/q51MzOCeNriIiI5IR5QH0g0Tm3CZgPNAROBr41s3hgIsHaeWv202qzGQ3cCLQn2ym0ZnY6MAd4zTk3OEezEMkD1GyK/DflBwYAvwNbCF7Ep0+4Fh46gnsNwe+s/BZ6jf+FXldERCTPcs6tB/YQbDJxzu0GfgQWhj61vILgtQmuAXaa2Z7QT+1sy/gK+As4jeDpsvt1Bs4G+md73p7cyEskEsw5F+kYRERERERExGP0yaaIiIiIiIiEnZpNERERERERCTs1myIiIiIiIhJ2ajZFREREREQk7NRsioiIiIiISNjFRTqAaJZ4UXdPXsp3x+LXIh2CHKPMgCdXRcwiHUHOifFych6VEIcG7RioRorkrIzMcP577LwjPlafhUWbw9VHjaaIiIiIiIiEnZpNERERERERCTs1myIiIiIiIhJ2ajZFREREREQk7NRsioiIiIiISNip2RQREREREZGwU7MpIiIiIiIiYadmU0RERERERMJOzaaIiIiIiIiEnZpNERERERERCTs1myIiIiIiIhJ2ajZFREREREQk7NRsioiIiIiISNip2RQREREREZGwU7OZB9zVth5LJjzE0ol96d6uHgDnJ53OvBH3sWhsbxa814tqVc8CoNmVF7J0Yl9mvX03xYsWBKDcGSUYOaBjpMI/Kgvnf06TG66lUcOreXvoEAD69LqPlsmNeeWlF7LmGzJ4EHNmz4pUmMfFi7lt2bKZOzrdQoumN9CyWSNGj3oXgJdfeI7WzZvwyEMPZs07feoU3hs5IlKhHrP+Dz9EgzpX0LJZ46xpL7/wHK2Tm/Bwn+jNC7y5LoJ385Ij+y/UR/DuOu7VvMCbuW3Zspk7b7uVVs0a0Tq5EWNCtf+VF5+jTYum9MtW+z+aNoXRUVQjvThe++X13NRsHoYF5eh7VKX8qXRsfgW1bx7IZTc+w3V1zuXsMiV46u5mPDXkY6q3GcATb0zjqbubAdC1TV1q3fQsb72/kBuvqwZA/7sa0X/QtJwM84RkZmby9FOPM2jwW0yaMp0ZH01j/bq15E9IYOKkqaxauYLU1FS2b9/GiuXLaXDlVZEO+ah5NbfY2Fjuuf9B3v9wOiPeG8v4se+xft1a1q5ZzfgPphAfH8+G9evw+XxM+fADWrdpF+mQj1rjZsm8Pnho1v3U1FTWrF7N+EkH5DU5uvLy6rro1byinepj+Hh1HfdqXuDd3OJiY7nnvl5MmDyN4aPGMWHc6KzaP/b9D4mPj+f79evx+XxMnTwpamqkV8cLoiO3qGw2zWyymS01s1Vmdkdo2h4ze8rMlpnZIjMrHZpePnR/hZk9aWZ7si3nATNbbGbLzeyx0LSyZrbOzN4FVgJlcjKXyuVOYfHKn0nzZZCZGWD+0u9p1uBCnIMiBRMAKFookc3bdwEQCATIHx9HgYR8ZPgzqXlRebb+vpsfUrbnZJgnZOWK5ZQpcxZnlClDfL58NLz+BuZ/Po90n49AIIDf7yc2JoZBr75Ct+49Ih3uMfFqbiVLluKcKlUBKFiwEOXKlWfz5t/w+zNwzuFLSyMuLp6R7wyjTdubiI+Pj3DER++SapdStGjRrPsxMfZ3Xr5gXu++M4w27aIrL6+ui17NK6eoPkZXfQTvruNezQu8m1uJkqWonFX7C1K2XHm2bN6M3+8P1UgfcfFxjBoxjBvbtScuSmqkV8cLoiO3qGw2gU7OuUuAakBPMzsZKAgscs5dAHwO3B6a92XgZefcecCm/Qsws2uAisBlwIXAJWZWJ/RwRWCQc66qc+6XnExk1Q+/UfOiChQvWpDEhHga1qrKGacU44HnJvL03c3Y8PETPHNPMv1e/RCAgcM+ZfrgHlxf51zGz1hC79sb8szQGTkZ4gnbtnUrp5x6Stb9UqVLs3XrVooVK06blsnUqVeflJQUAi6Q1eBECy/ntt9vv25i3do1XFLtMmrWrkvbVsmUKFmSQoULsWLFMupH0RHAgylYsBC16tSlTcu/81q5PPry8uq66NW8cpDqYxTVR/DuOu7VvMDbue3326+/sm7tGi6udik1a9WhfevmwRpZqBArVyynXoPoqZFeHq9oyC0uIq964nqaWXLodhmCxW8fsP9cmaXA1aHbNYBmodujgedCt68J/Xwbul8otJwU4Bfn3KKDvXDoSPEdAHFn1COuxIkN3LqftvL8O58yddBd7PXtY9m6TWRmBrijVW16Pf8Bk2d/R4urL+KNR9tzQ5fXmPPVWua0XwtAu0aXMXPBKiqeVYq7b7mSHbv3cv/AiaT5Mk4optzSq0/frNs9unXhkf6PMfTNN1i/bi3Va9SkRavWEYzuxHglt717/+L+e3py34N9KFSoEB06daZDp84APP7ow3S9qyeT3p/Aoi8WUjGpEp3v7BrhiI9P9rwe6/cwXbv35IOJE1j0ZTCv26M0L/DOunggr+YVBhGrjxDeGvlfro/g3XXcq3mBd3Lbu/cvet3bk/t69aZQoULc2qkzt4Zq5BOPPkyXbj2Y/P4EFn35BRWSkuh8R3TWSK+M18Hkpdyi7pNNM6sHXAXUCB2l/RZIADKccy40WyZHbqQNeMY5d2Hop4Jz7u3QY38d6knOuSHOuWrOuWon2mjuN2Lyl9Rs/yxX3/YSO3fvZcMv22jf6HImz/4OgPc//TbrAgj7JSbEc3Pjyxk8/nMe7nIDnR8ZyRff/Uib6y4NS0zhVKp0abZs3pJ1f9vWrZQuXTrr/tw5s6hStSp79+5l48YUBr7wMp9+MpO0tLRIhHtMvJxbRkYG99/Tk+tvaMyVV13zj8fWrlmNc46yZcvx6Scz+N/zL7FxYwopv/wcmWDDJHtesz6ZwbPPv8SmjSn8EgV5eXVd9GpeOSHS9RHCXyO9Xh/Bu+u4V/MCb+fmz8ig173/R8MbGtPgYLUfx1llyzHr05kMeO5Fft24Mc/Xfi+PVzTkFnXNJlAU2OGc22tmlYHqR5h/EdAidLtNtukzgU5mVgjAzE43s1Jhj/YolCxWCIAypxSjaYMLGPfxEjZv30XtSyoCUO+yJL4/4Dsn99xyFYPGzMPvD5CYEI/DEQgEKJCQL9fjP5Kq555HSsrPbNq0kYx9+5jx0XTq1m8ABBuaUe+OoEOnzqT70jEzAAKBTDIy8v4RaK/m5pzj8UcfptzZ5bnp1n9fyXHQay/TrXtP/H4/gcwAADExMfh8vtwONawGvfoy3XoE88oMBPMyi8GXlvfz8uq66NW8cojqI9FVH8G767hX8wLv5pZV+8udzU23dPjX44Nff4Wud/1fqPZnAmAxludrv1fHC6Ijt2g8jXYG0MXM1gDrCBbLw7kbGGVmfUPP3QXgnPvEzM4Bvgy9+XuAmwge9c1VY57rTPGTCpLhz+TuAePZtSeNu54YzcAHWhIXF0N6up/uT47Jmv/UkkWpdu5ZPD3kYwDeGDOPBaN6sSt1L63vHXqol4mYuLg4+vTtR9c7OhMIZNIsuQUVKgT/UBg35j2aNE0mMTGRpEqV8KX5aNGsMbVq16FIkSIRjvzIvJrbd99+w/SpH1KhYhJtWjYDoHvPe6hVpy5zZ8+iStVzKVkqeOSsUuXKtE5uTMWkSiRVqhzBqI9O7wfuZenixezcuYNrr6xLl249SG7RMiuvUvvzqlSZVqG8KlXO+3l5dV30al45RPUxyuojeHcd92pe4N3cln37DR9Nm0KFikm0axU8G79bz7upVbsun82ZxTlVzqVkqeBxp6RK53Bj8yZRUfu9Ol4QHbnZ32fWeJOZFQDSnHPOzNoAbZ1zTcOx7MSLunvyzdux+LVIhyDHKDPgyVWR0EE4T4rxcnIelRCHpwYtJ+sjqEaK5LSM0JlFXhMfG40nXv63Ha4+RuMnm8fqEuA1Cx6e3Ql0imw4IiIieYLqo4iI5CjPN5vOufnABZGOQ0REJC9RfRQRkZymz6lFREREREQk7NRsioiIiIiISNip2RQREREREZGwU7MpIiIiIiIiYadmU0RERERERMJOzaaIiIiIiIiEnZpNERERERERCTs1myIiIiIiIhJ2ajZFREREREQk7NRsioiIiIiISNip2RQREREREZGwU7MpIiIiIiIiYadmU0RERERERMLOnHORjiFq+fx48s0rds1TkQ4hx+z4pG+kQxABINXnj3QIOSIh3rvHMAvnj7FIxxBNPFsjL+0e6RByxI7Fr0U6BBEAAgFP7jo8rUA+O2R99O5fBSIiIiIiIhIxajZFREREREQk7NRsioiIiIiISNip2RQREREREZGwU7MpIiIiIiIiYadmU0RERERERMJOzaaIiIiIiIiEnZpNERERERERCTs1myIiIiIiIhJ2ajZFREREREQk7NRsioiIiIiISNip2RQREREREZGwU7MpIiIiIiIiYReRZtPMyprZyhx+jXpmNi0nXyMnLJz/OU1uuJZGDa/m7aFDAOjT6z5aJjfmlZdeyJpvyOBBzJk9K1JhHpW7ml/KkrdvZ+mwO+je4lIAmtetzNJhd/DXrIe4OOnUrHlrVD2Dr4d2ZsEbnSh/ejEAihbMz9Rn22IWkfCPmpfGLDvlFV15AaSm7ubhXnfTrnkj2rdozMrl3zHolee59cZknujXJ2u+mR9NZfzodyMY6dHbsmUzd952K62aNaJ1ciPGjArG/cqLz9GmRVP6PfRg1rwfTZvC6JEjIhVqWKg+Hp6Xtt+72tZjyYSHWDqxL93b1QPg/KTTmTfiPhaN7c2C93pRrepZADS78kKWTuzLrLfvpnjRggCUO6MEIwd0jFT4R8VL43Ugr+bm1bz6P/IQDepeQcvkxlnTXn7hOVo3b8LD2erI9KlTeC+K6kg05BV1n2yaWWykY8gpmZmZPP3U4wwa/BaTpkxnxkfTWL9uLfkTEpg4aSqrVq4gNTWV7du3sWL5chpceVWkQz6kKmVL0vGGC6ndbTiXdR7KddUrcvZpxVj103baPDqRBctT/jH//7W+nOQ+4+j1+ifc3vhiAHrfXItn31uIc5HI4Oh4acyyU17Rldd+Lw98hstr1GL0B9N4Z+z7lChZivVr1zBi3CTi4+L5YcN60n0+Ppoyieat2kY63KMSFxvLPff1YsLkaQwfNY4J40azft1a1q5Zzdj3PyQ+Pp7v16/H5/MxdfIkWrdpF+mQI8bL9RG8tf1WKX8qHZtfQe2bB3LZjc9wXZ1zObtMCZ66uxlPDfmY6m0G8MQb03jq7mYAdG1Tl1o3Pctb7y/kxuuqAdD/rkb0H5R3jxl4abwO5NXcvJoXQOOmybz+xtCs+6mpqaxZs5rxH0whPj6eDevX4fP5mPLhB1FVR6Ihr0g2m3Fm9p6ZrTGziWZWwMyuNLNvzWyFmQ0zs/wAZvazmf3PzL4BWpnZNWb2pZl9Y2YTzKxQaL6GZrY2NF/z/S9kZsXNbLKZLTezRWZ2fmh6fzMbYWbzzewXM2tuZs+GXn+GmcXn5huycsVyypQ5izPKlCE+Xz4aXn8D8z+fR7rPRyAQwO/3ExsTw6BXX6Fb9x65Gdoxq3zWySxe8xtp6X4yA475y1JoVrsS61L+YMPGP/81f4Y/QGJCPIkJ8WRkBih32kmcUbII85elHGTpeYeXxiw75RVdeQHsSU1l2bdLadSsBQDx8fkoUqQofr8f5xw+XxpxcXGMGTmcFje2Jy4+V3dvx61EyVJUrlIVgIIFC1K2XHm2bN6cLS8fcfFxjBoxjBvbRU9eR6D6eBBe2n4rlzuFxSt/Js2XQWZmgPlLv6dZgwtxDooUTACgaKFENm/fBUAgECB/fBwFEvKR4c+k5kXl2fr7bn5I2R7JNA7LS+N1IK/m5tW8AC6pdilFixbNuh8TY/j9GcE6kpZGXFw8774zjDZtbyI+iupINOQVyWazEjDIOXcOsBu4F3gHuNE5dx4QB3TNNv8fzrmLgVnAw8BVoftLgHvNLAEYCjQGLgFOyfbcx4BvnXPnAw8B2c8fKw80AJoAo4C5oddPA24Ia8ZHsG3rVk459e+wS5UuzdatWylWrDhtWiZTp159UlJSCLgA54T++MqrVv20nZrnlaF4kUQS88fR8PLynFGqyCHnHzj6C97u3YQH2l7B4ElLeKxTPfoP+yz3Aj5OXhqz7JRXdOUFsPm3TZxUrBhP9+9Lx3YtGPB4PyzGqFGzNh3bteDkEiUpWKgwq1euoE79KyMd7nH57ddfWbd2DRdXu5SaterQvnVzSpQsSaFChVi5Yjn1GkTPUfYjUH08CC9tv6t++I2aF1WgeNGCJCbE07BWVc44pRgPPDeRp+9uxoaPn+CZe5Lp9+qHAAwc9inTB/fg+jrnMn7GEnrf3pBnhs6IcBaH56XxOpBXc/NqXgdTsGAhatWuS5tWycE6UrgQK1cso34UfVp7MHkxr7iIvTJsdM4tDN0eBTwC/OScWx+aNgK4C3gpdH9c6Hd1oAqw0IJf5ssHfAlUDj1/A4CZjQLuCD2nFtACwDk3x8xONrP9nc/HzrkMM1sBxAL7994rgLJhy/YE9OrTN+t2j25deKT/Ywx98w3Wr1tL9Ro1adGqdQSjO7h1KX/w/NgvmfpsW/b6Mlj2w1YyA4c+H3b5D1up2/0dAGqeX4Ytf+7BzBj5SDIZ/kx6D57Nth1/5VL0Jy4ax+xoKK+8KzMzk/Vr13D3A32pet75vDTwGUYNf4vbu/Wk/a23ATDg8X7c1qU7UydN5OtFX1C+YhIdOneJcORHZ+/ev+h1b0/u69WbQoUKcWunztzaqTMATzz6MF269WDy+xNY9OUXVEhKovMdXY+wxDxN9fEYROP2u+6nrTz/zqdMHXQXe337WLZuE5mZAe5oVZtez3/A5Nnf0eLqi3jj0fbc0OU15ny1ljnt1wLQrtFlzFywiopnleLuW65kx+693D9wImm+jAhndXSicbyOlldz82peHTp1pkOojjz26MN0vasnH7w/gUVfLKRiUiVuvzM660heyyuSn2we2HnsPML8+zsNAz51zl0Y+qninLvtBOJIB3DOBYAM57K+IRjgIM24md1hZkvMbMn+L06HS6nSpdmyeUvW/W1bt1K6dOms+3PnzKJK1ars3buXjRtTGPjCy3z6yUzS0tLCGke4jPh4GTW7DOPqu0eyM9V30NNnD6b3TbV4ZuQC+t5Sm75DZjNs+nd0S740h6M9Pl4bs/2UV3TlBVCyVGlKlipN1fPOB6D+Vdewfu2arMfXr12Dw3Fm2bLMnTWTJ/73Ar9t2sjGlF8iFfJR82dk0Ove/6PhDY1pcNU1/3hs7ZrVOBxnlS3HrE9nMuC5F/l140ZSfvk5MsGGR1TWR1CNPBYjJn9JzfbPcvVtL7Fz9142/LKN9o0uZ/Ls7wB4/9Nvsy4QtF9iQjw3N76cweM/5+EuN9D5kZF88d2PtLku79VIr41Xdl7Nzat5HcnaNatxzlG2bDlmfTKDZ59/iU0bU/gluutInskrks3mmWZWI3S7HcHTfcqaWYXQtJuBeQd53iKg5v75zKygmSUBa0PPLx+aL/vVL+YD7UPz1wN+d87tPp6gnXNDnHPVnHPVbrv9jiM/4RhUPfc8UlJ+ZtOmjWTs28eMj6ZTt34DADIyMhj17gg6dOpMui+d0FFrAoFMMjLy5tHMkicVAKBMqSI0rV2JcbOPfIHF9tecx8yvvmdHqo8CCfEEAg7nHAUSIvkh/KF5bcz2U17RlRfAySVKUqr0KaT8/BMAS75eRNmzy2c9/tYbr3J71x74/X4yAwEALCYGny9v/5HgnOPxRx+mXLmzuemWDv96fPDrr9D1rv/D7/cTyMwEwGIMn8+Xy5GGVVTWR1CNPBYlixUCoMwpxWja4ALGfbyEzdt3UfuSigDUuyyJ7w/4TuY9t1zFoDHz8Ieuc+BwBAIBCiTky/X4j8Rr45WdV3Pzal5HMui1l+nWvWewPmZmr49RXUfyTF6R/At+HXCXmQ0DVgM9CRbKCWYWBywGBh/4JOfcdjPrAIzZf4EE4GHn3HozuwOYbmZ7CRbQwqHH+wPDzGw5sBe4NefSOn5xcXH06duPrnd0JhDIpFlyCypUCBadcWPeo0nTZBITE0mqVAlfmo8WzRpTq3YdihQ59HchI2lM/xYUL5JIRmaAu1+eya6/0mlSqxIv9LiGEkUL8MHTrVn+w1aaPDgWgMT8cdx87fk06jUGgFcmfMWkZ9qwz59Jh6cmRzCTQ/PamO2nvKIrr/3u6fUQjz38IP6MDE47/Qz69H8SgM/nzqZSlaqUKFkKgIpJlbmldTPKV0yiYlLlSIZ8RMu+/YaPpk2hQsUk2rVKBqBbz7upVbsun82ZxTlVzqVkqWBeSZXO4cbmTaiYVImkSnk7ryNQfTwIr22/Y57rTPGTCpLhz+TuAePZtSeNu54YzcAHWhIXF0N6up/uT47Jmv/UkkWpdu5ZPD3kYwDeGDOPBaN6sSt1L63vHXqol4kYr41Xdl7Nzat5AfTudS9LFy9m584dXHtlXbrc1YPk5i2ZO3sWVaqeS6lSwU9wK1WuTKvkxlRMqkSlKKgj0ZCXubz8fyXyOJ//X6c6eUKxa56KdAg5ZscnfY88k0guSPX5Ix1CjkiIj7r/qHXUCuePyeP/9Tdv8WyNvLR7pEPIETsWvxbpEEQACBzmGh+SNxXIZ4esj979q0BEREREREQiRs2miIiIiIiIhJ2aTREREREREQk7NZsiIiIiIiISdmo2RUREREREJOzUbIqIiIiIiEjYqdkUERERERGRsFOzKSIiIiIiImGnZlNERERERETCTs2miIiIiIiIhJ2aTREREREREQk7NZsiIiIiIiISdmo2RUREREREJOzUbIqIiIiIiEjYqdkUERERERGRsFOzKSIiIiIiImFnzrlIxxC1tqf6Pfnm+QOBSIeQY5Lf+DLSIeSISV1rRDqEHFE4IT7SIeSYtH2ZkQ4hRyTmi410CDmmUH6zSMcQTfak6w+MaFLyuqcjHUKO+W1a70iHkCO8vL+V6JIQxyHroz7ZFBERERERkbBTsykiIiIiIiJhp2ZTREREREREwk7NpoiIiIiIiISdmk0REREREREJOzWbIiIiIiIiEnZqNkVERERERCTs1GyKiIiIiIhI2KnZFBERERERkbBTsykiIiIiIiJhp2ZTREREREREwk7NpoiIiIiIiISdmk0REREREREJu7hIB3AkZtYf2AMUAT53zs3K4ddrBqx3zq3Oydc5mJSff6LfQ/dl3f/t1010vrM7v/++na++WECFpMo88vgzAMz8aCq7du6gdbtbcjvM4zJx7CimTX4fnOOGZi1o1fZm3nz1Bb76cgEVKlbmoceeBuCTj6eya+dOWrW9OcIRH16h/LE8dF0lzi5ZEJzjyY/WU/3sYjS54FR27s0A4I15P/Hlj39y/ulF6HVtRTIyHf2mrGHjjjQK5Y/lqWZVuHvcClyEczkUr43Zflu2bKZf3wf5848/MDOSW7Sm3U238MqLz7FwwedUqnQOjz/9PwA+mjaFnTt20O7mWyMc9ZF5df/h1fEKl/9SjfTquuC1vO5qfikdb7gQM2P49G957f3FNK9bmb631qHymSWo3W0436zfDECNqmfw8t0N2ecPcOuTk/jh1x0ULZifUY82p8mDY3B5tUACmZmZdGzfipKlSvP8K2/Q76EH+OH7DdSqXZeuPe4BYNjQwZSvUIG69a+KcLRHb+H8z/nfgKcIZAZIbtGK226/gz697mPDhvXUqVufnnffC8CQwYOoUDGJBldGR25ezQvyfm5R88mmc65fThfRkGZAlVx4nX85s2w53hn9Ae+M/oC3R04gISGBOvWvYv3aNYwYO4n4+Hh++H496T4fH02dRPPWbSMR5jH78YcNTJv8PoPfGc1b703kywXz+H79OtavW8Ow0R8QFx/Pj6G8Zkz9kORWbSId8hHdc1UFFv34J22GLuamYUv5+Y+/ABi7eBO3DF/KLcOX8uWPfwLQ7rIzuGfCCl6a/T3JF50KQMcrzmLElyl5ttH04pjtFxsbyz33PcjEydN5Z9RYJox7j/Xr1rJ2zWrGvT+FuPh4Nqxfh8/nY8rkD2jVpl2kQz4qXt1/eHW8wu2/UCO9ui54Ka8qZUvS8YYLqd1tOJd1Hsp11Sty9mnFWPXTdto8OpEFy1P+Mf//tb6c5D7j6PX6J9ze+GIAet9ci2ffW5inG02AcaNHUrZceQA2rF9H/vwJvDd+MqtXrWRPaiq/b9/OqpXLo6rRzMzM5OmnHmfQ4LeYNGU6Mz6axvp1a8mfkMDESVNZtXIFqampbN++jRXLl0dNQ+bVvCA6csuTzaaZ9TWz9Wa2AKgUmvaOmbUM3R5gZqvNbLmZPReaVt7MFpnZCjN70sz2hKbXM7Np2Zb9mpl1ONhyzOwKoAkw0My+M7PyuZv535YuXsTpp5ehSJGi+P1+nHP4fGnExcUxZtRwWtzYnri4+EiFd0xSfvqRKlXPIyEhkbi4OC68uBoL5s3Jyivdl0ZsXBzj3nuH5NZt83xeBfPHclGZokxZvgUAf8CxJz3zkPP7A46EuFjyx8Xiz3ScflICpYvk55uUXbkV8jHz2phlV7JkKc6pUhWAggULUa5cebZs/g2/P+Pv7Sw+npEjhnFju5uIj4+e3Pbz0v7jvzBex+q/WiO9ui54Ka/KZ53M4jW/kZbuJzPgmL8shWa1K7Eu5Q82bPzzX/Nn+AMkJsSTmBBPRmaAcqedxBklizB/WcpBlp53bNu6hS8WzKNJcgsA4uLiSE/3EQgE8Pv9xMTGMOSNV7m9S/cIR3psVq5YTpkyZ3FGmTLE58tHw+tvYP7n80j3/Z1bbEwMg159hW7de0Q63KPm1bwgOnLLc82mmV0CtAEuBK4HLj3g8ZOBZKCqc+584MnQQy8DLzvnzgM2HcXr/Gs5zrkvgCnAA865C51zP4Qnq2M3a+bHXHXt9RQoWJAaNWvTsX0LTi5RkoKFCrN65Qrq1LsyUqEds3LlK7L8u2/YtXMnPl8aixbOZ9fOHVS/ojadb2rFySVKUiiUV+0oyOu0ogns2JvBIzdUYkTHi3nouiQS4oObUqtLTmdUp0voe30ShfMHz1If8WUKjzaqzK01zmTCN7/SpU453vz85whmcGReG7ND+e3XTaxdu4aLq11GzVp1adc6mRIlS1KoUCFWrlhG/QbRc3QzOy/tP7Lz6ngdC9XIIK+uC9Ge16qftlPzvDIUL5JIYv44Gl5enjNKFTnk/ANHf8HbvZvwQNsrGDxpCY91qkf/YZ/lXsDH6cWBA+j+f/djMcHaX+7s8hQrVpxb27agVp16bNqYggsEqHxORE4COG7btm7llFNPybpfqnRptm7dSrFixWnTMpk69eqTkpJCwAWyDpBEA6/mBdGRW178zmZtYJJzbi+AmU054PFdgA94O3Q0dv8R2RoET+8BGA08d4TXOdRyIi4jYx8LP59Ll+53A9D+1ttof+ttAAx4oh+3denO1MkT+XrRF5SvkESHzl0iGO2RnVXubNre0okHet5BQkIiFZIqExMbS9tbOtH2lk4APPvko3S68y6mTX6fJV99wdkVkrjltjsjHPnBxcYYlU4pzAuffs+qzancc1V5bql+JhOX/sqwhb/gHNxZpyw9rzybpz5az4Ztf9F55LcAXFimKL//tQ+AJ5uegz/T8cqcH/gz9D3PvMJrY3Ywe/f+xQP39uT+Xn0oVKgQt3bqzK2dOgPw+KMP06VbTya9P4FFXy6kYlIlOt/RNcIRHx2v7T/28+p4HYf/fI306rrghbzWpfzB82O/ZOqzbdnry2DZD1vJDBz6fNjlP2ylbvd3AKh5fhm2/LkHM2PkI8lk+DPpPXg223b8lUvRH50Fn39GseLFqVylKkuXfJ01/Z4H+mTdvu//utG7b3+GvzWY79ev49LqV9CseatIhBsWvfr0zbrdo1sXHun/GEPffIP169ZSvUZNWrRqHcHojp9X84K8lVue+2TzSJxzfuAyYCLQCJhxhKf4+WeeCce5HADM7A4zW2JmS94dPvQYoz86ixYuIKlyFYqfXOIf09evXYNzjjPPKsvcWTN5YsAL/PbrRjam/JIjcYTTDU2bM+Td8bwyZASFixShzJlnZT22Yd0acI4yZ5Vl3uxP6P/M8/z260Y25dG8tqWmsz01nVWbUwGYs/Z3KpUuxJ97Mwg4cMCHyzZT5dR/H83teMWZDF/4C51rncVrc3/kw2WbaV3t9FzO4Oh4acwOlJGRwQP39uS6GxrT4Kpr/vHY2jWrAUfZsuWY9ekM/vfcS2zamELKLz9HJNZj5cX9h5fHK9zyUo0c9taQY4z+yLy6LngprxEfL6Nml2FcffdIdqb6Dnr67MH0vqkWz4xcQN9batN3yGyGTf+ObsmXHvmJuWz5d98wf95cml1/FY/0vo8li7/i0b69sh7/fO5sKp9ThbS0vfy6aSNPPfsic2d9gi8tLYJRH51SpUuzZfOWrPvbtm6ldOnSWffnzplFlapV2bt3Lxs3pjDwhZf59JOZpOXx3LyaF0RHbnmx2fwcaGZmiWZWGGic/UEzKwQUdc59BNwDXBB6aBHQInQ7+9VKfgGqmFl+MzsJuPIIy0kFCh8qOOfcEOdcNedctVs63n4CaR7arJkfcdW11/9r+luDX+X2rj3w+/1kBgIAmMXg8+X9jWHHn38AsHXLZj6fO4srs+X39uDX6NSleyiv4HcfYywGn88XkViP5M+/Mti6O50ziycCcGnZk/jpj72cXDBf1jx1k0rw4/Z/Ho29/tzSfPHDn+z2+ckfH0vAQcBB/vjYXI3/aHlpzLJzzvHEow9Trlx5brql478ef+P1l+l6V0/8fj+BzOB2FhMTHbmB9/YfXh+v4xA1NbJT5ztOIM2DLtuT64LX8ip5UgEAypQqQtPalRg3e+URn9P+mvOY+dX37Ej1USAhnkDA4ZyjQELeOwGvW897mTpzLpM/msUTA56n2qWX89hTzwLgz8hg7OiR3HzrbaT7fBgGBC/ikuHPW2cwHUzVc88jJeVnNm3aSMa+fcz4aDp16zcAggdERr07gg6dOpPuS8csmFsgkElGRt7Ozat5QXTklue2YufcN2Y2DlgGbAMWHzBLYeBDM0sADLg3NP1uYJSZ9SV4BHZXaHkbzWw8sBL4Cfj2CMsZCww1s55Ay9z+Tkpa2l4Wf/0FD/R99B/TP/9sNpXOqUqJkqUAqJhUmVtubEb5iklUTKqcmyEel34P3svu3TuJi43j7gf6Urhw8FO/+QfkVSGpMh3bJlO+QhIVkipFMuTDev7TDTzW+BziY41fd/p4cvo67r26AhVLFQRg8y4fA2ZsyJo/f1wMN5x3Cj3HLQdgzNebeKHVufgDwX+Hkhd5bcz2++7bb5g+7UMqVEyibatmANzV8x5q1a4bPAJY5VxKlgoeFUyqVJnWzRtTMakSSZXy/nbmxf2Hl8frePyXa6RX1wWv5TWmfwuKF0kkIzPA3S/PZNdf6TSpVYkXelxDiaIF+ODp1iz/YStNHhwLQGL+OG6+9nwa9RoDwCsTvmLSM23Y58+kw1OTI5jJsZs4fgzXN25KQmIiFZIq4fP5aN+qKTVq1cmqoXlZXFwcffr2o+sdnQkEMmmW3IIKFSoCMG7MezRpmkxiYiJJlSrhS/PRolljatWuQ5EieTs3r+YF0ZGbubx+bemjZGYFgDTnnDOzNkBb51zTnHzN7al+b7x5B/CHPvXwouQ3vox0CDliUtcakQ4hRxROyLtXXTxRafsOfQXjaJaYL29+Uh8OhfKHDgtHoUjUyD3pHvkD4z+i5HVPRzqEHPPbtN6RDiFHeHl/K9ElIY5D1sc898nmCbgEeM2CnxHvBDpFNhwREZE8QzVSRERynWeaTefcfP7+TomIiIiEqEaKiEgk5MULBImIiIiIiEiUU7MpIiIiIiIiYadmU0RERERERMJOzaaIiIiIiIiEnZpNERERERERCTs1myIiIiIiIhJ2ajZFREREREQk7NRsioiIiIiISNip2RQREREREZGwU7MpIiIiIiIiYadmU0RERERERMJOzaaIiIiIiIiEnZpNERERERERCTtzzkU6hqi16te/PPnmlS9dMNIh5JhAwJNDxtUvL4h0CDliUpfqkQ4hx8SYRTqEHBEb4828AIoViPVucjkg1efNHW58nI7TR5ti1w+MdAg54rfJ90Y6hBwR5+E64tX9R0Ichxw0b2YsIiIiIiIiEaVmU0RERERERMJOzaaIiIiIiIiEnZpNERERERERCTs1myIiIiIiIhJ2ajZFREREREQk7NRsioiIiIiISNip2RQREREREZGwizuamczsCqBs9vmdc+/mUEwiIiJRQzVSRETk4I7YbJrZSKA88B2QGZrsABVSERH5T1ONFBERObSj+WSzGlDFOedyOhgREZEooxopIiJyCEfznc2VwCk5HYiIiEgUUo0UERE5hEN+smlmUwmeClQYWG1mXwPp+x93zjXJ+fBERETyHtVIERGRIzvcabTP5VoUIiIi0UU1UkRE5AgOeRqtc26ec24ecP3+29mn5V6Ix87MeprZGjN7L9KxHMlrz/anQ/Mr+b9OrbKmPff4g9x7exvuvb0Nd7a9gXtvbwPAmpXfcU/n1jzQpT2/bUoB4K89qTz2QDcCgUBE4j9aC+d/TpMbrqVRw6t5e+gQAPr0uo+WyY155aUXsuYbMngQc2bPilSYx6z/Iw/RoO4VtExunDXt5Reeo3XzJjz80INZ06ZPncJ7I0dEIsRjUih/LE82OYfRnS7hvU6XUPW0wlmPtal2OgsfqE3RxOAxqnpJJzOq48UMans+RRKC004/KYHHG1eOSOxHK+Xnn+jYrkXWz7V1L2f86JG88coL3NommSf79cmad+ZHUxk/emQEoz02qam76dvrbto2b0S7Fo1Zufw7Br3yPLfcmMwTB+Q1bnR0Xb8mMzOTW9o0576eXQHo99ADtG/djDdefTFrnmFDBzNvbu7sP1Qjc196ejq3tGtN21bNaJ3ciDcHvQrAw30eoE3Lprz+yt/rwltD3uCzOdFTS7xaI72U113NLmbJkA4sHdKR7smXAFCscALTBrRixfDOTBvQipMK5QegWa0klg7pyKzn21K8cAIA5U49iZEPNT7k8vOSvLa/PVFe3ndA3t/OjuY7m1cfZNp14Q4kzLoBVzvn2h/vAszsqP4tzImqf21jHhnw2j+m3d/vf7wwdCwvDB1L9TpXUr12AwCmjB9J32depdNd9zNz6kQAJox8ixbtOxETk3f/ZWpmZiZPP/U4gwa/xaQp05nx0TTWr1tL/oQEJk6ayqqVK0hNTWX79m2sWL6cBldeFemQj1rjpsm8/sbQrPupqamsWbOa8R9MIT4+ng3r1+Hz+Zjy4Qe0btMugpEenbsblOern/6k3bCl3PrON/zyx14AShXOx2Vli7Flly9r3hYXncZtI79j8rLNXHNOKQBur3UWQ+b/HInQj9qZZcsxfPT7DB/9Pm+NHE9CQgJ16l/J+rWrGTF2EnHx8fzw/XrSfT4+mjqZ5q3bRDrko/bSwGe4vEYtxnwwjRFj36dEyVKsW7uGd8dNIi4unh82BPOaPmUSLVq1jXS4x2Tc6JGULVcegA3r15E/fwLvjZ/M6lUr2ZOayu/bt7Nq5XLq1s/1/YdqZC7Jly8fg98azpgJkxk9fhJfLFzAN0sXkz9/fsZO/JBVq1aE1oVtrFqxnHoNoqOWeLVGeimvKmVL0PH686ndYxSXdXmH6y4vz9mnncT9N17OZ9/+wnkd3+Kzb3/h/hsvB6Br04uo1WMkb320jBsbnANA/w616P/O/EimcdTy8P72uHh13wHRsZ0dskMxs65mtgKobGbLs/38BKzIvRCPjZkNBs4GPjazvmY2zMy+NrNvzaxpaJ6yZjbfzL4J/VwRml4vNH0KsDo34q16wSUULlL0oI855/jis0+p1aAhALFxcezz+Uj3+YiLjWPLrxv5ffsWzr2wWm6EetxWrlhOmTJncUaZMsTny0fD629g/ufzSPf5CAQC+P1+YmNiGPTqK3Tr3iPS4R6TS6pdStGif49fTIzh92fgnMOXlkZcXDzvvjOMNm1vIj4+PoKRHlnBfLFccEZRpq7YCoA/4NiTHvxPDj3rl2fQvJ/IfrlN5yBfbAwJcbH4AwEuOL0If/6VwaadvoMsPW9aungRp51ehiJFiuL3+3HOBbevuDjGjHqHFje2Iy4ub4/bfntSU1n27VIaN2sBQHx8PgoXKUpmVl5pxMXFMXrkcFre2J64PL4+Zrdt6xa+WDCPJsnB3OLi4khP/3v/ERMbw5A3XuX2Lt1zLSbVyNypkQfEToECBQHw+/1Z+9r09PR/rAuDB73Knd1yb104UV6tkV7Kq3KZ4ixeu5m0dD+ZAcf8FRtpVjOJRjUqMOrTVQCM+nQVja+oCEDAOfLHx1IgfxwZ/gA1zz2drTv+4offdkYwi6OTF/e3J8qr+w6Iju3scB+HjQYaAx+Gfu//ueREjobmNOdcF+A3oD5QEJjjnLssdH+gmRUEthE8qnsxcCPwSrZFXAz8n3MuKXcj/7fVy7/hpGLFOe2MMwFo0a4TLw94hA/GDOe65Bt5b9jrtOt0V4SjPLJtW7dyyql/X6yxVOnSbN26lWLFitOmZTJ16tUnJSWFgAtwTpWqEYz0xBUsWIhatevSplUyJUqWpFDhQqxcsYz6efiI7X6nnZTAzrQM+l6XxPBbLqL3tRVJiI+hVoXibN+Tzvfb//rH/CO/2shLrc+lZvnifLpmOx2uOJPhX6ZEKPrjM3vmx1x17fUUKFiQ6jXr0Kl9S04uUZKChQqzeuVy6tS7MtIhHrXfftvEScWK8VT/vnRo14JnHu9HTIxRo2ZtOrRrkS2vFdSpHz15Abw4cADd/+9+LHQGR7mzy1OsWHFubduCWnXqsWljCi4QoPI5VXIzLNXICMjMzKRd62Surl+Ly6tfwSXVLqNYsWLc1KYFderUZ2NKCoFAgMrnRE8t8WqN9FJeq37+nZrnnkHxwgkk5o+j4aVnc0bJwpQqVoAtfwZr45Y//6JUsQIADBz7FdP/15rrq5dn/Nw19G5/Bc+892UkUzhqeXR/e8K8uO+A6NjODnkajHNuF7DLzB484KFCZlbIORcNf1VeAzQxs/tD9xOAMwkW2tfM7EKC/4Q7e9H82jn3U65GeQgL5szM+lQToFyFSvzv9eD3rFYtW0qx4iXAOZ57/EHi4uLo0OVeTip+cqTCPWa9+vTNut2jWxce6f8YQ998g/Xr1lK9Rk1atGodweiOX4dOnenQqTMAjz36MF3v6skH709g0RcLqZhUidvv7BrhCA8u1oyk0oV4cfYPrN6cyv81OJvbrjiLC84owj0TVv5r/sW/7GTxyO8AaFi1FF/++CdnFkuk7aVnkOrL4KU5P5Luz7vfJc7IyGDh559xZ/e7AWh/ayfa39oJgAFP9OO2Lt2ZOnkiixd9SfkKSdza+c4IRntkmZmZrF+7hnse6EvV887npYHPMHL4W9zRrSftb70NgGce70fnLt2ZMmkiixd9QfmKSXTo3CXCkR/egs8/o1jx4lSuUpWlS77Omn7PA39/B/W+/+tG7779Gf7WYL5fv45Lq19Bs+atDra4sFGNjIzY2FhGj59E6u7d3H9PD77fsJ77ej2U9fg9Pbry0COP8fbQwWxYv47Lq9cguUV01hKv1shozGvdxj95fvzXTB3Qir2+DJb9sI3Mg1wrY/9/253zzS/M+Sb4ff92V1Vl5tc/UvH0Ytzd6lJ2pPq4/405pKX7czOFo5JX97fh8F/ad0De2s6O5ot+04Fpod+zgR+Bj3MyqDAyoIVz7sLQz5nOuTXAPcBW4AKC/5A7X7bn/HWQ5fy9QLM7zGyJmS2ZMGpYjgWemeln0YI51Kx/zb8ec84xcdTbtLr5dsa9O4Rb7vg/rr6hOdMnjcmxeE5EqdKl2bJ5S9b9bVu3Urp06az7c+fMokrVquzdu5eNG1MY+MLLfPrJTNLS0iIRbtisXbMa5xxly5Zj1iczePb5l9i0MYVffvk50qEd1LY96WxPTWf15lQAPlv3O0mlC3Fa0QRGdLiYiXdcSsnC+Rl2y0UUL/j3KZj542K4vmpp3v92M7fVPIsnP17H8l93c02VUpFK5agsWjifpMrnUPzkEv+Yvn7tGnCOM88qy9xZn/D4gOf59deNbEz5JUKRHp1SpUpTslRpqp53PgD1rrommEtI8LbjzLJlmTtrJk/87wV+3ZT381r+3TfMnzeXZtdfxSO972PJ4q94tG+vrMc/nzubyudUIS1tL79u2shTz77I3Fmf4Mu9/YdqZPYFZquRw98ekmOBFy5ShGqXXsaXXyzImvbZ3NlUrlKVvXv/YtPGjQwY+CKzP83VdeG4eLVGei2vETNWUPOukVx931h27vGx4dcdbNuxl1OKB0/PPKV4Qbbv3PuP5yTmj+Pma6oyeMq3PHxLTToP/JgvVv1KmwZ581PBKNjfnjAv7TsgOrazIzabzrnznHPnh35XBC4DouNcAJgJ9DAzAzCzi0LTiwKbnXMB4GYg9mgX6Jwb4pyr5pyr1uqmTmEPeL9lS7/i9DJlKVGy9L8e++yTaVx8eU0KFylKus9HTEwMZka6L29+V67queeRkvIzmzZtJGPfPmZ8NJ269YMXPcrIyGDUuyPo0Kkz6b50QkNFIJBJRkZGJMM+YYNee5lu3Xvi9/vJzAweAbWYGHx5dJz+/CuDbanpnFksEYBLzjqJ9Vv30GjQV7QcspiWQxazPTWdTu9+y59//T027S47gwnf/EZmwJE/LgbnIOAgIS7vXrQKYNbMj7jy2n9fNPStwa/SuWsP/H5/1lWeY8xI9+XtonNyiZKUKn0Kv/wc/NBp6deLKHt2+azHh75xkLxiYvDl8by69byXqTPnMvmjWTwx4HmqXXo5jz31LAD+jAzGjh7JzbfeRrrPhxHcf2RmZpLhz539h2rkP2WvkR1vuyOswe74809Sd+8GwOfz8dWiLylbthwQXBfGjHqXWzvcRnp6OqFSEhW1xKs10mt5lTwpeIpsmZKFaVqrIuPmrGH6ou+56ergqYk3XV2VaV9+/4/n3NPqMgZN/gZ/ZoDE/HE45wgEHAXy5/r1tY5KXt/fHi+v7jsgOrazY17bnXPfmNnlORFMDngCeAlYbmYxwE9AI2AQ8L6Z3QLM4AhHanPSC0/0YeWypaTu2knn1g1p06ELV13fjIVzP6F2tlNo90v3pTF35lT6Pfs6AE1atefJPj2Ji4vjnr5P53b4RyUuLo4+ffvR9Y7OBAKZNEtuQYUKwS/RjxvzHk2aJpOYmEhSpUr40ny0aNaYWrXrUKRIkQhHfmS9e93L0sWL2blzB9deWZcud/UguXlL5s6eRZWq51KqVPBgQaXKlWmV3JiKSZWoVCnv/muQF2f/wKONKhEXG8NvO9N4+uMNh52/RMF8VDmlMMO/CJ4xOPGb33j75gtJTffTZ9Kawz43ktLS9rLk6y95oO+j/5j++WezqXxOVUqUDH4qWzGpErfemEz5iklUSMq747bfPb0e4rGHH8SfkcFpp5/BQ/2fBEJHo6tUpWRWXpW5uXUzyldMomIU5HUoE8eP4frGTUlITKRCUiV8Ph/tWzWlRq06FC4cmf2HamTO+f337Tz6cB8CgUwCgQBXX9OQ2nXrAzB+3GgaNWlGQmIiFUPrwo0tmlCzVh0K5/Fa4tUa6bW8xjzSlOJFEsjwB7j71Vns+iud58Z+xaiHm3Brw/NJ2bqbm56akjX/qcULUq3SKTw96gsA3pj8DQtevZldf6XTuv+kSKVx3PLi/vZoeXXfAdGxnZlz7vAzmN2b7W4MwYsDnOycuzYnA4sGq3796/BvXpQqX7pgpEPIMYGAJ4eMq19ecOSZotCkLtUjHUKOidl/+NRjYmO8mRdAsQKx/0pONfLQUn3e3OHG5/GzNuTfil0/MNIh5IjfJt975JmiUJyH64hX9x8JcRxy0I7mk83C2W77CX4v5f0TDUpERMQDVCNFREQO4bDNppnFAoWdc/cfbj4REZH/GtVIERGRwzvkZ7lmFuecywRq5mI8IiIieZ5qpIiIyJEd7pPNrwl+9+Q7M5sCTCDbRQKccx/kcGwiIiJ5lWqkiIjIERzNdzYTgD+ABoAj+H+5HKBCKiIi/3WqkSIiIodwuGazVOgqeyv5u4Du58krzImIiBwl1UgREZEjOFyzGQsUgoNeylaFVERE/stUI0VERI7gcM3mZufc47kWiYiISPRQjRQRETmCw/1nUe/+R1UREZEToxopIiJyBIdrNq/MtShERESii2qkiIjIERyy2XTO/ZmbgYiIiEQL1UgREZEjO9wnmyIiIiIiIiLHRc2miIiIiIiIhJ2aTREREREREQk7NZsiIiIiIiISduac/vf08Xr80+89+eb1ql8h0iHkmAx/INIh5Ig96f5Ih5Ajhn6dEukQcsz9dctHOoQcERPj3f8IkhCnf3dyLDbv2ufJGlmsYL5Ih5AjvFofAfZlejO3creNjnQIOWLbqFsiHYIco8PVR32yKSIiIiIiImGnZlNERERERETCTs2miIiIiIiIhJ2aTREREREREQk7NZsiIiIiIiISdmo2RUREREREJOzUbIqIiIiIiEjYqdkUERERERGRsFOzKSIiIiIiImGnZlNERERERETCTs2miIiIiIiIhJ2aTREREREREQk7NZsiIiIiIiISdmo2RUREREREJOxyrNk0s7JmtjKnlh96jS9ycvm5KRDI5KMBPZj7Rv9/TF8yYTDj7m2RdX/dZ1OY9lQ35g56lEx/BgDbfljF0veH5Ga4x2zh/M9pcsO1NGp4NW8PDcbap9d9tExuzCsvvZA135DBg5gze1akwjxm6enp3NKuNW1bNaN1ciPeHPQqAA/3eYA2LZvy+isvZs371pA3+GxO9OQ2YfS7dLixGR3aJPP4w71IT0/nyUcepFO75gwd9HLWfO++/SbzP5sdwUiPjte3sf6PPESDulfQMrlx1rSXX3iO1s2b8PBDD2ZNmz51Cu+NHBGJEI+bF/cfqpHHx2v7pf28uI57uT4CpKbu5qEH7qZN80a0bd6YFcu+4/WXn+fm1sk8/kifrPlmTJ/KuPfejWCkR3bX9efw1cAmLBrYmGE9apM/PobX7qzBwv814ov/Nebde+pSMH8cAHdeW5lFAxsz8cEGxMcG24jqlUrxzC3VIpnCEXlxG9svr+cW1Z9sOueuiHQM4bJu7hSKlC7zj2l//LKB9L17/jHtpyWfcUOf1yhx9jlsXvMNzjlWfjyWcxu2zc1wj0lmZiZPP/U4gwa/xaQp05nx0TTWr1tL/oQEJk6ayqqVK0hNTWX79m2sWL6cBldeFemQj1q+fPkY/NZwxkyYzOjxk/hi4QK+WbqY/PnzM3bih6xatYI9qan8vn0bq1Ysp16D6Mht+7atvD9uNG+OGMs7YycRyMxk2uSJ5MufwLDRH7B29Ur27Enlj9+3s2bVCmrXuzLSIR+Rl7cxgMZNk3n9jaFZ91NTU1mzZjXjP5hCfHw8G9avw+fzMeXDD2jdpl0EIz02Xt5/5DQv1Ujw5n4JvLuOe7U+7vfSwGeofkUtxn4wjXfHvU+JkqVYv3YNI8dPIj4+nh82rCfd52P6lEm0aJ1368epxRK5s2Fl6j40neoPTCUmxmhxRTn6vLuEmg9O44oHp7Lp97+449rKALSuVY4avaby1frtXHnBaQA82Px8nn1/eSTTOCyvbmMQHbnldLMZa2ZDzWyVmX1iZolmdruZLTazZWb2vpkVADCzd8xssJktMbP1ZtYoNL2DmX1oZp+Z2QYze3T/ws1sT+h3vdDjE81srZm9Z2YWeuwSM5tnZkvNbKaZnRqa3tPMVpvZcjMbG5pW18y+C/18a2aFc/j9AWDvjt/5ddViKlxxbda0QCCTbye/zcXNOv1zZucIZGaSuS+dmNhYflo8l9OqXkL+grkS6nFZuWI5ZcqcxRllyhCfLx8Nr7+B+Z/PI93nIxAI4Pf7iY2JYdCrr9Cte49Ih3tMzIwCBQoC4Pf78fszcM6Rnp6elVtMbAyDB73Knd26RzjaY5OZ6Sc9PR2/34/P58M5x770v8csJiaWYW++Tsc7ukU61CPy+jYGcEm1SylatGjW/ZgYy1offWlpxMXF8+47w2jT9ibi4+MjGOmx8fL+A9XIY+al/dJ+Xl3HvVwf96Sm8t03S2ncLHhWTHx8PooULYrf7w/uc31pxMbFMXrkcFq2aU9cHt/nxsXGkJgvltgYo0D+OLbs2EtqWkbW4wn5YnE4AMwgPjaGxPxx+DMDtKl9Np9+9ys7/toXqfCPyKvbGERHbjndbFYEXnfOVQV2Ai2AD5xzlzrnLgDWALdlm78scBlwAzDYzBJC0y8LPfd8oJWZHeyz+ouAu4EqwNlATTOLB14FWjrnLgGGAU+F5u8NXOScOx/oEpp2P3CXc+5CoDaQdiLJH60l7w/homYdCdV+ANbPm8bp511OYtHi/5g3qW5jZj5/L3/t2EbJs6vw46JPSarTKDfCPG7btm7llFNPybpfqnRptm7dSrFixWnTMpk69eqTkpJCwAU4p0rVCEZ6fDIzM2nXOpmr69fi8upXcEm1yyhWrBg3tWlBnTr12ZiSQiAQoPI50ZNbyVKlufGmDrRucjUtrm9AoUKFaNnmJooWK87tN7fmitr1+HVTCi4QIKlylUiHe0Re38YOpmDBQtSqXZc2rZIpUbIkhQoXYuWKZdSPoiO24Pn9h2rkMfDafmk/L6/jXqyPAL/9tomTihXjqf59ubVtC555vB8xMUaNmrXp0LYFJ5coSaFChVm1YgV16+ftT9g370jj1WmrWPV6CzYMbsXuvfuYs3wzAIO6XMH3g1uRdFpR3pyxFoAhM9cx+4nrKXNyQRat20b7uuUZ8snaSKZwRF7exqIht7gcXv5PzrnvQreXEiyU55rZk8BJQCFgZrb5xzvnAsAGM/sRqBya/qlz7g8AM/sAqAUsOeC1vnbObQrN813otXYC5wKfhv7IjAU2h+ZfDrxnZpOByaFpC4EXzOw9ggV/0/EmfrQ2rfiahMJFOfnMimxdHzwFYe/OP0j5dgFX/d+Af81/9mUNOPuyBgCs+Hg0leo24bdVS/jx6zkULFaCi5M7YzHRcXZ0rz59s2736NaFR/o/xtA332D9urVUr1GTFq1aRzC6oxcbG8vo8ZNI3b2b++/pwfcb1nNfr4eyHr+nR1ceeuQx3h46mA3r13F59Rokt8jbuaXu3sXCeXMZO3kGhQoX5tHe9/HJx1Ppce/f3/3rc2937uvTj5HDhvDDhnVUu7wGjZq1jGDUB/df3sY6dOpMh06dAXjs0YfpeldPPnh/Aou+WEjFpErcfmfXCEd4/Dyy/1CNPAZe2i8dDS+s416sjxBsotevXcO9vfpS9bzzeXHgM4wc/hZ3dOvJTR2Cx4eeebwft3ftzpRJE/l60ReUr5hEx85djrDk3HdSwXxcf0kZzuvxAbv27uPdu+tyY61yjFvwE90Gf0GMGc91vIzmNcry3rwfGDv/R8bO/xEInj47eMZarr7wdNrWKc+vf/zFQyOX4FyEkzpKXtjGDiUv5ZbTfzGlZ7udSbC5fQfo7pw7D3gMSMg2z4GrpzvC9CO9lgGrnHMXhn7Oc85dE5rnBuB14GJgsZnFOecGAJ2BRGChmVXmAGZ2R+g0piVLpo89aNLHYvuPq9m04ism9+vIguH/Y+v65Ux/qiup239jymOdmdyvI/6MdD7s3/kfz9u78w9+/3k9ZS6owZo5k6jV6UHiEwuxZd2yE44p3EqVLs2WzVuy7m/bupXSpUtn3Z87ZxZVqlZl7969bNyYwsAXXubTT2aSlparB81PWOEiRah26WV8+cWCrGmfzZ1N5SpV2bv3LzZt3MiAgS8y+9NP8OXx3JZ+vYhTTzudk4oVJy4unjr1r2LV8r/XrQXz5pBUuQppe/fy268b6f/M88yb/Sk+X97L67+wjR3J2jWrcc5Rtmw5Zn0yg2eff4lNG1P45ZefIx3aEXl8/+HpGjnqnbcOmvTx8tJ+KTuPr+OAt+ojQKlSpSlZqjRVzzsfgPpXXsO6tWuyHl+3dg3OOc4sW5Y5n87kyf+9wK8bN7Ix5ZdIhXxI9c49lV+27+GP1HT8mY6pX6dweVKprMcDzjHxi59oevlZ/3jeKcUSuaRCCaYv2UiPG6rQ4aXP2fXXPuqde2pup3BEXt7GoiG3SByeLwxsDp2+0/6Ax1qZWYyZlSd4ms+60PSrzay4mSUCzQgeXT0a64CSZlYDwMzizayqmcUAZZxzc4EHgaJAITMr75xb4Zz7H7CYv48aZ3HODXHOVXPOVat2Q5tjSvxgLmrageZPvkuzx4dTq+ODlE46n1YDx9Pimfdo9vhwmj0+nLj4/DTt/8+ivXz6SC5odBMAmRn7MAwzw5/hO+GYwq3queeRkvIzmzZtJGPfPmZ8NJ269YOfHGVkZDDq3RF06NSZdF961mmOgUAmGRkZh1tsnrDjzz9J3b0bAJ/Px1eLvqRs2XIA+DMyGDPqXW7tcBvp6ensP4MzGnIrdcqprF65HJ8vDecc3yz+irP25+XPYOLYUbS9pWMwL4KJZebRvP4L29iRDHrtZbp174nf7yczMwCAxcTg8+X9XLy8/zgEz9TImzp0PvDhE+Kl/VJ2Xl3HvVofAU4uUZLSpU/hl59/AmDJ14soV6581uNDB73K7d164Pf7CQSC+9yYmJg8eeBj0x9/cWmFkiTmiwWg7rmnsu7XXZxd+u+vZF9frQzrf9v1j+c93PpCnhr/HQCJ+eJwOALOkZgvp0+aPHZe3cYgOnKLxBrxCPAVsD30O/sFBlKAr4EiQBfnnC/0xnwNvA+cAYxyzh14etBBOef2mVlL4BUzK0ow35eA9cCo0DQDXnHO7TSzJ8ysPhAAVgEfn2iyOeHPjT8AULxMBQDKVqvL9KfvokCxElS5Ku+dLhQXF0efvv3oekdnAoFMmiW3oEKFigCMG/MeTZomk5iYSFKlSvjSfLRo1phatetQpEiRCEd+ZL//vp1HH+5DIJBJIBDg6msaUrtufQDGjxtNoybNSEhMpGJSJXw+Hze2aELNWnUonMdzq3Lu+dS98mpuv7k1sbFxVKxUmUbJrQCYNGEs197QhISERMpXTMKX7qNj22SqX1GbwoXzdl5HK9q2MYDeve5l6eLF7Ny5g2uvrEuXu3qQ3Lwlc2fPokrVcylVKniks1LlyrRKbkzFpEpUqvSvXiHP8fL+4xBUIw/Bq/slr67jXq2P+93z4EM81vdBMjIyOO2MM+jb/0kA5oU+sS1ZMvjpYMVKlbmpdTMqVEyiYlLe2+cu+f53PvzqF+Y/0wh/IMDyn/9k+Oz1THvkGgonxmMGK3/ZwT1vf5X1nPPLBq91sOznPwGYsPAnFj3bhF//+IuXpqyKSB6H49VtDKIjN3N55MRqM3sHmOacm3jA9A5ANedcnrtU2eOffp833rww61W/QqRDyDEZ/kCkQ8gRe9L9kQ4hRwz9OiXSIeSY++uWP/JMUSgmxo48U5RKiCNiyUVjjdy8a58na2SxgvkiHUKO8Gp9BNiX6c3cyt02OtIh5Ihto26JdAhyjA5XH6PjKhciIiIiIiISVfLMidXOuQ6HmP4OwQsmiIiI/CepRoqISDTSJ5siIiIiIiISdmo2RUREREREJOzUbIqIiIiIiEjYqdkUERERERGRsFOzKSIiIiIiImGnZlNERERERETCTs2miIiIiIiIhJ2aTREREREREQk7NZsiIiIiIiISdmo2RUREREREJOzUbIqIiIiIiEjYqdkUERERERGRsFOzKSIiIiIiImFnzrlIxxC1UtMDevOiTHysN4+vZGQGIh1CjvDqeAEUa9A/whHkjB1z+kc6hByTEIdFOoZo4vPjyRqp/a1Izip2afdIh5Bjdix+LdIh5IjD1UftWURERERERCTs1GyKiIiIiIhI2KnZFBERERERkbBTsykiIiIiIiJhp2ZTREREREREwk7NpoiIiIiIiISdmk0REREREREJOzWbIiIiIiIiEnZqNkVERERERCTs1GyKiIiIiIhI2KnZFBERERERkbBTsykiIiIiIiJhp2ZTREREREREwk7NpoiIiIiIiISdp5tNMytrZu2O87l7wh3PkaSnp3NLu9a0bdmM1smNePP1VwF4uPcDtGnRlNdffjFr3reGvMFnc2bldojHzcu5LZz/OU1uuJZGDa/m7aFDAOjT6z5aJjfmlZdeyJpvyOBBzJkdHXlpvKJnvO5qeTlL3unG0hHd6N6qOgBPd72a70Z25+vhXRn35I0ULZQAQI1zy/D18K4sGHIH5c8oDkDRQglMff5mzCxiORyJ18Ysr4i2GgneXBe0v42u8drPq7l5Ka+72tZjyYSHWDqxL93b1QNg5ICOLBrbm0Vje7N2+mMsGtsbgBoXnM3X4/qw4L1elD+zJABFCyUyddBdebo+Qt4fM083m0BZ4KCF1MzicjeUI8uXLx+D3xrOmImTGT1+El8sXMA3SxaTP39+xr7/IatWrWBPaiq/b9/GquXLqdfgqkiHfNS8mltmZiZPP/U4gwa/xaQp05nx0TTWr1tL/oQEJk6ayqqVK0hNTWX79m2sWL6cBldGR14ar+gYryrlStGx0SXUvnMol3UazHU1kjj79OLMXvIjl3QYxGUd32DDpj944KZaAPxfmytI7vUevV6dwe1NqwHQ+5Y6PDtyPs65SKZySF4bszymLFFUI726Lmh/G13jBd7NzUt5VSl/Kh2bX0Htmwdy2Y3PcF2dczm7TAlu7j2c6m0GUL3NACbP/o4P53wHwP/d3IDkHm/Qa+BEbm8ZrJm9b2/Is29/kmfrI0THmOXJZjN0tHWNmQ01s1Vm9omZJZpZeTObYWZLzWy+mVUOzf+OmbXM9vz9R1wHALXN7Dszu8fMOpjZFDObA8w2s0JmNtvMvjGzFWbWNALpZjEzChQoCIDf78fvz8DhSE9PJxAI4Pf7iYmNYfDrr3Jnt+6RDPWYeTW3lSuWU6bMWZxRpgzx+fLR8PobmP/5PNJ9vqy8YmNiGPTqK3Tr3iPS4R41jVd0jFfls0qweM0m0tIzyMwMMP+7n2lW5xxmL/6BzMwAAF+v2sTpJYsAkOHPJDF/PIkJ8WT4A5Q7rRhnlCrC/O9+jmAWh+e1MQuH/2qN9Oq6oP1tdI0XeDc3L+VVudwpLF75M2m+UH1c+j3NGlz4j3laXH0x42csBUL1MSEfiQn5yPBnUu6MEpxR+iTmL90QgeiPXjSMWZ5sNkMqAq8756oCO4EWwBCgh3PuEuB+YNARltEbmO+cu9A5t/88lIuBls65uoAPSHbOXQzUB563CH9WnpmZSbtWyVxdrxaX17iCS6pdRrFixbjpxhbUqVufjSkpBFyAylWqRjLM4+LF3LZt3copp56Sdb9U6dJs3bqVYsWK06ZlMnXq1ScllNc5UZQXaLyiYbxW/bSNmuefRfEiiSTmj6dh9YqcUarIP+a55fqLmLnoewAGjlrA232TeaB9LQZ/8DWP3X4l/d+aE4nQj5rXxiyM/nM10svrgva30ZMXeDc3L+W16offqHlRBYoXLUhiQjwNa1XljFOKZT1e8+LybP0zlR9StgMwcNgnvP3EzTzQ6RoGj/2cx7o3pv+gaZEK/6hFw5jludNksvnJOfdd6PZSgqf7XAFMyFbr8h/Hcj91zv0Zum3A02ZWBwgApwOlgS3HGfMJi42NZfSESaTu3s399/Tg+w3rue/Bh7Iev6d7Vx7q9xhvDxnMhvXruLx6DZJbto5UuMfEy7kdqFefvlm3e3TrwiP9H2Pom2+wft1aqteoSYtWeT8vjVfeH691v/zO86MXMPX5m9nry2DZ91vIDPx9uk+vm2uTmRlg7KfLAVj+/Rbqdn0LgJoXnMWWP1IxjJH9W5LhD9D79Zls2/FXRHI5VtE6ZmH0n6yRB+OFdUH72+gar0Pxam7RmNe6n7by/DufMnXQXez17WPZuk1ZZ/wAtG5YjQkzlmTdX77+V+re+jwQbES3bN8VrI8DOpLhz6T3C5PY9mdqrudxvPLSmOXlTzbTs93OBIoDO0NHYPf/nBN63E8oFzOLAfIdZrnZ/5JqD5QELnHOXQhsBRIOF5SZ3WFmS8xsyfC3hhxTQseicJEiVLv0Mr5cuCBr2mdzZ1O5SlX27v2LTZs2MuC5F5k96xN8aWk5FkdO8FJupUqXZsvmv//u2rZ1K6VLl866P3fOLKpUrcrevXvZuDGFgS+8zKefzCQtj+eVncYrb4/XiOnfUvP2IVzdYzg7U31s2PgHADc1vJDrayTR4YkPDvq83rfU4ZkRn9O3Y136vvEpw6YupVuLy3Mz9KPixTELkzxfI/dfqCJc/gvrgva30TFeXs3Na3mNmPwlNds/y9W3vcTO3XvZ8Ms2AGJjY2ja4AImzvzmoM/r3bkhzwydQd87r6Pvy5MZNukLurWtl4uRH71oGLO83GweaDfwk5m1ArCgC0KP/QxcErrdBIgP3U4FCh9mmUWBbc65DDOrD5x1pCCcc0Occ9Wcc9U6dr7jONI4tB1//knq7t0A+Hw+vvryS8qWKweAPyODMaPe5daOt5Gens7+A9eBzEwyMjLCGkdO8GpuVc89j5SUn9m0aSMZ+/Yx46Pp1K3fAICMjAxGvTuCDp06k+5Lz7qaWSCQ9/PSeEXPeJU8KfhdrzKlitK0zjmMm7WCqy+rwL3tatKyzxjS0v8de/uGFzBz0QZ2pKZRIH88AedwzlEgIf5f80aaF8csh+S5Gnnb7eGtkV5dF7S/ja7xAu/m5rW8ShYrBECZU4rRtMEFjPs4+Elmg8srsf7nrfy6bee/ntO+8eXMXLCKHbv3UiAhH4GAwwXyZn2E6BizvHwa7cG0B94ws4cJFsuxwDJgKPChmS0DZvD3kdnlQGZo+jvAjgOW9x4w1cxWAEuAtTmewWH8/vt2Hn24D4HMTAKBAFdf25DadesDMH7saBo1aUZCYiIVkyrhS/NxY/Mm1Kxdh8JFihxhyZHn1dzi4uLo07cfXe/oTCCQSbPkFlSoUBGAcWPeo0nTZBITE0mqFMyrRbPG1KpdhyJ5PC+NV/SM15gnWlO8aAEy/Jnc/eJ0du3x8eLd15M/XyzTXrgFgK9Xb6Ln88HvniTmj+fmhhfS6L6RALwy/ksmPduefRmZdHj8/YjlcSheHLMc5Oka6dV1Qfvb6Bov8G5uXstrzHOdKX5SwWB9HDCeXXuCn+a1uvaSrAsDZZeYEM/NjS+nUbfXAHhl1BwmvdqNfRl+Ojz0Tm6GftSiYcwsL1/ON69LTQ/ozYsy8bHR9GH+0cvI9j0EL/HqeAEUa9A/whHkjB1z+kc6hByTEEfe/mdreYzPjydrpPa3Ijmr2KXRc9XlY7Vj8WuRDiFHHK4+as8iIiIiIiIiYadmU0RERERERMJOzaaIiIiIiIiEnZpNERERERERCTs1myIiIiIiIhJ2ajZFREREREQk7NRsioiIiIiISNip2RQREREREZGwU7MpIiIiIiIiYadmU0RERERERMJOzaaIiIiIiIiEnZpNERERERERCTs1myIiIiIiIhJ2ajZFREREREQk7NRsioiIiIiISNiZcy7SMUQtnx9Pvnk7/8qIdAg55qSC8ZEOQQSAVJ8/0iHkiDNvfifSIeSYtEmdLdIxRBOv1sh9/kCkQ8gR+eL0+YNITit2+f9FOoQckbb05UPWR+1ZREREREREJOzUbIqIiIiIiEjYqdkUERERERGRsFOzKSIiIiIiImGnZlNERERERETCTs2miIiIiIiIhJ2aTREREREREQk7NZsiIiIiIiISdmo2RUREREREJOzUbIqIiIiIiEjYqdkUERERERGRsFOzKSIiIiIiImGnZlNERERERETCTs2miIiIiIiIhF3UNptmVtbMVkY6jnBbOP9zmtxwLY0aXs3bQ4cA0KfXfbRMbswrL72QNd+QwYOYM3tWpMI8ZuNHv8utNzalQ5tmPPbwA6Snp/PEIw/SsV0yQwa9lDXfu2+/yfzPZkcu0OPg1TFTXtGVF0Bq6m4e7nU37Zo3on2Lxqxc/h2DXnmeW29M5ol+fbLmm/nRVMaPfjeCkR5Zj8bnsvTlFix5uTkj7q1P/vhYhvSow5rBN7LohWQWvZDM+WWLA9CselmWvtyCWU81onjh/ACUO6UwI+9rEMkUIsar9RG8uf2mp6dza7vWtGvVjNbJjXhz0KsAPNznAdq2bMrrr7yYNe/bQ97gsznRkRd4c7z282puyivv53VX27osGdebpeN7071tXQBGPnMri0Y/wKLRD7B2aj8WjX4AgBoXlOPrsQ+yYOR9lC9TEoCihRKZ+npXzCxX4o3aZtOLMjMzefqpxxk0+C0mTZnOjI+msX7dWvInJDBx0lRWrVxBamoq27dvY8Xy5TS48qpIh3xUtm/byvvj3mPIiHG8M3YygcwAUydPIH/+/AwfPYm1q1eyZ08qf/y+ndWrllO73pWRDvmoeXXMlFd05bXfywOf4fIatRj9wTTeGfs+JUqWYv3aNYwYN4n4uHh+2LCedJ+Pj6ZMonmrtpEO95BOK16AbjdUpeYDk6n2fx8QG2O0qnU2AA+N+Irq906i+r2TWP7znwB0vaEqtR6YzFufrOXG2uUB6N+uGv1HL4lYDhJ+Xt1+8+XLxxtvDWf0hMmMHj+JLxcu4Juli0nIn58xEz9k9aoV7ElN5fft21i5Yjn1GkRHXl4dL/Bubsor7+dVpfypdGxWg9q3Ps9lbZ/lutpVOfuMEtzcZwTV2w2keruBTJ6znA/nLgfg/26qT3LPwfR6fhK3t6wJQO/O1/DssE9xzuVKzBFvNs2soJlNN7NlZrbSzG40s35mtjh0f4iFWm8zuyQ03zLgrmzL6GBmH5jZDDPbYGbPZnvsGjP70sy+MbMJZlYoNH2Ama02s+Vm9lxoWqvQay4zs89z+a1g5YrllClzFmeUKUN8vnw0vP4G5n8+j3Sfj0AggN/vJzYmhkGvvkK37j1yO7wTkpnpJz09Hb/fT7ovDZwjPT2dQCBApt9PTEwsb7/5Gp3uuOvIC8tDvDpmyiu68gLYk5rKsm+X0qhZCwDi4/NRpEhR/H4/zjl8vjTi4uIYM3I4LW5sT1x8fIQjPry4WCMxXxyxMUZi/jg2/7n3kPMGnCN/fCwF8sWSkRmg5jml2bpjLz9s3p2LEYef6uM/eXX7NTMKFCgIgN/vx+/PCG6zoRrp9/uJiY1h8KBXubNb9whHe/S8Ol7g3dyUV97Pq3K50ixe+QtpvgwyMwPM/+Z7mjU4/x/ztLjqQsbP+AaADH8miQn5SEyIJ8OfSbkzTuaM0icxf+n3uRZzxJtNoCHwm3PuAufcucAM4DXn3KWh+4lAo9C8w4EezrkLDrKcC4EbgfOAG82sjJmVAB4GrnLOXQwsAe41s5OBZKCqc+584MnQMvoB14aW3yQnkj2cbVu3csqpp2TdL1W6NFu3bqVYseK0aZlMnXr1SUlJIeACnFOlam6Hd9xKlipNm5s60LrJVTS/vj4FCxWmZZubOalYMW6/uRVX1K7Hr5tScIEASZWrRDrcY+LVMVNe0ZUXwObfNnFSsWI83b8vHdu1YMDj/bAYo0bN2nRs14KTS5SkYKHCrF65gjr18/bZA7/9uZeXPlzB+iFt+GlYO3b/tY/Zy34FoH/7anz9YnOe7Xg5+eKCJWzg+8uY/tj1XH/pWYyf/wO9W1/EMxO+i2AGYaP6mI2Xt9/MzEzatU7mmvq1uLz6FVxS7TKKFSvGTW1aULtOfTamBGtk5XOiJy8vj5dXc1NeeT+vVd9vpuZFZ1O8aAESE+JpWLMKZ5QulvV4zYvKs/XPVH7YuB2AgcNn8fbjN/FAx6sZPO5zHuvWiP6DPsrVmONy9dUObgXwvJn9D5jmnJtvZi3MrBdQACgOrDKz+cBJzrn9R1RHAtdlW85s59wuADNbDZwFnARUARaGDv7mA74EdgE+4G0zmwZMCy1jIfCOmY0HPjhYsGZ2B3AHwGuD3uS22+8Iw1tweL369M263aNbFx7p/xhD33yD9evWUr1GTVq0ap3jMZyI1N27WDBvLmMnz6RQ4cI82vs+Pvl4Kj3u7Z01T+977+L+Po8yctibfL9hPdUur0HjZi0jGPWJifYxOxTllXdlZmayfu0a7n6gL1XPO5+XBj7DqOFvcXu3nrS/9TYABjzej9u6dGfqpIl8vegLyldMokPnLhGO/N9OKpiPRpedxTldxrHzr3RGP3AlbepWoN+oxWzZkUa+uBhe71aL+5pfwDPjv2XOsl+ZE2pG29WrwMylG6l4WhHubno+O/5K5/63viRtX2aEszouUVUfQ8tXjTwOsbGxjB4/idTdu3ngnh58v2E99/V6KOvxe3p05aFHHmPY0MFsWL+Oy6rXILlF3s/rYLwwXofi1dyUV96x7uetPD9iNlNf78betHSWrf+VzEAg6/HWDS9mwsxvsu4vX/8rdTsEv/dd86LybPl9N2bB73hm+AP0fnEy2/5MzdGYI/7JpnNuPXAxwaL6pJn1AwYBLZ1z5wFDgYSjWFR6ttuZBBtpAz51zl0Y+qninLvNOecHLgMmEjwqPCMUSxeCR3rLAEtDR3gPjHeIc66ac65auItoqdKl2bJ5S9b9bVu3Urp06az7c+fMokrVquzdu5eNG1MY+MLLfPrJTNLS0sIaR7gt+XoRp552OicVK05cXDy161/JyuXfZT2+YN4cKlWuQtrevfz660Yee+Z55s3+BJ8vb+cF3h0z5RVdeUHwDIKSpUpT9bzg6TT1r7qG9WvXZD2+fu0aHI4zy5Zl7qyZPPG/F/ht00Y2pvwSqZAPqcEFp/Pz1lR+3+3Dn+mYvOhnqlcqxZYdwXHY5w/w7uwNVKtY8h/PS8wXy80Nkhj88WoebnMJnV+ZxxdrttKmboVIpHHCoq0+huZTjTwBhYsU4ZJLL+PLLxZkTZs3dzbnVKnK3r1/sWnjRp4Z+CJzPv0EXx7Py8vj5dXclFd05DXiw0XUvOk5rr79VXbuTmNDSvBTzNjYGJrWv4CJn3xz0Of17nwNz7w1k753NKTvy1MYNukLurWpk+PxRrzZNLPTgL3OuVHAQIKFFeD30PdHWgI453YCO82sVujx9kex+EVATTOrEHqtgmaWFFpuUefcR8A9wAWhx8s7575yzvUDthMsqrmm6rnnkZLyM5s2bSRj3z5mfDSduvWDV1PMyMhg1Lsj6NCpM+m+9KwrSAUCmWRkZORmmMes9Cmnsnrlcny+NJxzfLP4K84qG7zYh9+fwYSxI2l7SyfS030Y+/MK5Pm8wLtjpryiKy+Ak0uUpFTpU0j5+ScgeJCn7Nnlsx5/641Xub1rD/x+f9ZRUIuJyZMHdTZu38NlSaVIzBcLQP3zT2Pdpp2cUiwxa54ml5/F6pQd/3jePc3OZ9C0VfgzHYn5YnHOEQg4CuTPCyfxHDvVx3/y6va7488/Sd0d/H6xz+fj60VfUrZsOQD8GRmMGfUut3S4jfT0dPZfPDIzCvLy6niBd3NTXtGRV8lihQAoc0oxmjY4n3EfLwWgwWVJrP95K79u2/Wv57RvdCkzF6xmx+69FEjIR8A5XMBRICFfjsebFyrwecBAMwsAGUBXoBmwEtgCLM42b0dgmJk54JMjLdg5t93MOgBjzCx/aPLDQCrwoZklEDy6e2/osYFmVjE0bTaw7MRSOzZxcXH06duPrnd0JhDIpFlyCypUqAjAuDHv0aRpMomJiSRVqoQvzUeLZo2pVbsORYoUyc0wj1mVc8+n7pVXc/vNrYmNjaVCpco0Tm4FwKQJY2l4Q1MSEhIpX7ESvnQfHdomU/2K2hQunLfzAu+OmfKKrrz2u6fXQzz28IP4MzI47fQz6NM/+HW7z+fOplKVqpQoWQqAikmVuaV1M8pXTKJiUuVIhnxQizdsZ9KXP/Hl88n4AwGW/fgHb3+ylg/7NaREkQTMYPlPf9Jj8N+f/pxarADVKpbk6fHfAvDGR6tZMLAZu/7aR+sBn0YqlROl+piNV7ff33/fTv+H+xAIZBIIBLjqmobUrlsfgPHjRnNDk2YkJCZSMakSPp+PNi2aULNWHQrn8by8Ol7g3dyUV3TkNWZgJ4oXLUiGP5O7B0xk157gQeNW117M+Jn//lQzMSGemxtfTqO7BgHwyqi5THrlTvZlZNKhb87/GzTLrcveepHPjyffvJ1/5c0jOeFwUsG8fQVO+e9I9fkjHUKOOPPmdyIdQo5Jm9Q5d/4pmUd4tUbu8weOPFMU2n/BLRHJOcUu/79Ih5Aj0pa+fMj6qD2LiIiIiIiIhJ2aTREREREREQk7NZsiIiIiIiISdmo2RUREREREJOzUbIqIiIiIiEjYqdkUERERERGRsFOzKSIiIiIiImGnZlNERERERETCTs2miIiIiIiIhJ2aTREREREREQk7NZsiIiIiIiISdmo2RUREREREJOzUbIqIiIiIiEjYqdkUERERERGRsFOzKSIiIiIiImFnzrlIxyBHwczucM4NiXQcOcGruSmv6OPV3JSXeJ1X1wXlFX28mptX8wLv5pZX8tInm9HjjkgHkIO8mpvyij5ezU15idd5dV1QXtHHq7l5NS/wbm55Ii81myIiIiIiIhJ2ajZFREREREQk7NRsRo+In3Odg7yam/KKPl7NTXmJ13l1XVBe0ceruXk1L/BubnkiL10gSERERERERMJOn2yKiIiIiIhI2KnZlKNmZmXNbGUOv0Y9M5uWk69xrMysv5ndb2aPm9lVufB6zcysSk6/zmFev6eZrTGz9yIVw8Hk0vr3RU4uP5JC71+743zunnDHcxwx5Pj4i5wI1Ujv18i8Wh9BNfJEqD7mLDWbHmZBeWKMzSw20jGcKOdcP+fcrFx4qWZAxJpNoBtwtXOu/fEuwMziwhhPrnHOXRHpGHJQWeCgxTRax0vkRKhGhtd/pEb+Z+sjeLpGlkX1McfkiZ3sf42ZTTazpWa2yszuCE3bY2ZPmdkyM1tkZqVD08uH7q8wsyezH0ExswfMbLGZLTezx0LTyprZOjN7F1gJlAlz+HFm9l7oyN5EMytgZlea2behGIeZWf5QLD+b2f/M7BuglZldY2Zfmtk3ZjbBzAqF5mtoZmtD8zXPll/x0Hu1PPQenB+a3t/MRpjZfDP7xcyam9mzodefYWbxJ5qkmfU1s/VmtgCoFJr2jpm1DN0eYGarQ7E9F5p20LE68Ei0mb1mZh0OthwzuwJoAgw0s+/MrPyJ5nKMeQ8GzgY+Dr0Hw8zs69D4Ng3NUzb03n8T+rkiW57zzWwKsDqHQow1s6GhbecTM0s0s9tD28EyM3vfzAqE4nnHzAab2ZLQWDYKTe9gZh+a2WdmtsHMHs2Wf/Yx+yy0jq8NrfMWeuwSM5sX2oZnmtmpoek9s43l2NC0uqFx/C70HhY+1oRD7/eag+RdPrS+Lw2975Wz5d3ywJyAAUDtUCz3hN6HKWY2B5htZoXMbHZoTFfsH+9wM7OCZjY9NF4rzexGM+sXGsOVZjbkgPd6mZktA+7KtowOZvZBKP8NZvZstscOtZ852DbbKvSay8zs85zIV46dqUaqRubBGml5vz7Cf6xGmurjMouG+uic008u/wDFQ78TCRa7kwEHNA5NfxZ4OHR7GtA2dLsLsCd0+xqCV5kyggcNpgF1CB6dCQDVcyDusqE4a4buDwMeBjYCSaFp7wJ3h27/DPQK3S4BfA4UDN1/EOgHJISeXzGUy3hgWmieV4FHQ7cbAN+FbvcHFgDxwAXAXuC60GOTgGYnmOclwAqgAFAE+B64H3gHaBkar3X8fYGtk44wVvX25xS6/xrQ4TDLeQdoGcH18+fQeD0N3LQ/NmA9UDD0viSEplcElmTL8y+gXA7FVRbwAxeG7o8HbgJOzjbPk0CPbO/jDILbR0VgU2h96wBsDr3/+7fBaqHnZB+zXcAZoed/CdQKrXNfACVD890IDAvd/g3If8BYTuXv7aUQEBfGvGcDFUPTLgfmHGz9Ocx62CH0nuzfH8UBRbJtr99nWzf3hHEcWwBDs90vuj+G0P2R/L0vXA7UCd0eCKzMFvuPoecmAL8QbBoOtZ851La2Ajg9+zT9RP4H1cjs665qZB6qkeTR+pht/ftP1cjD5Kz6mIfqoz7ZjIyeoSMRiwiuABWBfQR3xABLCW5AADWACaHbo7Mt45rQz7fAN0Dl0HIAfnHOLcqh2Dc65xaGbo8CrgR+cs6tD00bQbCg7zcu9Ls6wdNeFprZd8CtwFmhuH9yzm1wwTV6VLbn1iK4YeGcmwOcbGZFQo997JzLILgxxBLcYRK6X/YEc6wNTHLO7XXO7QamHPD4LsAHvG1mzQkWcjj0WB3KoZaTV1wD9A6N12cEd1pnEiwmQ81sBcF8s5/O9LVz7qccjOkn59x3odv7t5NzQ0cuVwDtgarZ5h/vnAs45zYQ3PlWDk3/1Dn3h3MuDfiA4Lp2oK+dc5uccwHgu9BrVQLOBT4NvS8PEyy2ENzxv2dmNxEsfgALgRfMrCfBnbWf43OwvK8AJoTieBM49TiW+6lz7s/QbQOeNrPlwCzgdKD0ccZ7OCuAqy34iU5t59wuoL6ZfRUawwZAVTM7ieB7tv+I6sgDljPbObfLOecj+EnBWRx6P3OobW0h8I6Z3U5wPyJ5g2qkauThlpMX5MX6CP/NGqn6mMfro85DzmVmVg+4CqjhnNtrZp8R3EllhAoJQCZHHhsDnnHOvXnA8ssSPHqWU9wB93cSPCpyKPtjMYIbbtvsD5rZhccZRzqAcy5gZtnfuwA5vF475/xmdhnBPyJaAt0J7gAOxc8/T1lPOM7l5DYDWjjn1v1joll/YCvBI+YxBHdS++XkugehcQ/JJHjU9R2CR+qXWfDUq3rZ5jlwfXVHmH6414oj+J6scs7VOMj8NxD8I7Ix0NfMznPODTCz6cD1BHfw1zrn1h4it8M5MJbSwE7n3IUHmTdrfbPg99HyHWa52cerPVASuMQ5l2FmPxNaV8PJObfezC4m+J48aWazCZ4CVM05tzG0fh3N6x5qfP61nwE42LbmnOtiZpcTHLulZnaJc+6PE0hPTpBqpGok0VEj82J9hP9mjVR9/Lc8VR/1yWbuKwrsCBXRygSPNBzOIoIfqwO0yTZ9JtAp2/nWp5tZqbBH+29nmtn+nUg7YAlQ1swqhKbdDMw7yPMWATX3z2fB89KTgLWh5+//3kX2jWA+wQ18/x8gv4eOoua0z4FmFjzvvzDBHWOW0Hte1Dn3EXAPwaIChx6rX4AqZpY/dDTqyiMsJxU45u/25YCZQA+zrO8HXBSaXhTYHDqaeTOR/0SoMLDZgt9DOvCiDa3MLCa0fp1N8FQRCB45LG5miQQvNrGQo7MOKLl/GzCzeDOrGipaZZxzcwmemlIUKGRm5Z1zK5xz/wMW8/dR4xO1G/jJzFqF4jAz27/+/EzwNDcIfrdp//ezjrReFQW2hQppfYJHPMPOzE4D9jrnRhE89efi0EO/h7aJlgDOuZ3ATjPbf0T9aC7IcdD9zKG2tdD4fOWc6wdsJ/zf35NjpxqJauQRlpMXamS01Ef479VI1ceDi1h91CebuW8G0MXM1hDcKI90Ks/dwCgz6xt67i4A59wnZnYO8GVoX7eH4HnqmTkU937rgLvMbBjBj+Z7EsxhggWv2LUYGHzgk5xz20NH1MZY6OIIBL9zs96CF4CYbmZ7CRbP/Rt8f2CYBU9b2EvwI/8c55z7xszGAcuAbQRzyq4w8KGZJRA8UnRvaPrdHHysNprZeILfe/iJ4Gldh1vOWIKn4fQk+N2CH8Kf5VF5AngJWB4qFD8BjYBBwPtmdgvBPHPjaO3hPAJ8RXBn+BX/LBgpwNcEv1fUxTnnC20vXwPvEzy9Z5RzbsnRvJBzbp8FLy7wipkVJbgPfYng93VGhaYZ8IpzbqeZPREqTAFgFfDxiSabTXvgDTN7mGDBHEtwnR1KcL1axj/HZzmQGZr+DrDjgOW9B0y14Kk6Swj+kZsTziN4cY8AkAF0JfjHzEpgC//c3joS3Ac44JMjLfhQ+xmCf0gcbFsbaGb7vws3m+D7J5GlGqkaGQ01MlrqI/w3a6Tq4wEiWR/3fxlU8igLXjUszTnnzKwNwS/XN410XPJvGqu8xczeIfiF/4kHTO9A8JSU7pGIS0TCR/vd6KGxyltUIyW36JPNvO8S4LXQqRo7gU6RDUcOQ2MlIpK7tN+NHhorkf8gfbIpIiIiIiIiYacLBImIiIiIiEjYqdkUERERERGRsFOzKSIiIiIiImGnZlPEY8ws08y+M7OVZjYhdAXA413WO6HLmGNmb5lZlcPMW8/Mrsh2v0vo8u8iIiJ5gmqkSO5SsyniPWnOuQudc+cC+4Au2R8M/a+3Y+ac6+ycW32YWeoBWYXUOTfYOffu8byWiIhIDlGNFMlFajZFvG0+UCF0RHW+mU0BVptZrJkNNLPFZrbczO4EsKDXzGydmc0CSu1fkJl9ZmbVQrcbmtk3ZrbMzGabWVmCBfue0BHj2mbW38zuD81/oZktCr3WJDMrlm2Z/zOzr81svZnVzt23R0RE/sNUI0VymP7PpohHhY7OXgfMCE26GDjXOfeTmd0B7HLOXWpm+YGFZvYJcBFQCagClAZWA8MOWG5JYChQJ7Ss4s65P81sMLDHOfdcaL4rsz3tXaCHc26emT0OPArcHXoszjl3mZldH5p+VZjfChERkX9QjRTJHWo2Rbwn0cy+C92eD7xN8NSdr51zP4WmXwOcv/+7JkBRoCJQBxjjnMsEfjOzOQdZfnXg8/3Lcs79ebhgzKwocJJzbl5o0ghgQrZZPgj9XgqUPaoMRUREjo9qpEguUrMp4j1pzrkLs08wM4C/sk8ieBR15gHzXZ/j0f1beuh3JtoniYhIzlKNFMlF+s6myH/TTKCrmcUDmFmSmRUEPgduDH1f5VSg/kGeuwioY2blQs8tHpqeChQ+cGbn3C5gR7bvmtwMzDtwPhERkTxCNVIkTHSEROS/6S2Cp+N8Y8FDutuBZsAkoAHB76GkAF8e+ETn3PbQ91k+MLMYYBtwNTAVmGhmTYEeBzztVmCwBS8x/yPQMQdyEhERCQfVSJEwMedcpGMQERERERERj9FptCIiIiIiIhJ2ajZFREREREQk7NRsioiIiIiISNip2RQREREREZGwU7MpIiIiIiIiYadmU0RERERERMJOzaaIiIiIiIiEnZpNERERERERCbv/BwrZ+kU1joX0AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 1080x360 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "import audplot\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "\n",
    "_, axs = plt.subplots(1, 2, figsize=[15, 5])\n",
    "\n",
    "axs[0].set_title('smile')\n",
    "audplot.confusion_matrix(\n",
    "    truth_smile, \n",
    "    pred_smile,\n",
    "    percentage=True,    \n",
    "    ax=axs[0],\n",
    ")\n",
    "\n",
    "axs[1].set_title('w2v2')\n",
    "audplot.confusion_matrix(\n",
    "    truth_w2v2, \n",
    "    pred_w2v2,\n",
    "    percentage=True,\n",
    "    ax=axs[1],\n",
    ")"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.16"
  },
  "toc": {
   "base_numbering": 1,
   "nav_menu": {},
   "number_sections": true,
   "sideBar": true,
   "skip_h1_title": false,
   "title_cell": "Table of Contents",
   "title_sidebar": "Contents",
   "toc_cell": false,
   "toc_position": {},
   "toc_section_display": true,
   "toc_window_display": false
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}