auto3dseg/notebooks/auto3dseg_hello_world.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# MONAI Auto3Dseg \"Hello World\" Example\n",
    "\n",
    "In this notebook, we will provide a simple demonstration of how to use Auto3Dseg AutoRunner to process a simulated datasets and generate results within minutes.\n",
    "\n",
    "## 1. Set up environment, imports and datasets"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 1.1 Set up Environment"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!python -c \"import monai\" || pip install -q \"monai-weekly[nibabel]\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 1.2 Set up imports"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Copyright (c) MONAI Consortium\n",
    "# Licensed under the Apache License, Version 2.0 (the \"License\");\n",
    "# you may not use this file except in compliance with the License.\n",
    "# You may obtain a copy of the License at\n",
    "#     http://www.apache.org/licenses/LICENSE-2.0\n",
    "# Unless required by applicable law or agreed to in writing, software\n",
    "# distributed under the License is distributed on an \"AS IS\" BASIS,\n",
    "# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n",
    "# See the License for the specific language governing permissions and\n",
    "# limitations under the License.\n",
    "\n",
    "import os\n",
    "import json\n",
    "import nibabel as nib\n",
    "import numpy as np\n",
    "\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "from monai.apps.auto3dseg import AutoRunner\n",
    "from monai.data import create_test_image_3d\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 1.3 Simulate a special datasets\n",
    "\n",
    "It is well known that AI takes time to train. To provide the \"Hello World!\" experience of Auto3D in this notebook, we will simulate a small datasets and run trainings only for multiple epocs. Due to the nature of AI, the performance shouldn't be highly expected, but the entire pipeline will be completed within minutes!\n",
    "\n",
    "`sim_datalist` provides the information of the simulated datasets. It lists 12 training and 2 testing images and labels. The training data are split into 3 folds. Each fold will use 8 images to train and 4 images to validate. The size of the dimension is defined by the `sim_dim` ."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "sim_datalist = {\n",
    "    \"testing\": [\n",
    "        {\"image\": \"test_image_001.nii.gz\", \"label\": \"test_label_001.nii.gz\"},\n",
    "        {\"image\": \"test_image_002.nii.gz\", \"label\": \"test_label_002.nii.gz\"},\n",
    "    ],\n",
    "    \"training\": [\n",
    "        {\"fold\": 0, \"image\": \"tr_image_001.nii.gz\", \"label\": \"tr_label_001.nii.gz\"},\n",
    "        {\"fold\": 0, \"image\": \"tr_image_002.nii.gz\", \"label\": \"tr_label_002.nii.gz\"},\n",
    "        {\"fold\": 0, \"image\": \"tr_image_003.nii.gz\", \"label\": \"tr_label_003.nii.gz\"},\n",
    "        {\"fold\": 0, \"image\": \"tr_image_004.nii.gz\", \"label\": \"tr_label_004.nii.gz\"},\n",
    "        {\"fold\": 1, \"image\": \"tr_image_005.nii.gz\", \"label\": \"tr_label_005.nii.gz\"},\n",
    "        {\"fold\": 1, \"image\": \"tr_image_006.nii.gz\", \"label\": \"tr_label_006.nii.gz\"},\n",
    "        {\"fold\": 1, \"image\": \"tr_image_007.nii.gz\", \"label\": \"tr_label_007.nii.gz\"},\n",
    "        {\"fold\": 1, \"image\": \"tr_image_008.nii.gz\", \"label\": \"tr_label_008.nii.gz\"},\n",
    "        {\"fold\": 2, \"image\": \"tr_image_009.nii.gz\", \"label\": \"tr_label_009.nii.gz\"},\n",
    "        {\"fold\": 2, \"image\": \"tr_image_010.nii.gz\", \"label\": \"tr_label_010.nii.gz\"},\n",
    "        {\"fold\": 2, \"image\": \"tr_image_011.nii.gz\", \"label\": \"tr_label_011.nii.gz\"},\n",
    "        {\"fold\": 2, \"image\": \"tr_image_012.nii.gz\", \"label\": \"tr_label_012.nii.gz\"},\n",
    "    ],\n",
    "}\n",
    "\n",
    "sim_dim = (64, 64, 64)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 1.4 Generate images and labels\n",
    "\n",
    "Now we can use MONAI `create_test_image_3d` and `nib.Nifti1Image` functions to generate the 3D simulated images under the work_dir"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/opt/conda/lib/python3.8/site-packages/tqdm/auto.py:22: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html\n",
      "  from .autonotebook import tqdm as notebook_tqdm\n"
     ]
    }
   ],
   "source": [
    "work_dir = \"./auto3dseg_work_dir\"\n",
    "if not os.path.isdir(work_dir):\n",
    "    os.makedirs(work_dir)\n",
    "\n",
    "dataroot_dir = os.path.join(work_dir, \"sim_dataroot\")\n",
    "if not os.path.isdir(dataroot_dir):\n",
    "    os.makedirs(dataroot_dir)\n",
    "\n",
    "datalist_file = os.path.join(work_dir, \"sim_datalist.json\")\n",
    "with open(datalist_file, 'w') as f:\n",
    "    json.dump(sim_datalist, f)\n",
    "\n",
    "for d in sim_datalist[\"testing\"] + sim_datalist[\"training\"]:\n",
    "    im, seg = create_test_image_3d(sim_dim[0], sim_dim[1], sim_dim[2], rad_max=10, num_seg_classes=1)\n",
    "    image_fpath = os.path.join(dataroot_dir, d[\"image\"])\n",
    "    label_fpath = os.path.join(dataroot_dir, d[\"label\"])\n",
    "    nib.save(nib.Nifti1Image(im, affine=np.eye(4)), image_fpath)\n",
    "    nib.save(nib.Nifti1Image(seg, affine=np.eye(4)), label_fpath)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 1.5 Inspect the simulated data\n",
    "\n",
    "Here we picked a slice of the 3D image in the test datasets to get an idea of the simulation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXYAAACpCAYAAAA/QRYrAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8qNh9FAAAACXBIWXMAAAsTAAALEwEAmpwYAAAeEElEQVR4nO3de5gcdZ3v8fdnOjOTy4RcCGCYBBJMggRRLtkAwUfumHBYWEVjAsfDAoK7eMEDoqB7XI767IG9KPiQlY2AgEe5CAo5iCJEWDzKJeEA0QQShlxIIOR+GQLJTHd/zx9VEzpz6e7prq6u6f6+nqeeTFdXV/8yz6e/U1X9q99PZoZzzrna0VDtBjjnnIuWF3bnnKsxXtidc67GeGF3zrka44XdOedqjBd255yrMV7Y+yBpqaRTqt0O5ypN0mpJZxSxnUmaVOJ7lPxa13+Dqt2ApDKzI6vdBuecK4UfsTvnXI3xwt6HrtNTSddL+oWk/y2pXdKfJU2RdJ2kjZLWSjor53UXS3ol3HalpC902+/XJa2X9Jakz+eeokpqlvSvkt6QtEHSrZKGxP1/d/VJ0nRJz0jaHmb0FklN3TY7O8z1Zkn/Iqkh5/WXhNnfJukxSYfG/F9wIS/sxflr4KfAKOBF4DGC310r8B3gP3K23QicA+wHXAz8QNKxAJJmAlcBZwCTgFO6vc8NwBTg6PD5VuDbFfj/ONebDPDfgTHAicDpwBXdtvkkMA04FjgPuARA0nnAN4FPAQcAfwDuiaXVricz86WXBVhNUICvBx7PWf/XwDtAKnw8HDBgZB/7eQi4Mvz5DuB/5Tw3KXztJEDALuCDOc+fCKyq9u/Cl9peurLey/qvAr/KeWzAzJzHVwALw59/A1ya81wD8C5waM5rJ1X7/1ovix+xF2dDzs/vAZvNLJPzGKAFQNIsSc9K2ippO3A2wREQwMHA2px95f58ADAUeCE8Fd4O/DZc71zFhZcYH5H0tqSdwD/xfna75GZ2DUGmAQ4Fbs7J7laCg5XWCjfb9cILe4QkNQMPAv8KHGRmI4FHCQIOsB4Yl/OS8Tk/byb4I3GkmY0MlxFm1lL5ljsHwI+AV4HJZrYfwaUVddsmN7OHAG+FP68FvpCT3ZFmNsTM/lTxVrsevLBHqwloBjYBaUmzgLNynr8fuFjSEZKGAv+j6wkzywI/JrgmfyCApFZJn4it9a7eDQd2Au9I+hDw971sc42kUZLGA1cC94XrbwWuk3QkgKQRkj4TR6NdT17YI2Rm7cBXCAr4NuACYEHO878Bfgg8CbQBz4ZP7Qn//UbX+vBU+Ang8Fga7xx8jSCz7QQHGff1ss3DwAvAS8CvgdsBzOxXwI3AvWF2/wLMqnyTXW8UfrHhqkDSEQQfgGYzS1e7Pc652uBH7DGT9Mmwv/oogiOc/+NF3TkXJS/s8fsCQV/31wn6Dfd2HdM550pWVmGXNFPSckltkq6NqlG1zMxmhr1dRpvZJ81sfbXb5HrybLu4SLojvIv9L308L0k/DLO4pOuGx3xKLuySUsA8gi9IpgJzJU0tdX/OJYVn28XsTmBmnudnAZPD5XKCbql5lXPEPh1oM7OVZtYB3Etwi7FzA51n28XGzJ4muKGrL+cBd1vgWWCkpLH59lnOsL2t7HsX2jrg+HwvaFKzDWZYGW/pBoJ2tm02s4F8x2y/su25rg9VzHVveWwluOGxVxUfj13S5QSnDwxmKMfr9Eq/pauyJ+yBNX0994lTh9mWrZke619YsucxM8t3Opoonuv6ky/XkDfbS4HdOavmm9n8iJu3j3IK+5vse3vxuHDdPsL/wHyA/TTaO83Xuc1b0/zptz2HDxl88KruY5JUU8Fse65dd3myvdvMppWx66Jqba5yrrEvAiZLmhiO2TyHnLssnetNFmOPpXssCePZdv1WwWwvAP5b2DvmBGBHod50JR+xm1la0pcIxiZPAXeY2dJS9+fqgwGdZKvdjLw8264UpWZb0j0EczOMkbQO+EegEcDMbiUYSPBsguFG3iWY5yGvsq6xm9mj4Zs6VxQDOi3ZhR08267/Ss22mc0t8LwBX+zPPn0yaxcrw+jEL0m72pOkbHthd7Eyg85kZN+5SCUp217YXawM0Wnd525wbuBLUra9sLtYGdDhY8+5GpSkbHthd7HKInabx87VniRlOxmtcHUj6DmQjKMa56KUpGx7YXexCq5Deuxc7UlStpPRClc3zESHpardDOcil6Rse2F3sQruzktG+J2LUpKy7YXdxSpJp6vORSlJ2U5GKyqsYfBgNGRI/o0yGTLt7cFdBq5ijOScrg50DcOGoaFD82+UzZDZtgOyPYeTddFKUrbrorBvnX0MW2e+l3eb9PYmpt64nvSatXm3c+XJInZnG6vdjJqw7oqPcvLsF/Jus2TrwbR8eRSZFa/H1Kr6laRs12Rhbxg6lIbRo/Y+3jlBzJqSf3C+pdvH0jF+f5oywSA+tqeDzObNfgQfseC265qMXcU1DB9OwwH7Yw3B3Y3vHN7BLa3P5X3NwtEpvjv5EoZmg1xrdwfpN9/yXFdAkrKdjFZE7L2Tj2TrF94hpSC8HxyxquBrDhu+hT9/s4k9ncMB2Pn6SKb8426y7e0VbWu9Ca5DJuN0daDZdu6RzL7udzQquKzy3aGvAU15X3Ni83tc9v1fsjXdAsBtr82g9eJ3yWzbVunm1p0kZbsmC3vn8BQnHrya5obiB7kfkupg+oHvz3z16K4jUSoZNxvUkiSFf6DZM6KBL45aTrO6TvfzF3WAoQ1NXDh8C7AFgFdax7I65fOzVkKSsu2Vy8UquDtvUI/FuYGu1GxLmilpuaQ2Sdf28vwhkp6U9KKkJZLOLrTPmirsqVGjYPpRtI9r2HsZplRDhnSw59hJpI6YDErGiG21oOuopvtSjEIfgHCb2ZKWSVoq6eeRNr5KUgccQOcZx9F+WJaGMj+yhw7ews6TJ6HjjvRcR6yUbEtKAfOAWcBUYK6kqd02+wfgfjM7hmCaxn8v1JaaOlTac8xhNHxrIx8evI5BKq9714zWVWy6voWXX/ggk699A9uzJ6JW1jez0k5Xcz4AZwLrgEWSFpjZspxtJgPXASeZ2TZJB0bU7Kra+fHD+N4//5hJjTtpVEtZ+/ryqKWc+28vc+GSizlodjPZ3bsjaqUrMdvTgTYzWwkg6V7gPGBZzjYG7Bf+PAJ4q9BOa6qwZxtF69AdjGzM37WxGM0NacYN3c6LQ5I/jdtAYqV3CSvmA3AZMM/MtgGY2cYym5sI2UYxtamdA1PlFXUIrrkf0dTE/sPejaBlLleJ2W4FcvtYrwOO77bN9cDvJH0ZGAacUWinNXUpxiVfcB2y19PVMZIW5yyXd3tpbx+A1m7bTAGmSPqjpGclzazYf8S5bsrIdiFzgTvNbBzBpNY/lZS3dtfUEbtLPkOkez9d3Wxm08rc/SBgMsGM7+OApyUdZWbby9yvcwWVmO03gfE5j8eF63JdCswEMLNnJA0GxgB9npH6EbuLlRl0Zht6LEUo5gOwDlhgZp1mtgpYQVDonau4ErO9CJgsaaKkJoIvRxd02+YN4HQASUcAg4FN+Xbqhd3FqoxeMcV8AB4iOFpH0hiCSzMrI2u8c3mUkm0zSwNfAh4DXiHo/bJU0ncknRtudjVwmaSXgXuAvzXLf+uwX4pxsTJEOtv/XjFmlpbU9QFIAXd0fQCAxWa2IHzuLEnLgAxwjZltibD5zvWpjGw/Cjzabd23c35eBpzUn316YXexCsbTKO1EsYgPgAFXhYtzsSon21GrucKeTcgv1vXOEB3Zmoudc4nKdjJaEZGhyzeyYv4RbDsCzjztxbJuUlq06RD2LDiQQ1ensc7ix5xx+RmUdLpaz0a+uJnTbrmGzLR2/jLjLlL5e7rldW/7KK6/bw6jXjX26yh4n4vrhyRlu6YKe3r1G4y68w0aZ59A5lQxqIw7pjdtG86Ue5aR2b4jugY6zETaz6r6JbO8jdYb29h4xQzSMzKkyujz8PTOwznshyvIbPavHqKWpGzXVGF3yRcc1SQj/M5FKUnZLljYJY0H7gYOImj7fDO7WdJo4D5gArAamN11K3e1KWPs6BzCIAXDAQxJdTIk1ZH3NVlrYGe6ee81+mxHCrI+GUHUgp4DyQj/QMt2Q9rYkNnDYAVZblEjQxvyD92bsSzbsu/RNTDG9o4hPslGhSQp28UcsaeBq83s/0kaDrwg6XHgb4GFZnZDONLetcA3KtfU4o340xo2bZ+IhZdi3pjVyNkn559CbOU7+7P1tkMYuqETgMk7O8i+6+NpRM5IzOkqAyzbH3j8LeZuu3rvqIztF+xgyfR78r7mhY4Ml/3wGlreDEp789ZOmna8XPG21qUEZbtgYTez9cD68Od2Sa8QjNFxHuHNIMBdwFMkIPwA6fVvM2j923sftxwxg9XT9s/7mrXbRzL+uQ1k2t6fbcmPa6KXpNPVgZbt9Ko1tKx6fzKYTcecyEsfzT/q6JPvfJiDF24lu+TVves815WRpGz36xq7pAnAMcBzwEHhBwPgbYLT2URq/dUa9jx3QN5txnVksHXr827jymeITELCn2sgZnvS7W9z1cIr8m6T2p1hUNuKmFpU35KU7aILu6QW4EHgq2a2UzmD9JuZSb3PbBGOZHY5wGCGltfaEqXXvQnrug8rsi8fnDcewXgayegS1qWUbCch15m2VTTmnGH2xbMdjyRlu6g/L5IaCYL/MzP7Zbh6g6Sx4fNj6WOkMTObb2bTzGxaI81RtNkNaMFRTfelaq0pMduea9dTcrJd8F0VHL7cDrxiZt/PeWoBcFH480XAw9E3z9UaM8hk1WOpBs+2i1KSsl3MpZiTgM8Bf5b0Urjum8ANwP2SLgXWALMr0kJXUwzIJKTnAJ5tF6EkZbuYXjH/F+jrz87p0TbH1b7qHcV059l20UpOtv3OUxcrM8gmpOeAc1FKUraT0QpXV5JyHdK5qJWSbUkzJS2X1BbeENfbNrMlLZO0VNLPC+3Tj9hdrJLU19e5KJWSbUkpYB5wJsHUjoskLQgn1+jaZjJwHXCSmW2TdGCh/fonzMXLwLLqsTg34JWW7elAm5mtNLMO4F6CO59zXQbM6xqvyMz6nMS6ixd2F7tsVj0W52pBCdluBdbmPF4Xrss1BZgi6Y+SnpU0s9BO/VKMi5UZmF+KcTUoT7bHSFqc83i+mc3vx64HAZMJxi8aBzwt6Sgz257vBc7Fyvwed1ej+sj2ZjOb1sdL3gTG5zweF67LtQ54zsw6gVWSVhAU+kV9tcMPnVzMel6D9GvsrjaUlO1FwGRJEyU1AXMI7nzO9RDhaKOSxhBcmlmZb6d+xO7iFX7B5FzNKSHbZpaW9CXgMSAF3GFmSyV9B1hsZgvC586StAzIANeYWd65Db2wu/iVWNjDL41uJvgA3GZmN/Sx3fnAA8Bfmdni3rZxriJKyLaZPQo82m3dt3N+NuCqcCmKF3YXrxKP2Ivp7xtuNxy4kmBcdefik6CzUb/G7uKXVc+lsGL6+wJ8F7gR2B1dg50rUmnZjpwXdhcvA2V7LkUo2N9X0rHAeDP7dWTtda5YpWc7cn4pxsWsz6OYsvr6SmoAvk8wEbVzVVC9I/TuvLC7+PW/ry8U7u87HPgw8FQ4td0HgAWSzvUvUF1sEnKPhhd2Fy8DlXZUs7e/L0FBnwNcsHe3ZjuAMV2PJT0FfM2LuotN6dmOnBf2AUKDBtHwwQnY4MZ91je0v0d61ZrgfuYBopTrjkX293UDTC3lGqp3Tb07L+wDRKp1LM0/3sGcDzy/z/r/+edzOORzQ8i++26VWtZPRsnXIQv19+22/pSS3sTFqmZyDWVlO2pe2BMoNXUKHQe27LOu/aAmvnLQfcxu2bHP+qcPaWPpxz9Candmn/XNa7YERzwJlJSjGhevWs81JCfbXtiTRmL5dcO4/2O37rO6UVkOb0wB+56y3jD2P2n70dNkuk3decHPrmTCPyT0A5CQ8LsY1UOuITHZ9sKeIA1HT6V90nCOmfA6xzU3FfWalobBHN3cc33qQ+3sOv949luyicxreccLipUS9AWTi0c95BqSlW0v7Any6uUt/OGcf2N0QxNQ3AegL8+fcBtbp6c5e97Xab0xYR+AhBzVuHjUS64hOdn2wp4A+quj2Hx0C1MOX8u4QS2FX1CElobBtDRA57HvsOXzJ3LgHzaRWd4Wyb7LYskJv6ususo1JCrbXtgTYOWnWnjpczfTqBRBT77o/OVjP2HHjN184ntfY0xCPgDKFN7GDXz1lmtITra9sFfT9KN489ThHHzcWwxtKO8UtS+NSjGiYTDtp+6iY/gMxi/YQGbF6xV5r6INrK7Jrr/qNdeQmGx7Ya+iN08dzstfuYWUKjsWW6NSrPj43ayf8Q7nr/0aw6v5AUjQ6aqrjLrMNZSc7UrMM+CjO1aTqHj4c6VU/W/sRXJGwHMVUoe5htKynTPPwCxgKjBX0tRetuvXPANe2F28EjS0qXORKi3bFZlnwAu7i1+2l8W5WtD/bFdknoGiC7uklKQXJT0SPp4o6TlJbZLuC2fYdsVqSGFVOIO0huC9qylJR+ye64jVca6hz2yPkbQ4Z7m86P29P8/A1f1pR3+O2K8EXsl5fCPwAzObBGwDLu3PG9ez9OnHsfb+Izjz088X3jhCIxqaGP/F12i7+yM0fPhDsb73XhZ0Ceu+VJHnOiJ1nWvIl+3NZjYtZ8mdQKY/8wysBk4gmGcg39wFxRV2SeOA/wLcFj4WcBrBN7QAdwF/U8y+HOyY2MTiE+7gprHxDhXerEbuP2whD550K3s+MCzW986VlCN2z3W06j3XUFK2984zEJ4dzgH2DkFtZjvMbIyZTTCzCcCzQMHJY4o9Yr8J+DrvXzHaH9huZunwcY/rQs71Rtb7UiU34bl2ESkl22HWuuYZeAW4v2ueAUnnltqWgv3YJZ0DbDSzFySd0t83CK8nXQ4wmKH9fbmrQUnoBeO5dpVQ4iQykc8zUMwNSicB50o6GxgM7EfQmX6kpEHhX5zu14VyGzIfmA+wn0Yn5L4sV1UJKOx4rl0lJCPbhS/FmNl1ZjYuvL4zB/i9mV0IPAl8OtzsIuDhirWyxgx6z3jivZGs6NwV+3sv6djNU+8eTmpPlRKYkH7snuvo1XWuITHZhvKGFPgGcK+k7wEvArdH06TaN/q3K5j32vm0zR3G65+9tfALIrI5s4sL511D6+93MOjVFVU7uEjCpZg8PNclqvdcQ3Ky3a/CbmZPAU+FP68kuGvK9VNmy1bYspUhp82I9X2zwPA3stgLS6s3VpFVvXtjD57raNR1riFR2fZBwFysusbTcK7WJCnbXthdvAyU9e8aXQ1KULa9sLvYJeWoxrmoJSXbPghYFbWszXLuazO5c+eBFX2fjGX5p82HM3f5XIZu7KjoexWj1J4DkmZKWh6O43JtL89fJWmZpCWSFko6NOq2u8LqNdeQnF4xXtiraOR9i+k8u53v/fpTFX2fNBnuv+10ms7bQuo/X67oexVUYpewIsetfhGYZmYfIRgW4J+jbbwrRl3mGhLV3dELexVZOk121y5GLxHHLJrDrdujv3v9mxs+wrTnL2LUik6yu3ZBtrpf2wdfMFmPpQgFx602syfN7N3w4bMENxi5mNVjrqGsbEfOC3sCjLr7WQ46v40bn5kV+b4f+uXHaP3MCpoei3dgpj4ZNKR7LkUoOG51N5cCvym9oa5cdZVrKCfbkfPCngRmWDrNmD82ctjjl0RyhHP1+mM57HeXcsBLaSydBkvGt/UQ/ZjVPfYv/VdgGvAvETXZlaLOcg3JuRTjvWISZPQdzzD6zhTzHjyZvzv+52Xt6+GnpjP56mcjalmE+u4SttnM8o0xXWjcagAknQF8CzjZzPaU01QXjbrINXh3R5eHZRm8YASTVv7dPqszLRnuO+vfmd7cuM/6u3eO4frfnU9D577T1hz8dEL6XXVTxk0ce8etJijoc4AL9tm3dAzwH8BMM9tYXktdpGo81+A3KLl8zBj9k2cY3W11atJE/vSxyUxvXr3P+gc2TOPwby0j294eWxPLYqV9oWRmaUld41angDu6xq0GFpvZAoJLLy3AL4I5M3jDzEoe09pFqNZzDSVnuxK8sA8UG7dw9y2z+PGofY9gWtYao95bVKVGlabUo5pC41ab2RllNczFr4ZyDaVlW9JMgiGjU8BtZnZDt+evAj4PpIFNwCVmtibfPr2wDxCZnTs54EfP9PpcMo4RimRAZkC12FVQzeQaSsp2zv0ZZxL09FokaYGZLcvZrOv+jHcl/T3B/Rmfzbdf7xXjYteQsR6Lc7WghGxX5P4MP2J3sVKCeg44F6USs93b/RnH59m+qPszvLC72MmP0F2N6iPbYyTl3kk1P5xasX/7fv/+jJMLbeuF3cXLDPyI3dWivrOd7x6Nityf4YXdxc4vxbhaVUK2K3J/hhd2Fy/zSzGuRpWQ7Urdn+GF3cXOj9hdrSrx5rvI78/wwu7i5f3YXa1KULa9sLtYCUOZ6o+d7VzUkpRtL+wuXn6N3dWqBGXbC7uLmUE2IUPgORep5GTbC7uLV4KuQzoXqQRl2wu7i50SclTjXNSSkm0v7C5eZpBJRvidi1SCsu2F3cUvIUc1zkUuIdn2wu7iZQbpKk3d7lwlJSjbRY3HLmmkpAckvSrpFUknShot6XFJr4X/jqp0Y10NMILT1e5LlXi2XWQSlO1iJ9q4GfitmX0I+CjwCnAtsNDMJgMLw8fOFRB2Ceu+VI9n20UkOdkuWNgljQA+DtwOYGYdZradYJaPu8LN7gL+pjJNdDXFgEym51IFnm0XqQRlu5gj9okEE6j+RNKLkm6TNAw4yMzWh9u8DRxUqUa6WmKJOV3Fs+0ilZxsF1PYBwHHAj8ys2OAXXQ7NTUzo4+5ZyVdLmmxpMWdFBwf3tU6A8tkeixVUnK2PdeuhwRlu5jCvg5YZ2bPhY8fIPgwbJA0FiD8t9cB4M1svplNM7NpjTRH0WY3kJkl5nSVMrLtuXY9JCjbBQu7mb0NrJV0eLjqdGAZsAC4KFx3EfBwRVroaoxh6XSPpRiSZkpaLqlNUo8vNCU1S7ovfP45SRPytsSz7SJVWrajzjUU34/9y8DPJDUBK4GLCf4o3C/pUmANMLvIfbl61vUFUz9JSgHzgDMJjrQXSVpgZstyNrsU2GZmkyTNAW4EPltg155tF40Ssl2pXBdV2M3sJYLZsbs7vZjXO9fFzEq97jgdaDOzlQCS7iXovZL7ATgPuD78+QHgFkkKr5P31Z6X8Gy7CJSY7Yrkuth+7M5FpsQvmFqBtTmP14Xret3GzNLADmD/CJrsXFFKyHZFcu1DCrhYtbPtsSfsgTG9PDVY0uKcx/PNbH5c7XKuXEnKdqyFvZ1tm5+wB3YBm+N83yKNwdvVH/nadWhfLzKzmSW+35vA+JzH48J1vW2zTtIgYASwpcT3K5rnuiRJbRf03bY+cw0lZ7siuY61sJvZAZIWm1lv1zSrytvVP1Vo1yJgsqSJBEGfA1zQbZuu3izPAJ8Gfp/vOmRUPNf9l9R2Qextq0iu/VKMGxDMLC3pS8BjQAq4w8yWSvoOsNjMFhAMDfBTSW3AVoIPiXOJValce2F3A4aZPQo82m3dt3N+3g18Ju52OVeOSuS6Gr1ikvqFmLerf5LarmpJ6u/D29V/SW5bURTDJUjnnHMx8n7szjlXY2Ir7IXGQ4ixHeMlPSlpmaSlkq4M1ydi1hxJqXAI2UfCxxPD8SHawvEimqrQJp9lKA/PdtHt82zHJJbCnjMewixgKjBX0tQ43rsXaeBqM5sKnAB8MWxLUmbNuZJgFp8uNwI/MLNJwDaCcSPi5rMM9cGz3S+e7biYWcUX4ETgsZzH1wHXxfHeRbTtYYIBeJYDY8N1Y4HlVWjLOIIgnQY8AojgRolBvf0eY2rTCGAV4fcxOeur/vtKwuLZLrotnu0Yl7guxRQzHkLswuEvjwGeIxmz5twEfB3omnZlf2C7BeNDQHV+bz7LUH6e7eLchGc7NnX75amkFuBB4KtmtjP3OQv+VMfaXUjSOcBGM3shzvctQlkzaLn4ebaLVrPZjquwFzMeQmwkNRIE/2dm9stwdVEzQlXQScC5klYD9xKcst4MjAzHh4Dq/N7KmkGrDni2C/Nsxyyuwr53PITwm+85BOMfxE6SCG7RfcXMvp/zVFVnzTGz68xsnJlNIPj9/N7MLgSeJBgfolrt8lmG8vNsF+DZroIYv6g4G1gBvA58q1pfKgAfIzi1WgK8FC5nE1zzWwi8BjwBjK5iG08BHgl/Pgx4HmgDfgE0V6E9RwOLw9/ZQ8CoJP2+qr14tvvVRs92DIvfeeqcczWmbr88dc65WuWF3TnnaowXduecqzFe2J1zrsZ4YXfOuRrjhd0552qMF3bnnKsxXtidc67G/H9AN6obKmDZKgAAAABJRU5ErkJggg==",
      "text/plain": [
       "<Figure size 432x288 with 4 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "img_nib = nib.load(os.path.join(dataroot_dir, sim_datalist['testing'][0]['image']))\n",
    "lbl_nib = nib.load(os.path.join(dataroot_dir, sim_datalist['testing'][0]['label']))\n",
    "img = np.array(img_nib.dataobj)\n",
    "lbl = np.array(lbl_nib.dataobj)\n",
    "plt.subplot(1, 2, 1)\n",
    "plt.imshow(img[32])\n",
    "plt.title('image')\n",
    "cbar = plt.colorbar(shrink=0.55)\n",
    "plt.subplot(1, 2, 2)\n",
    "plt.imshow(lbl[32])\n",
    "plt.title('label')\n",
    "cbar = plt.colorbar(shrink=0.55)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2. Test the Auto3Dsseg AutoRunner pipeline\n",
    "\n",
    "AutoRunner provides an interface to run the Auot3Dseg pipeline. It will automatically configure four different neural networks and perform multi-fold training to achieve the state-of-the-art performance. The module is highly configurable, but only requires minimal user inputs. In this example, we want to complete the piepline in two minutes, so we need to have some customizations\n",
    "\n",
    "### 2.1 Set up the AutoRunner with default settings"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2022-09-14 10:06:02,342 - INFO - Work directory ./auto3dseg_work_dir is used to save all results\n",
      "2022-09-14 10:06:02,344 - INFO - The output_dir is not specified. /workspace/monai/tutorial-in-dev/auto3dseg/notebooks/auto3dseg_work_dir/ensemble_output will be used to save ensemble predictions\n",
      "2022-09-14 10:06:02,345 - INFO - Directory /workspace/monai/tutorial-in-dev/auto3dseg/notebooks/auto3dseg_work_dir/ensemble_output is created to save ensemble predictions\n"
     ]
    }
   ],
   "source": [
    "runner = AutoRunner(\n",
    "    work_dir=work_dir,\n",
    "    input={\n",
    "        \"modality\": \"MRI\",\n",
    "        \"datalist\": datalist_file,\n",
    "        \"dataroot\": dataroot_dir,\n",
    "    }\n",
    ")\n",
    "# runner.run() # this will use the default settings and take hours/ to run "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.2 Override the training parameters so that we can complete the pipeline in minutes"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "train_param = {\n",
    "    \"CUDA_VISIBLE_DEVICES\": [0],\n",
    "    \"num_iterations\": 8,\n",
    "    \"num_iterations_per_validation\": 4,\n",
    "    \"num_images_per_batch\": 2,\n",
    "    \"num_epochs\": 2,\n",
    "    \"num_warmup_iterations\": 4\n",
    "}\n",
    "runner.set_training_params(train_param)\n",
    "runner.set_num_fold(num_fold=1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.3 Start the data anlaysis, algorithm generation, training, and model ensemble"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100%|██████████| 12/12 [00:01<00:00,  9.41it/s]\n",
      "algo_templates.tar.gz: 100%|██████████| 280k/280k [00:03<00:00, 81.9kB/s] "
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2022-09-14 10:06:09,788 - INFO - Downloaded: /tmp/tmpcs1plh4w/algo_templates.tar.gz\n",
      "2022-09-14 10:06:09,789 - INFO - Expected md5 is None, skip md5 check for file /tmp/tmpcs1plh4w/algo_templates.tar.gz.\n",
      "2022-09-14 10:06:09,790 - INFO - Writing into directory: /workspace/monai/tutorial-in-dev/auto3dseg/notebooks/auto3dseg_work_dir.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2022-09-14 10:06:10,218 - INFO - /workspace/monai/tutorial-in-dev/auto3dseg/notebooks/auto3dseg_work_dir/segresnet2d_0\n",
      "2022-09-14 10:06:10,435 - INFO - /workspace/monai/tutorial-in-dev/auto3dseg/notebooks/auto3dseg_work_dir/dints_0\n",
      "2022-09-14 10:06:10,625 - INFO - /workspace/monai/tutorial-in-dev/auto3dseg/notebooks/auto3dseg_work_dir/swinunetr_0\n",
      "2022-09-14 10:06:10,844 - INFO - /workspace/monai/tutorial-in-dev/auto3dseg/notebooks/auto3dseg_work_dir/segresnet_0\n",
      "2022-09-14 10:06:10,846 - INFO - Launching: python /workspace/monai/tutorial-in-dev/auto3dseg/notebooks/auto3dseg_work_dir/segresnet_0/scripts/train.py run --config_file='/workspace/monai/tutorial-in-dev/auto3dseg/notebooks/auto3dseg_work_dir/segresnet_0/configs/transforms_train.yaml','/workspace/monai/tutorial-in-dev/auto3dseg/notebooks/auto3dseg_work_dir/segresnet_0/configs/network.yaml','/workspace/monai/tutorial-in-dev/auto3dseg/notebooks/auto3dseg_work_dir/segresnet_0/configs/transforms_validate.yaml','/workspace/monai/tutorial-in-dev/auto3dseg/notebooks/auto3dseg_work_dir/segresnet_0/configs/hyper_parameters.yaml','/workspace/monai/tutorial-in-dev/auto3dseg/notebooks/auto3dseg_work_dir/segresnet_0/configs/transforms_infer.yaml' --num_iterations=8 --num_iterations_per_validation=4 --num_images_per_batch=2 --num_epochs=2 --num_warmup_iterations=4\n",
      "2022-09-14 10:06:21,929 - INFO - CompletedProcess(args=['python', '/workspace/monai/tutorial-in-dev/auto3dseg/notebooks/auto3dseg_work_dir/segresnet_0/scripts/train.py', 'run', \"--config_file='/workspace/monai/tutorial-in-dev/auto3dseg/notebooks/auto3dseg_work_dir/segresnet_0/configs/transforms_train.yaml','/workspace/monai/tutorial-in-dev/auto3dseg/notebooks/auto3dseg_work_dir/segresnet_0/configs/network.yaml','/workspace/monai/tutorial-in-dev/auto3dseg/notebooks/auto3dseg_work_dir/segresnet_0/configs/transforms_validate.yaml','/workspace/monai/tutorial-in-dev/auto3dseg/notebooks/auto3dseg_work_dir/segresnet_0/configs/hyper_parameters.yaml','/workspace/monai/tutorial-in-dev/auto3dseg/notebooks/auto3dseg_work_dir/segresnet_0/configs/transforms_infer.yaml'\", '--num_iterations=8', '--num_iterations_per_validation=4', '--num_images_per_batch=2', '--num_epochs=2', '--num_warmup_iterations=4'], returncode=0, stdout=b\"[info] number of GPUs: 1\n",
      "[info] world_size: 1\n",
      "train_files: 8\n",
      "val_files: 4\n",
      "num_epochs 2\n",
      "num_epochs_per_validation 1\n",
      "[info] training from scratch\n",
      "[info] amp enabled\n",
      "----------\n",
      "epoch 1/2\n",
      "learning rate is set to 2e-05\n",
      "[2022-09-14 10:06:18] 1/4, train_loss: 1.2363\n",
      "[2022-09-14 10:06:18] 2/4, train_loss: 1.2216\n",
      "[2022-09-14 10:06:18] 3/4, train_loss: 1.0369\n",
      "[2022-09-14 10:06:18] 4/4, train_loss: 1.0711\n",
      "epoch 1 average loss: 1.1415, best mean dice: -1.0000 at epoch -1\n",
      "1 / 4 tensor([[0.6527]], device='cuda:0')\n",
      "2 / 4 tensor([[0.5636]], device='cuda:0')\n",
      "3 / 4 tensor([[0.5988]], device='cuda:0')\n",
      "4 / 4 tensor([[0.5912]], device='cuda:0')\n",
      "evaluation metric - class 1: 0.6015638709068298\n",
      "avg_metric 0.6015638709068298\n",
      "saved new best metric model\n",
      "current epoch: 1 current mean dice: 0.6016 best mean dice: 0.6016 at epoch 1\n",
      "----------\n",
      "epoch 2/2\n",
      "learning rate is set to 0.0\n",
      "[2022-09-14 10:06:19] 1/4, train_loss: 0.9718\n",
      "[2022-09-14 10:06:19] 2/4, train_loss: 0.9847\n",
      "[2022-09-14 10:06:20] 3/4, train_loss: 0.9438\n",
      "[2022-09-14 10:06:20] 4/4, train_loss: 0.9388\n",
      "epoch 2 average loss: 0.9598, best mean dice: 0.6016 at epoch 1\n",
      "1 / 4 tensor([[0.7008]], device='cuda:0')\n",
      "2 / 4 tensor([[0.6137]], device='cuda:0')\n",
      "3 / 4 tensor([[0.6392]], device='cuda:0')\n",
      "4 / 4 tensor([[0.6402]], device='cuda:0')\n",
      "evaluation metric - class 1: 0.6484684944152832\n",
      "avg_metric 0.6484684944152832\n",
      "saved new best metric model\n",
      "current epoch: 2 current mean dice: 0.6485 best mean dice: 0.6485 at epoch 2\n",
      "train completed, best_metric: 0.6485 at epoch: 2\n",
      "0.6484684944152832\n",
      "\", stderr=b'')\n",
      "2022-09-14 10:06:21,942 - INFO - Launching: python /workspace/monai/tutorial-in-dev/auto3dseg/notebooks/auto3dseg_work_dir/segresnet2d_0/scripts/train.py run --config_file='/workspace/monai/tutorial-in-dev/auto3dseg/notebooks/auto3dseg_work_dir/segresnet2d_0/configs/transforms_train.yaml','/workspace/monai/tutorial-in-dev/auto3dseg/notebooks/auto3dseg_work_dir/segresnet2d_0/configs/network.yaml','/workspace/monai/tutorial-in-dev/auto3dseg/notebooks/auto3dseg_work_dir/segresnet2d_0/configs/transforms_validate.yaml','/workspace/monai/tutorial-in-dev/auto3dseg/notebooks/auto3dseg_work_dir/segresnet2d_0/configs/hyper_parameters.yaml','/workspace/monai/tutorial-in-dev/auto3dseg/notebooks/auto3dseg_work_dir/segresnet2d_0/configs/transforms_infer.yaml' --num_iterations=8 --num_iterations_per_validation=4 --num_images_per_batch=2 --num_epochs=2 --num_warmup_iterations=4\n",
      "2022-09-14 10:06:35,376 - INFO - CompletedProcess(args=['python', '/workspace/monai/tutorial-in-dev/auto3dseg/notebooks/auto3dseg_work_dir/segresnet2d_0/scripts/train.py', 'run', \"--config_file='/workspace/monai/tutorial-in-dev/auto3dseg/notebooks/auto3dseg_work_dir/segresnet2d_0/configs/transforms_train.yaml','/workspace/monai/tutorial-in-dev/auto3dseg/notebooks/auto3dseg_work_dir/segresnet2d_0/configs/network.yaml','/workspace/monai/tutorial-in-dev/auto3dseg/notebooks/auto3dseg_work_dir/segresnet2d_0/configs/transforms_validate.yaml','/workspace/monai/tutorial-in-dev/auto3dseg/notebooks/auto3dseg_work_dir/segresnet2d_0/configs/hyper_parameters.yaml','/workspace/monai/tutorial-in-dev/auto3dseg/notebooks/auto3dseg_work_dir/segresnet2d_0/configs/transforms_infer.yaml'\", '--num_iterations=8', '--num_iterations_per_validation=4', '--num_images_per_batch=2', '--num_epochs=2', '--num_warmup_iterations=4'], returncode=0, stdout=b\"[info] number of GPUs: 1\n",
      "[info] world_size: 1\n",
      "train_files: 8\n",
      "val_files: 4\n",
      "num_epochs 2\n",
      "num_epochs_per_validation 1\n",
      "[info] training from scratch\n",
      "[info] amp enabled\n",
      "----------\n",
      "epoch 1/2\n",
      "learning rate is set to 0.025\n",
      "[2022-09-14 10:06:28] 1/4, train_loss: 0.5923\n",
      "[2022-09-14 10:06:28] 2/4, train_loss: 0.6014\n",
      "[2022-09-14 10:06:28] 3/4, train_loss: 0.6480\n",
      "[2022-09-14 10:06:28] 4/4, train_loss: 0.5376\n",
      "epoch 1 average loss: 0.5948, best mean dice: -1.0000 at epoch -1\n",
      "1 / 4 tensor([[0.4825]], device='cuda:0')\n",
      "2 / 4 tensor([[0.5447]], device='cuda:0')\n",
      "3 / 4 tensor([[0.4675]], device='cuda:0')\n",
      "4 / 4 tensor([[0.4297]], device='cuda:0')\n",
      "evaluation metric - class 1: 0.4810941815376282\n",
      "avg_metric 0.4810941815376282\n",
      "saved new best metric model\n",
      "current epoch: 1 current mean dice: 0.4811 best mean dice: 0.4811 at epoch 1\n",
      "----------\n",
      "epoch 2/2\n",
      "learning rate is set to 0.0015625\n",
      "[2022-09-14 10:06:31] 1/4, train_loss: 0.4854\n",
      "[2022-09-14 10:06:32] 2/4, train_loss: 0.3979\n",
      "[2022-09-14 10:06:32] 3/4, train_loss: 0.4743\n",
      "[2022-09-14 10:06:32] 4/4, train_loss: 0.5052\n",
      "epoch 2 average loss: 0.4657, best mean dice: 0.4811 at epoch 1\n",
      "1 / 4 tensor([[0.5429]], device='cuda:0')\n",
      "2 / 4 tensor([[0.5752]], device='cuda:0')\n",
      "3 / 4 tensor([[0.5219]], device='cuda:0')\n",
      "4 / 4 tensor([[0.4922]], device='cuda:0')\n",
      "evaluation metric - class 1: 0.5330679416656494\n",
      "avg_metric 0.5330679416656494\n",
      "saved new best metric model\n",
      "current epoch: 2 current mean dice: 0.5331 best mean dice: 0.5331 at epoch 2\n",
      "train completed, best_metric: 0.5331 at epoch: 2\n",
      "0.5330679416656494\n",
      "\", stderr=b'')\n",
      "2022-09-14 10:06:35,389 - INFO - Launching: python /workspace/monai/tutorial-in-dev/auto3dseg/notebooks/auto3dseg_work_dir/swinunetr_0/scripts/train.py run --config_file='/workspace/monai/tutorial-in-dev/auto3dseg/notebooks/auto3dseg_work_dir/swinunetr_0/configs/transforms_train.yaml','/workspace/monai/tutorial-in-dev/auto3dseg/notebooks/auto3dseg_work_dir/swinunetr_0/configs/network.yaml','/workspace/monai/tutorial-in-dev/auto3dseg/notebooks/auto3dseg_work_dir/swinunetr_0/configs/transforms_validate.yaml','/workspace/monai/tutorial-in-dev/auto3dseg/notebooks/auto3dseg_work_dir/swinunetr_0/configs/hyper_parameters.yaml','/workspace/monai/tutorial-in-dev/auto3dseg/notebooks/auto3dseg_work_dir/swinunetr_0/configs/transforms_infer.yaml' --num_iterations=8 --num_iterations_per_validation=4 --num_images_per_batch=2 --num_epochs=2 --num_warmup_iterations=4\n",
      "2022-09-14 10:06:48,294 - INFO - CompletedProcess(args=['python', '/workspace/monai/tutorial-in-dev/auto3dseg/notebooks/auto3dseg_work_dir/swinunetr_0/scripts/train.py', 'run', \"--config_file='/workspace/monai/tutorial-in-dev/auto3dseg/notebooks/auto3dseg_work_dir/swinunetr_0/configs/transforms_train.yaml','/workspace/monai/tutorial-in-dev/auto3dseg/notebooks/auto3dseg_work_dir/swinunetr_0/configs/network.yaml','/workspace/monai/tutorial-in-dev/auto3dseg/notebooks/auto3dseg_work_dir/swinunetr_0/configs/transforms_validate.yaml','/workspace/monai/tutorial-in-dev/auto3dseg/notebooks/auto3dseg_work_dir/swinunetr_0/configs/hyper_parameters.yaml','/workspace/monai/tutorial-in-dev/auto3dseg/notebooks/auto3dseg_work_dir/swinunetr_0/configs/transforms_infer.yaml'\", '--num_iterations=8', '--num_iterations_per_validation=4', '--num_images_per_batch=2', '--num_epochs=2', '--num_warmup_iterations=4'], returncode=0, stdout=b\"[info] number of GPUs: 1\n",
      "[info] world_size: 1\n",
      "train_files: 8\n",
      "val_files: 4\n",
      "num_epochs 2\n",
      "num_epochs_per_validation 1\n",
      "[info] training from scratch\n",
      "[info] amp enabled\n",
      "----------\n",
      "epoch 1/2\n",
      "learning rate is set to 0.0001\n",
      "[2022-09-14 10:06:42] 1/4, train_loss: 1.3725\n",
      "[2022-09-14 10:06:42] 2/4, train_loss: 1.3171\n",
      "[2022-09-14 10:06:43] 3/4, train_loss: 1.1689\n",
      "[2022-09-14 10:06:43] 4/4, train_loss: 1.0789\n",
      "epoch 1 average loss: 1.2344, best mean dice: -1.0000 at epoch -1\n",
      "1 / 4 tensor([[0.3653]], device='cuda:0')\n",
      "2 / 4 tensor([[0.3513]], device='cuda:0')\n",
      "3 / 4 tensor([[0.3798]], device='cuda:0')\n",
      "4 / 4 tensor([[0.3683]], device='cuda:0')\n",
      "evaluation metric - class 1: 0.36617928743362427\n",
      "avg_metric 0.36617928743362427\n",
      "saved new best metric model\n",
      "current epoch: 1 current mean dice: 0.3662 best mean dice: 0.3662 at epoch 1\n",
      "----------\n",
      "epoch 2/2\n",
      "learning rate is set to 6.25e-06\n",
      "[2022-09-14 10:06:45] 1/4, train_loss: 1.0401\n",
      "[2022-09-14 10:06:45] 2/4, train_loss: 1.0208\n",
      "[2022-09-14 10:06:45] 3/4, train_loss: 1.0062\n",
      "[2022-09-14 10:06:46] 4/4, train_loss: 1.0117\n",
      "epoch 2 average loss: 1.0197, best mean dice: 0.3662 at epoch 1\n",
      "1 / 4 tensor([[0.4009]], device='cuda:0')\n",
      "2 / 4 tensor([[0.3918]], device='cuda:0')\n",
      "3 / 4 tensor([[0.4300]], device='cuda:0')\n",
      "4 / 4 tensor([[0.4121]], device='cuda:0')\n",
      "evaluation metric - class 1: 0.4086832106113434\n",
      "avg_metric 0.4086832106113434\n",
      "saved new best metric model\n",
      "current epoch: 2 current mean dice: 0.4087 best mean dice: 0.4087 at epoch 2\n",
      "train completed, best_metric: 0.4087 at epoch: 2\n",
      "0.4086832106113434\n",
      "\", stderr=b'Detected call of `lr_scheduler.step()` before `optimizer.step()`. In PyTorch 1.1.0 and later, you should call them in the opposite order: `optimizer.step()` before `lr_scheduler.step()`.  Failure to do this will result in PyTorch skipping the first value of the learning rate schedule. See more details at https://pytorch.org/docs/stable/optim.html#how-to-adjust-learning-rate\n",
      "')\n",
      "2022-09-14 10:06:48,305 - INFO - Launching: python /workspace/monai/tutorial-in-dev/auto3dseg/notebooks/auto3dseg_work_dir/dints_0/scripts/search.py run --config_file='/workspace/monai/tutorial-in-dev/auto3dseg/notebooks/auto3dseg_work_dir/dints_0/configs/transforms_train.yaml','/workspace/monai/tutorial-in-dev/auto3dseg/notebooks/auto3dseg_work_dir/dints_0/configs/network.yaml','/workspace/monai/tutorial-in-dev/auto3dseg/notebooks/auto3dseg_work_dir/dints_0/configs/transforms_validate.yaml','/workspace/monai/tutorial-in-dev/auto3dseg/notebooks/auto3dseg_work_dir/dints_0/configs/hyper_parameters_search.yaml','/workspace/monai/tutorial-in-dev/auto3dseg/notebooks/auto3dseg_work_dir/dints_0/configs/network_search.yaml','/workspace/monai/tutorial-in-dev/auto3dseg/notebooks/auto3dseg_work_dir/dints_0/configs/hyper_parameters.yaml','/workspace/monai/tutorial-in-dev/auto3dseg/notebooks/auto3dseg_work_dir/dints_0/configs/transforms_infer.yaml' --searching#num_iterations=8 --searching#num_iterations_per_validation=4 --searching#num_images_per_batch=2 --searching#num_epochs=2 --searching#num_warmup_iterations=4\n",
      "2022-09-14 10:07:38,374 - INFO - CompletedProcess(args=['python', '/workspace/monai/tutorial-in-dev/auto3dseg/notebooks/auto3dseg_work_dir/dints_0/scripts/search.py', 'run', \"--config_file='/workspace/monai/tutorial-in-dev/auto3dseg/notebooks/auto3dseg_work_dir/dints_0/configs/transforms_train.yaml','/workspace/monai/tutorial-in-dev/auto3dseg/notebooks/auto3dseg_work_dir/dints_0/configs/network.yaml','/workspace/monai/tutorial-in-dev/auto3dseg/notebooks/auto3dseg_work_dir/dints_0/configs/transforms_validate.yaml','/workspace/monai/tutorial-in-dev/auto3dseg/notebooks/auto3dseg_work_dir/dints_0/configs/hyper_parameters_search.yaml','/workspace/monai/tutorial-in-dev/auto3dseg/notebooks/auto3dseg_work_dir/dints_0/configs/network_search.yaml','/workspace/monai/tutorial-in-dev/auto3dseg/notebooks/auto3dseg_work_dir/dints_0/configs/hyper_parameters.yaml','/workspace/monai/tutorial-in-dev/auto3dseg/notebooks/auto3dseg_work_dir/dints_0/configs/transforms_infer.yaml'\", '--searching#num_iterations=8', '--searching#num_iterations_per_validation=4', '--searching#num_images_per_batch=2', '--searching#num_epochs=2', '--searching#num_warmup_iterations=4'], returncode=0, stdout=b\"[info] number of GPUs: 1\n",
      "[info] world_size: 1\n",
      "train_files_w: 4\n",
      "train_files_a: 4\n",
      "val_files: 4\n",
      "num_epochs 4\n",
      "num_epochs_warmup 2\n",
      "num_epochs_per_validation 2\n",
      "[info] amp enabled\n",
      "----------\n",
      "epoch 1/4\n",
      "learning rate is set to 0.025\n",
      "[2022-09-14 10:06:58] 1/2, train_loss: 0.7565\n",
      "[2022-09-14 10:06:59] 2/2, train_loss: 0.7478\n",
      "epoch 1 average loss: 0.7522, best mean dice: -1.0000 at epoch -1\n",
      "----------\n",
      "epoch 2/4\n",
      "learning rate is set to 0.025\n",
      "[2022-09-14 10:07:01] 1/2, train_loss: 0.7479\n",
      "[2022-09-14 10:07:02] 2/2, train_loss: 0.7597\n",
      "epoch 2 average loss: 0.7538, best mean dice: -1.0000 at epoch -1\n",
      "1 / 4 tensor([[0.1909]], device='cuda:0')\n",
      "2 / 4 tensor([[0.1381]], device='cuda:0')\n",
      "3 / 4 tensor([[0.1919]], device='cuda:0')\n",
      "4 / 4 tensor([[0.1288]], device='cuda:0')\n",
      "evaluation metric - class 1: 0.16243486106395721\n",
      "avg_metric 0.16243486106395721\n",
      "saved new best metric model\n",
      "current epoch: 2 current mean dice: 0.1624 best mean dice: 0.1624 at epoch 2\n",
      "----------\n",
      "epoch 3/4\n",
      "learning rate is set to 0.025\n",
      "[2022-09-14 10:07:16] 1/2, train_loss: 0.7378\n",
      "[2022-09-14 10:07:17] 1/2, train_loss_arch: 0.7343\n",
      "[2022-09-14 10:07:18] 2/2, train_loss: 0.7439\n",
      "[2022-09-14 10:07:19] 2/2, train_loss_arch: 0.7561\n",
      "epoch 3 average loss: 0.7408, best mean dice: 0.1624 at epoch 2\n",
      "epoch 3 average arch loss: 0.7452, best mean dice: 0.1624 at epoch 2\n",
      "----------\n",
      "epoch 4/4\n",
      "learning rate is set to 0.00625\n",
      "[2022-09-14 10:07:21] 1/2, train_loss: 0.7396\n",
      "[2022-09-14 10:07:23] 1/2, train_loss_arch: 1.0639\n",
      "[2022-09-14 10:07:24] 2/2, train_loss: 0.7139\n",
      "[2022-09-14 10:07:25] 2/2, train_loss_arch: 1.0797\n",
      "epoch 4 average loss: 0.7267, best mean dice: 0.1624 at epoch 2\n",
      "epoch 4 average arch loss: 1.0718, best mean dice: 0.1624 at epoch 2\n",
      "1 / 4 tensor([[0.2357]], device='cuda:0')\n",
      "2 / 4 tensor([[0.1475]], device='cuda:0')\n",
      "3 / 4 tensor([[0.1962]], device='cuda:0')\n",
      "4 / 4 tensor([[0.1431]], device='cuda:0')\n",
      "evaluation metric - class 1: 0.18062789738178253\n",
      "avg_metric 0.18062789738178253\n",
      "saved new best metric model\n",
      "current epoch: 4 current mean dice: 0.1806 best mean dice: 0.1806 at epoch 4\n",
      "train completed, best_metric: 0.1806 at epoch: 4\n",
      "\", stderr=b'')\n",
      "2022-09-14 10:07:38,376 - INFO - Launching: python /workspace/monai/tutorial-in-dev/auto3dseg/notebooks/auto3dseg_work_dir/dints_0/scripts/train.py run --config_file='/workspace/monai/tutorial-in-dev/auto3dseg/notebooks/auto3dseg_work_dir/dints_0/configs/transforms_train.yaml','/workspace/monai/tutorial-in-dev/auto3dseg/notebooks/auto3dseg_work_dir/dints_0/configs/network.yaml','/workspace/monai/tutorial-in-dev/auto3dseg/notebooks/auto3dseg_work_dir/dints_0/configs/transforms_validate.yaml','/workspace/monai/tutorial-in-dev/auto3dseg/notebooks/auto3dseg_work_dir/dints_0/configs/hyper_parameters_search.yaml','/workspace/monai/tutorial-in-dev/auto3dseg/notebooks/auto3dseg_work_dir/dints_0/configs/network_search.yaml','/workspace/monai/tutorial-in-dev/auto3dseg/notebooks/auto3dseg_work_dir/dints_0/configs/hyper_parameters.yaml','/workspace/monai/tutorial-in-dev/auto3dseg/notebooks/auto3dseg_work_dir/dints_0/configs/transforms_infer.yaml' --training#num_iterations=8 --training#num_iterations_per_validation=4 --training#num_images_per_batch=2 --training#num_epochs=2 --training#num_warmup_iterations=4\n",
      "2022-09-14 10:07:56,477 - INFO - CompletedProcess(args=['python', '/workspace/monai/tutorial-in-dev/auto3dseg/notebooks/auto3dseg_work_dir/dints_0/scripts/train.py', 'run', \"--config_file='/workspace/monai/tutorial-in-dev/auto3dseg/notebooks/auto3dseg_work_dir/dints_0/configs/transforms_train.yaml','/workspace/monai/tutorial-in-dev/auto3dseg/notebooks/auto3dseg_work_dir/dints_0/configs/network.yaml','/workspace/monai/tutorial-in-dev/auto3dseg/notebooks/auto3dseg_work_dir/dints_0/configs/transforms_validate.yaml','/workspace/monai/tutorial-in-dev/auto3dseg/notebooks/auto3dseg_work_dir/dints_0/configs/hyper_parameters_search.yaml','/workspace/monai/tutorial-in-dev/auto3dseg/notebooks/auto3dseg_work_dir/dints_0/configs/network_search.yaml','/workspace/monai/tutorial-in-dev/auto3dseg/notebooks/auto3dseg_work_dir/dints_0/configs/hyper_parameters.yaml','/workspace/monai/tutorial-in-dev/auto3dseg/notebooks/auto3dseg_work_dir/dints_0/configs/transforms_infer.yaml'\", '--training#num_iterations=8', '--training#num_iterations_per_validation=4', '--training#num_images_per_batch=2', '--training#num_epochs=2', '--training#num_warmup_iterations=4'], returncode=0, stdout=b\"[info] number of GPUs: 1\n",
      "[info] world_size: 1\n",
      "train_files: 8\n",
      "val_files: 4\n",
      "num_epochs 2\n",
      "num_epochs_per_validation 1\n",
      "[info] training from scratch\n",
      "[info] amp enabled\n",
      "----------\n",
      "epoch 1/2\n",
      "learning rate is set to 0.2\n",
      "[2022-09-14 10:07:47] 1/4, train_loss: 0.6825\n",
      "[2022-09-14 10:07:48] 2/4, train_loss: 0.6256\n",
      "[2022-09-14 10:07:48] 3/4, train_loss: 0.6048\n",
      "[2022-09-14 10:07:49] 4/4, train_loss: 0.6135\n",
      "epoch 1 average loss: 0.6316, best mean dice: -1.0000 at epoch -1\n",
      "1 / 4 tensor([[0.3849]], device='cuda:0')\n",
      "2 / 4 tensor([[0.2863]], device='cuda:0')\n",
      "3 / 4 tensor([[0.3075]], device='cuda:0')\n",
      "4 / 4 tensor([[0.3176]], device='cuda:0')\n",
      "evaluation metric - class 1: 0.32408225536346436\n",
      "avg_metric 0.32408225536346436\n",
      "saved new best metric model\n",
      "current epoch: 1 current mean dice: 0.3241 best mean dice: 0.3241 at epoch 1\n",
      "----------\n",
      "epoch 2/2\n",
      "learning rate is set to 0.0125\n",
      "[2022-09-14 10:07:51] 1/4, train_loss: 0.5886\n",
      "[2022-09-14 10:07:52] 2/4, train_loss: 0.5935\n",
      "[2022-09-14 10:07:53] 3/4, train_loss: 0.5752\n",
      "[2022-09-14 10:07:53] 4/4, train_loss: 0.5734\n",
      "epoch 2 average loss: 0.5827, best mean dice: 0.3241 at epoch 1\n",
      "1 / 4 tensor([[0.3974]], device='cuda:0')\n",
      "2 / 4 tensor([[0.2958]], device='cuda:0')\n",
      "3 / 4 tensor([[0.3169]], device='cuda:0')\n",
      "4 / 4 tensor([[0.3279]], device='cuda:0')\n",
      "evaluation metric - class 1: 0.33449363708496094\n",
      "avg_metric 0.33449363708496094\n",
      "saved new best metric model\n",
      "current epoch: 2 current mean dice: 0.3345 best mean dice: 0.3345 at epoch 2\n",
      "train completed, best_metric: 0.3345 at epoch: 2\n",
      "0.33449363708496094\n",
      "\", stderr=b'')\n",
      "0\n",
      "[info] checkpoint /workspace/monai/tutorial-in-dev/auto3dseg/notebooks/auto3dseg_work_dir/swinunetr_0/model_fold0/best_metric_model.pt loaded\n",
      "2022-09-14 10:07:58,635 INFO image_writer.py:194 - writing: /workspace/monai/tutorial-in-dev/auto3dseg/notebooks/auto3dseg_work_dir/swinunetr_0/prediction_testing/test_image_001/test_image_001_seg.nii.gz\n",
      "[info] checkpoint /workspace/monai/tutorial-in-dev/auto3dseg/notebooks/auto3dseg_work_dir/dints_0/model_fold0/best_metric_model.pt loaded\n",
      "2022-09-14 10:08:00,584 INFO image_writer.py:194 - writing: /workspace/monai/tutorial-in-dev/auto3dseg/notebooks/auto3dseg_work_dir/dints_0/prediction_testing/test_image_001/test_image_001_seg.nii.gz\n",
      "1\n",
      "[info] checkpoint /workspace/monai/tutorial-in-dev/auto3dseg/notebooks/auto3dseg_work_dir/swinunetr_0/model_fold0/best_metric_model.pt loaded\n",
      "2022-09-14 10:08:01,396 INFO image_writer.py:194 - writing: /workspace/monai/tutorial-in-dev/auto3dseg/notebooks/auto3dseg_work_dir/swinunetr_0/prediction_testing/test_image_002/test_image_002_seg.nii.gz\n",
      "[info] checkpoint /workspace/monai/tutorial-in-dev/auto3dseg/notebooks/auto3dseg_work_dir/dints_0/model_fold0/best_metric_model.pt loaded\n",
      "2022-09-14 10:08:03,239 INFO image_writer.py:194 - writing: /workspace/monai/tutorial-in-dev/auto3dseg/notebooks/auto3dseg_work_dir/dints_0/prediction_testing/test_image_002/test_image_002_seg.nii.gz\n",
      "Auto3Dseg picked the following networks to ensemble:\n",
      "swinunetr_0\n",
      "dints_0\n",
      "2022-09-14 10:08:03,250 INFO image_writer.py:194 - writing: /workspace/monai/tutorial-in-dev/auto3dseg/notebooks/auto3dseg_work_dir/ensemble_output/test_image_001/test_image_001_ensemble.nii.gz\n",
      "2022-09-14 10:08:03,266 INFO image_writer.py:194 - writing: /workspace/monai/tutorial-in-dev/auto3dseg/notebooks/auto3dseg_work_dir/ensemble_output/test_image_002/test_image_002_ensemble.nii.gz\n",
      "2022-09-14 10:08:03,279 - INFO - Auto3Dseg ensemble prediction outputs are saved in /workspace/monai/tutorial-in-dev/auto3dseg/notebooks/auto3dseg_work_dir/ensemble_output.\n",
      "2022-09-14 10:08:03,280 - INFO - Auto3Dseg pipeline is complete successfully.\n"
     ]
    }
   ],
   "source": [
    "runner.run()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.4 Inspect the algo ensembled predctions and compare it with the ground truth"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX4AAAEVCAYAAADn6Y5lAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8qNh9FAAAACXBIWXMAAAsTAAALEwEAmpwYAABFo0lEQVR4nO29e5wU5ZX//z7dXGYYUUDut4BcBHQTNxLBH15YTAyigLIbf5iYb6JmMbvRJIKJFzb5Gk02mGjQ7CYxbFBjolFjFFFZjDcW44KiWRe5JGYEDIhyUVCYcYDpOt8/qqqnpqd7pnumu6p6+rxfr3pNd1V19enu5/PM85w6zzmiqhiGYRiVQyJqAwzDMIxwsY7fMAyjwrCO3zAMo8Kwjt8wDKPCsI7fMAyjwrCO3zAMo8Kwjj9PRGSjiEyN2g7DKCdEZJuIfDKP81RERrfzPdr92kqlS9QGlAuqekLUNhiGYRQDG/EbhmFUGNbx54k/ZRWRG0TktyLyaxE5ICKvichYEblORHaLyHYROTvwuktEZLN37hYRuTzjut8UkbdFZKeIfCk4bRWR7iJyi4j8VUR2icgdIlId9mc3jI4iIqeIyBoR2e+1938XkW4Zp83wNLJXRH4oIonA6y/1dLRPRJ4UkY+E/BE6Fdbxt4+ZwK+A3sD/AE/ifpdDgBuBnwfO3Q2cBxwNXAIsFpGPA4jIdGA+8ElgNDA1430WAWOBk7zjQ4Bvl+DzGEapSQFXAX2BU4GzgH/OOOcCYCLwcWA2cCmAiMwGrgfmAP2A54HfhGJ1J0UsV09+iMg24EvAacAUVf2Ut38mbiM8RlVTItIT+ADorar7s1xnGfCcqt4uIncCu1T1Ou/YaOAvwBjgDeAg8FFVfcM7fipwn6qOLOVnNYxi4etGVZ/O2P914ExVvcB7rsA5qrrSe/7PwN+r6lki8p/AQ6q61DuWwNXGeFV903vtGFWtDetzlTs24m8fuwKPPwT2qmoq8BzgKAAROUdE1orIeyKyH5iBO+oBGAxsD1wr+Lgf0AN4xZse7wdWevsNo6zw3KGPi8g7IvIB8K806cAn2P7fxNUHwEeA2wM6eA8Q3Bmw0Q6s4y8hItId+B1wCzBAVXsBK3AbLcDbwNDAS4YFHu/F/Sdygqr28rZjVPWo0ltuGEXnZ8CfcEfmR+O6biTjnGD7Hw7s9B5vBy4P6KCXqlar6n+X3OpOinX8paUb0B3YAzSKyDnA2YHjDwKXiMh4EekBfMs/oKoO8B+49wT6A4jIEBH5dGjWG0bx8F2gB0VkHPBPWc75hoj0FpFhwNeAB7z9dwDXicgJACJyjIh8JgyjOyvW8ZcQVT0AfBW3g98HfBZYHjj+n8CPgeeAWmCtd+iQ9/caf783PX4aOD4U4w2juFyN2/4P4A5oHshyzqPAK8CrwBPAUgBVfQS4Gbjf08EG4JzSm9x5sZu7MUJExuM26u6q2hi1PYZhdE5sxB8xInKBF6/fG3dU85h1+oZhlBLr+KPnctxY/zdwY52z+T4NwzCKRuQdv4hMF5E/i0itiFwbtT1ho6rTvWidPqp6gaq+HbVNRvyodJ0YxSVSH7+IJIHXgU8BO4B1wEWquikyowwjZphOjGITdXbOU4BaVd0CICL34y7Vztmgu0l3raImJPOMTA6wb6+q2iKycClIJ6aRaCkHjUTd8Q+h+Wq9HcCkzJNEZB4wD6CKHkySs8KxzmjB0/rQm1HbUIG0qRPTSHwoB41E7uPPB1VdoqoTVXViV7pHbY5hxA7TiFEIUXf8b9F8mfZQb59hGE2YToyiEnXHvw4YIyIjvdzccwmsbDUMAzCdGEUmUh+/qjaKyBW4+eyTwJ2qujFKmwwjbphOjGIT9c1dVHUFbsZKwzByYDoxiknUrh7DMAyjFUTkTq+s64Ycx0VEfuwt7lvvV/hrDev4DcMw4s3dwPRWjp+DW7VvDG5I78/auqB1/IZhGDFGVVfjVh3LxWzgHnVZC/QSkUGtXTNyH7/Refj03/XQve856ed/XH/oSVVtbaRiGBVHUCd/XH9oI9AQOLxEVZcUeMlsC/yG4Fb4y4p1/EbR2P1eI//1nwPTz48Zsj2zpqphVDxBnRwzZHuDqk4M2wbr+I2iocAhUm2eZxiVTAl0UvACP+v4AySqqnh73sdJ5VjxPmBdA8lVfwzVpnLCQWlQp+0TjbIlUVND7bc+ilOVPavvkFUO1cteCtmq8qIEOlkOXOEl75sEvN9Wenfr+IHksX1ABHodzejPvM7g6veznvdMl0/wkQ3HAqAH63AaGrKeV6k4KtQ7ErUZRrERocuA/iCC068XT1x0C2O7Zs/+ObLbPCa86LoxnA8O4NTVhWlpWVCoTkTkN8BUoK+I7AD+L9AVQFXvwF3fMQO3Pnc9cElb16z4jj/Zuzcf3NuL/j0OkJADDKw6kPPck2dt4OA57nRg64Mfp/+//3dYZpYFCjRoMmozjCKT7N+PS1evYUK3d0iKMqpLdc5z181czJ4Zbqc2+74FjLx+TVhmlg2F6kRVL2rjuAJfKcSGiu/4SQj9exxgaI/9bZ7aq+uH9Or6IQC13UpsVxniIDSoNanOhogwods7jO/Wo81z+yZr6Ov1aU736Io8xZk46KSi4/gTPXpA72NISOENNNXdcxEZaRyEOu2W3ozyJ9GzJ07/3iTbo5Fqh6TnIjKaCOokKiq643/7SyfR467W3Tu5OHnWBt6/txfJ3r1LYFl54qjQoF3Tm1H+1C48kdsfW9qqeycX62Yu5rLn15Lsa1G9QYI6iYqKnpenupPzRm5b9Or6If2q6zhcZJvKGUVocKzD70w4VZrzRm5b9E3WMK7bLkjYiD9IHHRS0R2/UVxSJKhzrPqTYbRGHHRiHb9RNFSFQ+biMYxWiYNOrOM3ioYTgymsYcSdOOjEOn6jaDgI9ebqMYxWiYNOrOM3ioZq9CMZw4g7cdBJRYdzDljXwDO/+wT7jxQeqvb4xr/hzd+OQj/8sASWlScOwiGnS3ozyp8hqxxGLpvH3lThqRcu3jaVOffORw9a2oYgQZ1ERUWrM7nqj3zktT4cPKd7ekVuvhz1v1X0//f/xlKSNeHmILGFW52J6mUvMWHtAPbMkPSK3HxZu2YcoxauMY1kEAedVHTHbxQX9UYyhmHkJg46CeXdRWQYcA8wADdH0RJVvV1E+gAPACOAbcCFqrovDJt8tK6erQ9+nNpu7oKuk2dtyDn6f3zj33DU/1YBMOAVc/FkogiHreNvN3HViXPgILPvW4DTXUlVO6ybuZi+yeyLui7eNpW1a8YBMHi1jfWzEQedhPXujcACVf2jiPQEXhGRp4AvAs+o6iIRuRa4FrgmJJsAcBoa0lk2k71788aZfelXnd0necxLVfT/iWXkzIWjwocpu7nbAWKpE6euLp1lMzmgP89+crC7IjcL654bz6iFlpGzNeKgk1A6fq8owNve4wMishm3JuRs3DzTAL8EVhFyxx8ktW8fx1xEzjQMAz/8o/krW0FVOJyyEX97KQedpHbv4e4pp+RMw3DcwfWmkTaIg05Cf3cRGQH8LfAiMCBQKeYd3CluttfMA+YBVNF2atiOkNoXqqepU+EAhx3Lx18MCtVJaBpRJbVnT+muXwHEQSehdvwichTwO+DrqvqBBNK1qqqKZM/96lWdXwJwtPSxJN8xRVVosBF/h2mPTkwj5UMcdBJaHL+IdMVtzPeq6sPe7l0iMsg7PgjYHZY9RvFxEA6nkumtLURkuoj8WURqPd915vEvisgeEXnV275UEsNjhOmk8xPUSVSE0vGLO2RZCmxW1R8FDi0HvuA9/gLwaBj2GKVBFY44yfTWGiKSBH4CnANMAC4SkQlZTn1AVU/ytl8U3+r4YDqpDII6iYqw5htTgM8Dr4nIq96+64FFwIMichnwJnBhSPYYJUARDjfm3aROAWpVdQuAiNyPexNzU4nMKwdMJxVAgTopCWFF9fwByFWN4awwbDBKjyo0Os0mkX1F5OXA8yWeLxrcaJXtgWM7gElZLvv3InIG8Dpwlapuz3JOp8B0Uhlk0Uno2J04o4gIjalmDXqvqk7swAUfA36jqodE5HLcUMZpHbHQMKKnhU5Cp6KTtBnFRRWONCbTWxu8BQwLPB/q7QtcT99V1UPe018AJxfNWMOIiKBOosI6fqNoqAopJ5He2mAdMEZERopIN2Au7k3MNH4ki8csYHNRDTaMCAjqJCrM1WMUFSeVX2FtVW0UkSuAJ4EkcKeqbhSRG4GXVXU58FURmYWbyuA93NQFhlH25KuTUmEdv1E0VMFpzH8Uo6orgBUZ+74deHwdcF3RDDSMGFCoTkqBdfxG8VDBifimlWHEnhjoxDp+o7g40U5hDaMsiFgn1vEbxUNBG63jN4xWUdCIffw2LzeKh4KkJL0ZhpEFBWkUJM9BUilyWtmI3ygiAtbhG0Yb5K+TQE6rT+Gubl8nIstVNTO1yQOqekW+FljHXy6I0GVAfwik6HU+OIBTl71aWCR4I37DiIRy0AgUqpOS5LSyjr9MSPbvx6Wr1zCh2zvpfbPvW5AuiRcXrOM3oqJcNALNdNJaPisoUU4r6/hjSv2cSew8IzByqXI4s/qRZkWup0zbwOrbJqefJxuEUTetj26EoyCpaN7aqDzKUiOQqZOO5rOCduS0so4/bnjT1R1nK1tn/TzjYE2zZ3cNfx6GP59+vvlwPfN/fSmJrdsjadhiHb8RBmWsEShYJ3nltAo8/QXwg7YualE9McOfrq47d3HBrx3btYrbn1jKGws/WgLL8sOPVsg3YsEwCqXcNQIFRfWUJKeVdfwxon7OJDZ/ZwRnVr/dbLqaL0lJMLZrDVOmbWDrolNJ1BR+jQ7hjWT8zTCKTdlrBJrppM1TVRsBP6fVZuBBP6eVl8cK3JxWG0Xkf4GvkkdOK+v444AIXQYO8KauS9rVoIPcNfx5Hp17K4wcFnrDFqdpM4yi0Yk0AoVpRFVXqOpYVR2lqt/z9n3bS2SIql6nqieo6sdU9e9U9U9tXdM6/hjQkalrLqKY0opCorFpM4xi0Vk0As11EhV2czcGiAgTur3T4VFMEH9K63TXol2zTezmrlEiOo1GIBY6sY7fKCpRN2jDKAei1ol1/EbxUHPxGEabxEAnofr4RSQpIv8jIo97z0eKyIte8qEHvHCliqJ+ziQ23fAR+iVLM908fcrG0KIXRO3mbkcxjbSkM2kEmuskKsK+ufs1mseY3gwsVtXRwD7gspDtiZydZ0hRohRycdfw53lk7o+Qo0Jq1BbO2VFMIxl0No1A9BoJreMXkaHAubgryxARwV1W/JB3yi+B88OyxygBFtXTIUwjFUIMonrCHPHfBnwT8Cc4xwL7vQUK4CYfGpLthSIyT0ReFpGXj3Co5IYa7cQWcHWU2zCNdH4KWMBVKkLp+EXkPGC3qr7Snter6hJVnaiqE7vSvcjWRUuyQdh8uJ6UlsbhtztVx2uHBoNT+pA1wTr+9mIayU1n0gg010lUhDXinwLMEpFtwP2409fbgV4i4kcWtUg+VAmMumk9C869hDcaPyzJ9Scvn889p00ktXdvSa7fDIVEo6Y3oyBMIznoVBqBZjqJilA6fm9J8VBVHYGbZOhZVf0c8BzwD95pXwAeDcOeOOHU1SHv7ielpUlqlmgQUnv2gIbQyBQSqabNyB/TSG46lUagmU6iIuqUDdcA80WkFtefuTRie4wO4E5hNb0ZRcE00skI6iQqQl/ApaqrgFXe4y24pcWMzoBC4kjURpQ/ppFOTgx0EvWI3wCcAweZfd8CLvnr6UW75t5UHcc9cjmDV4e4SkQhkdL0ZhjFotNoBJrpJCqs448BTl0dI69fwwurTixK9MLuVB1P1w9l3A1bqH70pSJZ2Taiaq4eoyR0Fo1Ac51EhXX8MaJY0QuhRyn4KCSOaHozjGJT9hqBZjqJCuv4Y4RTV4du28F5917drimtP3Ud+nsNN0ohgI34jVLSGTQCFXhz12id9JR20amsn/tUen9XHMZ2rSIpTf+rd6fqeCeVTD/fdMiduqb27AnVZh/xfJeGUUrKWSMQD51Yxx9TRt20nmsXz2ra0bc3t624k7FdmxJJTV4+n3E3bm06x9Fopq4+qsgRS8tphENZagRioRPr+GOKU1cHdXXp54mDdZx379XNqgUNXe2Q2rU7CvOy461INIwwKEuNQCx0Yh1/meBPb+OOpGzEb0RDuWgEoteJdfxG0RBV5IjlajCM1oiDTqzjN4qHgjTaiN8wWiUGOrGO3ygeqpCyEb9htEoMdGIdv1E8FOSwdfyG0Sox0Il1/EYRiX4kYxjxJ3qdWMdvFA9VaLRiu4bRKjHQiXX8RvFQhSPW8RtGq8RAJ9bxG8VDiXwkYxixJwY6sY7fKB6qqI34DaN1YqATy85pFA9V9PDh9NYWIjJdRP4sIrUicm2W491F5AHv+IsiMqIUZhtGqAR0kg+l0Il1/Ebx8G9a+VsriEgS+AlwDjABuEhEJmScdhmwT1VHA4uBm0tgtWGES1AnbVAqnVjHbxQNVcU50pje2uAUoFZVt6jqYeB+YHbGObOBX3qPHwLOEhEpqtGGETJBneRBSXRiPn6jaBxg35NPOw/2DeyqEpGXA8+XqOoS7/EQYHvg2A5gUsYl0+eoaqOIvA8cC0ScV9cw2k+GTlrTCJRIJ6F1/CLSC/gFcCLufe1LgT8DDwAjgG3Ahaq6LyybjOKiqtOjtqHcMZ10fuKgkzBdPbcDK1V1HPAxYDNwLfCMqo4BnvGeG5XBW8CwwPOh3r6s54hIF+AY4N1QrIsO04kRpCQ6CaXjF5FjgDOApQCqelhV99PcN/VL4Pww7DFiwTpgjIiMFJFuwFxgecY5y4EveI//AXhWNaIiqSFgOjGyUBKdhDXiHwnsAe4Skf8RkV+ISA0wQFXf9s55BxiQ7cUiMk9EXhaRl49wKCSTjVKiqo3AFcCTuKPaB1V1o4jcKCJ+Pb2lwLEiUgvMp/OPdNutE9NI56RUOpEwBlAiMhFYC0xR1RdF5HbgA+BKVe0VOG+fqvZu7VpHSx+dJGeV1F4jN0/rQ6+o6sSo7eiMFEsnppFoKQeNhDXi3wHsUNUXvecPAR8HdonIIADvb8yKYxpGqJhOjFAIpeNX1XeA7SJyvLfrLGATzX1TXwAeDcMew4gjphMjLMKM478SuNe7QbEFuAT3H8+DInIZ8CZwYYj2GEYcMZ0YJScUH38xEZE9QB3xW8TTl/jZBMW36yOq2q+I1zOKTIw1AvHUScVppOw6fgAReTluN0/iaBPE1y6jtMT1d4+jXXG0qdRYrh7DMIwKwzp+wzCMCqNcO/4lbZ8SOnG0CeJrl1Fa4vq7x9GuONpUUsrSx28YhmG0n3Id8RuGYRjtxDp+wzCMCqOsOv62ak+GZMMwEXlORDaJyEYR+Zq3/wYReUtEXvW2GRHYtk1EXvPe/2VvXx8ReUpE/uL9bTUXklH+mE7atK3idVI2Pn6v9uTrwKdwc5qsAy5S1U0h2zEIGKSqfxSRnsAruGlyLwQOquotYdqTYds2YKKq7g3s+wHwnqou8jqB3qp6TVQ2GqXFdJKXbduocJ2U04g/n9qTJUdV31bVP3qPD+CmSh0Sth0FYLncKwvTSfuoKJ2UU8efrfZkpA1JREYAfwv42RSvEJH1InJnRFNFBX4vIq+IyDxvX141D4xOg+mkbSpeJ+XU8ccKETkK+B3wdVX9APgZMAo4CXgbuDUCs05T1Y8D5wBfEZEzgge9qjzl4dszOgWmk3hSTh1/PrUnQ0FEuuI25ntV9WEAVd2lqilVdYD/wJ1yh4qqvuX93Q084tlgudwrC9NJG5hOyqvjz6f2ZMkREcEtdbZZVX8U2D8ocNoFwIaQ7arxbqLhles727PBcrlXFqaT1u0ynRBuPv4OoaqNIuLXnkwCd6rqxghMmQJ8HnhNRF719l0PXCQiJ+FOEbcBl4ds1wDgEVdvdAHuU9WVIrIOy+VeMZhO2sR0QhmFcxqGYRjFoZxcPYZhGEYRsI7fMAyjwrCO3zAMo8Kwjt8wDKPCsI7fMAyjwrCO3zAMo8Kwjt8wDKPCaLPj93JXf7KYbyoiU0VkRzGvGQYiMkJEVEQKWvgmIl8UkT+Uyq6wCbYJEbleRH7RzutsFJGpxbQtakTkeC/P+wER+WrU9oSFacOlXLRRNit3jXiiqv+az3kicjewQ1X/JfDaE0plV4R8E3hOVU+K2hAjWuKsjbJ39XiFJ4x2UugIzWiTjwDtSpFQ7N/CtNExOrU2VLXVDTefxnXAJmAfcBdQ5R3rDTwO7PGOPQ4MDby2j3f+Tu/4Mm//VNz/cP55X/WuP9R7/k3clK07gS/h5vUY7R27Gze16wqgDvgkMB5YBezHFd2swLVXAV8KPP8i8IfAcwW+DPzFe/1PaEplkQRuAfYCW4CveOd3yfFdDQMe9r6Pd4F/z/Get+PmTP8AtzLR6YFjpwAve8d2AT/y9lcBv/auux83GdeAdvxmU3FztF+Dm3f8V7gDgGuBN7zrPwj0CVzv87j5S94FFnrX/6R37Abg14FzTwP+27Nxu/fZ5wFHgMPAQeCxgJ3+dboDt3m/+U7vcfcMmxfgZk18G7ikrbYb9gY8C6SABu9zjgWOAe7x2sSbwL8AiUC7eAFY7H233/W+h1uAv3q//x1AdeA9TBumjdvooDby7fg3eD9cH9yG+l3v2LHA3wM9gJ7Ab/E6d+/4E8ADuP8gugJnBo31Hn8b+CPQz3s+3fvST/Cu+2taNu73cZNAJbz3rcVNANUNmAYcAI4voHE/DvQChuM2zOnesS8Dfwp89ufI0bhxhfC/uCKuwW2Mp+V4z4u9766L94O9Q1PjWwN83nt8FDDZe3w58Jj3nSSBk4Gj2/GbTQUagZtxG1Q18DVgLW4K3+7Az4HfeOdPwG2QZ3jHfuS9vkXjxh3tHgAu8n7vY4GTAr/bd7PY6V/nRs+G/kA/XIHclGHzjd51ZwD1uOXxIu/wMz5TZnu7BzfTY09gBG5ZxMsC7aIRuNJrC9Ve+1nu/W49vd/8+6YN0wZF1Ea+Hf+XA89nAG/kOPckYJ/3eBDgZDPAM/Yt74v6A3BM4NideA3dez6alo37nsDx073GkQjs+w1wQwGN+7TA8weBa73Hz2Z89rPJ3bhPxRVGtmPN3jPL8X3Ax7zHq4HvAH0zzrnU+8E/2pHfzPvuD+OJydu3GTgr8HwQ7iikC+4/5vsDx2q812dr3NcBj+Sw6W5ab9xvADMCxz4NbAvY/GHwu8Ud3Uxu67sIewu2N9xO6DAwIXD8cmBVoF38NXBMcEfqozLa1VbThmmjmNrI18cfLOX2JjAYQER6iMjPReRNEfnA+2F6eb7FYbjFi/fluGYv3GnO91X1/cD+wRnvF3ycbd9gYLu6hR2CNhZSbu6dwON63NFENlvebOUaw4A3VbWxrTcTkatFZLOIvC8i+3HdAX29w5fhugj+JCLrROQ8b/+vcFPt3i8iO0XkB16hi1xk/c089qhqQ+D5R3BT1e737NmM67IYQMZ3oKp1uNPabAzDbaTtYTDNv99Mm9/N+G6Dv1Nc6Ys7Csv8XMG2Gfyd+uGOWl8J/BYrvf1g2jBtZLe5YG3k2/EHK/oMx/UzgTsVOx6YpKpH4055wB25bAf6iEivHNfcB5wH3CUiUwL738adVmV7bx8NPN4JDBOR4GcZTlPVoTpcMfkMzGFPNt6m5WfPxXZgeFs3hETkdFw/7YW4s6FeuNNzAVDVv6jqRbjTupuBh0SkRlWPqOp3VHUC8P/hfnf/p5W3yvWbQcuyctuBc1S1V2CrUrdSUbPvQER64E5Ts7Edt6xeNjLfM5OduCLLZXM5shd3dJj5uYIVsTTj/A+BEwK/wzGq6ovYtGHayGZzweTb8X9FRIaKSB/cGxgPePt74jbU/d6x/+u/QN3Cxf8J/FREeotI1yy1LVcBnwMeFhG/BNuDwCUiMt77Ir/Vhm0v4v6H+6b3HlOBmcD93vFXgTne7GQ07qghXx4Evup99t64N3ly8RJuQ1jkVfmpyviH5tMT1ye3B+giIt8GjvYPisjFItLPG6Xt93Y7IvJ3IvI33mzqA9wOxSE3uX6zbNwBfE9EPuLZ0E9EZnvHHgLOE5HTxK3odCO52829wCdF5EIR6SIix3pFN8C9GXdcKzb8BvgX77374k6jf93K+bFHVVO4beh7ItLT+37nk+NzaVM5wsUi0h9ARIaIyKe9U0wbpo2iaCPfjv8+4Pe4d+/fwI0+APfucjXuSGUt7rQ0yOdxf4Q/4fqdvp55YVV9CtdH95iIfFxV/xP4Me7NolrvugCHshmmqodxG/M5nh0/Bf6Pqv7JO2Uxrt9tF/BL3B8gX/4Ddwr5v7g3oB/OdaIn8pm4fte/4t5p//+znPok7vf0Ou6UrYHmU8/pwEYROYgb4TBXVT/EHY09hNuwNwP/hTvFzUWu3ywbt+PeUPy9iBzA/c4neZ9rI27Exn244t3nfbZs38FfcX2mC4D3cDuWj3mHlwITvCnzsiwv/y5uxMZ64DXc77s1m8uFK3FH1ltw72fdh+urz8U1eO3ec58+jTurxrRh2qBI2oh9BS4RGY97F757Pj5Cw109iHvT7umobTFKh2mjcEwbLrFcwCUiF4hId28KeTNubKs1bKPiMW0YxSDyjl9EpovIn0WkVkR8P+HluK6hN3DvoP9TZAYaRgzwdYLr230f04bRASJ19Xg3Y14HPoXrG1sHXKSqmyIzyjBihunEKDZRj/hPAWpVdYt3I+p+YHYbrzGMSsN0YhSVqJMQDaH5XfsdeHfMc9G3T1JHDGttbYZRSl5Zf2ivqvZr+0yjiBSkk27SXauoKblRRnYOsC/2Gom6488LEZmHu8qX4UO68NKT2dattM7eVB2nLJ/PmSdv4odDVnLK8vkkGqTg66SqHdbNXEzfZHkKK/g93DX8+YJfnxxU29oKTSMighqpogeT5KyCr5GoqaH2Wx9l0NoUNU9tpPZbH8WpKtwVnGwQRt20HqeuruDXxoHg91C97KWCX/+0PhR7jUTd8b9F81V0Q2m+qhEAVV0CLAGY+LHcLXF3qo53U8LYrlUkpbkXa09KOP5n77P6Syfw9DkbGHfDFlJ79hRkbKJnTxg5hPfOhb5lmvDW/x5WXTEe2tHxG5HQpk6CGjla+uTUSKJnT6RHNandeyDj/p4cVcP35tzH9V0vYsyWYTxx0S2M7VrYAGd3qo5n64dyz60ToUw7fv97+Fbqs4xYFrU1pSFqH/86YIyIjPRWvs3FXSzRLiYvn8+Ccy/hjcYPWxwb27WK259YiqTgntMmktq7t+Dr1y48kdsfW8qoLtXtNTFy/O9h3bmLozbFyJ+i6aR24Ylc9vxakn37tjiW2r2Hu6ecgiZpdzufvHx+u/UVF/zv4bh/XR+1KSUj0hG/qjaKyBW4K/aSwJ3eariC8N0XQ3+v6LYdnHfv1UyZtiGrS2fwaqfgkb6PU6UFj4DiRlISZf8ZKo1i6MR3X5wx9TXOrH6bq28aybCVI7O6dKZO3tDuNpJokHbrKzaolv9naIOoXT2o6grcwhHtZlcqwbgbt5LatRsHGHn9Gl5YdCpPX9A+l04ukg3C+sNu4r7ByVRefv6UOvzpyCEGJJ2yvS9gRE9HdZLoeRTL597K+G49gBq2zlrCSGce49rp0slFqtqhyyA315vzwYH8/PwidBnQH+fAwbK9L1BuRO3qKRmjblpf9CnnqJvWc+2kWVw7aRafeOKqvF7z+pEGvnHuF/M+v5IQkTtFZLeIbMhxXETkx97ivvUi8vGwbezMrJu5uOiuy3UzF3PLmoe5Zc3DvLHwo3m9Jtm/H5euXpP3+ZVGKXQS+Yi/vfjunUSDkGwQjjvY3B/n1NUV/eZS8JrDVo5gdN2XATh9ysasETIXb5vKuufGc9y29QxbeQwjnXllHRFUAu4G/h23SlU2zgHGeNsk3LKCrYb7Gk347h2nSnGqHPolm9/M7ZusKXqQQvCaU6ZtYPVtkwEYssrJGiFTP2cSO85Wzqx+hCnTNvDColPLOiKoRNxNkXVSth2/H52SeHc/6jik6utDff/qZS8xapn7+IVFp7J57pMtoonWrhnHqIVrcLzzx6/pz7OfHMy06p3pzr+1SKTOjqquFpERrZwyG7eilOJmq+wlIoO8lN9GG/jRKeO67aIrDr0T4QYl3DX8+XTk2Mhu8xi/pn+LaKKdZwhbZ/0cqOGu4c+zee6TzP/1pSS2bk93/q1FIlUCpdBJ2fY0fnTKLWsezhmlEBajblqfM5ooiB8tEHT7tBaJVG58+u9qdOLHqtKbiGwQkZcD27wCL5lt4VIh1aMqGr+9XTtpVuRtbN3MxXnp1Nd10O3TWiRSORLUSRE0Au3QSdmO+P3olLQ7pT660Cunro6EF03kdG8akQxenVELwosWGLZyZNpNNHS10ywSqT2LquLCnvcaeWFlU0W4HoO3NajqxAhNqmy89ua7U/pEOMzrm6xJRxMlGpoKUU2d3Nxt7es66CaaOrl5JFJ7FlXFiaBOotJI2Xb8PkF3SpQ4dXWMvH5NXufWPLOZ418M+PmP7smYf9vKqj7lvahKUQ4VN0NwXgv8jNYJulOipG/SjSbKh5uHrOSd859KP9+ZSvLo9B/zmXevKvtFVXHQSdl3/OVI7cITWT731hb73Ztv5Xvj1wHqNVXMSy4HrhCR+3FvVr1v/v3KYPLy+Yy7cWuL/ccdXB/5IK+jxEEn1vGHSHARjRtP3blwVGko4OabiPwGmAr0FZEduDWbuwKo6h24ceszcMsM1gOXFNlkI2YEF2Omdu2O2pySEAedlH3H364FI0Um36iD5otoOh+K0KD5O5JV9aI2jituTVOjA7Rn4WGxyTd6LbgYs7MSB52UbVSPT3sWjBSbzhZ10F5SCHXaNb0Z8aA9Cw+LTWeKXusoQZ1ERdl3/H2TNfRLKuc+cRUDXyzcb5aoqWHLolP58PxT2m3DoLUprv/t59A21hI4Bw4y+74FXPLX09v9XnFGERqcLunNiAdOXR16sI6/XDWKaScVXrRrb6qO4x65vEPt9syTN7H5ymPajCwakHTY/J0RHdJj3AnqJCrKvuOHpulh9aOFh3n57pedp7f/q6h5ZjOjfvwGTisdf6JnTxJ+9M6r49v9XnHGUaFBu6Y3Iz747Xzp8D8U/FpfX6vXntDu9795yEpWTL+t1UVku1N16eidXZ8o07zneRDUSVTYsKwI1C48kUVz7mXplEk5E8IFI3nKPXonFw5CvXaP2gwjhkxePp9xP93PbSvuzJkQLhjJ0xmid3IRB51UdMdfP2cSO88QZv5ufsvFVgXgVCnjuu2CRO6KXk6Vdtqbuj7uFNZG+p2Ji7dNZe2aceh1LRdbFUKiQZB395PS3BpJNEinvqnrEwedlH3HvztVx2uHhoKTX3hUMAJn1ykJvnfO/W4Wzw6kbk42CK8dGpy3DZ0VBzEXTwxJ9OyJ0783ScmvfQYjcF54dSzjfrmf2x9b2qHUzalqpyAbOjNx0EnZ+/gLrfgTjMApVurmUqSALkccFeqc7unNiAeFVo4LRuAUK3VzKVJAlytBnURF2Y/4C634M2htiutTn+O4+vVFS93c2nX8RVtTJ7/W4feJOw7CIXP1xI5CK8edefImVl05nj6J4qVubu066UVbHXC3lhNx0EnZd/yFUr3sJUYsI7QbR35q3GnVO+mMN3SDuAtTKq5JdTqa0imH0179FOts3d5pb+gGiYNOyt7VE3eypWLurDiaoD7VPb0ZRj5kS8XcmQnqJCo6dcefz+KsYizgahUvNW6ioVN/1YA/he2S3oz4k8/irGIs4GoNPxVzMKV5Zyaok6go+94oVe2QHNAfpHmYWKJnTxjlFpLefjbpcxI9ezY733fF7Dw9kfU6xSLZIGw+XE9KO+9kVhXr+GNIrra3O1XH0/VDGXfDFl5YdWL6nN2pumbn+66Y1WtPKGkbzqXlzkZQJ1ERSscvIsNE5DkR2SQiG0Xka97+PiLylIj8xfvbu9Br56rsE4xkCJ6TmVfHd8VokpLm28m3Slc545Dgw1S39GYURql0kqvtBSPigudk5tXxXTGSoqRtON8qXeVOUCdREdaIvxFYoKoTgMnAV0RkAnAt8IyqjgGe8Z4XRN9kTdbFU34kQ1ISzc5pkVfHc8UMWeXklW+nvTh1delKW502V4+N+DtKSXTi1NVlXTyVjohTbXZOZl4d3xVz+pSNeeXbaS9+la7NN43s3Ll6YjDiD+WdvaIAb3uPD4jIZtyakLNx80wD/BJYBVxT6PUTKNqvD10SCbfw+u6W4Z3+OTVPbWTEsroW0QNhRPv4VbpeWHQq6+c+1eJ4VClzi4WDcNjpvDlWSk0pdaKqvHZ4EEdwC6+P7VqV85wfDllJ3yxRPWFE+/hVukY685jw4sAWx6NKvV5M4qCT0P/leNXi/xZ4ERgQqBTzDjCgPdf0p6IpFTYdHsjSKZNynnPevVfnXSKxVIy6aT3XLp7VYv/m74zIuzRdHFEVGlIWx18Miq0T36VJQqBvb25bcWfOc665aWTk7XDdzMXsmdHS1z/7vgWR67ejxEEnoXb8InIU8Dvg66r6gQRu4qiqimRfz+1Vnp8HMHxIS5P9qShAv6RblHnaSc3ziuxzPmT64/MZmkfq5vo5k9h1SoJRN60vyegi14KvRMNxRX+vMHEQDqdsxN9R2qOToEaqyJITynNpAiQO1nHevVczOEMLiR49+MuC0Uw7qe3Fhhdvm8oLr45l3czFJZml5lrw1Rkif+Kgk9A6fhHpituY71XVh73du0RkkKq+LSKDgKwZmlR1CbAEYOLHqlr95XMVdPYjExLv7oc2KnbtOiXBjRfczz23TizKyt588aMv2qpSFFdUiXwKW+60VydBjRwtfVrViO9ybPHeXoTbuG67WH/Y3ZfL/fjCq2MZd8cHvHcuRVnZmy9+5E9b1e7iTBx0EkrHL+6QZSmwWVV/FDi0HPgCsMj7+2ipbAi6g3xyTRtH3bSee24NP/fOqJvWs+BXl7SaujbOKMKhlN3UbS9R66SZO8gjl/tx3czFvHcuoefeWTdzMf/1yUGtpkCPO3HQSVjvPgX4PPCaiLzq7bsetyE/KCKXAW8CF5bKgKA7yCfXtLFYOXwKxamro0sbqWvjjKpwxFw9HSFanQTcQT653I/FyuFTKLmi+MqJOOgkrKiePwC5fqmzCrlWvTrsTdWVdfRLZ0UVGlPl56KKC8XSiXTtQqJ7TdlHv3RW4qCTslPpW5t7VUTem3JEvZtW/mZEw9Bx+ysm7005EtRJVJSdQ1YbGysi70054o5krMOPmmpJ8GEniH7prMRBJ2XX8ReTYIRA4qij0pW5gtECwYpdeUcRiNAlI+dIZ1h40haKkDJXT6ciGGn2rvNhujJXMOosWLEr32i0lDr86cghUgHPVrkvYMyXOOikolXaWg4fn1z7WyPZvx+Xrl7DLWseTm8VMfVWSKUS6c0of1rL4eOTa39rvH6kgW+c+0WunTQrvVWMCzegk6goO3UeGlbToaLPF2+byshl89I3iP3cIGdMfS1rnpB8CqlnIiJM6PYO47v1SG/5LDypnzOJTTd8hH7J8pymq4KTSqQ3Ixpee68fgztQzap+ziS2LjqVRE1NsxxTQ3+vWfNN5VNIPRMHgb37SO3and7yceFevG0qc+6djx4s39lzUCdRUXbqHNVrFzcPebLg16XUYePhD1n33Hgm3Pgme1JuI/UXfC0d/of04+1nQ5dBA+kyaCBOVWECKrSwdZCdZwhbZy0p4+mu4KSaNiMaqnYdoubpjYW/UIQuAwew42zlkbk/Qo5y26G/4Kv60Zea8k2tOpH1hxtYf7iBZENhv/XuVB2vHRoMTuEaWbtmHCMWrilzt2n0Gik7H78f1VNoLhF3ankJx21bDz2PavXcYJ6QfkllVwH/mWsXnsgTF91SmUWlFdRG+pEzdNx+3vjsRwvOaeO7KM+sfrjNNh/MN3XcwbY1FWTy8vmMu2FL6AskY0MMdFJ2HX/DoO5MO2lTwa/rl1Q2X3kMiYbeOFVOq+6UlotT6tj8nREMWzmC6mUvtfo+uQpbnz5lIy8sOrVk+X9igQI20o+c2n0D+EQeOaky0YN1XPfQ53C6K8kGcTv0HGRb5Dj7vgVMmbbBzeLZCul00BkMWeUwstu8kuX/iQ0x0EnZdfxNrp7CGkbLHD75v95/7ei6LzNqWY6TvEieXK6hu4Y/z8a5v+cbi2e3uirYj3YYkHSaNf5iR0HsTtXxjhdSVsxoCrWOP3J8V0+hXv7MHD6FvN5/7erbJnupm1vit+FcrqHqZS8xYe0A9syQ1lcF+1o7cLD5IKrI0XSJnj1J+O6uIkflRa2TspuXx3UBlz9NXnfu4g5dx492yPyMxY6CmLx8fvGjKVSQxqatLURkuoj8WURqRaRFcRER+aKI7BGRV73tS8UxtHMT1wVcfhs+7l9zzyTywdda5mcsdjRd7cITSxOVp/lrBEqjk7Lr+OO6gMuP5OnoyNmPdsj8jP7+A1NGUnvVKPRgHYmGRLsLYZ958qb0dYathNG/+TKjf/PljlUH86ew/tYKIpIEfgKcA0wALvKqTWXygKqe5G2/aL9xlUO1JGKZvthvwx0dOftay/yM/v7v7ZzBjBVfp19ScboriZoatiw6teCqXoPWptLXmTJtA7W3Tab2tskdrw4W1EkblEon8etBY0xHi0H7VcASNU3/HDKLv/vnZLqM/P2ZERe7UgnG3bi1WbHsFnZ7EU17U02Cu3nISm684H6kRw+ql73EqKtf5PgfbmXVq+Pb9dl8xGna2uAUoFZVt6jqYeB+3EpTRhmTq7B7vvhVwIJtNbP4u39OpsvI358ZuZfoeRTL597K9rPJrV8voimozZpnNruppx3XVfv6Z37K8vMXs+sTHV91m6dGoEQ6sY6/ADpaDNpPDR2cNmYuEPPPyXQZ5drv01ox92zuo2ChbSiSq0rJdPX0FZGXA9u8wNlDgO2B5zu8fZn8vYisF5GHRGRY+40zwqC1dpgPfmrozLYavKZ/TqbLKNd+n9b0m819VLvwRG5/bGk6Qq9YrqqgTmhdI1AinZTdzd0o6WhKWD81dHCKmrlAzD/n4m1TWbtmXPbrZIm4OPjpE9l5hjDjoQXN9p8+ZSM/HLKSzVce0ywa6syTN/HCghMZdVO9W2i7SK4qaR5MsldVJ3bgco8Bv1HVQyJyOW692WkduJ5RYjqcWtxLDZ1oGJXe1WKBmHdO/ZxJ7Dwj+/tki9z7+vZzWbtmHLoQoOn6Q1Y51Dy1kese+hwDA9FQg9am+PTRV6WjjIrlqoJmOumoRqAdOrGOv0DS7pqDdekGUOiirVS1Qxe/CliV08y940c+rHtuPKP+ZW2LKIU0R/cEYOPhgdC3NzvOVh6ffhvfmDSb1K6mAk2rb5tM3wufb7HuIRhllEgk2r3oLIgoSP7RCm8BwZHJUG9fGlV9N/D0F8APOmSgEQq+y6Vfcmd6IOEu2hqa96KtZIOw/nBD+nEz944XvbPjbKV25s9aRLv57PSi1py+vdl0eCDrnhvP8f+2hVvWPsL4bk3lKUenvsyoZS2rkgWjjBwKs7814qAT6/gLJFvR9kIXbWUuEOudqOb2J5bSJwGvH5H0QjPxpp8Tur2T9Tozfzefu8//NPMefYLTqnYVtNAsSNEWnSkkGvM+ex0wRkRG4jbkucBngyf45Qa9p7OAzR0z0AiDbEXbC120lblALFVf77px6pt0cWb1w2m9sHdf1uvodXD5I4+zZPYMjnuzsIVmQYq66CwGOrGOv0CC7ppETQ213/ooZ0x9raBSiZkLxPam6pj++HwS3s0q/UeA3gBc89hxaXfNKcvnc+bJm9KPh6520Dff4pu//TxTpm1wXToZC81aWxQzIOmw+TsjmHZSYfa3Rp43rFDVRhG5AngSSAJ3qupGEbkReFlVlwNfFZFZQCPwHvDFohhplJaAu2Zvqs5tq79vWd2rNTIXiCVqavjLgtE4Xsntax5rqgym/whDVh1DzVMbqf3WRxm0NpV+PHXya5xWtYurvtqLYSt7UfPUxhYLzVpbXOkcOMjs+xYw9MVUUUs9Rq0T6/g7gF+celr1TgpdUBZcqOVH5mj9hzByCLc/tpReCXi2fij3nDaRFxacyLNz1jPup/tZ/Y8n8OwM97G89z4c3ZMx/7aV1T1OYM/5T1I78w5GO19m/Bo33XTNM5sZVzuE//rkIM6sfrtZ55+rMH270RY+/tZPV10BrMjY9+3A4+uA64plnhE+e1LC8T97H7ZuL3hBWXChlh+Zc2xS07rQhoa0Xj7d7SrG1Q5j0Zx7uabrZxnjPR7TbTc7U0kenf5j5tR9nbFrahi5cC0vfH8ym+c+ydiuVdw8ZCWr5qzn7l9NJ7FtR7POP1dh+g4RA51YVE8HyBaBkC/ZIm2CUQTBqJtRN613XTrLnkBSpB8HF6pIinTkQ2a66csfeZyl508v+cI38aaw/mYY2SLZ8iVbpE1QF0G9rJu5ON3ONUn6cXDBoyZJ6yIz3bSvqTAWvgV1EhU24m8np0/ZyOrFkwCYdlLhaaL93EHTTtrUzOXSJwFjlv0Tclj488JRwKh01ME3f/t5Br+YQt57n+O77uZ7O2ekI380AX/+x97MeGgBp0/ZyNUDfw8JwalSxnTdA+/uJ9HQJz31PvPkTW3mVGkPhYxkjM7NkFUOx6f+GYDBHcgdNPDFVDOXi9bXs+X7k3G6aTqKzXeHXvXVXkw7aUO6zR+YMrIp8seBax77LLrQta3n2jdJqbhRQ56mgi7cQWtTbebmai9R68Q6/nZy1/Dnc+YkyQffzbI7VceelFA78w6SkmDjYdftU3vVKB6a68bUf6buKkYEog50QP/0QpVRC919Wxedynfn3MfdF0xndfIEzpmxHvq69wn8yB+nyklPvVddMT6r/e2pppSmwCms0bmpXvZS7txWeeC7WRI9eyJHuS4aVEkMHMDyubcyY8XXmXDTXwFYVTOevsObotc2H25a8Lh11s8BGLlsHuN++gFfengF16Q+y5haN9oHmiJ/kg2SduF+K/VZRmSxv11V+YLEQCehdvze8uOXgbdU9TzvTvX9wLHAK8DnvdVpFcPk5fMZ99P93LbizmY3WDOjGoL+0fRClfqm/aNuWs/d97gRPlcvvzgd7RN8fFrVrmYRRNnuS+SyJy8UxFw8HcI00pLahSeyaM69LJ0yqdkN1szouGB79l1MwXa+buZi/vCpASyZPQOd1xTtE3x83JvNI4iy3ZfIZU/exEAnYY/4v4YbanS09/xmYLGq3i8idwCXAT8L2aZIOfPkTay6cjx9Em51oXXPjXc7+ixpb9NoywgJp66OLt509fQpG1mdPIGrl1/M4NVOehrrL2ABd2ocdPUEoy/8Kkv5pNgNIkCi/YWfDBfTSAaD1qa4PvU5jqtfT/2cSew4W+mX1Czp05vwo++C9E3WpF1AQ1b14urUxeg8mDq5yTXkL4QE1x0UdPUEo/jOrH6bq28aybCVIwt2B8VBJ6F1/CIyFDgX+B4wX0QEd3WZH5P6S+AGKqRR+1E9Pxyykr7DnwdqWLtmHKMWrik8+iGDu4Y/z8aBv29azDVwAAAvvDqWCbdsA2B18gTWD3wq/ZpNh4a6ccp79uBAmyl2sxKDKWw5YxrJwIvqqXlqIyOW1eHgV6n7OYVG0WVSvewljl87IL2Ya6M3h9p1SoJHznddrHNSX+f4FwemX+P068UTF93i/UPJI1V7LmKgkzBH/LcB3wR6es+PBfarqj/pyZWDolPiVwTbfOUxxQ2pbIXg1Hjm79y0zGkcLcriFIvm6RC3YRpJ40f1XPfQ54ofUpmDoItVr4Nb1jzcZI9o0SrrRa2TUDp+ETkP2K2qr4jI1Ha8fh4wD6CKHm2cHX/8PDz6j80jgopVpSvoMgrixvC7Lh1JwJ+vcxfBZE5p24vEYCRTrhRTI8OHdKHj+SOjxc/Dc81jxzWLCCpWla6gyyiIU1dHAqj91kfBUWb+bj7Q0jXaEeKgk7Di+KcAs0RkG+6NqmnA7UAvEfH/+bTIQeGjqktUdaKqTuxK9zDsLQnBgu9jf7CF78/4DV8b+HQ65exdw59vlnI5LwLpZP18JmvXjGPUj9/Aqa9vkUfIj+pxqh3euPAO3rjwDnaensh6zUILzbtTWE1vRkEUTSP9ji3jbj9Q8P3x83/EmLv20/Olv6bTKVcve6lZyuV8CKYl9/Ni7TxDWDH9NnonqlsUf/ejerp8KIxasJZRC9ayeu0JWa9ZaKF5oJlOoiKUjl9Vr1PVoao6AjfXxLOq+jngOeAfvNO+ADwahj1REUzr6kfmXDtpVofS2AYXufjXDC5UyUwt21Z65+A125Oi2RZwtQ/TiEuw7flt9ZY1D3coHXpwsaR/zeCCx8wU5W2ldw5es70pmqPWSNQrd6/BvYlVi+vPXBqxPSXFX7T1+vf+hq3fn4zW13Ngykg2X3mMF3aWH8GKQsFFLokePfjLgtHgwPW//RxaX58u/u7H5PvRDsFp8ulTNrJ10akkamqonzOJzd8Z0SK9Qz6IQiLVtBlFoaI04rfnyQ8uYPSjX6ZPAr63c0a6PedLsDJdcLHkPudDpj8+H02Q1l26+Lsfk+9FzQXdrUNWOYxcNo+9qTou3jaV8+9dgGakd8iXoE6iIvQFXKq6CljlPd6CW2GmIuidqObx6ben8/N8Y/Fsdp2SYMX0W+mdcEfk2dI+Z9Jsgcmj9Yz58RacAweb9v/2s4woIDroruHPs3nuk8z/9aXegpcltCtqwpvCGh2jkjXi1De150TPo9gzQ3jh1bGM//EbpLyOP1va50yCCxV7D/uvtO78/X+64uiC2nn1spcYv6Y/z35yMOueG1+QvloQA51EPeKvKLLl58msWJRPbpPgVDSbq6c90898XEBtopA4ounNMAolW36ezMpZ+eTICrbnbK6e9rTzjuirGQGdRIWlbAiRbPl5hq0knTMEslfpakFgAVcCmioHZVnYlS/ZFrwUimAuHqNjZMvPM2XahnTuKfekllW6MmnenuvSuutQO++AvoLEQSfW8YdI8zTI7uORzjzGeTlDjrCbrjiM7VrVvErXBwdw6uvT1bjUcdJ5QrKljU307EnCiwwqODKnI6hF8xgdI9ie/YWELyw6lXPmuLmnuiQS6fYfrNI1OJmid6I6XY3L11FSElnTj+9O1fGOV6GrXZE5HSEGOrGOP2KC+UN4dz/07c1tK+5stthq9n0LGH37G+lqXJsOD2w1T0jtwhNZPvdWoGUOk5LiTWENo5gE81Ad33V3uv0HF1tt/s4IVky/rakal6ejXKP7ycvnM+7GrUDLXFglJwY6sY4/Yvoma5pVCPLdPn2TPdJ5SKZM28DqHiekqw4lG4Tj6pv7GYOFp6dOfi1dU/TibVN54dWxHV7wkhcxuGlldD6cujoSgUpz6ZTjgXxWw1aOYGbd/HT1OqfKaREp5y+cBBi62knXpq6fM4ldpyQ6vHAyb2KgE+v4Y4A/FR1d92XGrnEjFo6wKz1dvXnISlbNWM/dF0x3q24BHN2TxNE909cIpp8N8sKrYxl3xwe8dy45E1oVCwESjZalzSg+vgto9W2T+eb5T6L9+jRz+9Q8s5kxtUP40sMrGNPN7dB3ppLsTDWkrxFMYx5k1ykJbrzgfu65dWLuxIhFJA46sY4/ZviRAyQkPV2d/vh8xv1kX3qqm41cLp11Mxfz3rkULcdIq6gi1vEbJcaPzEmppN0+f1kwmh/+/a+aXKZZyOXSGXXTeu65dWJxCqnnQwx0Yh1/jAhW9QLS09UzT97E6nlumuVCc4a0lrq26ChIo7l6jNIRrOoFTW7PQWtTXN3VTbM8ZFWvgnJPtZoCvRTEQCfW8ceI7FW9atwFVgOfZP55l7KqJnvlrFigRD6SMTo32ap6Od7+sWv688XnX+RbddkrZ8WGGOjEOv4yIVtFobghqsgRC+Q3oiFbZbo4EgedWMdfJhRjgVXpUSRlHb8REUVaYFV6oteJdfxG8YjBFNYwYk8MdGIdv1E8VOGI5WM2jFaJgU6s4zeKhyo0WsdvGK0SA51Yx28UDwUazcdvGK0SA51Yx28UD1U4ciRqKwwj3sRAJ9bxG8VDFTVXj2G0Tgx0UnaFWKRrl3BTDRv549+08rc2EJHpIvJnEakVkWuzHO8uIg94x18UkRGlMLuzUa9O+KmGjfwJ6iQPSqGTsuv4h47b37EqUUbpUEUPH0lvrSEiSeAnwDnABOAiEZmQcdplwD5VHQ0sBm4ugdWdjrc29+p4lSijdAR00hal0knZdfzVXmEFI36oKtp4JL21wSlArapuUdXDwP3A7IxzZgO/9B4/BJwlIjaUbQNtbAwnvbDRLoI6yYOS6MR8/EbROMC+J59qfKBvYFeViLwceL5EVf1SSEOA7YFjO4BJNCd9jqo2isj7wLFASGkUDaP4ZOikNY1AiXQSWscvIr2AXwAn4gY0XQr8GXgAGAFsAy5U1X1h2WQUF1WdHrUN5Y7ppPMTB52E6eq5HVipquOAjwGbgWuBZ1R1DPCM99yoDN4ChgWeD/X2ZT1HRLoAxwDvhmJddJhOjCAl0UkoHb+IHAOcASwFUNXDqrqf5r6pXwLnh2GPEQvWAWNEZKSIdAPmAsszzlkOfMF7/A/As6raaRP+m06MLJREJ2GN+EcCe4C7ROR/ROQXIlIDDFDVt71z3gEGhGSPETGq2ghcATyJO6p9UFU3isiNIjLLO20pcKyI1ALz6fwjXdOJ0YxS6SQsH38X4OPAlar6oojcToZxqqoikvW/lIjMA+YBDB9i96M7C6q6AliRse/bgccNwGfCtitC2q2ToEaq6BGGrUZIlEInYY34dwA7VPVF7/lDuA18l4gMAvD+Zi0oq6pLVHWiqk7sd2xYdQQNI3TarZOgRrrSPTSDjfIklI5fVd8BtovI8d6us4BNNPdNfQF4NAx7DCOOmE6MsAjTb3IlcK93g2ILcAnuP54HReQy4E3gwhDtMYw4YjoxSk5oHb+qvgpMzHLorLBsMIy4YzoxwkDKLTpORPYAdcRv9WZf4mcTFN+uj6hqvyJezygyMdYIxFMnFaeRsuv4AUTkZVXNNiqKjDjaBPG1yygtcf3d42hXHG0qNWWXpM0wDMPoGNbxG4ZhVBjl2vEvafuU0ImjTRBfu4zSEtffPY52xdGmklKWPn7DMAyj/ZTriN8wDMNoJ9bxG4ZhVBhl1fG3VXQ4JBuGichzIrJJRDaKyNe8/TeIyFsi8qq3zYjAtm0i8pr3/i97+/qIyFMi8hfvb++w7TLCxXTSpm0Vr5Oy8fF7RYdfBz6Fm8xqHXCRqm4K2Y5BwCBV/aOI9ARewc2PfiFwUFVvCdOeDNu2ARNVdW9g3w+A91R1kdcJ9FbVa6Ky0SgtppO8bNtGheuknEb8+RQdLjmq+raq/tF7fAA3R/aQsO0oACviUVmYTtpHRemknDr+bEWHI21IIjIC+FvAT6N7hYisF5E7I5oqKvB7EXnFy88OVsSj0jCdtE3F66ScOv5YISJHAb8Dvq6qHwA/A0YBJwFvA7dGYNZpqvpx4BzgKyJyRvCgV46tPHx7RqfAdBJPyqnjz6focCiISFfcxnyvqj4MoKq7VDWlqg7wH7hT7lBR1be8v7uBRzwb8ip2Y3QaTCdtYDopr44/n6LDJUdEBLfG5WZV/VFg/6DAaRcAG0K2q8a7iYZXp/VszwYr4lFZmE5at8t0QriFWDqEqjaKiF90OAncqaobIzBlCvB54DURedXbdz1wkYichDtF3AZcHrJdA4BHXL3RBbhPVVeKyDqsiEfFYDppE9MJZRTOaRiGYRSHcnL1GIZhGEXAOn7DMIwKwzp+wzCMCsM6fsMwjArDOn7DMIwKwzp+wzCMCsM6fsMwjArj/wE/Ne9lckif+QAAAABJRU5ErkJggg==",
      "text/plain": [
       "<Figure size 432x288 with 8 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "image_name = sim_datalist['testing'][0]['image'].split(\".\")[0]\n",
    "prediction_nib = nib.load(os.path.join(work_dir, 'ensemble_output', image_name, image_name+\"_ensemble\"+\".nii.gz\"))\n",
    "pred = np.array(prediction_nib.dataobj)\n",
    "\n",
    "plt.subplot(2, 2, 1)\n",
    "plt.imshow(img[32])\n",
    "plt.title('image')\n",
    "cbar = plt.colorbar(shrink=0.8)\n",
    "plt.subplot(2, 2, 2)\n",
    "plt.imshow(lbl[32])\n",
    "plt.title('label')\n",
    "cbar = plt.colorbar(shrink=0.8)\n",
    "plt.subplot(2, 2, 3)\n",
    "plt.imshow(pred[32, :, :, 0])\n",
    "plt.title('background class prediction')\n",
    "cbar = plt.colorbar(shrink=0.8)\n",
    "plt.subplot(2, 2, 4)\n",
    "plt.imshow(pred[32, :, :, 1])\n",
    "plt.title('foreground class prediction')\n",
    "cbar = plt.colorbar(shrink=0.8)\n",
    "# set the spacing between subplots\n",
    "plt.subplots_adjust(left=0.1,\n",
    "                    bottom=0.1,\n",
    "                    right=0.9,\n",
    "                    top=0.9,\n",
    "                    wspace=0.4,\n",
    "                    hspace=0.4)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3. Conclusion\n",
    "\n",
    "Here we complete the entire pipeline. As you may notice, we can see the algorithms start to learn the datasets and the prediction of background and foreground voxels. Auto3Dseg and AutoRunner are highly configurable. To get better results, you can increase the training time, apply different ensemble method, or use hyper-parameter optimization through the AutoRunner or Auto3Dseg module APIs.\n",
    "\n",
    "As a \"Hello World\" notebook, we will end the experiment with simulated datasets here."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3.8.13 ('base')",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.13"
  },
  "orig_nbformat": 4,
  "vscode": {
   "interpreter": {
    "hash": "d4d1e4263499bec80672ea0156c357c1ee493ec2b1c70f0acce89fc37c4a6abe"
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}