imdb-checkpoint.ipynb

{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "%reload_ext autoreload\n",
    "%autoreload 2\n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "from fastai.text import *\n",
    "from fastai.vision import *\n",
    "from fastai.callbacks.hooks import *"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "path=Path('D:/Ritika Agarwal/fatsai-v3/dataset/imdb_sample/imdb_sample')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[WindowsPath('D:/Ritika Agarwal/fatsai-v3/dataset/imdb_sample/imdb_sample/classes.txt'),\n",
       " WindowsPath('D:/Ritika Agarwal/fatsai-v3/dataset/imdb_sample/imdb_sample/models'),\n",
       " WindowsPath('D:/Ritika Agarwal/fatsai-v3/dataset/imdb_sample/imdb_sample/texts.csv'),\n",
       " WindowsPath('D:/Ritika Agarwal/fatsai-v3/dataset/imdb_sample/imdb_sample/tmp'),\n",
       " WindowsPath('D:/Ritika Agarwal/fatsai-v3/dataset/imdb_sample/imdb_sample/train.csv'),\n",
       " WindowsPath('D:/Ritika Agarwal/fatsai-v3/dataset/imdb_sample/imdb_sample/valid.csv')]"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "path.ls()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "df=pd.read_csv(path/'texts.csv')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "      label                                               text  is_valid\n",
      "0  negative  Un-bleeping-believable! Meg Ryan doesn't even ...     False\n",
      "1  positive  This is a extremely well-made film. The acting...     False\n",
      "2  negative  Every once in a long while a movie will come a...     False\n",
      "3  positive  Name just says it all. I watched this movie wi...     False\n",
      "4  negative  This movie succeeds at being one of the most u...     False\n"
     ]
    }
   ],
   "source": [
    "print(df.head())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'This is a extremely well-made film. The acting, script and camera-work are all first-rate. The music is good, too, though it is mostly early in the film, when things are still relatively cheery. There are no really superstars in the cast, though several faces will be familiar. The entire cast does an excellent job with the script.<br /><br />But it is hard to watch, because there is no good end to a situation like the one presented. It is now fashionable to blame the British for setting Hindus and Muslims against each other, and then cruelly separating them into two countries. There is some merit in this view, but it\\'s also true that no one forced Hindus and Muslims in the region to mistreat each other as they did around the time of partition. It seems more likely that the British simply saw the tensions between the religions and were clever enough to exploit them to their own ends.<br /><br />The result is that there is much cruelty and inhumanity in the situation and this is very unpleasant to remember and to see on the screen. But it is never painted as a black-and-white case. There is baseness and nobility on both sides, and also the hope for change in the younger generation.<br /><br />There is redemption of a sort, in the end, when Puro has to make a hard choice between a man who has ruined her life, but also truly loved her, and her family which has disowned her, then later come looking for her. But by that point, she has no option that is without great pain for her.<br /><br />This film carries the message that both Muslims and Hindus have their grave faults, and also that both can be dignified and caring people. The reality of partition makes that realisation all the more wrenching, since there can never be real reconciliation across the India/Pakistan border. In that sense, it is similar to \"Mr & Mrs Iyer\".<br /><br />In the end, we were glad to have seen the film, even though the resolution was heartbreaking. If the UK and US could deal with their own histories of racism with this kind of frankness, they would certainly be better off.'"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df['text'][1]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [],
   "source": [
    "data_lm = TextDataBunch.from_csv(path, 'texts.csv')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [],
   "source": [
    "data_lm.save()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#Tokenization\n",
    "The first step of processing we make texts go through is to split the raw sentences into words, or more exactly tokens. The easiest way to do this would be to split the string on spaces, but we can be smarter:\n",
    "\n",
    "we need to take care of punctuation\n",
    "some words are contractions of two different words, like isn't or don't\n",
    "we may need to clean some parts of our texts, if there's HTML code for instance\n",
    "To see what the tokenizer had done behind the scenes, let's have a look at a few texts in a batch."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [],
   "source": [
    "data = TextDataBunch.load(path)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table>  <col width='90%'>  <col width='10%'>  <tr>\n",
       "    <th>text</th>\n",
       "    <th>target</th>\n",
       "  </tr>\n",
       "  <tr>\n",
       "    <th>xxbos xxup the xxup shop xxup around xxup the xxup corner is one of the xxunk and most feel - good romantic comedies ever made . xxmaj there 's just no getting around that , and it 's hard to actually put one 's feeling for this film into words . xxmaj it 's not one of those films that tries too hard , nor does it come up with</th>\n",
       "    <th>positive</th>\n",
       "  </tr>\n",
       "  <tr>\n",
       "    <th>xxbos xxmaj now that xxmaj che(2008 ) has finished its relatively short xxmaj australian cinema run ( extremely limited xxunk screen in xxmaj xxunk , after xxunk ) , i can xxunk join both xxunk of \" xxmaj at xxmaj the xxmaj movies \" in taking xxmaj steven xxmaj soderbergh to task . \\n\\n xxmaj it 's usually satisfying to watch a film director change his style / subject ,</th>\n",
       "    <th>negative</th>\n",
       "  </tr>\n",
       "  <tr>\n",
       "    <th>xxbos xxmaj many neglect that this is n't just a classic due to the fact that it 's the first xxup 3d game , or even the first xxunk - up . xxmaj it 's also one of the first xxunk games , one of the xxunk definitely the first ) truly claustrophobic games , and just a pretty well - xxunk gaming experience in general . xxmaj with graphics</th>\n",
       "    <th>positive</th>\n",
       "  </tr>\n",
       "  <tr>\n",
       "    <th>xxbos i really wanted to love this show . i truly , honestly did . \\n\\n xxmaj for the first time , gay viewers get their own version of the \" xxmaj the xxmaj bachelor \" . xxmaj with the help of his obligatory \" xxunk \" xxmaj xxunk , xxmaj james , a good looking , well - to - do thirty - something has the chance of love</th>\n",
       "    <th>negative</th>\n",
       "  </tr>\n",
       "  <tr>\n",
       "    <th>xxbos \\n\\n i 'm sure things did n't exactly go the same way in the real life of xxmaj homer xxmaj hickam as they did in the film adaptation of his book , xxmaj rocket xxmaj boys , but the movie \" xxmaj october xxmaj sky \" ( an xxunk of the book 's title ) is good enough to stand alone . i have not read xxmaj hickam 's</th>\n",
       "    <th>positive</th>\n",
       "  </tr>\n",
       "</table>\n"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "data.show_batch()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Help on method from_csv in module fastai.text.data:\n",
      "\n",
      "from_csv(path:Union[pathlib.Path, str], csv_name, valid_pct:float=0.2, test:Union[str, NoneType]=None, tokenizer:fastai.text.transform.Tokenizer=None, vocab:fastai.text.transform.Vocab=None, classes:Collection[str]=None, header='infer', text_cols:Union[int, Collection[int], str, Collection[str]]=1, label_cols:Union[int, Collection[int], str, Collection[str]]=0, label_delim:str=None, **kwargs) -> fastai.basic_data.DataBunch method of builtins.type instance\n",
      "    Create a `TextDataBunch` from texts in csv files.\n",
      "\n"
     ]
    }
   ],
   "source": [
    "help(TextDataBunch.from_csv)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Help on Vocab in module fastai.text.transform object:\n",
      "\n",
      "class Vocab(builtins.object)\n",
      " |  Contain the correspondance between numbers and tokens and numericalize.\n",
      " |  \n",
      " |  Methods defined here:\n",
      " |  \n",
      " |  __getstate__(self)\n",
      " |  \n",
      " |  __init__(self, itos:Collection[str])\n",
      " |      Initialize self.  See help(type(self)) for accurate signature.\n",
      " |  \n",
      " |  __setstate__(self, state:dict)\n",
      " |  \n",
      " |  numericalize(self, t:Collection[str]) -> List[int]\n",
      " |      Convert a list of tokens `t` to their ids.\n",
      " |  \n",
      " |  textify(self, nums:Collection[int], sep=' ') -> List[str]\n",
      " |      Convert a list of `nums` to their tokens.\n",
      " |  \n",
      " |  ----------------------------------------------------------------------\n",
      " |  Class methods defined here:\n",
      " |  \n",
      " |  create(tokens:Collection[Collection[str]], max_vocab:int, min_freq:int) -> 'Vocab' from builtins.type\n",
      " |      Create a vocabulary from a set of `tokens`.\n",
      " |  \n",
      " |  ----------------------------------------------------------------------\n",
      " |  Data descriptors defined here:\n",
      " |  \n",
      " |  __dict__\n",
      " |      dictionary for instance variables (if defined)\n",
      " |  \n",
      " |  __weakref__\n",
      " |      list of weak references to the object (if defined)\n",
      "\n"
     ]
    }
   ],
   "source": [
    "help(data.vocab)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['xxunk',\n",
       " 'xxpad',\n",
       " 'xxbos',\n",
       " 'xxfld',\n",
       " 'xxmaj',\n",
       " 'xxup',\n",
       " 'xxrep',\n",
       " 'xxwrep',\n",
       " 'the',\n",
       " ',']"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data.vocab.itos[:10]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Text xxbos xxmaj best of the xxmaj best 4 , is better than 3 , but just barely . xxmaj basically , i say this because part 4 does n't xxunk parts 1 / 2 ( like 3 does ) , ( xxunk . their is no reference to xxmaj tommy xxmaj lee having siblings ) . \n",
       "\n",
       " xxmaj anyway , i liked the xxmaj russian plot line of the story , and especially xxmaj xxunk xxmaj xxunk - xxmaj xxunk 's bit part as xxmaj boris . xxmaj aside from that though and a few fighting scenes , the movie is nothing special . xxmaj the limited budget is also very noticeable ( especially in the xxunk blow - up scene ) . \n",
       "\n",
       " xxmaj also , part 4 does not really have a moral or say anything like part 3 did , there are a couple of more better known actors ( xxmaj hudson , xxmaj xxunk ) in part 4 , but alas nothing like the beginning of the series ( and even these characters have very small roles ) . \n",
       "\n",
       " xxmaj alas , it seems xxmaj best of the xxmaj best is the xxmaj xxunk show , and to be truthful , he can not carry a movie . \n",
       "\n",
       " xxmaj saw on tape , xxmaj xxunk"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data.train_ds[0][0]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([  2,   4, 156,  13,   8,   4, 156, 264,   9,  15], dtype=int64)"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data.train_ds[0][0].data[:10]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "path=Path('D:/Ritika Agarwal/fatsai-v3/dataset/imdb/imdb/')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[WindowsPath('D:/Ritika Agarwal/fatsai-v3/dataset/imdb/imdb/imdb.vocab'),\n",
       " WindowsPath('D:/Ritika Agarwal/fatsai-v3/dataset/imdb/imdb/models'),\n",
       " WindowsPath('D:/Ritika Agarwal/fatsai-v3/dataset/imdb/imdb/README'),\n",
       " WindowsPath('D:/Ritika Agarwal/fatsai-v3/dataset/imdb/imdb/test'),\n",
       " WindowsPath('D:/Ritika Agarwal/fatsai-v3/dataset/imdb/imdb/tmp_clas'),\n",
       " WindowsPath('D:/Ritika Agarwal/fatsai-v3/dataset/imdb/imdb/tmp_lm'),\n",
       " WindowsPath('D:/Ritika Agarwal/fatsai-v3/dataset/imdb/imdb/train'),\n",
       " WindowsPath('D:/Ritika Agarwal/fatsai-v3/dataset/imdb/imdb/unsup')]"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "path.ls()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "data_lm = (TextList.from_folder(path)\n",
    "           #Inputs: all the text files in path\n",
    "            .filter_by_folder(include=['train', 'test', 'unsup']) \n",
    "           #We may have other temp folders that contain text files so we only keep what's in train and test\n",
    "            .random_split_by_pct(0.1)\n",
    "           #We randomly split and keep 10% (10,000 reviews) for validation\n",
    "            .label_for_lm()           \n",
    "           #We want to do a language model so we label accordingly\n",
    "            .databunch(bs=32))\n",
    "data_lm.save('tmp_lm')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "data_lm = TextLMDataBunch.load(path, 'tmp_lm', bs=32)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table>  <col width='5%'>  <col width='95%'>  <tr>\n",
       "    <th>idx</th>\n",
       "    <th>text</th>\n",
       "  </tr>\n",
       "  <tr>\n",
       "    <th>0</th>\n",
       "    <th>xxbos xxmaj this is not good . xxmaj it 's some sort of generic , b - movie , noir epic ... with all of those ideas in conflict . xxmaj twenty \" developments \" get teased out , to no discernible entertainment value . xxmaj it 's a very , very long movie trying hard to believe it has peaks and structure , but the meaninglessness begins piling up</th>\n",
       "  </tr>\n",
       "  <tr>\n",
       "    <th>1</th>\n",
       "    <th>put on some of the most pointless shows i 've ever seen xxmaj in my xxunk xxunk only thing that keeps me watching xxup xxunk anymore is the occasional bits of the old shows and the anime on 4kids . xxmaj it was such a huge mistake to pull this show off of the schedule . xxmaj this brings me back to my first point ... xxup bring xxup it</th>\n",
       "  </tr>\n",
       "  <tr>\n",
       "    <th>2</th>\n",
       "    <th>ag \" , everyone was white , whereas in \" xxup ch \" they 're all black . xxmaj moreover , this one has a xxmaj motown soundtrack . xxmaj specifically , the movie focuses on several working - class xxmaj african - xxmaj american students in 1964 xxmaj chicago and their antics . xxmaj the movie deals mainly with home life and relationships . xxmaj in their apartments ,</th>\n",
       "  </tr>\n",
       "  <tr>\n",
       "    <th>3</th>\n",
       "    <th>of xxmaj american women who can not discern logic from nonsense . \\n\\n xxmaj in xxup xxunk , it is not only the boyfriends of the week who are xxunk overlords , it is the women themselves . i have no objection to a story about the most wealthy , powerful women on earth , but i do n't think such a story can work as a series . i</th>\n",
       "  </tr>\n",
       "  <tr>\n",
       "    <th>4</th>\n",
       "    <th>family is one of the few white families around . xxmaj it gives a sense of what life was like both for the colonists and for the natives with whom they associated . xxmaj it 's a sense consistent with another movie i 've seen about xxmaj africa in a similar time period ( xxmaj xxunk in xxmaj afrika ( 2001 ) ) , but i have no way of</th>\n",
       "  </tr>\n",
       "</table>\n"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "data_lm.show_batch()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "\n",
    "learn = language_model_learner(data_lm, pretrained_model=URLs.WT103_1, drop_mult=0.3)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "ename": "RuntimeError",
     "evalue": "CUDA out of memory. Tried to allocate 91.63 MiB (GPU 0; 4.00 GiB total capacity; 1.45 GiB already allocated; 0 bytes free; 30.23 MiB cached)",
     "output_type": "error",
     "traceback": [
      "\u001b[1;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[1;31mRuntimeError\u001b[0m                              Traceback (most recent call last)",
      "\u001b[1;32m~\\Anaconda3\\lib\\site-packages\\fastai\\basic_train.py\u001b[0m in \u001b[0;36mfit\u001b[1;34m(epochs, model, loss_func, opt, data, callbacks, metrics)\u001b[0m\n\u001b[0;32m     95\u001b[0m         \u001b[0mexception\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0me\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m---> 96\u001b[1;33m         \u001b[1;32mraise\u001b[0m \u001b[0me\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m     97\u001b[0m     \u001b[1;32mfinally\u001b[0m\u001b[1;33m:\u001b[0m \u001b[0mcb_handler\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mon_train_end\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mexception\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;32m~\\Anaconda3\\lib\\site-packages\\fastai\\basic_train.py\u001b[0m in \u001b[0;36mfit\u001b[1;34m(epochs, model, loss_func, opt, data, callbacks, metrics)\u001b[0m\n\u001b[0;32m     85\u001b[0m                 \u001b[0mxb\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0myb\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mcb_handler\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mon_batch_begin\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mxb\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0myb\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m---> 86\u001b[1;33m                 \u001b[0mloss\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mloss_batch\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mmodel\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mxb\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0myb\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mloss_func\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mopt\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mcb_handler\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m     87\u001b[0m                 \u001b[1;32mif\u001b[0m \u001b[0mcb_handler\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mon_batch_end\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mloss\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m:\u001b[0m \u001b[1;32mbreak\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;32m~\\Anaconda3\\lib\\site-packages\\fastai\\basic_train.py\u001b[0m in \u001b[0;36mloss_batch\u001b[1;34m(model, xb, yb, loss_func, opt, cb_handler)\u001b[0m\n\u001b[0;32m     26\u001b[0m         \u001b[0mloss\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mcb_handler\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mon_backward_begin\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mloss\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m---> 27\u001b[1;33m         \u001b[0mloss\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mbackward\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m     28\u001b[0m         \u001b[0mcb_handler\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mon_backward_end\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;32m~\\Anaconda3\\lib\\site-packages\\torch\\tensor.py\u001b[0m in \u001b[0;36mbackward\u001b[1;34m(self, gradient, retain_graph, create_graph)\u001b[0m\n\u001b[0;32m    101\u001b[0m         \"\"\"\n\u001b[1;32m--> 102\u001b[1;33m         \u001b[0mtorch\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mautograd\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mbackward\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mself\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mgradient\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mretain_graph\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mcreate_graph\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m    103\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;32m~\\Anaconda3\\lib\\site-packages\\torch\\autograd\\__init__.py\u001b[0m in \u001b[0;36mbackward\u001b[1;34m(tensors, grad_tensors, retain_graph, create_graph, grad_variables)\u001b[0m\n\u001b[0;32m     89\u001b[0m         \u001b[0mtensors\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mgrad_tensors\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mretain_graph\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mcreate_graph\u001b[0m\u001b[1;33m,\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m---> 90\u001b[1;33m         allow_unreachable=True)  # allow_unreachable flag\n\u001b[0m\u001b[0;32m     91\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;31mRuntimeError\u001b[0m: CUDA out of memory. Tried to allocate 512.75 MiB (GPU 0; 4.00 GiB total capacity; 1.46 GiB already allocated; 0 bytes free; 26.73 MiB cached)",
      "\nDuring handling of the above exception, another exception occurred:\n",
      "\u001b[1;31mRuntimeError\u001b[0m                              Traceback (most recent call last)",
      "\u001b[1;32m<ipython-input-8-792767d417ae>\u001b[0m in \u001b[0;36m<module>\u001b[1;34m()\u001b[0m\n\u001b[1;32m----> 1\u001b[1;33m \u001b[0mlearn\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mlr_find\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m",
      "\u001b[1;32m~\\Anaconda3\\lib\\site-packages\\fastai\\train.py\u001b[0m in \u001b[0;36mlr_find\u001b[1;34m(learn, start_lr, end_lr, num_it, stop_div, **kwargs)\u001b[0m\n\u001b[0;32m     30\u001b[0m     \u001b[0mcb\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mLRFinder\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mlearn\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mstart_lr\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mend_lr\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mnum_it\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mstop_div\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m     31\u001b[0m     \u001b[0ma\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mint\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mnp\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mceil\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mnum_it\u001b[0m\u001b[1;33m/\u001b[0m\u001b[0mlen\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mlearn\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mdata\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mtrain_dl\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m---> 32\u001b[1;33m     \u001b[0mlearn\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mfit\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0ma\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mstart_lr\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mcallbacks\u001b[0m\u001b[1;33m=\u001b[0m\u001b[1;33m[\u001b[0m\u001b[0mcb\u001b[0m\u001b[1;33m]\u001b[0m\u001b[1;33m,\u001b[0m \u001b[1;33m**\u001b[0m\u001b[0mkwargs\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m     33\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m     34\u001b[0m \u001b[1;32mdef\u001b[0m \u001b[0mto_fp16\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mlearn\u001b[0m\u001b[1;33m:\u001b[0m\u001b[0mLearner\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mloss_scale\u001b[0m\u001b[1;33m:\u001b[0m\u001b[0mfloat\u001b[0m\u001b[1;33m=\u001b[0m\u001b[1;36m512.\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mflat_master\u001b[0m\u001b[1;33m:\u001b[0m\u001b[0mbool\u001b[0m\u001b[1;33m=\u001b[0m\u001b[1;32mFalse\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m->\u001b[0m\u001b[0mLearner\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;32m~\\Anaconda3\\lib\\site-packages\\fastai\\basic_train.py\u001b[0m in \u001b[0;36mfit\u001b[1;34m(self, epochs, lr, wd, callbacks)\u001b[0m\n\u001b[0;32m    172\u001b[0m         \u001b[0mcallbacks\u001b[0m \u001b[1;33m=\u001b[0m \u001b[1;33m[\u001b[0m\u001b[0mcb\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mself\u001b[0m\u001b[1;33m)\u001b[0m \u001b[1;32mfor\u001b[0m \u001b[0mcb\u001b[0m \u001b[1;32min\u001b[0m \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mcallback_fns\u001b[0m\u001b[1;33m]\u001b[0m \u001b[1;33m+\u001b[0m \u001b[0mlistify\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mcallbacks\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    173\u001b[0m         fit(epochs, self.model, self.loss_func, opt=self.opt, data=self.data, metrics=self.metrics,\n\u001b[1;32m--> 174\u001b[1;33m             callbacks=self.callbacks+callbacks)\n\u001b[0m\u001b[0;32m    175\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    176\u001b[0m     \u001b[1;32mdef\u001b[0m \u001b[0mcreate_opt\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mself\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mlr\u001b[0m\u001b[1;33m:\u001b[0m\u001b[0mFloats\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mwd\u001b[0m\u001b[1;33m:\u001b[0m\u001b[0mFloats\u001b[0m\u001b[1;33m=\u001b[0m\u001b[1;36m0.\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m->\u001b[0m\u001b[1;32mNone\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;32m~\\Anaconda3\\lib\\site-packages\\fastai\\basic_train.py\u001b[0m in \u001b[0;36mfit\u001b[1;34m(epochs, model, loss_func, opt, data, callbacks, metrics)\u001b[0m\n\u001b[0;32m     95\u001b[0m         \u001b[0mexception\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0me\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m     96\u001b[0m         \u001b[1;32mraise\u001b[0m \u001b[0me\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m---> 97\u001b[1;33m     \u001b[1;32mfinally\u001b[0m\u001b[1;33m:\u001b[0m \u001b[0mcb_handler\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mon_train_end\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mexception\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m     98\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m     99\u001b[0m loss_func_name2activ = {'cross_entropy_loss': F.softmax, 'nll_loss': torch.exp, 'poisson_nll_loss': torch.exp,\n",
      "\u001b[1;32m~\\Anaconda3\\lib\\site-packages\\fastai\\callback.py\u001b[0m in \u001b[0;36mon_train_end\u001b[1;34m(self, exception)\u001b[0m\n\u001b[0;32m    272\u001b[0m     \u001b[1;32mdef\u001b[0m \u001b[0mon_train_end\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mself\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mexception\u001b[0m\u001b[1;33m:\u001b[0m\u001b[0mUnion\u001b[0m\u001b[1;33m[\u001b[0m\u001b[0mbool\u001b[0m\u001b[1;33m,\u001b[0m\u001b[0mException\u001b[0m\u001b[1;33m]\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m->\u001b[0m\u001b[1;32mNone\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    273\u001b[0m         \u001b[1;34m\"Handle end of training, `exception` is an `Exception` or False if no exceptions during training.\"\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m--> 274\u001b[1;33m         \u001b[0mself\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;34m'train_end'\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mexception\u001b[0m\u001b[1;33m=\u001b[0m\u001b[0mexception\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m    275\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    276\u001b[0m \u001b[1;32mclass\u001b[0m \u001b[0mAverageMetric\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mCallback\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;32m~\\Anaconda3\\lib\\site-packages\\fastai\\callback.py\u001b[0m in \u001b[0;36m__call__\u001b[1;34m(self, cb_name, call_mets, **kwargs)\u001b[0m\n\u001b[0;32m    197\u001b[0m         \u001b[1;34m\"Call through to all of the `CallbakHandler` functions.\"\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    198\u001b[0m         \u001b[1;32mif\u001b[0m \u001b[0mcall_mets\u001b[0m\u001b[1;33m:\u001b[0m \u001b[1;33m[\u001b[0m\u001b[0mgetattr\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mmet\u001b[0m\u001b[1;33m,\u001b[0m \u001b[1;34mf'on_{cb_name}'\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;33m**\u001b[0m\u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mstate_dict\u001b[0m\u001b[1;33m,\u001b[0m \u001b[1;33m**\u001b[0m\u001b[0mkwargs\u001b[0m\u001b[1;33m)\u001b[0m \u001b[1;32mfor\u001b[0m \u001b[0mmet\u001b[0m \u001b[1;32min\u001b[0m \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mmetrics\u001b[0m\u001b[1;33m]\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m--> 199\u001b[1;33m         \u001b[1;32mreturn\u001b[0m \u001b[1;33m[\u001b[0m\u001b[0mgetattr\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mcb\u001b[0m\u001b[1;33m,\u001b[0m \u001b[1;34mf'on_{cb_name}'\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;33m**\u001b[0m\u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mstate_dict\u001b[0m\u001b[1;33m,\u001b[0m \u001b[1;33m**\u001b[0m\u001b[0mkwargs\u001b[0m\u001b[1;33m)\u001b[0m \u001b[1;32mfor\u001b[0m \u001b[0mcb\u001b[0m \u001b[1;32min\u001b[0m \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mcallbacks\u001b[0m\u001b[1;33m]\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m    200\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    201\u001b[0m     \u001b[1;32mdef\u001b[0m \u001b[0mset_dl\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mself\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mdl\u001b[0m\u001b[1;33m:\u001b[0m\u001b[0mDataLoader\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;32m~\\Anaconda3\\lib\\site-packages\\fastai\\callback.py\u001b[0m in \u001b[0;36m<listcomp>\u001b[1;34m(.0)\u001b[0m\n\u001b[0;32m    197\u001b[0m         \u001b[1;34m\"Call through to all of the `CallbakHandler` functions.\"\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    198\u001b[0m         \u001b[1;32mif\u001b[0m \u001b[0mcall_mets\u001b[0m\u001b[1;33m:\u001b[0m \u001b[1;33m[\u001b[0m\u001b[0mgetattr\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mmet\u001b[0m\u001b[1;33m,\u001b[0m \u001b[1;34mf'on_{cb_name}'\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;33m**\u001b[0m\u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mstate_dict\u001b[0m\u001b[1;33m,\u001b[0m \u001b[1;33m**\u001b[0m\u001b[0mkwargs\u001b[0m\u001b[1;33m)\u001b[0m \u001b[1;32mfor\u001b[0m \u001b[0mmet\u001b[0m \u001b[1;32min\u001b[0m \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mmetrics\u001b[0m\u001b[1;33m]\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m--> 199\u001b[1;33m         \u001b[1;32mreturn\u001b[0m \u001b[1;33m[\u001b[0m\u001b[0mgetattr\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mcb\u001b[0m\u001b[1;33m,\u001b[0m \u001b[1;34mf'on_{cb_name}'\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;33m**\u001b[0m\u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mstate_dict\u001b[0m\u001b[1;33m,\u001b[0m \u001b[1;33m**\u001b[0m\u001b[0mkwargs\u001b[0m\u001b[1;33m)\u001b[0m \u001b[1;32mfor\u001b[0m \u001b[0mcb\u001b[0m \u001b[1;32min\u001b[0m \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mcallbacks\u001b[0m\u001b[1;33m]\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m    200\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    201\u001b[0m     \u001b[1;32mdef\u001b[0m \u001b[0mset_dl\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mself\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mdl\u001b[0m\u001b[1;33m:\u001b[0m\u001b[0mDataLoader\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;32m~\\Anaconda3\\lib\\site-packages\\fastai\\callbacks\\lr_finder.py\u001b[0m in \u001b[0;36mon_train_end\u001b[1;34m(self, **kwargs)\u001b[0m\n\u001b[0;32m     43\u001b[0m         \u001b[1;31m# restore the valid_dl we turned off on `__init__`\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m     44\u001b[0m         \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mdata\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mvalid_dl\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mvalid_dl\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m---> 45\u001b[1;33m         \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mlearn\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mload\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;34m'tmp'\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m     46\u001b[0m         \u001b[1;32mif\u001b[0m \u001b[0mhasattr\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mlearn\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mmodel\u001b[0m\u001b[1;33m,\u001b[0m \u001b[1;34m'reset'\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m:\u001b[0m \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mlearn\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mmodel\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mreset\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m     47\u001b[0m         \u001b[0mprint\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;34m'LR Finder is complete, type {learner_name}.recorder.plot() to see the graph.'\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;32m~\\Anaconda3\\lib\\site-packages\\fastai\\basic_train.py\u001b[0m in \u001b[0;36mload\u001b[1;34m(self, name, device, strict, with_opt)\u001b[0m\n\u001b[0;32m    229\u001b[0m         \u001b[1;34m\"Load model and optimizer state (if `with_opt`) `name` from `self.model_dir` using `device`.\"\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    230\u001b[0m         \u001b[1;32mif\u001b[0m \u001b[0mdevice\u001b[0m \u001b[1;32mis\u001b[0m \u001b[1;32mNone\u001b[0m\u001b[1;33m:\u001b[0m \u001b[0mdevice\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mdata\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mdevice\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m--> 231\u001b[1;33m         \u001b[0mstate\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mtorch\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mload\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mpath\u001b[0m\u001b[1;33m/\u001b[0m\u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mmodel_dir\u001b[0m\u001b[1;33m/\u001b[0m\u001b[1;34mf'{name}.pth'\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mmap_location\u001b[0m\u001b[1;33m=\u001b[0m\u001b[0mdevice\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m    232\u001b[0m         \u001b[1;32mif\u001b[0m \u001b[0mset\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mstate\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mkeys\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m)\u001b[0m \u001b[1;33m==\u001b[0m \u001b[1;33m{\u001b[0m\u001b[1;34m'model'\u001b[0m\u001b[1;33m,\u001b[0m \u001b[1;34m'opt'\u001b[0m\u001b[1;33m}\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    233\u001b[0m             \u001b[0mget_model\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mmodel\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mload_state_dict\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mstate\u001b[0m\u001b[1;33m[\u001b[0m\u001b[1;34m'model'\u001b[0m\u001b[1;33m]\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mstrict\u001b[0m\u001b[1;33m=\u001b[0m\u001b[0mstrict\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;32m~\\Anaconda3\\lib\\site-packages\\torch\\serialization.py\u001b[0m in \u001b[0;36mload\u001b[1;34m(f, map_location, pickle_module)\u001b[0m\n\u001b[0;32m    365\u001b[0m         \u001b[0mf\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mopen\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mf\u001b[0m\u001b[1;33m,\u001b[0m \u001b[1;34m'rb'\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    366\u001b[0m     \u001b[1;32mtry\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m--> 367\u001b[1;33m         \u001b[1;32mreturn\u001b[0m \u001b[0m_load\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mf\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mmap_location\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mpickle_module\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m    368\u001b[0m     \u001b[1;32mfinally\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    369\u001b[0m         \u001b[1;32mif\u001b[0m \u001b[0mnew_fd\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;32m~\\Anaconda3\\lib\\site-packages\\torch\\serialization.py\u001b[0m in \u001b[0;36m_load\u001b[1;34m(f, map_location, pickle_module)\u001b[0m\n\u001b[0;32m    536\u001b[0m     \u001b[0munpickler\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mpickle_module\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mUnpickler\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mf\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    537\u001b[0m     \u001b[0munpickler\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mpersistent_load\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mpersistent_load\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m--> 538\u001b[1;33m     \u001b[0mresult\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0munpickler\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mload\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m    539\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    540\u001b[0m     \u001b[0mdeserialized_storage_keys\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mpickle_module\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mload\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mf\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;32m~\\Anaconda3\\lib\\site-packages\\torch\\serialization.py\u001b[0m in \u001b[0;36mpersistent_load\u001b[1;34m(saved_id)\u001b[0m\n\u001b[0;32m    502\u001b[0m             \u001b[1;32mif\u001b[0m \u001b[0mroot_key\u001b[0m \u001b[1;32mnot\u001b[0m \u001b[1;32min\u001b[0m \u001b[0mdeserialized_objects\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    503\u001b[0m                 deserialized_objects[root_key] = restore_location(\n\u001b[1;32m--> 504\u001b[1;33m                     data_type(size), location)\n\u001b[0m\u001b[0;32m    505\u001b[0m             \u001b[0mstorage\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mdeserialized_objects\u001b[0m\u001b[1;33m[\u001b[0m\u001b[0mroot_key\u001b[0m\u001b[1;33m]\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    506\u001b[0m             \u001b[1;32mif\u001b[0m \u001b[0mview_metadata\u001b[0m \u001b[1;32mis\u001b[0m \u001b[1;32mnot\u001b[0m \u001b[1;32mNone\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;32m~\\Anaconda3\\lib\\site-packages\\torch\\serialization.py\u001b[0m in \u001b[0;36mrestore_location\u001b[1;34m(storage, location)\u001b[0m\n\u001b[0;32m    385\u001b[0m     \u001b[1;32melif\u001b[0m \u001b[0misinstance\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mmap_location\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mtorch\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mdevice\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    386\u001b[0m         \u001b[1;32mdef\u001b[0m \u001b[0mrestore_location\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mstorage\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mlocation\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m--> 387\u001b[1;33m             \u001b[1;32mreturn\u001b[0m \u001b[0mdefault_restore_location\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mstorage\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mstr\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mmap_location\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m    388\u001b[0m     \u001b[1;32melse\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    389\u001b[0m         \u001b[1;32mdef\u001b[0m \u001b[0mrestore_location\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mstorage\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mlocation\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;32m~\\Anaconda3\\lib\\site-packages\\torch\\serialization.py\u001b[0m in \u001b[0;36mdefault_restore_location\u001b[1;34m(storage, location)\u001b[0m\n\u001b[0;32m    111\u001b[0m \u001b[1;32mdef\u001b[0m \u001b[0mdefault_restore_location\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mstorage\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mlocation\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    112\u001b[0m     \u001b[1;32mfor\u001b[0m \u001b[0m_\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0m_\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mfn\u001b[0m \u001b[1;32min\u001b[0m \u001b[0m_package_registry\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m--> 113\u001b[1;33m         \u001b[0mresult\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mfn\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mstorage\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mlocation\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m    114\u001b[0m         \u001b[1;32mif\u001b[0m \u001b[0mresult\u001b[0m \u001b[1;32mis\u001b[0m \u001b[1;32mnot\u001b[0m \u001b[1;32mNone\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    115\u001b[0m             \u001b[1;32mreturn\u001b[0m \u001b[0mresult\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;32m~\\Anaconda3\\lib\\site-packages\\torch\\serialization.py\u001b[0m in \u001b[0;36m_cuda_deserialize\u001b[1;34m(obj, location)\u001b[0m\n\u001b[0;32m     93\u001b[0m     \u001b[1;32mif\u001b[0m \u001b[0mlocation\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mstartswith\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;34m'cuda'\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m     94\u001b[0m         \u001b[0mdevice\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mvalidate_cuda_device\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mlocation\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m---> 95\u001b[1;33m         \u001b[1;32mreturn\u001b[0m \u001b[0mobj\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mcuda\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mdevice\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m     96\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m     97\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;32m~\\Anaconda3\\lib\\site-packages\\torch\\_utils.py\u001b[0m in \u001b[0;36m_cuda\u001b[1;34m(self, device, non_blocking, **kwargs)\u001b[0m\n\u001b[0;32m     74\u001b[0m         \u001b[1;32melse\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m     75\u001b[0m             \u001b[0mnew_type\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mgetattr\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mtorch\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mcuda\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0m__class__\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0m__name__\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m---> 76\u001b[1;33m             \u001b[1;32mreturn\u001b[0m \u001b[0mnew_type\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0msize\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mcopy_\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mself\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mnon_blocking\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m     77\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m     78\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;32m~\\Anaconda3\\lib\\site-packages\\torch\\cuda\\__init__.py\u001b[0m in \u001b[0;36m_lazy_new\u001b[1;34m(cls, *args, **kwargs)\u001b[0m\n\u001b[0;32m    494\u001b[0m     \u001b[1;31m# We need this method only for lazy init, so we can remove it\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    495\u001b[0m     \u001b[1;32mdel\u001b[0m \u001b[0m_CudaBase\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0m__new__\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m--> 496\u001b[1;33m     \u001b[1;32mreturn\u001b[0m \u001b[0msuper\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0m_CudaBase\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mcls\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0m__new__\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mcls\u001b[0m\u001b[1;33m,\u001b[0m \u001b[1;33m*\u001b[0m\u001b[0margs\u001b[0m\u001b[1;33m,\u001b[0m \u001b[1;33m**\u001b[0m\u001b[0mkwargs\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m    497\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    498\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;31mRuntimeError\u001b[0m: CUDA out of memory. Tried to allocate 91.63 MiB (GPU 0; 4.00 GiB total capacity; 1.45 GiB already allocated; 0 bytes free; 30.23 MiB cached)"
     ]
    }
   ],
   "source": [
    "learn.lr_find()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}