N) RoadMap 14 - Classification 1 - Pytorch model zoo.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# RoadMap 14 - Classification 1 - Pytorch model zoo\n",
    "\n",
    "    1. Available Models\n",
    "\n",
    "    2. Using models for image classification\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import torchvision.models as models"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 1. Models\n",
    "    \n",
    "    1. Alexnet\n",
    "    2. Vgg11\n",
    "    3. Vgg11 With batch normalization\n",
    "    4. Vgg13\n",
    "    5. Vgg13 with batch normalization\n",
    "    6. Vgg16\n",
    "    7. Vgg16 with batch normalization\n",
    "    8. Vgg19\n",
    "    9. Vgg19 with batch normalization\n",
    "    10. Resnet-18\n",
    "    11. Resnet-34\n",
    "    12. Resnet-50\n",
    "    13. Resnet-101\n",
    "    14. Resnet-152\n",
    "    15. Squeezenet-1.0\n",
    "    16. Squeezenet-1.1\n",
    "    17. Densenet-121\n",
    "    18. Densenet-161\n",
    "    19. Densenet-169\n",
    "    20. Densenet-201\n",
    "    21. Inception-v3\n",
    "    \n",
    "    \n",
    "# 2. Using a model for Image Classification\n",
    "\n",
    "    1. Alexnet\n",
    "    2. Vgg11\n",
    "    3. Vgg13\n",
    "    4. Vgg16\n",
    "    5. Vgg19\n",
    "    6. Squeezenet-1.1\n",
    "    "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Understanding models"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "# untrained model\n",
    "alexnet = models.alexnet()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['__call__', '__class__', '__delattr__', '__dict__', '__dir__', '__doc__', '__eq__', '__format__', '__ge__', '__getattr__', '__getattribute__', '__gt__', '__hash__', '__init__', '__init_subclass__', '__le__', '__lt__', '__module__', '__ne__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__setstate__', '__sizeof__', '__str__', '__subclasshook__', '__weakref__', '_apply', '_backward_hooks', '_buffers', '_forward_hooks', '_forward_pre_hooks', '_get_name', '_load_from_state_dict', '_load_state_dict_pre_hooks', '_modules', '_named_members', '_parameters', '_register_load_state_dict_pre_hook', '_register_state_dict_hook', '_save_to_state_dict', '_slow_forward', '_state_dict_hooks', '_tracing_name', '_version', 'add_module', 'apply', 'avgpool', 'buffers', 'children', 'classifier', 'cpu', 'cuda', 'double', 'dump_patches', 'eval', 'extra_repr', 'features', 'float', 'forward', 'half', 'load_state_dict', 'modules', 'named_buffers', 'named_children', 'named_modules', 'named_parameters', 'parameters', 'register_backward_hook', 'register_buffer', 'register_forward_hook', 'register_forward_pre_hook', 'register_parameter', 'requires_grad_', 'share_memory', 'state_dict', 'to', 'train', 'training', 'type', 'zero_grad']\n"
     ]
    }
   ],
   "source": [
    "print(dir(alexnet))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Pre-trained model\n",
    "alexnet = models.alexnet(pretrained=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Downloading: \"https://download.pytorch.org/models/densenet169-b2777c0a.pth\" to /home/abhi/.torch/models/densenet169-b2777c0a.pth\n",
      "100.0%\n",
      "Downloading: \"https://download.pytorch.org/models/densenet201-c1103571.pth\" to /home/abhi/.torch/models/densenet201-c1103571.pth\n",
      "100.0%\n",
      "/home/abhi/.virtualenvs/frameworks/local/lib/python2.7/site-packages/scipy/special/__init__.py:640: RuntimeWarning: numpy.dtype size changed, may indicate binary incompatibility. Expected 96, got 88\n",
      "  from ._ufuncs import *\n",
      "/home/abhi/.virtualenvs/frameworks/local/lib/python2.7/site-packages/scipy/linalg/basic.py:17: RuntimeWarning: numpy.dtype size changed, may indicate binary incompatibility. Expected 96, got 88\n",
      "  from ._solve_toeplitz import levinson\n",
      "/home/abhi/.virtualenvs/frameworks/local/lib/python2.7/site-packages/scipy/linalg/__init__.py:207: RuntimeWarning: numpy.dtype size changed, may indicate binary incompatibility. Expected 96, got 88\n",
      "  from ._decomp_update import *\n",
      "/home/abhi/.virtualenvs/frameworks/local/lib/python2.7/site-packages/scipy/special/_ellip_harm.py:7: RuntimeWarning: numpy.dtype size changed, may indicate binary incompatibility. Expected 96, got 88\n",
      "  from ._ellip_harm_2 import _ellipsoid, _ellipsoid_norm\n",
      "/home/abhi/.virtualenvs/frameworks/local/lib/python2.7/site-packages/scipy/interpolate/_bsplines.py:10: RuntimeWarning: numpy.dtype size changed, may indicate binary incompatibility. Expected 96, got 88\n",
      "  from . import _bspl\n",
      "/home/abhi/.virtualenvs/frameworks/local/lib/python2.7/site-packages/scipy/sparse/lil.py:19: RuntimeWarning: numpy.dtype size changed, may indicate binary incompatibility. Expected 96, got 88\n",
      "  from . import _csparsetools\n",
      "/home/abhi/.virtualenvs/frameworks/local/lib/python2.7/site-packages/scipy/sparse/csgraph/__init__.py:165: RuntimeWarning: numpy.dtype size changed, may indicate binary incompatibility. Expected 96, got 88\n",
      "  from ._shortest_path import shortest_path, floyd_warshall, dijkstra,\\\n",
      "/home/abhi/.virtualenvs/frameworks/local/lib/python2.7/site-packages/scipy/sparse/csgraph/_validation.py:5: RuntimeWarning: numpy.dtype size changed, may indicate binary incompatibility. Expected 96, got 88\n",
      "  from ._tools import csgraph_to_dense, csgraph_from_dense,\\\n",
      "/home/abhi/.virtualenvs/frameworks/local/lib/python2.7/site-packages/scipy/sparse/csgraph/__init__.py:167: RuntimeWarning: numpy.dtype size changed, may indicate binary incompatibility. Expected 96, got 88\n",
      "  from ._traversal import breadth_first_order, depth_first_order, \\\n",
      "/home/abhi/.virtualenvs/frameworks/local/lib/python2.7/site-packages/scipy/sparse/csgraph/__init__.py:169: RuntimeWarning: numpy.dtype size changed, may indicate binary incompatibility. Expected 96, got 88\n",
      "  from ._min_spanning_tree import minimum_spanning_tree\n",
      "/home/abhi/.virtualenvs/frameworks/local/lib/python2.7/site-packages/scipy/sparse/csgraph/__init__.py:170: RuntimeWarning: numpy.dtype size changed, may indicate binary incompatibility. Expected 96, got 88\n",
      "  from ._reordering import reverse_cuthill_mckee, maximum_bipartite_matching, \\\n",
      "/home/abhi/.virtualenvs/frameworks/local/lib/python2.7/site-packages/scipy/spatial/__init__.py:95: RuntimeWarning: numpy.dtype size changed, may indicate binary incompatibility. Expected 96, got 88\n",
      "  from .ckdtree import *\n",
      "/home/abhi/.virtualenvs/frameworks/local/lib/python2.7/site-packages/scipy/spatial/__init__.py:96: RuntimeWarning: numpy.dtype size changed, may indicate binary incompatibility. Expected 96, got 88\n",
      "  from .qhull import *\n",
      "/home/abhi/.virtualenvs/frameworks/local/lib/python2.7/site-packages/scipy/spatial/_spherical_voronoi.py:18: RuntimeWarning: numpy.dtype size changed, may indicate binary incompatibility. Expected 96, got 88\n",
      "  from . import _voronoi\n",
      "/home/abhi/.virtualenvs/frameworks/local/lib/python2.7/site-packages/scipy/spatial/distance.py:122: RuntimeWarning: numpy.dtype size changed, may indicate binary incompatibility. Expected 96, got 88\n",
      "  from . import _hausdorff\n",
      "/home/abhi/.virtualenvs/frameworks/local/lib/python2.7/site-packages/scipy/optimize/_trlib/__init__.py:1: RuntimeWarning: numpy.dtype size changed, may indicate binary incompatibility. Expected 96, got 88\n",
      "  from ._trlib import TRLIBQuadraticSubproblem\n",
      "/home/abhi/.virtualenvs/frameworks/local/lib/python2.7/site-packages/scipy/optimize/_numdiff.py:10: RuntimeWarning: numpy.dtype size changed, may indicate binary incompatibility. Expected 96, got 88\n",
      "  from ._group_columns import group_dense, group_sparse\n",
      "/home/abhi/.virtualenvs/frameworks/local/lib/python2.7/site-packages/scipy/stats/_continuous_distns.py:18: RuntimeWarning: numpy.dtype size changed, may indicate binary incompatibility. Expected 96, got 88\n",
      "  from . import _stats\n",
      "Downloading: \"https://download.pytorch.org/models/inception_v3_google-1a9a5a14.pth\" to /home/abhi/.torch/models/inception_v3_google-1a9a5a14.pth\n",
      "100.0%\n"
     ]
    }
   ],
   "source": [
    "# Other models\n",
    "\n",
    "#vgg11 = models.vgg11(pretrained=True)\n",
    "#vgg11_bn = models.vgg11_bn(pretrained=True)\n",
    "#vgg13 = models.vgg13(pretrained=True)\n",
    "#vgg13_bn = models.vgg13_bn(pretrained=True)\n",
    "#vgg16 = models.vgg16(pretrained=True)\n",
    "#vgg16_bn = models.vgg16_bn(pretrained=True)\n",
    "#vgg19 = models.vgg19(pretrained=True)\n",
    "#vgg19_bn = models.vgg19_bn(pretrained=True)\n",
    "#resnet18 = models.resnet18(pretrained=True)\n",
    "#resnet34 = models.resnet34(pretrained=True)\n",
    "#resnet50 = models.resnet50(pretrained=True)\n",
    "#resnet101 = models.resnet101(pretrained=True)\n",
    "#resnet152 = models.resnet152(pretrained=True)\n",
    "#squeezenet1_0 = models.squeezenet1_0(pretrained=True)\n",
    "#squeezenet1_1 = models.squeezenet1_1(pretrained=True)\n",
    "#densenet121 = models.densenet121(pretrained=True)\n",
    "#densenet161 = models.densenet161(pretrained=True)\n",
    "densenet169 = models.densenet169(pretrained=True)\n",
    "densenet201 = models.densenet201(pretrained=True)\n",
    "inception_v3 = models.inception_v3(pretrained=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Using pre-trained models for image classification"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "import io\n",
    "import requests\n",
    "import time\n",
    "from PIL import Image\n",
    "from torchvision import models, transforms\n",
    "from torch.autograd import Variable"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Getting image and labels\n",
    "LABELS_URL = 'https://s3.amazonaws.com/outcome-blog/imagenet/labels.json'\n",
    "IMG_URL = 'https://s3.amazonaws.com/outcome-blog/wp-content/uploads/2017/02/25192225/cat.jpg'\n",
    "response = requests.get(IMG_URL)\n",
    "img_pil = Image.open(io.BytesIO(response.content))\n",
    "labels = {int(key):value for (key, value)\n",
    "          in requests.get(LABELS_URL).json().items()}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Creating preprocessing transformations\n",
    "\n",
    "normalize = transforms.Normalize(\n",
    "   mean=[0.485, 0.456, 0.406],\n",
    "   std=[0.229, 0.224, 0.225]\n",
    ")\n",
    "preprocess = transforms.Compose([\n",
    "   transforms.Resize(256),\n",
    "   transforms.CenterCrop(224),\n",
    "   transforms.ToTensor(),\n",
    "   normalize\n",
    "])\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "torch.Size([3, 224, 224])\n"
     ]
    }
   ],
   "source": [
    "# Preprocess Image\n",
    "img_tensor = preprocess(img_pil)\n",
    "print(img_tensor.shape)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Using Alexnet classifier\n",
      "Predicted Label:  tabby, tabby cat\n",
      "Time Taken:  0.08603048324584961  seconds\n"
     ]
    }
   ],
   "source": [
    "# Alexnet\n",
    "\n",
    "# Initialize model\n",
    "model = models.alexnet(pretrained=True)\n",
    "\n",
    "# Applying a forward pass\n",
    "# The input to the network needs to be an autograd Variable. \n",
    "# To make it 4 dimension use unsqueeze\n",
    "input_data = img_tensor.unsqueeze(0)\n",
    "img_variable = Variable(input_data)\n",
    "start = time.time()\n",
    "fc_out = model(img_variable)\n",
    "time_d = time.time() - start\n",
    "\n",
    "print(\"Using Alexnet classifier\")\n",
    "print(\"Predicted Label: \", labels[fc_out.data.numpy().argmax()])\n",
    "print(\"Time Taken: \", time_d, \" seconds\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Using VGG11 classifier\n",
      "Predicted Label:  tabby, tabby cat\n",
      "Time Taken:  0.1313014030456543  seconds\n"
     ]
    }
   ],
   "source": [
    "# VGG11\n",
    "\n",
    "# Initialize model\n",
    "model = models.vgg11(pretrained=True)\n",
    "\n",
    "# Applying a forward pass\n",
    "# The input to the network needs to be an autograd Variable. \n",
    "# To make it 4 dimension use unsqueeze\n",
    "input_data = img_tensor.unsqueeze(0)\n",
    "img_variable = Variable(input_data)\n",
    "start = time.time()\n",
    "fc_out = model(img_variable)\n",
    "time_d = time.time() - start\n",
    "\n",
    "print(\"Using VGG11 classifier\")\n",
    "print(\"Predicted Label: \", labels[fc_out.data.numpy().argmax()])\n",
    "print(\"Time Taken: \", time_d, \" seconds\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# VGG13\n",
    "\n",
    "# Initialize model\n",
    "model = models.vgg13(pretrained=True)\n",
    "\n",
    "# Applying a forward pass\n",
    "# The input to the network needs to be an autograd Variable. \n",
    "# To make it 4 dimension use unsqueeze\n",
    "input_data = img_tensor.unsqueeze(0)\n",
    "img_variable = Variable(input_data)\n",
    "start = time.time()\n",
    "fc_out = model(img_variable)\n",
    "time_d = time.time() - start\n",
    "\n",
    "print(\"Using VGG13 classifier\")\n",
    "print(\"Predicted Label: \", labels[fc_out.data.numpy().argmax()])\n",
    "print(\"Time Taken: \", time_d, \" seconds\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Using VGG16 classifier\n",
      "Predicted Label:  Egyptian cat\n",
      "Time Taken:  1.05129289627  seconds\n"
     ]
    }
   ],
   "source": [
    "# VGG16\n",
    "\n",
    "# Initialize model\n",
    "model = models.vgg16(pretrained=True)\n",
    "\n",
    "# Applying a forward pass\n",
    "# The input to the network needs to be an autograd Variable. \n",
    "# To make it 4 dimension use unsqueeze\n",
    "input_data = img_tensor.unsqueeze(0)\n",
    "img_variable = Variable(input_data)\n",
    "start = time.time()\n",
    "fc_out = model(img_variable)\n",
    "time_d = time.time() - start\n",
    "\n",
    "print(\"Using VGG16 classifier\")\n",
    "print(\"Predicted Label: \", labels[fc_out.data.numpy().argmax()])\n",
    "print(\"Time Taken: \", time_d, \" seconds\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Using VGG19 classifier\n",
      "Predicted Label:  tabby, tabby cat\n",
      "Time Taken:  1.27219319344  seconds\n"
     ]
    }
   ],
   "source": [
    "# VGG19\n",
    "\n",
    "# Initialize model\n",
    "model = models.vgg19(pretrained=True)\n",
    "\n",
    "# Applying a forward pass\n",
    "# The input to the network needs to be an autograd Variable. \n",
    "# To make it 4 dimension use unsqueeze\n",
    "input_data = img_tensor.unsqueeze(0)\n",
    "img_variable = Variable(input_data)\n",
    "start = time.time()\n",
    "fc_out = model(img_variable)\n",
    "time_d = time.time() - start\n",
    "\n",
    "print(\"Using VGG19 classifier\")\n",
    "print(\"Predicted Label: \", labels[fc_out.data.numpy().argmax()])\n",
    "print(\"Time Taken: \", time_d, \" seconds\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/home/abhi/.virtualenvs/frameworks/local/lib/python2.7/site-packages/torchvision/models/squeezenet.py:94: UserWarning: nn.init.kaiming_uniform is now deprecated in favor of nn.init.kaiming_uniform_.\n",
      "  init.kaiming_uniform(m.weight.data)\n",
      "/home/abhi/.virtualenvs/frameworks/local/lib/python2.7/site-packages/torchvision/models/squeezenet.py:92: UserWarning: nn.init.normal is now deprecated in favor of nn.init.normal_.\n",
      "  init.normal(m.weight.data, mean=0.0, std=0.01)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Using Suqeezenet v1.1 classifier\n",
      "Predicted Label:  tabby, tabby cat\n",
      "Time Taken:  0.0576660633087  seconds\n"
     ]
    }
   ],
   "source": [
    "# Squeezenet-v1.1\n",
    "\n",
    "# Initialize model\n",
    "model = models.squeezenet1_1(pretrained=True)\n",
    "\n",
    "# Applying a forward pass\n",
    "# The input to the network needs to be an autograd Variable. \n",
    "# To make it 4 dimension use unsqueeze\n",
    "input_data = img_tensor.unsqueeze(0)\n",
    "img_variable = Variable(input_data)\n",
    "start = time.time()\n",
    "fc_out = model(img_variable)\n",
    "time_d = time.time() - start\n",
    "\n",
    "print(\"Using Suqeezenet v1.1 classifier\")\n",
    "print(\"Predicted Label: \", labels[fc_out.data.numpy().argmax()])\n",
    "print(\"Time Taken: \", time_d, \" seconds\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Author - Tessellate Imaging - https://www.tessellateimaging.com/\n",
    "\n",
    "## Monk Library - https://github.com/Tessellate-Imaging/monk_v1\n",
    "\n",
    "    Monk is an opensource low-code tool for computer vision and deep learning\n",
    "\n",
    "### Monk features\n",
    "- low-code\n",
    "- unified wrapper over major deep learning framework - keras, pytorch, gluoncv\n",
    "- syntax invariant wrapper\n",
    "\n",
    "\n",
    "### Enables\n",
    "- to create, manage and version control deep learning experiments\n",
    "- to compare experiments across training metrics\n",
    "- to quickly find best hyper-parameters\n",
    "\n",
    "\n",
    "### At present it only supports transfer learning, but we are working each day to incorporate\n",
    "- GUI based custom model creation\n",
    "- various object detection and segmentation algorithms\n",
    "- deployment pipelines to cloud and local platforms\n",
    "- acceleration libraries such as TensorRT\n",
    "- preprocessing and post processing libraries\n",
    "\n",
    "## To contribute to Monk AI or Pytorch RoadMap repository raise an issue in the git-repo or dm us on linkedin \n",
    " - Abhishek - https://www.linkedin.com/in/abhishek-kumar-annamraju/\n",
    " - Akash - https://www.linkedin.com/in/akashdeepsingh01/"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.9"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}