#1 Computer Vision Workshop Intro.ipynb

{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import IPython.display as D"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Computer Vision Workshop Intro\n",
    "\n",
    "![](https://computervisionworkshop.blob.core.windows.net/media/chocolates.jpg)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This intro is based on the work detecting complex policies in the following [real life code story](https://www.microsoft.com/developerblog/2017/07/31/using-object-detection-complex-image-classification-scenarios/)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# The AI Computer Vision Revolution \n",
    "\n",
    "Recent developments in computer vision have changed the computer vision landscape. Many scenarios that were once considered possible only in  sci-fi have recently become as easy as consuming an API."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**XKCD in September 24th 2014**\n",
    " \n",
    "![](https://imgs.xkcd.com/comics/tasks.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Microsoft Developer Blog [Bird Detection with Azure ML](https://www.microsoft.com/developerblog/2017/10/24/bird-detection-with-azure-ml-workbench/)  on October 24th 2017**\n",
    "\n",
    "![](https://codestoryedge.azureedge.net/developerblog/wp-content/uploads/kittywake_data.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Yet in practice not every computer vision problem is related to birds, flowers, cats and dogs.\n",
    "\n",
    "Today we are going to review a real world computer vision use case from the retail sector and are going to compare and contrast some of the different approaches and technologies availble to solve the problem. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Real Life Code\n",
    "\n",
    "Today we are going to go through a real world complex image classification use case and explore the spectrum of solutions to solve the problem.\n",
    "\n",
    "We recently partnered with a large manufacturer of confectionery products in Central & Eastern Europe, to build a machine learning model which validates whether distributors are stocking chocolates correctly. The company we worked with has a huge distribution network of supermarket chains across over fourteen countries. Each of these distributors is required to arrange chocolates on their stands according to standardized policies. Each policy describes what shelf a given chocolate should be on and in what order it should be stocked.\n",
    "\n",
    "There are huge costs associated with “routine” audit activities to enforce these policies. Our partner wanted to develop a system in which an auditor or store manager could take a picture and be told immediately whether the shelf was stocked correctly, like in the image below.\n",
    "\n",
    "![](https://computervisionworkshop.blob.core.windows.net/media/policy.jpg)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Even with state of the art methods this is challenging for a multiude of reasons such as\n",
    "\n",
    "- Image quality\n",
    "- Image angle \n",
    "- Subtle Policy Violations \n",
    "![](https://computervisionworkshop.blob.core.windows.net/media/obvious violation.jpg)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Even though some images of misstocked shelves such as the image on the right are flagrant violations of policy others such as the violation on the left are much more subtle. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Potential Technology Solutions to this Problem"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Microsoft provides a spectrum of AI services that can be used for solving Computer Vision Tasks like this one, each solution can be operationalizaed on Azure. From [Cognitive Servies](https://azure.microsoft.com/en-us/services/cognitive-services/) to [Azure ML workbench](https://azure.microsoft.com/en-us/services/machine-learning-services/) and [CNTK/Cognitive Toolkit](https://www.microsoft.com/en-us/cognitive-toolkit/) each techonology and approach has different advantages and tradeoffs that fit the spectrum of computer vision use cases."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Cognitive Services \n",
    "\n",
    "Azure Cognitive Services are a set of out of the box API's and high level services that help you get started with AI tasks with out needing any domain expierence.\n",
    "\n",
    "![](https://www.programmableweb.com/sites/default/files/microsoft%20computer%20vision%20API.jpg)\n",
    "![](https://computervisionworkshop.blob.core.windows.net/media/cognitive services.jpg)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Azure ML Workbench & ML Studio\n",
    "\n",
    "![](https://azurecomcdn.azureedge.net/cvt-f50521786d7494d1129069f701d8751293420aaad404b4aa8260a68606055807/images/page/services/machine-learning-studio/simple-scalable-cutting-edge.jpg)\n",
    "\n",
    "**Microsoft Azure Machine Learning Studio** is a collaborative, drag-and-drop tool you can use to build, test, and deploy predictive analytics solutions on your data. Machine Learning Studio publishes models as web services that can easily be consumed by custom apps or BI tools such as Excel. It is typically used less for **Computer Vision Scenarios**.\n",
    "\n",
    "**Azure ML Workbench** is a downloadable desktop application and command-line interface for Windows and MacOS. Built-in data preparation learns your data preparation steps as you perform them. Project management, run history, and notebook integration bolsters your productivity. Take advantage of the best open source frameworks, including TensorFlow, Cognitive Toolkit, Spark ML, and scikit-learn.\n",
    "\n",
    "![](https://docs.microsoft.com/en-us/azure/machine-learning/preview/media/scenario-big-data/architecture.png)\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Cognitive Toolkit CNTK\n",
    "\n",
    "![](https://msdnshared.blob.core.windows.net/media/2017/03/033117_1829_MicrosoftUp1.png)\n",
    "\n",
    "The Microsoft Cognitive Toolkit—previously known as CNTK—empowers you to harness the intelligence within massive datasets through deep learning by providing uncompromised scaling, speed, and accuracy with commercial-grade quality and compatibility with the programming languages and algorithms you already use. Hear about the team that developed the Cognitive Toolkit, or read more below.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In the following workshop we are going to explore some of these technologies and architectures to solve our own complex policy classification scenario."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Workshop Challenge "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this workshop we will building a complex policy classifier similar to the one outlined the above scenario. For an image to be stocked validly it must be stocked in the following order.\n",
    "\n",
    "- Top Shelf (Soda)\n",
    "- Middle Shelf (Juice)\n",
    "- Bottom Shelf (water)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Valid Example (Soda, Juice, Water)\n",
    "\n",
    "![](dataset/Classification/Valid/IMG_4321.JPG)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Invalid Example \n",
    "\n",
    "![](dataset/Classification/Invalid/IMG_4194.JPG)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Workshop Goals\n",
    "\n",
    "By the end of this workshop you should better \n",
    "\n",
    "- Understand the AI Computer Vision Landscape \n",
    "\n",
    "- Be able to build an deploy a custom vision service model\n",
    "\n",
    "- Build a CNN image recognition model with CNTK and transfer learning \n",
    "\n",
    "- Use VoTT and FasterRCNN to annotate and train an object detection model for complex classficiation scenarios\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}