building-blocks-of-gans.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Beginning of GANS ERA\n",
    "Machine learning (ML) and deep learning can be described by two terms: generative and discriminative modeling."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Descriptive modelling\n",
    "1. We create a machine learning model that uses conv layers or other to understand division of data\n",
    "2. Next we collect a dataset that has both training + validation\n",
    "3. Train the model\n",
    "4. Now predict which datapiont belongs to particular class"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Generative Modeling\n",
    "\n",
    "1. create model that learn how to reproduce painings\n",
    "2. collect train + val data\n",
    "3. Train your model\n",
    "4. use model to produce examples that use similarity metrics to verify ability of model"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## How it Works"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* Discriminative models get their power from more data\n",
    "* Not designed to work with unlabeled data<p>\n",
    "\n",
    "* Generative create a probabilistic model of each class in order to estimate distribution\n",
    "* Ability to use unlabeled data since it learns labels during training"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Backprop, for short, allows an artificial neural network to adjust the weights of each layer at\n",
    "every epoch of training."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Basics"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "1. **Counterfeiter (generator) goal**: Produce products so that the cop can not dicriminate b/w real and fake\n",
    "\n",
    "2. **Cop(discriminator)** : Detect anomalous products by using prior experience to classify real and fake"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* Generator do it by maximizing the likelihood that the discriminator misclassifies its output as real\n",
    "* Optimize towards towards a goal of 0.5, where discriminator can't distimguish between real and generated"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Double feedback loop"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* The discriminator is in a feedback loop with the ground truth of the images, which we know.\n",
    "* The generator is in a feedback loop with the discriminator.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Building Block - Generator\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's try to understand how we create a generator conceptually\n",
    "\n",
    "<ul><li>First generators take sample from latent space and create a relationhip between latent space and output\n",
    "    <li> Create a NN [from input to output]\n",
    "        <li> we will train the generator in adversarial mode where we connect generator and discriminator\n",
    "            <li> Generator can be used for inference after training"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Latent Space"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Latent means hidden**\n",
    "<p> You observe some data which in the space and you want to map to a latent space where similar points are closer together\n",
    "    <br>**In other words, the latent space is the space where your features lie**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "class Generator:\n",
    "    def __init__(self):\n",
    "        slef.initVaiable = 1\n",
    "        \n",
    "    def loss_func(self):\n",
    "        return\n",
    "   \n",
    "    #to contruct actual model of NN\n",
    "    def buildModel(self):\n",
    "        return\n",
    "\n",
    "    def trainmodel(self,X,Y):\n",
    "        return"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Discriminator"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We discussed previously that it is used to determine whether the output of generator and a real image are real or fake"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "1. Build a conv NN to classifiy real or fake (binary)\n",
    "2. create dataset of real dataset and fake dataset by generator\n",
    "3. train discriminator on real and fake data\n",
    "4. Learn to balance training of discriminator with generator"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}