Blank+Exercise.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's build our our own machine learning algorithm that uses the principles in the first Obama/Trump tweet example to learn what Trump tweets and Obama tweets and be able to predict whether an unknown tweet is Trump or Obama.\n",
    "\n",
    "We have 9 trump tweets and 10 Obama tweets. We will use the first 6 of each to teach our system what is an Obama Tweet and what is a Trump tweet. We will then give the system our 3 Trump tweets and our 4 Obama tweets and ask it to tell us who tweeted. \n",
    "\n",
    "## 1 Load the training data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2 Create the training and test sets:\n",
    "\n",
    "Before we start coding our machine learning algorithm we need to prepare our data. Typically 80% of any machine learning or NLP activity is data preparation. Here we will take our 19 tweets and create a 'training set' of 12 tweets and a test set of 7 tweets - 4 Obama and 3 trump.\n",
    "\n",
    "We will train our algorithm with the 12 training tweets by giving feeding it each tweet and telling it the author so the algorithm can learn what an Obama tweet and a Trump tweet is like.\n",
    "\n",
    "Once the machine has been trained we will give it the remaining 7 tweets without telling it who is author and see how accurately it classifies the 7 unknown tweets. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3 Probability\n",
    "\n",
    "In this corpus there are 6 training tweets for Trump and 6 for Obama so by inspection we can deduce that:\n",
    "\n",
    "P(Obama) = 0.5\n",
    "\n",
    "P(Trump) = 0.5\n",
    "\n",
    "Thus without applying Bayes Theorem there is a 50/50 chance the training tweets are either Trump or Obama.\n",
    "\n",
    "## 4 Prior Probability\n",
    "\n",
    "### 4.1 Naive Bayes Learning\n",
    "\n",
    "Now let's calculate the probabilty of Obama and Trump using each word in our corpus. We will concatenate all of the tweets for an author (Obama then Trump) then create we will iterate through every word in every tweet and build a learnings dictionary that will be the probability of that word being used by its author.\n",
    "\n",
    "For example the word grateful may occur two times in a total of 130 words so our dictionary entry would look like:\n",
    "\n",
    "`grateful: 0.015` "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 4.2 Naive Bayes Likelihood\n",
    "\n",
    "We'll create an algorithm that uses the data we have learned to calculate the likelihood of each of our TEST tweets being either Obama or Trump. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 5 Naive Bayes Classification\n",
    "\n",
    "Let's use our probabilty and likelihood in Bayes Theorem to classify our test tweets:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Exercises\n",
    "\n",
    "We'll do a short exercise that builds a machine learning Naive Bayes Classifier so you get some hands-on experience of using Jupytr Notebook, Python and NLP/Machine Learning.\n",
    "\n",
    "NOTE: Python has no curly braces so indentation is very important. Copy and paste carefully.\n",
    "\n",
    "### Exercise 1:\n",
    "\n",
    "Using our nlp_exercise_code.txt copy and paste the code from sections 1, 2, 4.1, 4.2 and 5. When you have pasted the code click the run button at the top of the page and you will see the output of that section of code appear below the code snippet (except for 4.2 - it's just a method so understand the code). Take a good look at the output so you understand what the machine is doing.\n",
    "\n",
    "### Exercise 2:\n",
    "\n",
    "If you've got it right then you should see every result as __Trump or Obama - equal probability: 0.00 0.00__ which isn't very helpful. This is because in every tweet there is at least one word that the predicted author did not use. For example, if the tweet contains \"@csforall\" and we are predicting the likelihood of Trump using that word our __p_likelihood__ algorithm will return 0 meaning the product of all the words in that likelihood will be 0: 0.0 * 0.1 * 0.2 will always be 0. To fix this we apply Laplace Smoothing:\n",
    "\n",
    "change this line in p_likelihood to apply Laplace Smoothing: \n",
    "\n",
    "`likelihood = likelihood * (learnings.get(word,0.0));` \n",
    "\n",
    "to \n",
    "\n",
    "`likelihood = likelihood * (learnings.get(word,0.0) + 1.0);`\n",
    "\n",
    "Now run the code in sections 4.2 and 5 again. You should see classifications now with the first two Obama tweets being classified incorrectly as Trump tweets. Note that generally the classifications are very similar and not far off 50%."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Exercise 3:\n",
    "\n",
    "Our data set is very small and in most cases all but the most common words (stop words) are present only once. We added the Laplace Smoothing as 1 too so in many cases a word we have applied smoothing to because it's not in our training set.\n",
    "\n",
    "Let's attempt to offset that bias by multiplying every real word occrence by 10 and reducing the effect of the Laplace Smoothing:\n",
    "\n",
    "change this line in p_words \n",
    "\n",
    "`learnings[word] = ( float(words.count(word))) / float(len(words))`\n",
    "\n",
    "to\n",
    "\n",
    "`learnings[word] = ( float(words.count(word) * 10.0)) / float(len(words))`\n",
    "\n",
    "Now run the code in sections 4.1, 4.2 and 5 again and we should now see more significant differences in our classification. The first tweet will still be incorrectly predicted.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Exercise 4:\n",
    "\n",
    "Stopwords are the everyday words like 'a', 'the', 'and', 'but' etc.. if we remove these words then we are left with the more significant words which should increase the accuracy of our predictions:\n",
    "\n",
    "In `def clean` uncomment\n",
    "\n",
    "`stopwords = [\"a\", \"about\", \"above\", \"above\", \"across\"....]`\n",
    "\n",
    "and comment out\n",
    "\n",
    "`stopwords = []`\n",
    "\n",
    "This will remove any stop words from the tweets. Run sections 2,4.1,4.2 and 5 again. Now we have two tweets that are exact matches - 50/50. Why is that ?\n",
    "    "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}